PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications

Kuntai Du; Bowen Wang; Chen Zhang; Yi‐Ming Cheng; Qing Lan; H. S. Sang; Yihua Cheng; Jiayi Yao; Xiaoxuan Liu; Yong Qiao; Ion Stoica; Junchen Jiang

doi:10.1145/3731569.3764834

Related publications

Preprint2025

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications

Kuntai Du, Bowen Wang, Chen Zhang, Yi‐Ming Cheng, Qing Lan, H. S. Sang, Yihua Cheng, Jiayi Yao, Xiaoxuan Liu, Yong Qiao, Ion Stoica, Junchen Jiang

Preprint2023

SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads

Alind Khare, Dhruv Garg, Sukrit Kalra, Snigdha Grandhi, Ion Stoica, Alexey Tumanov

Preprint2025

AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model

Quanlong Zheng, Yanhao Zhang, Hongmei Ji, Jian Ma, Zhitong Zheng, Zhenyi Lin, Hongxia Deng, Xin Zou, Xiaojie Yin, Ran Wang, Liang Cai, Haijing Liu, Yuqing Qiu, Ke Chen, Zheng Li, Chi Xie, Huafei Li, Chenxing Li, Chuangchuang Wang, Kai Tang, Zhiguang Zhu, Weijun Gao, Rui Wang, Jun Wu, Chao Liu,

Article2024

Large Language Model (LLM)-Enabled Graphs in Dynamic Networking

Geng Sun, Yixian Wang, Dusit Niyato, Jiacheng Wang, Xinying Wang, H Vincent Vincent Poort, Khaled B. Letaief

Preprint2024

Large Language Model (LLM)-enabled Graphs in Dynamic Networking

Geng Sun, Yixian Wang, Dusit Niyato, Jiacheng Wang, Xinying Wang, H Vincent Vincent Poort, Khaled B. Letaief

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications

Discussion(0)

Related publications

PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications

SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads

AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model

Large Language Model (LLM)-Enabled Graphs in Dynamic Networking

Large Language Model (LLM)-enabled Graphs in Dynamic Networking