LLM

Compute Or Load KV Cache? Why Not Both?
Recent advancements in Large Language Models (LLMs) have significantly increased context window sizes, enabling sophisticated …
Shuowei Jin, Xueshen Liu, Qingzhao Zhang, Z. Morley Mao
PDF
Eagle: Efficient Training-Free Router for Multi-LLM Inference
The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection …
Zesen Zhao, Shuowei Jin, Z. Morley Mao
PDF
AutoSpec: Automated Generation of Neural Network Specifications
The increasing adoption of neural networks in learning-augmented systems highlights the importance of model safety and robustness, …
Shuowei Jin, Francis Y. Yan, Cheng Tan, Anuj Kalia, Xenofon Foukas, Z. Morley Mao
PDF
Adaptive Skeleton Graph Decoding
Large language models (LLMs) have seen significant adoption for natural language tasks, owing their success to massive numbers of model …
Shuowei Jin, Yongji Wu, Haizhong Zheng, Qingzhao Zhang, Matthew Lentz, Z. Morley Mao, Atul Prakash, Feng Qian, Danyang Zhuo
PDF