Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving

Hui Zeng, Daming Zhao, Pengfei Yang, WenXuan Hou, Tianyang Zheng, Hui Li, Weiye Ji, Jidong Zhai
January, 2025
Cite DOI URL
Type
Journal article
Publication
CoRR
Jidong Zhai
Jidong Zhai
Professor
(长聘教授、博士生导师)