UltraAttn: Efficiently Parallelizing Attention through Hierarchical Context-Tiling

Haoyu Yang, Zan Zong, Yuyang Jin, Kinman Lei, Jiaao He, Qigang Yang, Jidong Zhai
January, 2025
Cite DOI URL
Type
Conference paper
Publication
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2025, St. Louis, MO, USA, November 16-21, 2025
Haoyu Yang
Haoyu Yang
Ph.D. Student
Zan Zong
Zan Zong
Postdoc
Yuyang Jin
Yuyang Jin
Researcher
Kinman Lei
Kinman Lei
Ph.D. Students
Jiaao He
Jiaao He
Ph.D. Student
Qigang Yang
Qigang Yang
Ph.D. Student
Jidong Zhai
Jidong Zhai
Professor
(长聘教授、博士生导师)