SmartMoE: Efficiently Training Sparsely-Activated Models through Combining Offline and Online Parallelization

Mingshu Zhai, Jiaao He, Zixuan Ma, Zan Zong, Runqing Zhang, Jidong Zhai
January, 2023
Cite URL
Type
Conference paper
Publication
Proceedings of the 2023 USENIX Annual Technical Conference, USENIX ATC 2023, Boston, MA, USA, July 10-12, 2023
Mingshu Zhai
Mingshu Zhai
Ph.D. Student
Jiaao He
Jiaao He
Ph.D. Student
Zixuan Ma
Zixuan Ma
Ph.D. Student
Zan Zong
Zan Zong
Postdoc
Runqing Zhang
Runqing Zhang
Ph.D. Student
Jidong Zhai
Jidong Zhai
Professor
(长聘教授、博士生导师)