Mixture of experts

Large-scale pretrained AI models have shown state-of-the-art accuracy in a series of important applications. As the size of pretrained …