Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
Eunbi Choi, Kibong Choi, Seokhee Hong +62 authors
K-EXAONE is a multilingual language model with a Mixture-of-Experts architecture that achieves competitive performance on various benchmarks while supporting multiple languages and long-context windows.
Tiwei Bie, Maosong Cao, Kun Chen +28 authors
LLaDA2.0 converts auto-regressive models into discrete diffusion large language models with a novel training scheme, achieving superior performance and efficiency at scale.
Songhao Wu, Ang Lv, Ruobing Xie +1 authors
Researchers propose a novel router redesign for Mixture-of-Experts models that aligns router rows with the principal singular directions of expert matrices using Manifold Power Iteration to improve model effectiveness.
Vishesh Tripathi, Abhay Kumar
Grouped Query Experts (GQE) improves Transformer efficiency by selectively activating query heads based on token content while maintaining key-value cache benefits of grouped-query attention.
DeepSeek-AI, Qihao Zhu, Daya Guo +37 authors
DeepSeek-Coder-V2, a Mixture-of-Experts language model, excels in code-specific tasks by enhancing coding and mathematical reasoning capabilities while expanding language support and context length.
Alexander Amini, Anna Banaszak, Harold Benoit +30 authors
LFM2, a family of compact foundation models, achieves high efficiency and performance on-device through hardware-in-the-loop architecture search and advanced training techniques, supporting various tasks including multimodal applications.
Shuailei Ma, Jiaqi Liao, Xinyang Wang +24 authors
LingBot-Video presents a DiT-based video pretraining framework with Mixture-of-Experts architecture, specialized data augmentation, and multi-dimensional reward system for embodied intelligence applications.
Damai Dai, Chengqi Deng, Chenggang Zhao +14 authors
The DeepSeekMoE architecture improves expert specialization in Mixture-of-Experts models by segmenting experts and isolating shared ones, achieving better performance and computational efficiency compared to GShard and other models.
Dahyun Kim, Chanjun Park, Sanghoon Kim +15 authors
A novel technique, depth up-scaling (DUS), efficiently enhances large language models (LLMs) without complex changes, building SOLAR 10.7B that outperforms existing open-source models in various NLP tasks, including instruction-following.
Haotian Zhang, Mingfei Gao, Zhe Gan +20 authors
MM1.5 enhances multimodal language models with a data-centric approach, supporting diverse tasks like text-rich image understanding and video comprehension through careful data curation and training strategies.
Zihan Wang, Deli Chen, Damai Dai +3 authors
Research on parameter-efficient fine-tuning for Large Language Models with Mixture-of-Experts architecture reveals that task-specific expert tuning can enhance performance and efficiency.
Junyao Yang, Yucheng Shi, Zhongzhi Li +4 authors
T1 is a 122B Mixture-of-Experts model trained with reinforcement learning to execute long-horizon terminal tasks in a cloud sandbox, achieving state-of-the-art results through stable actor-critic optimization and out-of-distribution training.
Junghwan Lim, Joon Son Chung, Sungmin Lee +24 authors
Motif 3 is a large sparse mixture-of-experts language model using grouped differential latent attention and specialized training techniques to achieve strong reasoning, coding, and long-context performance.
Róbert Csordás, Piotr Piękos, Kazuki Irie +1 authors
SwitchHead reduces memory and compute for Transformers by using Mixture-of-Experts layers, matching performance with significant speedup.
DiffusionGemma Team, Adrien Ali Taïga, James Assiene +41 authors
DiffusionGemma is a fine-tuned mixture-of-experts language model that uses discrete diffusion to generate text blocks in parallel, achieving high speed while preserving capabilities like multimodal inputs and reasoning.
Sangjin Choi, Sukmin Cho, Yifan Xiong +3 authors
ELDR is an expert-locality-aware decode router for prefill-decode disaggregated Mixture-of-Experts serving that improves performance by predicting expert activations and routing requests accordingly.
Elias Frantar, Dan Alistarh
QMoE enables efficient execution of trillion-parameter MoE models on affordable hardware through compression and bespoke GPU decoding.
Zeyue Xue, Guanglu Song, Qiushan Guo +4 authors
A text-conditional image diffusion model named RAPHAEL generates high-quality and artistic images by stacking mixture-of-experts layers, outperforming existing models in style switching and zero-shot FID score.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号