Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
Qwen, An Yang, Baosong Yang +39 authors
Qwen2.5, an enhanced series of large language models, demonstrates superior performance across various benchmarks and use cases through extensive pre-training and advanced post-training techniques.
Shaobo Wang, Xuan Ouyang, Tianyi Xu +9 authors
OPUS is a dynamic data selection framework that improves pre-training efficiency by scoring data candidates based on optimizer-induced update projections in a stable proxy-derived target space, achieving superior performance with reduced computational overhead.
Vladimir Arkhipkin, Vladimir Korviakov, Nikolai Gerasimenko +22 authors
Kandinsky 5.0 is a family of state-of-the-art generative models for high-resolution images and short videos, featuring model lineups with varying parameters and enhanced training techniques to achieve superior quality and performance.
Jiawei Zhao, Zhenyu Zhang, Beidi Chen +3 authors
Gradient Low-Rank Projection (GaLore) improves memory efficiency for training large language models without sacrificing performance, enabling pre-training of 7B models on consumer GPUs.
Weimin Xiong, Shuhao Gu, Bowen Ye +5 authors
A large-scale GUI dataset was created by automatically extracting interaction trajectories from internet videos, enabling improved performance in GUI agents through pre-training on this diverse collection.
Shen Nie, Fengqi Zhu, Zebin You +7 authors
LLaDA, a diffusion model trained from scratch, outperforms autoregressive models in benchmarks and demonstrates strong instruction-following capabilities, challenging the dominance of ARMs in LLMs.
Daixuan Cheng, Yuxian Gu, Shaohan Huang +3 authors
Instruction Pre-Training enhances language models by generating and incorporating instruction-response pairs into unsupervised multitask pre-training.
Xiaomi LLM-Core Team, Bingquan Xia, Bowen Shen +61 authors
MiMo-7B, a large language model optimized for reasoning tasks, enhances pre-training with data mixing and Multi-Token Prediction, and post-training with reinforcement learning on math and programming problems, achieving superior performance over larger models.
Daixuan Cheng, Shaohan Huang, Furu Wei
Transforming raw corpora into reading comprehension texts enhances large language models' question-answering abilities across various domains and improves performance on general benchmarks.
Yang Shen, Xiu-Shen Wei, Yifan Sun +6 authors
An auto-regressive vision-language model trained with explanatory instructions achieves instruction-level zero-shot capabilities and strong zero-shot generalization in computer vision tasks.
Yixin Ye, Zhen Huang, Yang Xiao +3 authors
LIMO, a new model, achieves high mathematical reasoning performance using minimal training data, challenging the notion that extensive datasets are necessary for complex reasoning.
Wenyi Hong, Weihan Wang, Ming Ding +22 authors
The CogVLM2 family of visual language models, including CogVLM2, CogVLM2-Video, and GLM-4V, advances image and video understanding with enhanced architectures and training methods.
Fuzhao Xue, Yukang Chen, Dacheng Li +15 authors
LongVILA, a full-stack solution for long-context vision-language models, introduces Multi-Modal Sequence Parallelism for efficient training and inference, and a five-stage training pipeline that enhances long video processing and captioning performance.
Xuekai Zhu, Daixuan Cheng, Hengli Li +7 authors
The use of synthetic data in language model training leads to model collapse, which is mitigated by token-level editing of human-produced data to create semi-synthetic data.
Tiannan Wang, Jiamin Chen, Qingrui Jia +43 authors
Weaver, a family of specialized large language models, achieves superior writing capabilities through pre-training and fine-tuning methods, surpasses GPT-4 in various writing tasks, and supports retrieval-augmented generation and tool usage.
Viraat Aryabumi, Yixuan Su, Raymond Ma +6 authors
The inclusion of code data during pre-training significantly improves LLMs' performance across various natural language and world knowledge tasks, beyond code generation.
Noveen Sachdeva, Benjamin Coleman, Wang-Cheng Kang +6 authors
Data-efficient methods like Ask-LLM and Density sampling improve model quality and training efficiency in large language models by optimizing data selection and coverage.
Zheng Cai, Maosong Cao, Haojiong Chen +97 authors
InternLM2 is an open-source LLM that outperforms predecessors through innovative pre-training and optimization techniques, including Supervised Fine-Tuning and Conditional Online Reinforcement Learning from Human Feedback.
Yiheng Xu, Hongjin Su, Chen Xing +13 authors
Lemur and Lemur-Chat are open-source language models combining language and coding capabilities, achieving state-of-the-art performance across various benchmarks and agent tasks.
Wenyu Du, Tongxu Luo, Zihan Qiu +5 authors
A depthwise stacking operator called $G_{\text{stack}}$ accelerates LLM pre-training, improving performance with less computational expense and providing guidelines for effective use.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号