Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
606 篇论文 · 按点赞排序
Ang Lv, Jin Ma, Yiyuan Ma +1 authors
An expert-router coupling (ERC) loss aligns router decisions with expert capabilities in Mixture-of-Experts (MoE) models by enforcing constraints on internal activations, improving performance and computational efficiency.
Jiuhai Chen, Zhiyang Xu, Xichen Pan +10 authors
A diffusion transformer is used in a unified multimodal model framework to improve image generation while maintaining image understanding capabilities.
MiniCPM Team, Chaojun Xiao, Yuxuan Li +72 authors
MiniCPM4, a highly efficient large language model for end-side devices, achieves superior performance using innovations in sparse attention, pre-training datasets, training algorithms, and inference systems.
Xun Liang, Jiawei Yang, Yezhaohui Wang +9 authors
SurveyX enhances automated survey generation through innovative preprocessing and post-processing methods, improving content and citation quality beyond existing systems.
Zhenru Zhang, Chujie Zheng, Yangzhen Wu +6 authors
Process Reward Models improve performance in mathematical reasoning by integrating Monte Carlo estimation with LLM-as-a-judge, addressing biases in Best-of-N evaluation and step-wise error identification.
Tianyu Fu, Zihan Min, Hanling Zhang +4 authors
Cache-to-Cache (C2C) enables direct semantic communication between LLMs using neural network projections, improving accuracy and reducing latency compared to text-based communication.
Ling Yang, Ye Tian, Bowen Li +4 authors
MMaDA, a multimodal diffusion foundation model, achieves superior performance through a unified architecture, mixed long chain-of-thought fine-tuning, and a unified policy-gradient-based RL algorithm.
Alex L. Zhang, Tim Kraska, Omar Khattab
We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference strategy that treats long prompts as part of an external environment and allows the LLM to programmatically examine, decompose, and recursively call itself over snippets of the prompt. We find that RLMs successfully handle inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of base LLMs and common long-context scaffolds across four diverse long-context tasks, while having comparable (or cheaper) cost per query.
Qiushi Sun, Jingyang Gong, Yang Liu +6 authors
A unified multimodal code corpus and model enable high-quality code generation from both text and visual inputs, outperforming commercial models.
Violet Xiang, Charlie Snell, Kanishk Gandhi +11 authors
The Meta Chain-of-Thought framework extends traditional CoT by explicitly modeling reasoning, enabling more powerful and human-like reasoning in LLMs through process supervision, synthetic data generation, and reinforcement learning.
Hoiyeong Jin, Hyojin Jang, Jeongho Kim +6 authors
InsertAnywhere framework enhances video object insertion by generating geometrically consistent and visually coherent scenarios through 4D aware mask generation and diffusion-based synthesis.
Yiping Wang, Qing Yang, Zhiyuan Zeng +11 authors
Reinforcement learning with verifiable reward using one training example significantly enhances math reasoning capabilities of large language models.
Jinyuan Fang, Yanwen Peng, Xi Zhang +12 authors
A survey of self-evolving AI agents that adapt to dynamic environments through automatic enhancement based on interaction data and feedback.
Jing Xiong, Liyang Fan, Hui Shen +4 authors
Denoising Positional Encoding (DoPE) enhances length generalization in Transformer models by detecting and mitigating noisy frequency bands in positional embeddings, improving retrieval accuracy and reasoning stability.
Runqi Qiao, Qiuna Tan, Minghan Yang +10 authors
V-Thinker, a multimodal reasoning assistant using reinforcement learning, enhances image-interactive thinking by synthesizing datasets and aligning perception for improved performance in vision-centric tasks.
Yiwen Huang, Aaron Gokaslan, Volodymyr Kuleshov +1 authors
A new regularization approach and simplified architecture for GANs, named R3GAN, surpasses existing methods including StyleGAN2 and diffusion models across multiple datasets.
Justin Cui, Jie Wu, Ming Li +6 authors
A method is proposed to enhance long-horizon video generation by using sampled segments from self-generated long videos to guide student models, maintaining quality and consistency without additional supervision or retraining.
Shizhe Diao, Yu Yang, Yonggan Fu +12 authors
Pre-training datasets are typically collected from web content and lack inherent domain divisions. For instance, widely used datasets like Common Crawl do not include explicit domain labels, while manually curating labeled datasets such as The Pile is labor-intensive. Consequently, identifying an optimal pre-training data mixture remains a challenging problem, despite its significant benefits for pre-training performance. To address these challenges, we propose CLustering-based Iterative Data Mixture Bootstrapping (CLIMB), an automated framework that discovers, evaluates, and refines data mixtures in a pre-training setting. Specifically, CLIMB embeds and clusters large-scale datasets in a semantic space and then iteratively searches for optimal mixtures using a smaller proxy model and a predictor. When continuously trained on 400B tokens with this mixture, our 1B model exceeds the state-of-the-art Llama-3.2-1B by 2.0%. Moreover, we observe that optimizing for a specific domain (e.g., Social Sciences) yields a 5% improvement over random sampling. Finally, we introduce ClimbLab, a filtered 1.2-trillion-token corpus with 20 clusters as a research playground, and ClimbMix, a compact yet powerful 400-billion-token dataset designed for efficient pre-training that delivers superior performance under an equal token budget. We analyze the final data mixture, elucidating the characteristics of an optimal data mixture. Our data is available at: https://research.nvidia.com/labs/lpr/climb/
Yikun Wang, Zuyan Liu, Ziyi Wang +3 authors
GeoVista, an agentic model integrating tool invocation and reinforcement learning, achieves high geolocalization performance on GeoBench, outperforming open-source models and matching closed-source models.
Junyu Zhang, Runpei Dong, Han Wang +8 authors
AlphaOne dynamically modulates reasoning in large models by introducing $\alpha$ moment and Bernoulli process for slow thinking, improving efficiency and capability across diverse domains.
Yuchen Fan, Kaiyan Zhang, Heng Zhou +15 authors
LLMs can serve as efficient simulators for RL tasks by leveraging internal knowledge, reducing reliance on external search engines and improving sim-to-real transfer.
Ibragim Badertdinov, Alexander Golubev, Maksim Nekrashevich +6 authors
A novel pipeline extracts real-world, interactive software engineering tasks from GitHub to create SWE-rebench, improving the evaluation of reinforcement learning models in SWE.
Jindong Jiang, Xiuyu Li, Zhijian Liu +13 authors
STORM integrates a temporal encoder using the Mamba State Space Model to enhance video understanding and reduce computational costs in multimodal large language models.
Siyuan Li, Luyuan Zhang, Zedong Wang +8 authors
MergeVQ and MergeAR enhance MIM with VQ by integrating token merging and KV cache compression, improving both image generation and visual representation learning.
Asaf Yehudai, Lilach Eden, Alan Li +5 authors
This survey analyzes evaluation methodologies for large language model-based agents, covering fundamental capabilities, application-specific benchmarks, and generalist agents, highlighting trends and gaps in the field.
Enze Zhang, Jiaying Wang, Mengxi Xiao +7 authors
A new evaluation framework, DITING, and a reasoning-driven multi-agent evaluation framework, AgentEval, are introduced to assess the quality of web novel translations, revealing that Chinese-trained LLMs outperform larger foreign models.
Jintao Zhang, Kaiwen Zheng, Kai Jiang +5 authors
TurboDiffusion accelerates video generation by 100-200x using attention acceleration, step distillation, and quantization, while maintaining video quality.
Kaichen Zhang, Keming Wu, Zuhao Yang +5 authors
OpenMMReasoner, a two-stage training approach combining supervised fine-tuning and reinforcement learning, enhances multimodal reasoning performance through rigorous data curation and improved training strategies.
Roman Abramov, Felix Steinbauer, Gjergji Kasneci
Grokking with synthetic data augmentation in knowledge graphs enhances Transformer models' multi-step factual reasoning on real-world tasks, achieving high accuracy on benchmarks.
Zhihong Shao, Yuxiang Luo, Chengda Lu +6 authors
A self-verifying large language model for theorem proving improves mathematical reasoning by incentivizing rigorous step-by-step derivations and achieves high scores in international competitions.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号