Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
Tianzhu Ye, Li Dong, Yuqing Xia +4 authors
Diff Transformer improves large language models by selectively focusing attention on relevant context and reducing noise, leading to better performance in scaling, long-context modeling, key information retrieval, and in-context learning.
Hanyu Guo, Jiedong Yang, Chao Chen +3 authors
TransitLM dataset enables end-to-end transit route planning using large language models trained on structured transit data, eliminating the need for traditional map-based approaches.
Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4 authors
Analysis of Large Language Models reveals that even minor tokens are crucial for context, and introduces LLM-Microscope for assessing token-level nonlinearity and contextual memory.
B. Y. Yan, Chaofan Li, Hongjin Qian +2 authors
GAM, a novel framework that employs JIT compilation principles, improves memory efficiency and task completion by leveraging a lightweight memorizer and researcher in conjunction with reinforcement learning.
Jing Liang, Hongyao Tang, Yi Ma +9 authors
Training-inference mismatch in reinforcement learning for large language models leads to instability, which is addressed through a new policy optimization objective and framework that ensures consistent policy improvements between training and inference phases.
Zhiyu Li, Shichao Song, Chenyang Xi +36 authors
MemOS, a memory operating system for Large Language Models, addresses memory management challenges by unifying plaintext, activation-based, and parameter-level memories, enabling efficient storage, retrieval, and continual learning.
Guanting Dong, Hangyu Mao, Kai Ma +11 authors
Agentic Reinforced Policy Optimization (ARPO) enhances multi-turn reasoning in large language models by balancing long-horizon capabilities and tool interactions, using entropy-based adaptive rollouts and advantage attribution.
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63 authors
StarCoder2, a large language model for code developed through a collaboration with Software Heritage, outperforms other models of similar size on various benchmarks and matches or outperforms larger models in specific areas.
Yifei Wang, Yueqi Wang, Zhenrui Yue +6 authors
FASA is a novel framework that uses query-aware token eviction and functional sparsity in RoPE to reduce KV cache memory usage while maintaining high performance in long-context LLM tasks.
Runze Liu, Junqi Gao, Jian Zhao +5 authors
Test-Time Scaling improves Large Language Model performance by optimizing inference computation based on policy models, Process Reward Models, and problem difficulty.
Zhi Zhou, Yuhao Tan, Zenan Li +4 authors
A theoretical framework for sampling-based test-time scaling in large language models reveals limitations in self-consistency and perplexity, and introduces RPC to improve reasoning performance and reduce sampling costs.
Zhiyuan Hu, Yucheng Wang, Yufei He +7 authors
Reinforcement learning for large language models is enhanced by a rollout-level objective that rewards rare high-level reasoning strategies, improving diverse solution discovery without sacrificing initial performance.
Xuyang Liu, Zichen Wen, Shaobo Wang +13 authors
The focus in AI research is shifting from model-centric to data-centric compression, with token compression identified as key to improving efficiency in handling long-context scenarios.
Melisa Russak, Umar Jamil, Christopher Bryant +4 authors
Writing in the Margins (WiM) enhances Large Language Models' performance on long input sequences and retrieval tasks by using chunked prefill and marginal information, improving accuracy and F1-score without fine-tuning.
Sergey Pletenev, Maria Marina, Nikolay Ivanov +6 authors
EverGreenQA, a multilingual QA dataset with evergreen labels, is introduced to benchmark LLMs on temporality encoding and assess their performance through verbalized judgments and uncertainty signals.
Yang Liu, Enxi Wang, Yufei Gao +6 authors
MEDS is a memory-enhanced dynamic reward shaping framework that improves sampling diversity in reinforcement learning for large language models by identifying and penalizing recurrent error patterns through clustering of historical behavioral signals.
Runming He, Zhen Hao Wong, Hao Liang +4 authors
DataFlow-Harness guides LLM agents to build persistent, editable data pipelines via incremental mutations and live platform grounding, achieving high reliability with lower cost and latency.
Zonghang Li, Tao Li, Wenjiao Feng +2 authors
Prima.cpp, a distributed inference system, enables running large language models on home devices using CPU/GPU and low memory, overcoming hardware limitations.
Xufang Luo, Yuge Zhang, Zhiyuan He +5 authors
Agent Lightning is a flexible RL framework for training LLMs in various agents, using a hierarchical RL algorithm and decoupling execution from training to handle complex interactions.
Guochao Jiang, Jingyi Song, Guofeng Quan +3 authors
Dynamic Variance-adaptive Advantage Optimization (DVAO) addresses training instability in multi-reward reinforcement learning by adaptively weighting objectives based on empirical reward variance, maintaining bounded advantage magnitudes and improving multi-objective performance.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号