Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
50 篇论文 · 按点赞排序
David Grangier, Angelos Katharopoulos, Pierre Ablin +1 authors
The study evaluates various machine learning approaches under constrained inference and training budgets, demonstrating that hyper-networks and mixture of experts perform well with large pretraining budgets, while small models trained on importance sampled datasets are better with large specialization budgets.
Zhiyong Wu, Chengcheng Han, Zichen Ding +5 authors
OS-Copilot framework enables FRIDAY, a self-improving generalist agent, to automate a wide range of computer tasks with enhanced generalization and minimal supervision.
Jun Zhan, Junqi Dai, Jiasheng Ye +13 authors
AnyGPT is a multimodal language model using discrete representations to process and generate content across various modalities with performance on par with specialized models.
Gilles Baechler, Srinivas Sunkara, Maria Wang +7 authors
ScreenAI, a vision-language model optimized with a flexible patching strategy, achieves state-of-the-art performance on UI and infographic tasks with 5B parameters.
Noveen Sachdeva, Benjamin Coleman, Wang-Cheng Kang +6 authors
Data-efficient methods like Ask-LLM and Density sampling improve model quality and training efficiency in large language models by optimizing data selection and coverage.
Shihan Dou, Yan Liu, Haoxiang Jia +13 authors
StepCoder, a reinforcement learning framework, enhances code generation by breaking tasks into subtasks and optimizing only executed code segments, using the APPS+ dataset for training.
Nikhil Bhendawade, Irina Belousova, Qichen Fu +3 authors
Speculative Streaming is a method that incorporates drafting into a target language model, enhancing decoding speed without quality loss while using fewer parameters.
Yuri Kuratov, Aydar Bulatov, Petr Anokhin +3 authors
Recursively augmented GPT-2 fine-tuning significantly extends the processing capability of generative models to handle long document sequences beyond $10^6$ elements.
Long Zhao, Nitesh B. Gundavarapu, Liangzhe Yuan +16 authors
VideoPrism, a pretrained video encoder, achieves top performance across various video understanding tasks by utilizing global-local distillation and token shuffling of semantic video embeddings enhanced with associated text.
Hao Liu, Wilson Yan, Matei Zaharia +1 authors
Large-scale training of a transformer model for long video and language sequences using RingAttention and masked sequence packing improves performance on retrieval and long-form understanding tasks.
Fanqi Wan, Ziyi Yang, Longguang Zhong +3 authors
FuseChat extends the FuseLLM framework for knowledge fusion of chat LLMs through lightweight fine-tuning and parameter merging, achieving superior performance across various domains.
Xing Han Lù, Zdeněk Kasner, Siva Reddy
A benchmark and retrieval-inspired model for efficient conversational web navigation by digital agents across diverse real-world websites is introduced, highlighting challenges in generalization to unseen sites.
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran +3 authors
A new math instruction tuning dataset, OpenMathInstruct-1, leverages the Mixtral model to achieve competitive performance on math reasoning benchmarks using open-source LLMs.
Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta +2 authors
ReadAgent, an LLM-based prompting system, enhances effective context length up to 20x through memory episodes and gist memories, outperforming baselines on long-document reading comprehension tasks.
Jian Xie, Kai Zhang, Jiangjie Chen +5 authors
TravelPlanner is a new benchmark for evaluating the planning capabilities of language agents in complex, real-world travel planning scenarios, demonstrating that current models still struggle with information gathering and constraint management.
Jiacheng Liu, Sewon Min, Luke Zettlemoyer +2 authors
Modernized n-gram models, specifically $\infty$-gram models, enhance text analysis and neural LLMs by providing high accuracy in next-token prediction and reducing language modeling perplexities.
Johan Obando-Ceron, Ghada Sokar, Timon Willi +6 authors
Incorporating Soft MoE modules into value-based networks enhances scalability and performance across different training regimes and model sizes in reinforcement learning.
Ziheng Jiang, Haibin Lin, Yinmin Zhong +29 authors
MegaScale is a production system designed to train large language models at scale using more than 10,000 GPUs, focusing on efficiency, stability, and fault tolerance through advanced observability and diagnostic tools.
Shangmin Guo, Biao Zhang, Tianlin Liu +9 authors
Online AI feedback improves direct alignment from preferences methods by providing real-time feedback from a model, outperforming both offline DAP and RLHF methods.
Chiyu Zhang, Yifei Sun, Jun Chen +7 authors
The SPAR framework enhances content recommendations by using pretrained language models, poly-attention layers, and large language models to effectively process long user engagement histories and predict user-item interactions.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号