Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan +81 authors
Phi-3-mini, a compact 3.8 billion parameter language model, achieves competitive performance with larger models through an enhanced training dataset and alignment.
50 篇论文 · 按点赞排序
Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao +8 authors
OpenELM, an open language model, uses layer-wise scaling to enhance accuracy and is released with comprehensive training and evaluation frameworks, enabling transparent and reproducible research.
Opher Lieber, Barak Lenz, Hofit Bata +19 authors
Jamba, a hybrid Transformer-Mamba MoE model, achieves state-of-the-art performance with high throughput and minimal memory usage, supporting large context lengths.
Tsendsuren Munkhdalai, Manaal Faruqui, Siddharth Gopal
A novel Infini-attention mechanism allows Transformer-based LLMs to handle infinitely long inputs with limited memory and compute, enabling efficient long-context modeling and fast inference.
David Raposo, Sam Ritter, Blake Richards +3 authors
Transformer-based models can dynamically allocate computational resources to specific positions in input sequences, leading to efficient performance with reduced FLOPs.
Zhengxuan Wu, Aryaman Arora, Zheng Wang +4 authors
Representation Finetuning (ReFT) methods, exemplified by Low-rank Linear Subspace ReFT (LoReFT), achieve high efficiency and performance by adapting representations in frozen base models, outperforming state-of-the-art Parameter-efficient Fine-tuning (PEFT) methods.
Zhenghao Lin, Zhibin Gou, Yeyun Gong +8 authors
Rho-1, a novel language model using Selective Language Modeling, improves efficiency and performance by selectively training on useful tokens rather than all tokens in the corpus.
Alexey Gorbatovski, Boris Shaposhnikov, Alexey Malakhov +5 authors
A new method, Trust Region DPO (TR-DPO), is proposed to improve policy alignment in reinforcement learning, outperforming Direct Preference Optimization (DPO) by up to 19% on key datasets by updating the reference policy during training.
Keen You, Haotian Zhang, Eldon Schoop +5 authors
Ferret-UI, a multimodal large language model tailored for mobile UI screens, enhances understanding and interaction through region annotations and a comprehensive dataset of UI tasks, outperforming existing models including GPT-4V.
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich +10 authors
LayerSkip, an end-to-end method, accelerates large language model inference through layer dropout and self-speculative decoding without auxiliary layers.
Elizaveta Goncharova, Anton Razzhigaev, Matvey Mikhalchuk +6 authors
The OmniFusion model, leveraging pretrained LLMs and visual adapters, demonstrates superior performance across multiple visual-language benchmarks compared to open-source alternatives.
Keyu Tian, Yi Jiang, Zehuan Yuan +2 authors
Visual AutoRegressive (VAR) modeling improves image generation performance and speed over autoregressive and diffusion transformer models, achieving superior results in image quality, data efficiency, and scalability.
Pat Verga, Sebastian Hofstatter, Sophia Althammer +6 authors
Evaluating Large Language Models using a Panel of smaller evaluators outperforms single large evaluators with reduced bias and cost.
Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach +3 authors
LLM2Vec transforms decoder-only LLMs into effective text encoders using bidirectional attention, masked next token prediction, and unsupervised contrastive learning, achieving state-of-the-art performance on text embedding tasks.
Xuezhe Ma, Xiaomeng Yang, Wenhan Xiong +7 authors
Megaldon, an enhanced neural architecture, provides efficient long-sequence modeling by combining and improving upon components from mega and transformer models, achieving better pretraining efficiency and performance.
Corby Rosset, Ching-An Cheng, Arindam Mitra +3 authors
A Direct Nash Optimization (DNO) algorithm is introduced for post-training large language models using preference feedback, achieving state-of-the-art performance with polynomial parameters.
Xun Wu, Shaohan Huang, Wenhui Wang +1 authors
MH-MoE enhances SMoE by splitting tokens into sub-tokens processed by diverse experts in parallel, improving expert activation, context understanding, and overfitting, and demonstrating effectiveness across various modeling tasks.
Zhe Chen, Weiyun Wang, Hao Tian +24 authors
InternVL 1.5, an open-source multimodal language model, improves visual understanding and high-resolution image processing through continuous learning, dynamic tile division, and a high-quality bilingual dataset, achieving state-of-the-art results in several benchmarks.
Wei Chen, Zhiyuan Li
A new on-device method improves the performance of a large language model, matching GPT-4's accuracy and latency while reducing context length and enhancing deployment on edge devices.
Shengnan An, Zexiong Ma, Zeqi Lin +2 authors
FILM-7B enhances long-context processing by using information-intensive training with synthesized datasets, improving retrieval and performance on real-world tasks without compromising short-context performance.
Ye Tian, Baolin Peng, Linfeng Song +4 authors
AlphaLLM enhances Large Language Models through integration with Monte Carlo Tree Search and critic models, improving performance in complex reasoning tasks without additional annotations.
Tianbao Xie, Danyang Zhang, Jixuan Chen +14 authors
OSWorld provides a scalable, real computer environment for evaluating multimodal agents across various operating systems and tasks, highlighting significant gaps in current LLM/VLM capabilities.
Hyungjoo Chae, Yeonghyeon Kim, Seungone Kim +8 authors
Think-and-Execute is a novel framework that enhances algorithmic reasoning in large language models by decomposing the reasoning process into task-level logic expressed as pseudocode and instance-specific execution.
Ming Li, Taojiannan Yang, Huafeng Kuang +4 authors
ControlNet++ enhances text-to-image generation controllability through pixel-level cycle consistency optimization using an efficient noise-disturbed reward strategy.
Aleksandar Botev, Soham De, Samuel L Smith +59 authors
RecurrentGemma, using Griffin architecture, achieves excellent language performance with efficient memory usage and fewer training tokens compared to Gemma-2B.
Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren +17 authors
Gecko is a compact and versatile text embedding model that distills knowledge from large language models to achieve strong retrieval performance on benchmarks.
Wei Huang, Xudong Ma, Haotong Qin +7 authors
LLaMA3 performance degrades significantly when quantized to low bit-width, highlighting challenges in low-bit quantization for large language models.
Zichen Liu, Yihao Meng, Hao Ouyang +4 authors
Dynamic Typography generates coherent and semantically meaningful text animations using neural displacement fields, end-to-end optimization, and perceptual loss regularization, outperforming baseline methods.
Lifan Yuan, Ganqu Cui, Hanbin Wang +12 authors
Eurus, a suite of reasoning-optimized large language models, achieves state-of-the-art performance on various benchmarks through UltraInteract, a large-scale, high-quality alignment dataset, and a novel reward modeling objective.
Wenhao Huang, Chenghao Peng, Zhixu Li +4 authors
AutoCrawler combines LLMs with crawlers to efficiently handle diverse and changing web environments using a hierarchical HTML structure for progressive learning.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号