Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Shuming Ma, Hongyu Wang, Lingxiao Ma +7 authors
A 1-bit LLM variant, BitNet b1.58, achieves comparable performance to full-precision models with reduced computational costs and introduces new scaling laws and hardware design opportunities.
598 篇论文 · 按点赞排序
Qwen, An Yang, Baosong Yang +39 authors
Qwen2.5, an enhanced series of large language models, demonstrates superior performance across various benchmarks and use cases through extensive pre-training and advanced post-training techniques.
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan +81 authors
Phi-3-mini, a compact 3.8 billion parameter language model, achieves competitive performance with larger models through an enhanced training dataset and alignment.
Alexey Dontsov, Dmitrii Korzh, Alexey Zhavoronkin +6 authors
CLEAR benchmark evaluates multimodal unlearning methods across textual and visual data, highlighting challenges and demonstrating the effectiveness of $\ell_1$ regularization on LoRA weights in mitigating catastrophic forgetting.
Linrui Tian, Qi Wang, Bang Zhang +1 authors
EMO, a direct audio-to-video synthesis framework, enhances realism and expressiveness in talking head video generation by capturing nuanced audio-facial relationships without intermediate models.
Jiawei Zhao, Zhenyu Zhang, Beidi Chen +3 authors
Gradient Low-Rank Projection (GaLore) improves memory efficiency for training large language models without sacrificing performance, enabling pre-training of 7B models on consumer GPUs.
Yaowei Zheng, Richong Zhang, Junhao Zhang +2 authors
LlamaFactory is a unified framework enabling efficient fine-tuning of large language models across various tasks using a web-based user interface.
Tianzhu Ye, Li Dong, Yuqing Xia +4 authors
Diff Transformer improves large language models by selectively focusing attention on relevant context and reducing noise, leading to better performance in scaling, long-context modeling, key information retrieval, and in-context learning.
Aeree Cho, Grace C. Kim, Alexander Karpekov +5 authors
Transformer Explainer is an interactive visualization tool that allows non-experts to understand the inner workings of the GPT-2 model through real-time experimentation and visualization in a web browser.
An Yang, Baosong Yang, Binyuan Hui +55 authors
The Qwen2 series, comprising 0.5 to 72 billion parameter models, surpasses prior open models across language understanding, generation, multilingualism, coding, math, and reasoning, with exceptional performance in benchmarks like MMLU, GPQA, HumanEval, GSM8K, BBH, MT-Bench, Arena-Hard, and LiveCodeBench.
Benjamin Warner, Antoine Chaffin, Benjamin Clavié +11 authors
ModernBERT, an optimized encoder-only transformer model, demonstrates superior performance and efficiency in diverse classification and retrieval tasks compared to previous models.
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux +23 authors
Mixtral 8x7B, a Sparse Mixture of Experts language model, achieves superior performance across benchmarks by using a selective architecture that leverages fewer active parameters.
Zhe Chen, Weiyun Wang, Yue Cao +37 authors
InternVL 2.5, an advanced multimodal large language model, showcases competitive performance across various benchmarks, including multimodal reasoning and understanding, and is the first open-source model to surpass 70% on the MMMU benchmark using Chain-of-Thought reasoning.
Binyuan Hui, Jian Yang, Zeyu Cui +14 authors
Qwen2.5-Coder series demonstrates state-of-the-art code generation, completion, reasoning, and repair capabilities using the Qwen2.5 architecture with over 5.5 trillion tokens of training data.
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova +4 authors
Transformer decoders exhibit near-perfect linear relationships between layers, which can be reduced with cosine-similarity-based regularization, leading to improved performance on benchmarks.
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63 authors
StarCoder2, a large language model for code developed through a collaboration with Software Heritage, outperforms other models of similar size on various benchmarks and matches or outperforms larger models in specific areas.
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho +3 authors
A study on Self-Rewarding Language Models shows that using LLM-as-a-Judge prompting for iterative DPO training enhances both instruction-following and self-reward generation, leading to superior performance compared to existing systems.
Hongyin Luo, Wei Sun
A new integer addition-based algorithm reduces energy consumption in floating point tensor multiplications with comparable precision across various tasks.
Orr Zohar, Xiaohan Wang, Yann Dubois +9 authors
A comprehensive study reveals effective design principles for video understanding in Large Multimodal Models, introducing Apollo, which achieves superior performance across different model sizes.
Zhihong Shao, Peiyi Wang, Qihao Zhu +6 authors
DeepSeekMath 7B improves mathematical reasoning through enhanced data pre-training and Group Relative Policy Optimization, achieving high scores on MATH benchmark without external tools.
Melisa Russak, Umar Jamil, Christopher Bryant +4 authors
Writing in the Margins (WiM) enhances Large Language Models' performance on long input sequences and retrieval tasks by using chunked prefill and marginal information, improving accuracy and F1-score without fine-tuning.
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal +15 authors
The multi-turn online reinforcement learning approach SCoRe improves LLM self-correction using self-generated data and regularization, achieving state-of-the-art performance.
Yuzhang Tian, Jianbo Zhao, Haoyu Dong +8 authors
SpreadsheetLLM introduces SheetCompressor and Chain of Spreadsheet to enhance LLMs' performance on spreadsheet tasks through efficient encoding and understanding.
Andreas Steiner, André Susano Pinto, Michael Tschannen +15 authors
PaliGemma 2 integrates a SigLIP-So400m vision encoder with Gemma 2 models of varying sizes and resolutions, advancing transfer performance across diverse vision-language tasks, including OCR and captioning.
Chameleon Team
Chameleon is a mixed-modal early-fusion token-based model that achieves state-of-the-art performance across various tasks, including image captioning, text generation, and long-form mixed-modal generation, using a unified architecture.
Zechun Liu, Changsheng Zhao, Forrest Iandola +9 authors
MobileLLM leverages deep, thin architectures and advanced techniques to achieve high accuracy in sub-billion parameter models suitable for mobile devices.
Hugo Laurençon, Andrés Marafioti, Victor Sanh +1 authors
A comprehensive tutorial on building vision-language models, detailing the development of Idefics3-8B with an improved dataset enhancing document understanding.
Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier +28 authors
A study of Multimodal Large Language Models finds a mix of pre-training data and image encoder design crucial for SOTA performance on multimodal benchmarks.
Chris Lu, Cong Lu, Robert Tjarko Lange +3 authors
The AI Scientist is a framework that enables independent scientific research through automatic idea generation, experimentation, and paper writing using large language models.
Siming Huang, Tianhao Cheng, Jason Klein Liu +16 authors
OpenCoder is a top-tier open-access code LLM that provides comprehensive reproducible resources, including data, pipelines, and protocols, to advance code AI research.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号