Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Yukang Chen, Shengju Qian, Haotian Tang +4 authors
LongLoRA extends LLM context sizes efficiently using sparse local attention and parameter-efficient fine-tuning, demonstrating strong results with minimal computational cost.
50 篇论文 · 按点赞排序
Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai +5 authors
CulturaX is a large multilingual dataset designed to improve the training and quality of large language models through rigorous data cleaning and deduplication.
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne +9 authors
Large language models demonstrate strong compression capabilities, outperforming domain-specific compressors and offering new insights into scaling laws, tokenization, and in-context learning.
Daixuan Cheng, Shaohan Huang, Furu Wei
Transforming raw corpora into reading comprehension texts enhances large language models' question-answering abilities across various domains and improves performance on general benchmarks.
Vincent Perot, Kai Kang, Florian Luisier +7 authors
LMDX adapts large language models for document information extraction with grounding guarantees, setting a new state-of-the-art on VRDU and CORD benchmarks.
Chenyang Si, Ziqi Huang, Yuming Jiang +1 authors
A method called FreeU improves diffusion U-Net models' generation quality by re-weighting skip connections and backbone features without additional training.
Runpei Dong, Chunrui Han, Yuang Peng +11 authors
DreamLLM, a framework for Multimodal Large Language Models, directly samples in the multimodal space to enhance comprehension and creation synergy, enabling free-form interleaved content generation.
Tengchao Lv, Yupan Huang, Jingye Chen +11 authors
Kosmos-2.5, a unified multimodal model, generates spatially-aware and structured text from text-intensive images using a Transformer architecture and task-specific prompts.
Jon Saad-Falcon, Joe Barrow, Alexa Siu +3 authors
PDFTriage enables LLMs to retrieve context effectively from structured documents like PDFs and web pages for better document QA performance.
Qingyan Guo, Rui Wang, Junliang Guo +6 authors
EvoPrompt, a framework using evolutionary algorithms, automates prompt optimization for large language models, achieving better performance than human-engineered prompts.
Chunyuan Li, Zhe Gan, Zhengyuan Yang +4 authors
The survey covers the development and advancements in multimodal foundation models that integrate vision and language, transitioning from specialized models to versatile assistants through various training methods and unified architectures.
Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu +4 authors
The Chain-of-Verification method reduces hallucinations in language models by having them draft, plan verification questions, answer independently, and then generate a final verified response.
Sean O'Brien, Mike Lewis
Contrastive Decoding, a simple and training-free text generation method, outperforms other decoding techniques on various reasoning tasks and logical benchmarks by improving perceived quality and reducing reasoning errors.
Qihang Fan, Huaibo Huang, Mingrui Chen +2 authors
The proposed RMT model, combining RetNet and Transformer architectures, introduces explicit spatial distance priors and coordinate-wise decomposition to achieve exceptional performance in computer vision tasks.
Haoran Xu, Young Jin Kim, Amr Sharaf +1 authors
A novel fine-tuning approach for moderate-sized language models significantly improves translation performance with less parallel data compared to traditional methods.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng +10 authors
A large-scale dataset of one million real-world conversations with state-of-the-art LLMs provides a variety of applications including content moderation, safety benchmarking, and instruction-following model training.
Yevgen Chebotar, Quan Vuong, Alex Irpan +22 authors
Q-Transformer, a novel reinforcement learning method leveraging a Transformer for scalable Q-function representation, excels in offline multi-task robotic manipulation by efficiently modeling complex action sequences.
Parsa Kavehzadeh, Mojtaba Valipour, Marzieh Tahaei +3 authors
Sorted Fine-Tuning extends SortedNet to generative NLP tasks, enhancing efficiency and speed of large language models without the need for multiple models or complex pretraining.
Aiyuan Yang, Bin Xiao, Bingning Wang +49 authors
Baichuan 2, a series of 7 billion and 13 billion parameter multilingual LLMs, achieves competitive performance on benchmarks and excels in specialized domains, with open-source pre-training checkpoints released.
Longhui Yu, Weisen Jiang, Han Shi +7 authors
MetaMath, a fine-tuned language model specializing in mathematical reasoning, outperforms existing models on benchmarks like GSM8K and MATH through the use of a new dataset, MetaMathQA.
Yadong Lu, Chunyuan Li, Haotian Liu +3 authors
Scaling LLaVA to 33B and 65B/70B parameters enhances performance, with LoRA/QLoRA tuning comparable to full-model fine-tuning, and highlights the importance of higher image resolutions and multimodal data mixing.
Jianing Yang, Xuweiyi Chen, Shengyi Qian +4 authors
LLM-Grounder, a zero-shot, open-vocabulary pipeline using Large Language Models, enhances 3D visual grounding by decomposing complex queries and evaluating object relations in 3D scenes, achieving state-of-the-art accuracy on ScanRefer.
Mitchell Wortsman, Jaehoon Lee, Justin Gilmer +1 authors
Replacing the attention softmax with ReLU and dividing by sequence length in vision transformers can maintain or improve performance compared to using softmax alone.
Juntao Li, Zecheng Tang, Yuyang Ding +9 authors
OpenBA, a 15B bilingual asymmetric seq2seq model, achieves competitive performance on natural language processing tasks using an efficient three-stage training strategy and minimal pre-training data.
Zecheng Tang, Chenfei Wu, Juntao Li +1 authors
LayoutNUWA enhances graphic layout generation by treating it as a code generation task, integrating LLMs to incorporate semantic information and achieve state-of-the-art performance on multiple datasets.
Elias Frantar, Carlos Riquelme, Neil Houlsby +2 authors
Parameter sparsity in Transformers affects their scaling behavior on large datasets; optimal sparsity increases with data volume and different structures can impact performance.
Hoagy Cunningham, Aidan Ewart, Logan Riggs +2 authors
Using sparse autoencoders, this work identifies and removes overcomplete directions in language model activations, improving interpretability and model editability.
Meghana Moorthy Bhat, Rui Meng, Ye Liu +2 authors
A question-generation method using abstractive summaries reveals performance gaps between large and small LLMs in long-form question answering tasks.
Shiyi Zhu, Jing Ye, Wei Jiang +3 authors
A new self-attention structure called Collinear Constrained Attention (CoCA) improves the extrapolating performance of Transformer models without fine-tuning, even for significantly increased sequence lengths.
Kaixin Ma, Hongming Zhang, Hongwei Wang +2 authors
The proposed LASER agent improves LLM performance in interactive decision-making tasks by modeling them as state space exploration, enabling flexible back-tracking and better error recovery.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号