Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
598 篇论文 · 按点赞排序
Luca Soldaini, Rodney Kinney, Akshita Bhagia +33 authors
Dolma, a three trillion token English corpus, and its open-source data curation toolkit facilitate open research in language model pretraining by providing detailed insights into data curation practices.
Xuezhe Ma, Xiaomeng Yang, Wenhan Xiong +7 authors
Megaldon, an enhanced neural architecture, provides efficient long-sequence modeling by combining and improving upon components from mega and transformer models, achieving better pretraining efficiency and performance.
Pierre Colombo, Telmo Pires, Malik Boudiaf +7 authors
Two large legal language models, SaulLM-54B and SaulLM-141B, based on the Mixtral architecture, are introduced for domain-specific adaptation in the legal sector using continued pretraining, specialized protocols, and preference alignment with synthetic data.
Yiheng Liu, Hao He, Tianle Han +18 authors
The paper reviews techniques for cost-efficient training and deployment of large language models, covering aspects like data preprocessing, parallel training, model fine-tuning, and inference optimizations including model compression and memory scheduling.
Liqiang Jing, Zhehui Huang, Xiaoyang Wang +6 authors
DSBench evaluates large language and vision-language models on realistic data science tasks, revealing significant performance gaps that indicate the need for improved autonomous agents.
01. AI, Alex Young, Bei Chen +28 authors
The Yi model family, based on transformer architecture, showcases strong performance across benchmarks and modalities through optimized data and scalable infrastructure.
Wenting Zhao, Xiang Ren, Jack Hessel +3 authors
Chatbots such as GPT-4 and ChatGPT are now serving millions of users. Despite their widespread use, there remains a lack of public datasets showcasing how these tools are used by a population of users in practice. To bridge this gap, we offered free access to ChatGPT for online users in exchange for their affirmative, consensual opt-in to anonymously collect their chat transcripts and request headers. From this, we compiled WildChat, a corpus of 1 million user-ChatGPT conversations, which consists of over 2.5 million interaction turns. We compare WildChat with other popular user-chatbot interaction datasets, and find that our dataset offers the most diverse user prompts, contains the largest number of languages, and presents the richest variety of potentially toxic use-cases for researchers to study. In addition to timestamped chat transcripts, we enrich the dataset with demographic data, including state, country, and hashed IP addresses, alongside request headers. This augmentation allows for more detailed analysis of user behaviors across different geographical regions and temporal dimensions. Finally, because it captures a broad range of use cases, we demonstrate the dataset's potential utility in fine-tuning instruction-following models. WildChat is released at https://wildchat.allen.ai under AI2 ImpACT Licenses.
Chunqiu Steven Xia, Yinlin Deng, Soren Dunn +1 authors
Agentless, a two-phase approach to software development automation, achieves high performance and low cost compared to complex autonomous agents on the SWE-bench Lite benchmark.
Siheng Li, Cheng Yang, Zesen Cheng +4 authors
A novel method enables LLMs to self-improve in long-context reasoning through sampling, scoring with Minimum Bayes Risk, and supervised fine-tuning, outperforming expert-annotated data approaches.
Xun Liang, Hanyu Wang, Yezhaohui Wang +8 authors
Controllable Text Generation techniques for Large Language Models ensure predefined control conditions and high-quality text output, covering content and attribute control through various methods like retraining, fine-tuning, and latent manipulation.
Xin Men, Mingyu Xu, Qingyu Zhang +5 authors
A new layer pruning method called ShortGPT significantly improves model pruning performance by removing redundant layers based on Block Influence scores.
Machel Reid, Nikolay Savinov, Denis Teplyashin +668 authors
Gemini 1.5 Pro, a multimodal mixture-of-experts model, achieves near-perfect recall and state-of-the-art performance in long-context tasks, including QA and ASR, surpassing previous models in token prediction and translation capabilities.
Fan Zhou, Zengzhi Wang, Qian Liu +2 authors
ProX, a novel framework treating data refinement as a programming task, enhances large language model pre-training by generating fine-grained operations for each example, outperforming traditional human-crafted rule-based methods significantly across various benchmarks and domains.
Ziyan Jiang, Xueguang Ma, Wenhu Chen
The LongRAG framework improves information retrieval and answer extraction by using longer retrieval units and leveraging long-context LLMs, leading to better performance on benchmark datasets.
Qianqian Xie, Dong Li, Mengxi Xiao +36 authors
Open-FinLLMs, a series of financial LLMs, enhance performance through comprehensive pre-training and instruction fine-tuning, demonstrating superior results in both zero-shot and few-shot evaluations, and excels in understanding tables and charts.
Ming Li, Yanhong Li, Tianyi Zhou
Investigating gradient patterns in LLMs during fast vs. slow training reveals that slow thinking (chain-of-thoughts) enhances learning stability and distinguishes correct reasoning paths.
Lihe Yang, Bingyi Kang, Zilong Huang +3 authors
Depth Anything is a robust monocular depth estimation model built on a large-scale dataset with data augmentation and auxiliary supervision strategies, achieving state-of-the-art results on various datasets and enhancing depth-conditioned ControlNet.
Yunfei Chu, Jin Xu, Qian Yang +9 authors
Qwen2-Audio, a large-scale audio-language model, enhances instruction-following and audio analysis through natural language prompts and DPO optimization.
Zeyi Sun, Ziyang Chu, Pan Zhang +6 authors
X-Prompt, an auto-regressive large-vision language model, uses in-context learning to achieve competitive performance in seen and unseen image generation tasks.
Yuzhe Yang, Yifei Zhang, Yan Hu +10 authors
A user-centric framework evaluates LLMs in financial tasks through human feedback and task-specific interactions, achieving high correlation with human preferences.
Shengbang Tong, Ellis Brown, Penghao Wu +11 authors
Cambrian-1, a vision-centric multimodal LLM family, introduces the Spatial Vision Aggregator (SVA) and a new benchmark CV-Bench to enhance visual grounding and representation learning in multimodal systems.
Chunting Zhou, Lili Yu, Arun Babu +7 authors
Transfusion combines language modeling and diffusion training to develop a multi-modal transformer model that scales effectively across text and image data, matching the performance of specialized models.
Zicheng Lin, Tian Liang, Jiahao Xu +6 authors
The proposed cDPO approach identifies and rewards critical tokens to improve the reasoning accuracy of LLMs by utilizing contrastive estimation between positive and negative models.
Damai Dai, Chengqi Deng, Chenggang Zhao +14 authors
The DeepSeekMoE architecture improves expert specialization in Mixture-of-Experts models by segmenting experts and isolating shared ones, achieving better performance and computational efficiency compared to GShard and other models.
Jiuhai Chen, Jianwei Yang, Haiping Wu +4 authors
Florence-VL enhances multimodal language models with improved visual features from Florence-2, achieving state-of-the-art results across various benchmarks through depth-breath fusion and a novel training recipe.
Junying Chen, Ruyi Ouyang, Anningzhe Gao +9 authors
The refinement of medical image-text pairs using multimodal large language models enhances medical multimodal capabilities and data quality, leading to superior performance in benchmarks.
Ziyu Liu, Tao Chu, Yuhang Zang +8 authors
A new benchmark (MMDU) and instruction tuning dataset (MMDU-45k) are introduced to evaluate and enhance large vision-language models' performance in multi-turn and multi-image conversations, highlighting the need for improved conversational instruction tuning data.
Lianghui Zhu, Bencheng Liao, Qian Zhang +3 authors
A new vision backbone using bidirectional Mamba blocks (Vim) outperforms established vision transformers like DeiT in terms of performance and efficiency on high-resolution image tasks.
Trung Dao, Thuan Hoang Nguyen, Thanh Le +4 authors
Enhancements to SwiftBrush through improved weight initialization, efficient LoRA training, and novel clamped CLIP loss achieve state-of-the-art image quality while maintaining generative diversity.
Corby Rosset, Ching-An Cheng, Arindam Mitra +3 authors
A Direct Nash Optimization (DNO) algorithm is introduced for post-training large language models using preference feedback, achieving state-of-the-art performance with polynomial parameters.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号