Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Jiawei Zhao, Zhenyu Zhang, Beidi Chen +3 authors
Gradient Low-Rank Projection (GaLore) improves memory efficiency for training large language models without sacrificing performance, enabling pre-training of 7B models on consumer GPUs.
50 篇论文 · 按点赞排序
Yaowei Zheng, Richong Zhang, Junhao Zhang +2 authors
LlamaFactory is a unified framework enabling efficient fine-tuning of large language models across various tasks using a web-based user interface.
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63 authors
StarCoder2, a large language model for code developed through a collaboration with Software Heritage, outperforms other models of similar size on various benchmarks and matches or outperforms larger models in specific areas.
Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier +28 authors
A study of Multimodal Large Language Models finds a mix of pre-training data and image encoder design crucial for SOTA performance on multimodal benchmarks.
Chenglei Si, Yanzhe Zhang, Zhengyuan Yang +2 authors
Evaluates the performance of multimodal LLMs, including GPT-4V and Gemini Pro Vision, on converting visual designs to code using a new Design2Code task and benchmarks.
Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf +8 authors
SaulLM-7B, a large language model with 7 billion parameters, excels in legal text comprehension and generation using instructional fine-tuning on a legal corpus.
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham +10 authors
A model-stealing attack is introduced that can extract detailed information such as the embedding projection layer from black-box language models with minimal cost.
Andrey Gromov, Kushal Tirumala, Hassan Shapourian +2 authors
Layer pruning of pre-trained LLMs with parameter-efficient finetuning methods shows minimal performance degradation and significant resource savings in both finetuning and inference.
Eric Zelikman, Georges Harik, Yijia Shao +3 authors
Quiet-STaR is an extension of the Self-Taught Reasoner that teaches language models to infer and generate rationales for each token, improving their ability to predict difficult tokens and answer questions without task-specific fine-tuning.
Byung-Kwan Lee, Beomchan Park, Chae Won Kim +1 authors
MoAI, a new LLVM, integrates outputs from external CV models to enhance VL tasks without expanding model size or curating extra datasets.
Zhengqing Yuan, Ruoxi Chen, Zhaoxu Li +4 authors
A new multi-agent framework Mora replicates Sora's generalist video generation capabilities using several advanced visual AI agents, achieving similar performance across various tasks.
Kai Mei, Zelong Li, Shuyuan Xu +3 authors
AIOS, an operating system embedding large language models, addresses resource allocation, context switching, and concurrency challenges for intelligent agents, demonstrating reliability and efficiency.
Patrick Esser, Sumith Kulal, Andreas Blattmann +14 authors
Improved noise sampling techniques and a transformer-based architecture enhance rectified flow models for high-resolution text-to-image synthesis, outperforming existing methods.
Tianjun Zhang, Shishir G. Patil, Naman Jain +4 authors
Retrieval Augmented FineTuning (RAFT) enhances pre-trained large language models' in-domain performance by filtering out irrelevant documents and citing relevant passages.
Axel Sauer, Frederic Boesel, Tim Dockhorn +3 authors
Latent Adversarial Diffusion Distillation (LADD) simplifies the training and improves the performance of diffusion models by using generative features from latent diffusion models, enabling fast and high-resolution image synthesis.
01. AI, Alex Young, Bei Chen +28 authors
The Yi model family, based on transformer architecture, showcases strong performance across benchmarks and modalities through optimized data and scalable infrastructure.
Xin Men, Mingyu Xu, Qingyu Zhang +5 authors
A new layer pruning method called ShortGPT significantly improves model pruning performance by removing redundant layers based on Block Influence scores.
Machel Reid, Nikolay Savinov, Denis Teplyashin +668 authors
Gemini 1.5 Pro, a multimodal mixture-of-experts model, achieves near-perfect recall and state-of-the-art performance in long-context tasks, including QA and ASR, surpassing previous models in token prediction and translation capabilities.
Hakim Sidahmed, Samrat Phatale, Alex Hutcheson +16 authors
Parameter-efficient reinforcement learning using LoRA in RLHF matches conventional RLHF performance with reduced computational cost and memory usage.
Soham De, Samuel L. Smith, Anushan Fernando +14 authors
Hawk and Griffin, models incorporating gated linear recurrences, achieve high performance with lower computational resources and better hardware efficiency compared to existing architectures.
Takuya Akiba, Makoto Shing, Yujin Tang +2 authors
An evolutionary algorithm automates model merging, discovering effective combinations of open-source models without additional training data or compute, achieving state-of-the-art performance on benchmarks and culturally-aware tasks.
Hugo Laurençon, Léo Tronchon, Victor Sanh
A synthetic dataset of HTML code and screenshots is introduced to enhance the ability of vision-language models to convert screenshots into HTML code.
Qihang Fan, Quanzeng You, Xiaotian Han +5 authors
ViTAR enhances Vision Transformers' scalability across resolutions through dynamic token integration and fuzzy positional encoding, improving accuracy and reducing computational costs.
Changyue Liao, Mo Sun, Zihan Yang +4 authors
Fuyou enables efficient fine-tuning of large models on low-end GPUs by optimizing SSD-CPU communication and computation, achieving higher GPU utilization compared to ZeRO-Infinity.
Hengxing Cai, Xiaochen Cai, Shuwen Yang +14 authors
Uni-SMART, a multimodal analysis model, outperforms LLMs in understanding scientific literature with diverse content and has applications in patent infringement and chart analysis.
Shangda Wu, Xu Tan, Zili Wang +3 authors
bGPT, a model using next byte prediction, simulates various digital processes with high accuracy, matching specialized models in performance and offering insights into algorithm and hardware behavior.
Renrui Zhang, Dongzhi Jiang, Yichi Zhang +8 authors
MathVerse is a comprehensive visual math benchmark with CoT evaluation to assess MLLMs' true understanding and reasoning quality from visual diagrams.
Adam Ibrahim, Benjamin Thérien, Kshitij Gupta +5 authors
Simple continual learning strategies, including learning rate re-warming, re-decaying, and data replay, can match the performance of full re-training for large language models with much lower computational cost.
Gemma Team, Thomas Mesnard, Cassidy Hardin +105 authors
Gemma, a family of lightweight and high-performing language models, outperforms similarly sized open models across text-based tasks and emphasizes the importance of responsible model development and safety.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen +14 authors
Chronos, a framework using pretrained transformer-based models for time series forecasting, outperforms classical methods and achieves comparable zero-shot performance on unseen datasets using tokenized time series data.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号