Neural Network Diffusion
Kai Wang, Zhaopan Xu, Yukun Zhou +4 authors
Diffusion models can generate high-performing neural network parameters using an autoencoder, producing new subsets of network parameters with comparable or improved performance.
Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
598 篇论文 · 按点赞排序
Emu3: Next-Token Prediction is All You Need
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo +22 authors
Emu3, a transformer-based multimodal model trained exclusively with next-token prediction, outperforms existing diffusion and compositional models in generation and perception tasks.
GenEx: Generating an Explorable World
Taiming Lu, Tianmin Shu, Junfei Xiao +8 authors
GenEx generates 3D environments from a single image, enabling AI agents to explore and interact with a consistent, expansive space through guided generative imagination.
Design2Code: How Far Are We From Automating Front-End Engineering?
Chenglei Si, Yanzhe Zhang, Zhengyuan Yang +2 authors
Evaluates the performance of multimodal LLMs, including GPT-4V and Gemini Pro Vision, on converting visual designs to code using a new Design2Code task and benchmarks.
Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency
Jianwen Jiang, Chao Liang, Jiaqi Yang +3 authors
An audio-only conditioned video diffusion model, Loopy, improves natural and high-quality human video generation without auxiliary spatial signals.
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
Pan Zhang, Xiaoyi Dong, Yuhang Cao +26 authors
A framework combining disentangled perception, reasoning, and memory mechanisms enables real-time interaction for long-term multimodal AI systems.
TinyLlama: An Open-Source Small Language Model
Peiyuan Zhang, Guangtao Zeng, Tianduo Wang +1 authors
TinyLlama, a compact 1.1B language model, leverages FlashAttention to achieve high performance in downstream tasks with enhanced computational efficiency.
Instruction Pre-Training: Language Models are Supervised Multitask Learners
Daixuan Cheng, Yuxian Gu, Shaohan Huang +3 authors
Instruction Pre-Training enhances language models by generating and incorporating instruction-response pairs into unsupervised multitask pre-training.
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
Yuan Yao, Tianyu Yu, Ao Zhang +20 authors
MiniCPM-V presents a series of efficient Multimodal Large Language Models optimized for end-side deployment, offering high performance and practical usability compared to larger models.
Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing
Vadim Titov, Madina Khalmatova, Alexandra Ivanova +2 authors
A modified diffusion sampling process using self-guidance and noise rescaling achieves high-quality image editing with preserved structure and appearance without requiring diffusion model fine-tuning.
Rho-1: Not All Tokens Are What You Need
Zhenghao Lin, Zhibin Gou, Yeyun Gong +8 authors
Rho-1, a novel language model using Selective Language Modeling, improves efficiency and performance by selectively training on useful tokens rather than all tokens in the corpus.
Law of Vision Representation in MLLMs
Shijia Yang, Bohan Zhai, Quanzeng You +3 authors
Correlation between cross-modal alignment and vision representation improves performance in multimodal large language models, enabling identification and training of optimal vision representation with reduced computational cost.
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output
Pan Zhang, Xiaoyi Dong, Yuhang Zang +24 authors
A large-vision language model with long-contextual capabilities supports various comprehension and composition tasks, achieving GPT-4V level performance using 7B parameters.
Training Large Language Models to Reason in a Continuous Latent Space
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su +4 authors
A new reasoning paradigm, Coconut, utilizes continuous representations in latent space to enhance LLM performance on complex reasoning tasks, particularly those requiring backtracking.
Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss
Zesen Cheng, Hang Zhang, Kehan Li +6 authors
A tile-based computation strategy optimizes contrastive loss by partitioning calculations, enabling large batch sizes without significant memory or speed sacrifices.
Sapiens: Foundation for Human Vision Models
Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez +5 authors
Sapiens, a family of vision models for human-centric tasks, achieves superior performance with self-supervised pretraining and can be easily fine-tuned for 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction.
SaulLM-7B: A pioneering Large Language Model for Law
Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf +8 authors
SaulLM-7B, a large language model with 7 billion parameters, excels in legal text comprehension and generation using instructional fine-tuning on a legal corpus.
Attention Heads of Large Language Models: A Survey
Zifan Zheng, Yezhaohui Wang, Yuxin Huang +4 authors
This survey explores the interpretability and underlying mechanisms of attention heads within Large Language Models to understand their reasoning processes.
Are Your LLMs Capable of Stable Reasoning?
Junnan Liu, Hongwei Liu, Linchen Xiao +6 authors
G-Pass@k and LiveMathBench assess Large Language Models' reasoning capabilities and consistency using complex mathematical problems.
An Introduction to Vision-Language Modeling
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay +38 authors
Introduction to vision-language models (VLMs) covering their applications, training, evaluation, and extension to videos, addressing challenges in mapping visual data to language.
Learn Your Reference Model for Real Good Alignment
Alexey Gorbatovski, Boris Shaposhnikov, Alexey Malakhov +5 authors
A new method, Trust Region DPO (TR-DPO), is proposed to improve policy alignment in reinforcement learning, outperforming Direct Preference Optimization (DPO) by up to 19% on key datasets by updating the reference policy during training.
Structured 3D Latents for Scalable and Versatile 3D Generation
Jianfeng Xiang, Zelong Lv, Sicheng Xu +6 authors
A 3D generation method using a unified SLAT representation and rectified flow transformers achieves high-quality results across different formats and conditions.
Stealing Part of a Production Language Model
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham +10 authors
A model-stealing attack is introduced that can extract detailed information such as the embedding projection layer from black-box language models with minimal cost.
LoRA Learns Less and Forgets Less
Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes +9 authors
LoRA, a parameter-efficient finetuning method for large language models, underperforms full finetuning in target domains but provides better regularization and maintains diverse generation compared to other techniques.
ShowUI: One Vision-Language-Action Model for GUI Visual Agent
Kevin Qinghong Lin, Linjie Li, Difei Gao +6 authors
ShowUI is a vision-language-action model that enhances GUI assistants by using UI-guided token selection and interleaved vision-language-action streaming, achieving high accuracy and efficiency in zero-shot screenshot grounding across different environments.
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
Alexander Nikulin, Ilya Zisman, Alexey Zemtsov +3 authors
XLand-100B is a large-scale dataset for in-context reinforcement learning, containing extensive learning histories in the XLand-MiniGrid environment.
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems
Philippe Laban, Alexander R. Fabbri, Caiming Xiong +1 authors
The SummHay task evaluates LLMs and RAG systems on summarizing long-context documents by identifying relevant insights and citing sources, revealing challenges even for systems with document relevance signals.
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
Min Shi, Fuxiao Liu, Shihao Wang +12 authors
Mixture of vision encoders and resolutions in multimodal large language models improves performance through concatenation of visual tokens and a Pre-Alignment mechanism, leading to superior results on benchmarks.
GPT-4o System Card
OpenAI, Aaron Hurst, Adam Lerer +416 authors
GPT-4o is an omnimodal autoregressive model trained to handle text, audio, image, and video inputs, offering high-performance outputs across these modalities, with particular strengths in vision and audio.
Baichuan-Omni Technical Report
Yadong Li, Haoze Sun, Mingan Lin +24 authors
Baichuan-Omni, a 7B open-source Multimodal Large Language Model, excels in processing image, video, audio, and text, showcasing competitive performance across multimodal benchmarks.