Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay +38 authors
Introduction to vision-language models (VLMs) covering their applications, training, evaluation, and extension to videos, addressing challenges in mapping visual data to language.
50 篇论文 · 按点赞排序
Sean McLeish, Arpit Bansal, Alex Stein +8 authors
Transformers achieve state-of-the-art performance on large arithmetic tasks and other reasoning tasks by addressing positional tracking with embeddings and integrating architectural modifications.
Byung-Kwan Lee, Chae Won Kim, Beomchan Park +1 authors
Meteor, an efficient large language and vision model, enhances understanding and answering capabilities by embedding multifaceted rationales using the Mamba architecture, leading to improved vision language performance without increasing model size or using additional vision encoders.
Ge Zhang, Scott Qu, Jiaheng Liu +42 authors
MAP-Neo, a fully open-sourced bilingual LLM with 7B parameters, achieves performance comparable to existing state-of-the-art LLMs, enhancing transparency and fostering innovation in the open research community.
Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman +9 authors
The Phased Consistency Model (PCM) addresses limitations in Latent Consistency Models (LCM) and outperforms them in high-resolution, text-conditioned image and few-step text-to-video generation.
Chunjiang Ge, Sijie Cheng, Ziming Wang +6 authors
ConvLLaVA addresses excessive visual tokens and quadratic complexity in high-resolution multimodal models by using ConvNeXt as a hierarchical backbone, optimizing pretrained ConvNeXt for high resolution, and adding a successive stage for further compression, achieving competitive performance.
Boshi Wang, Xiang Yue, Yu Su +1 authors
Transformers can learn implicit reasoning through extensive training, with varying degrees of generalization for different reasoning types, and parametric memory proves more effective for complex reasoning tasks compared to non-parametric memory.
Andreas Koukounas, Georgios Mastrapas, Michael Günther +11 authors
A novel multi-task contrastive training method improves CLIP model performance on both text-image and text-text retrieval tasks.
Mu Cai, Jianwei Yang, Jianfeng Gao +1 authors
Large Multimodal Models (LMMs) such as LLaVA have shown strong performance in visual-linguistic reasoning. These models first embed images into a fixed large number of visual tokens and then feed them into a Large Language Model (LLM). However, this design causes an excessive number of tokens for dense visual scenarios such as high-resolution images and videos, leading to great inefficiency. While token pruning/merging methods do exist, they produce a single length output for each image and do not afford flexibility in trading off information density v.s. efficiency. Inspired by the concept of Matryoshka Dolls, we propose M3: Matryoshka Multimodal Models, which learns to represent visual content as nested sets of visual tokens that capture information across multiple coarse-to-fine granularities. Our approach offers several unique benefits for LMMs: (1) One can explicitly control the visual granularity per test instance during inference, e.g. , adjusting the number of tokens used to represent an image based on the anticipated complexity or simplicity of the content; (2) M3 provides a framework for analyzing the granularity needed for existing datasets, where we find that COCO-style benchmarks only need around ~9 visual tokens to obtain accuracy similar to that of using all 576 tokens; (3) Our approach provides a foundation to explore the best trade-off between performance and visual token length at sample level, where our investigation reveals that a large gap exists between the oracle upper bound and current fixed-scale representations.
Chunjing Gan, Dan Yang, Binbin Hu +9 authors
MetRag enhances retrieval-augmented generation by integrating similarity and utility-oriented thoughts, employing an LLM as a task adaptive summarizer, and leveraging multi-layered thoughts for knowledge-intensive tasks.
Viraat Aryabumi, John Dang, Dwarak Talupuru +14 authors
Aya 23 is a powerful multilingual language model for 23 languages, outperforming previous models on a wide range of tasks.
Wenyu Du, Tongxu Luo, Zihan Qiu +5 authors
A depthwise stacking operator called $G_{\text{stack}}$ accelerates LLM pre-training, improving performance with less computational expense and providing guidelines for effective use.
Paolo Glorioso, Quentin Anthony, Yury Tokpanov +4 authors
Zamba, a 7B SSM-transformer hybrid model, achieves competitive performance with minimal parameter cost and efficient inference by combining a Mamba backbone with a shared attention module.
Bin Lei, Yuchen Li, Qiuwu Chen
AutoCoder, a Large Language Model surpassing GPT-4 Turbo and GPT-4o, uses a novel training method \textsc{AIEV-Instruct} to enhance code interpretation and execution validation.
Christopher Rae, Joseph K. L. Lee, James Richings
Introducing 2-stage backpropagation enhances the throughput of pipeline parallelism in training large DNNs by reducing idle compute time.
Aaron Defazio, Xingyu, Yang +4 authors
A new schedule-free method achieves state-of-the-art performance in optimization without requiring a stopping time or additional hyper-parameters, by unifying scheduling and iterate averaging.
Shenao Zhang, Donghan Yu, Hiteshi Sharma +4 authors
A bilevel objective for self-exploration in preference optimization enhances Large Language Models' performance by systematically exploring diverse responses and reducing bias towards unseen extrapolations.
Yixiao Zhang, Yukara Ikemiya, Woosung Choi +7 authors
Instruct-MusicGen, a finetuned MusicGen model with integrated text and audio modules, efficiently follows editing instructions with minimal additional parameters and achieves superior performance in text-to-music editing.
Chankyu Lee, Rajarshi Roy, Mengyao Xu +4 authors
NV-Embed, a decoder-only large language model with latent attention and instruction-tuned contrastive training, achieves state-of-the-art performance across various text embedding and retrieval tasks.
Jiachen Li, Weixi Feng, Tsu-Jui Fu +4 authors
T2V-Turbo improves video generation quality and speed by integrating differentiable reward models into consistency distillation, achieving superior results to models with iterative sampling.
Anthony Sarah, Sharath Nittur Sridhar, Maciej Szankin +1 authors
A method using one-shot NAS and genetic algorithms identifies smaller, more efficient LLaMA2-7B architectures with reduced size and increased throughput for various tasks, complemented by quantization.
Shaohua Wu, Jiangang Luo, Xi Chen +12 authors
Yuan 2.0-M32, using a mixture-of-experts architecture with an Attention Router, surpasses Llama3-70B on MATH and ARC-Challenge benchmarks with lower computational requirements and parameters.
Ling-Hao Chen, Shunlin Lu, Ailing Zeng +4 authors
MotionLLM, a unified video-motion framework, leverages Large Language Models for effective human behavior understanding using diverse datasets and evaluation benchmarks.
Bolin Ni, JingCheng Hu, Yixuan Wei +4 authors
Xwin-LM is a suite of alignment methodologies for large language models that includes supervised finetuning, reward modeling, rejection sampling finetuning, and direct preference optimization, showing consistent improvements on evaluation benchmarks.
Winnie Street, John Oliver Siy, Geoff Keeling +7 authors
Large language models exhibit varying levels of higher-order theory of mind, with some models surpassing adult performance on recursive mental and emotional reasoning tasks.
Wenqi Ouyang, Yi Dong, Lei Yang +2 authors
A novel method extends image editing to videos by aligning motion and refining appearance, maintaining visual and motion integrity with high temporal consistency.
Weiyu Li, Jiarui Liu, Rui Chen +4 authors
CraftsMan uses a 3D native diffusion model for efficient generation and refinement of high-quality 3D models with regular mesh topologies and interactive user edits.
Runqian Wang, Soumya Ghosh, David Cox +4 authors
Trans-LoRA facilitates data-free transfer of LoRA modules across base models using synthetic data generated by large language models, enabling lossless or improved performance across various tasks and model families.
Ruizhi Shao, Youxin Pang, Zerong Zheng +2 authors
We present a novel approach for generating high-quality, spatio-temporally coherent human videos from a single image under arbitrary viewpoints. Our framework combines the strengths of U-Nets for accurate condition injection and diffusion transformers for capturing global correlations across viewpoints and time. The core is a cascaded 4D transformer architecture that factorizes attention across views, time, and spatial dimensions, enabling efficient modeling of the 4D space. Precise conditioning is achieved by injecting human identity, camera parameters, and temporal signals into the respective transformers. To train this model, we curate a multi-dimensional dataset spanning images, videos, multi-view data and 3D/4D scans, along with a multi-dimensional training strategy. Our approach overcomes the limitations of previous methods based on GAN or UNet-based diffusion models, which struggle with complex motions and viewpoint changes. Through extensive experiments, we demonstrate our method's ability to synthesize realistic, coherent and free-view human videos, paving the way for advanced multimedia applications in areas such as virtual reality and animation. Our project website is https://human4dit.github.io.
Jialong Wu, Shaofeng Yin, Ningya Feng +4 authors
Interactive VideoGPT is a scalable multimodal autoregressive transformer for world modeling that integrates visual observations, actions, and rewards, enabling competitive video prediction, planning, and reinforcement learning.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号