Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier +28 authors
A study of Multimodal Large Language Models finds a mix of pre-training data and image encoder design crucial for SOTA performance on multimodal benchmarks.
47 篇论文 · 按点赞排序
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham +10 authors
A model-stealing attack is introduced that can extract detailed information such as the embedding projection layer from black-box language models with minimal cost.
Eric Zelikman, Georges Harik, Yijia Shao +3 authors
Quiet-STaR is an extension of the Self-Taught Reasoner that teaches language models to infer and generate rationales for each token, improving their ability to predict difficult tokens and answer questions without task-specific fine-tuning.
Byung-Kwan Lee, Beomchan Park, Chae Won Kim +1 authors
MoAI, a new LLVM, integrates outputs from external CV models to enhance VL tasks without expanding model size or curating extra datasets.
Machel Reid, Nikolay Savinov, Denis Teplyashin +668 authors
Gemini 1.5 Pro, a multimodal mixture-of-experts model, achieves near-perfect recall and state-of-the-art performance in long-context tasks, including QA and ASR, surpassing previous models in token prediction and translation capabilities.
Hugo Laurençon, Léo Tronchon, Victor Sanh
A synthetic dataset of HTML code and screenshots is introduced to enhance the ability of vision-language models to convert screenshots into HTML code.
Changyue Liao, Mo Sun, Zihan Yang +4 authors
Fuyou enables efficient fine-tuning of large models on low-end GPUs by optimizing SSD-CPU communication and computation, achieving higher GPU utilization compared to ZeRO-Infinity.
Adam Ibrahim, Benjamin Thérien, Kshitij Gupta +5 authors
Simple continual learning strategies, including learning rate re-warming, re-decaying, and data replay, can match the performance of full re-training for large language models with much lower computational cost.
Gemma Team, Thomas Mesnard, Cassidy Hardin +105 authors
Gemma, a family of lightweight and high-performing language models, outperforms similarly sized open models across text-based tasks and emphasizes the importance of responsible model development and safety.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen +14 authors
Chronos, a framework using pretrained transformer-based models for time series forecasting, outperforms classical methods and achieves comparable zero-shot performance on unseen datasets using tokenized time series data.
Haoyu Lu, Wen Liu, Bo Zhang +11 authors
DeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.
Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma +8 authors
Branch-Train-MiX (BTX) method enhances Large Language Models by asynchronously training experts in parallel and integrating them using Mixture-of-Expert layers with token-level routing for improved accuracy and efficiency.
Xiwei Hu, Rui Wang, Yixiao Fang +3 authors
ELLA, an Efficient Large Language Model Adapter, enhances text-to-image diffusion models by integrating powerful Large Language Models through a Timestep-Aware Semantic Connector, improving dense prompt comprehension and generation quality.
Enric Corona, Andrei Zanfir, Eduard Gabriel Bazavan +3 authors
VLOGGER generates audio-driven human videos from a single image using a diffusion-based method that includes 3D motion and text-to-image models, outperforming existing methods in quality, identity, and consistency.
Zilong Chen, Yikai Wang, Feng Wang +2 authors
V3D leverages video diffusion models to generate high-quality, multi-view consistent 3D objects and scenes from single images with rapid reconstruction times.
Kunchang Li, Xinhao Li, Yi Wang +4 authors
VideoMamba, an adaptation of Mamba, addresses local and global challenges in video understanding through a linear-complexity operator, self-distillation, and compatibility with multiple modalities, setting a new benchmark.
Liang Chen, Haozhe Zhao, Tianyu Liu +4 authors
FastV optimizes Large Vision-Language Models by learning adaptive attention in early layers and pruning visual tokens to reduce computational costs without sacrificing performance.
Jaerin Lee, Daniel Sungho Jung, Kanggeon Lee +1 authors
StreamMultiDiffusion is a real-time region-based text-to-image generation framework that achieves fast inference speeds using a multi-prompt stream batch architecture, enabling interactive image generation.
Haiyang Wang, Hao Tang, Li Jiang +5 authors
GiT is a versatile multi-task vision model using a vanilla ViT architecture that overcomes task-specific limitations and achieves competitive performance across various visual tasks through a universal language interface.
Sahand Sharifzadeh, Christos Kaplanis, Shreya Pathak +5 authors
A novel method using Large Language Models and text-to-image models generates synthetic image-text pairs, enhancing Visual-Language Model performance with less human-labeled data and improved efficiency.
Marco De Nadai, Francesco Fabbri, Paul Gigioli +11 authors
A scalable recommendation system integrating heterogeneous graph neural networks and a two-tower model enhances personalized audiobook recommendations by leveraging user preferences and reducing model complexity, resulting in improved streaming rates and benefits to other content types.
Wendi Zheng, Jiayan Teng, Zhuoyi Yang +6 authors
CogView3, a cascaded text-to-image diffusion model using relay diffusion, improves performance and reduces computational costs compared to SDXL.
Sun Ao, Weilin Zhao, Xu Han +6 authors
BurstAttention is a distributed attention framework that optimizes memory and communication for processing long sequences, reducing overhead and improving speed compared to existing solutions.
Uriel Singer, Amit Zohar, Yuval Kirstain +4 authors
Emu Video Edit (EVE) uses unsupervised distillation, specifically Factorized Diffusion Distillation, to perform video editing by aligning image editing and video generation adapters without needing supervised data.
Ruiyi Wang, Haofei Yu, Wenxin Zhang +5 authors
Interactive learning method SOTOPIA-π improves social intelligence in language agents through behavior cloning and self-reinforcement training, enhancing goal completion and safety without sacrificing general QA ability.
Zhengyi Wang, Yikai Wang, Yifei Chen +6 authors
A convolutional generative model that integrates geometric priors and leverages Flexicubes for efficient high-fidelity 3D mesh generation from a single image.
Anson Ho, Tamay Besiroglu, Ege Erdil +6 authors
Analysis of language model evaluations shows that compute requirements halve faster than hardware improvements, with compute making the larger contribution to performance gains.
Yabo Zhang, Yuxiang Wei, Xianhui Lin +5 authors
VideoElevator enhances text-to-video diffusion models by decomposing sampling into temporal and spatial refinement, using capabilities of text-to-image diffusion models.
Zeyu Liu, Weicong Liang, Zhanhao Liang +4 authors
Customizing the ByT5 text encoder with Glyph-ByT5 and integrating it with SDXL significantly improves text rendering accuracy and paragraph rendering in design and real images.
Sayash Kapoor, Rishi Bommasani, Kevin Klyman +22 authors
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号