Matryoshka Diffusion Models
Jiatao Gu, Shuangfei Zhai, Yizhe Zhang +2 authors
Matryoshka Diffusion Models use a NestedUNet architecture for joint denoising at multiple resolutions, enabling efficient high-resolution image and video synthesis.
Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
本周最热53 赞
A Picture is Worth a Thousand Words: Principled Recaptioning Improves Image Generation
Eyal Segalis, Dani Valevski, Danny Lumen +2 authors
Relabeling a text-to-image dataset with a specialized automatic captioning model improves image quality and semantic alignment in diffusion models.
50 篇论文 · 按点赞排序
In-Context Learning Creates Task Vectors
Roee Hendel, Mor Geva, Amir Globerson
In-Context Learning in Large Language Models can be understood as compressing a training set into a task vector that modulates a transformer for output generation.
CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images
Aaron Gokaslan, A. Feder Cooper, Jasmine Collins +6 authors
A transfer learning technique and efficient training recipe using Creative Commons images enable high-quality text-to-image models with synthetic captions and reduced computational requirements.
JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Lianghui Zhu, Xinggang Wang, Xinlong Wang
Large Language Models fine-tuned as scalable judges (JudgeLM) achieve state-of-the-art performance in evaluating open-ended benchmarks through a comprehensive dataset and benchmark, enhancing judgment efficiency and accuracy.
DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior
Jingxiang Sun, Bo Zhang, Ruizhi Shao +4 authors
DreamCraft3D uses a hierarchical approach with diffusion models and tailored 3D priors to generate high-fidelity and coherent 3D objects from 2D references, addressing geometry consistency and texture fidelity.
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
Elias Frantar, Dan Alistarh
QMoE enables efficient execution of trillion-parameter MoE models on affordable hardware through compression and bespoke GPU decoding.
HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models
Fuxiao Liu, Tianrui Guan, Zongxia Li +4 authors
HallusionBench is a benchmark that highlights language hallucination and visual illusion issues in vision-language models (VLMs), showcasing the limitations of current state-of-the-art models like GPT-4V and LLaVA-1.5.
Contrastive Prefence Learning: Learning from Human Feedback without RL
Joey Hejna, Rafael Rafailov, Harshit Sikchi +4 authors
A new regret-based algorithm, Contrastive Preference Learning (CPL), learns optimal policies directly from human preferences without learning a reward function, addressing optimization challenges in Reinforcement Learning from Human Feedback (RLHF).
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding
Haoxiang Wang, Pavan Kumar Anasosalu Vasu, Fartash Faghri +6 authors
A unified model combining CLIP and SAM using multi-task learning, continual learning, and distillation techniques achieves state-of-the-art zero-shot semantic segmentation performance with reduced computational and data requirements.
Wonder3D: Single Image to 3D using Cross-Domain Diffusion
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin +8 authors
Wonder3D leverages cross-domain diffusion models and geometry-aware normal fusion for high-quality, efficient, and consistent 3D mesh generation from single-view images.
An Early Evaluation of GPT-4V(ision)
Yang Wu, Shilong Wang, Hao Yang +4 authors
GPT-4V demonstrates strong visual understanding but has limitations in language comprehension, handling sensitive data, modalities like depth and audio, and fine visual nuances.
ConvNets Match Vision Transformers at Scale
Samuel L. Smith, Andrew Brock, Leonard Berrada +1 authors
ConvNets pre-trained on a large dataset match the performance of Vision Transformers on ImageNet with comparable computational resources.
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models
Ruida Wang, Wangchunshu Zhou, Mrinmaya Sachan
Synthesis Step by Step (S3) iteratively refines synthesized pseudo training examples using a large language model to reduce distributional discrepancies and improve small model performance on NLP tasks.
Woodpecker: Hallucination Correction for Multimodal Large Language Models
Shukang Yin, Chaoyou Fu, Sirui Zhao +7 authors
Woodpecker, a training-free method, corrects hallucinations in multimodal large language models by extracting key concepts, formulating questions, validating visual knowledge, generating visual claims, and correcting inconsistencies.
DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics
Kaiwen Zheng, Cheng Lu, Jianfei Chen +1 authors
A novel ODE solver for diffusion probabilistic models optimizes sampling efficiency and sample quality by minimizing discretization errors and introducing empirical model statistics, achieving faster and better results in both pixel-space and latent-space models.
SALMONN: Towards Generic Hearing Abilities for Large Language Models
Changli Tang, Wenyi Yu, Guangzhi Sun +6 authors
SALMONN, an integrated multimodal model combining a pre-trained text-based LLM with speech and audio encoders, demonstrates competitive performance and emergent abilities in various speech and audio tasks.
Democratizing Reasoning Ability: Tailored Learning from Large Language Model
Zhaoyang Wang, Shaohan Huang, Yuxuan Liu +8 authors
A tailored learning approach uses interactive multi-round distillation from large language models to enhance reasoning abilities in smaller LMs, fostering democratization through self-reflection and customized training.
HyperFields: Towards Zero-Shot Generation of NeRFs from Text
Sudarshan Babu, Richard Liu, Avery Zhou +3 authors
HyperFields, a method using a dynamic hypernetwork and NeRF distillation, generates text-conditioned NeRFs efficiently across multiple scenes with potentially fast fine-tuning.
ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search
Yuchen Zhuang, Xiang Chen, Tong Yu +5 authors
ToolChain*, an A* search-based planning algorithm, enhances the efficiency of LLM-based autonomous agents in navigating expansive action spaces by pruning incorrect API function calls.
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Zichang Liu, Jue Wang, Tri Dao +8 authors
DejaVu system predicts and exploits contextual sparsity on large language models to significantly reduce inference latency without affecting quality.
Controlled Decoding from Language Models
Sidharth Mudgal, Jong Lee, Harish Ganapathy +10 authors
Controlled decoding is an off-policy reinforcement learning method that uses a prefix scorer to guide language model generation towards high rewards and can handle multiple objectives without additional complexity.
LLM-FP4: 4-Bit Floating-Point Quantized Transformers
Shih-yang Liu, Zechun Liu, Xijie Huang +2 authors
LLM-FP4 quantizes both weights and activations of large language models to 4-bit floating-point, achieving near-full-precision performance with post-training quantization.
DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design
Kevin Lin, Zhengyuan Yang, Linjie Li +2 authors
DEsignBench evaluates text-to-image models in visual design contexts with human and automatic assessments across various design criteria.
DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation
Bangbang Yang, Wenqi Dong, Lin Ma +4 authors
A novel indoor scene texturing framework uses diffusion-based methods for text-driven texture generation with high detail and spatial coherence, addressing challenges in 3D spatial applications like XR/VR through dual texture alignment and a separated inpainting strategy.
Teaching Language Models to Self-Improve through Interactive Demonstrations
Xiao Yu, Baolin Peng, Michel Galley +2 authors
TriPosT trains smaller models to self-improve by interacting with larger language models to collect feedback, narrowing the performance gap in math and reasoning tasks.
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models
Zhihan Zhang, Shuohang Wang, Wenhao Yu +6 authors
Auto-Instruct automatically improves the quality of instructions for large language models by generating and ranking diverse candidates using a scoring model trained on various NLP tasks.
Detecting Pretraining Data from Large Language Models
Weijia Shi, Anirudh Ajith, Mengzhou Xia +5 authors
Researchers propose Min-K% Prob, a method for detecting pretraining data in large language models without needing knowledge of the pretraining corpus, using a benchmark that supports gold truth detection.
CLEX: Continuous Length Extrapolation for Large Language Models
Guanzheng Chen, Xin Li, Zaiqiao Meng +2 authors
CLEX, a method for continuous length extrapolation, extends the context window of Transformer-based LLMs beyond the training length with minimal performance loss.
TiC-CLIP: Continual Training of CLIP Models
Saurabh Garg, Mehrdad Farajtabar, Hadi Pouransari +5 authors
Web-scale Time-Continual (TiC) benchmarks are introduced to evaluate and improve the temporal robustness of vision-language models through efficient continual learning methods.