Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova +4 authors
Transformer decoders exhibit near-perfect linear relationships between layers, which can be reduced with cosine-similarity-based regularization, leading to improved performance on benchmarks.
36 篇论文 · 按点赞排序
Jihwan Kim, Junoh Kang, Jinyoung Choi +1 authors
FIFO-Diffusion generates long text-conditional videos through iterative denoising with latent partitioning and lookahead strategies to minimize the training-inference gap.
Ting Jiang, Shaohan Huang, Shengyue Luo +8 authors
MoRA, a high-rank updating method using square matrices, enhances the ability of large language models to learn and memorize new knowledge, especially in memory-intensive tasks, compared to LoRA.
Huajian Xin, Daya Guo, Zhihong Shao +6 authors
Generating extensive Lean 4 proof data from mathematical competition problems improved DeepSeekMath 7B's theorem-proving capabilities over GPT-4 and other methods.
Jian Hu, Xibin Wu, Weixun Wang +3 authors
OpenRLHF is an open-source RLHF framework that enhances training efficiency and accessibility for researchers and practitioners.
Joshua Engels, Isaac Liao, Eric J. Michaud +2 authors
Research explores multi-dimensional features in language models, discovering interpretable circular representations in GPT-2, Mistral 7B, and Llama 3 8B, which are used for modular arithmetic tasks.
Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka +31 authors
INDUS, a suite of domain-specific LLMs trained on curated scientific corpora, surpasses general-purpose and existing domain-specific models on specialized NLP tasks and new scientific benchmarks.
William Brandon, Mayank Mishra, Aniruddha Nrusimha +2 authors
Cross-Layer Attention modifies transformer-based autoregressive large language models to reduce KV cache size while maintaining accuracy, enabling longer sequence lengths and larger batch sizes during inference.
Eloi Alonso, Adam Jelley, Vincent Micheli +4 authors
DIAMOND, a reinforcement learning agent using a diffusion world model, achieves top performance on Atari 100k by capturing visual details effectively.
Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti +5 authors
A novel approach using model-based clustering and zero-shot routing improves the generalization of adapters in large language models for new tasks.
Octo Model Team, Dibya Ghosh, Homer Walke +15 authors
Octo, a large transformer-based policy trained on extensive robotic datasets, demonstrates versatility and efficient fine-tuning for diverse robotic platforms and tasks.
Zhenwei Shao, Zhou Yu, Jun Yu +5 authors
A systematic study of lightweight large multimodal models (LMMs) led to the development of the Imp family, which achieves superior performance compared to larger models and is capable of high-speed inference on mobile devices.
Huanjin Yao, Wenhao Wu, Taojiannan Yang +7 authors
The Dense Connector enhances Multimodal Large Language Models by integrating multi-layer visual features, improving performance across image and video benchmarks.
Chong Mou, Mingdeng Cao, Xintao Wang +3 authors
ReVideo is a novel approach to video editing that precisely modifies both content and motion through a three-stage training strategy and spatiotemporal adaptive fusion module.
Yue Han, Junwei Zhu, Keke He +7 authors
Face-Adapter, an adaptive framework for pre-trained diffusion models, achieves high-fidelity face editing with efficient identity and attribute control, demonstrating performance competitive with fully fine-tuned models in face reenactment and swapping.
Haoyi Wu, Kewei Tu
A method reducing key-value cache memory in transformer architectures improves inference throughput without sacrificing performance in language modeling tasks.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley +2 authors
LiteVAE, an autoencoder family for LDMs using the 2D discrete wavelet transform, enhances scalability and efficiency without compromising quality, offering improved training dynamics and reconstruction quality.
Nadav Timor, Jonathan Mamou, Daniel Korat +6 authors
Distributed speculative inference (DSI) accelerates large language model inference faster than speculative inference (SI) and traditional inference methods, supporting a wider range of drafters and achieving significant speedups.
Tianwei Yin, Michaël Gharbi, Taesung Park +4 authors
DMD2 enhances one-step image generation by removing regression loss, integrating GAN loss, and enabling multi-step sampling, setting new benchmarks and achieving exceptional visual quality.
Kaelan Donatella, Samuel Duffield, Maxwell Aifer +3 authors
A new hybrid digital-analog algorithm efficiently implements natural gradient descent, achieving superior performance in training neural networks compared to existing digital methods.
Jialong Guo, Xinghao Chen, Yehui Tang +1 authors
New methods improve transformer efficiency by progressively replacing LayerNorm with re-parameterized BatchNorm and using a simplified linear attention module, achieving strong performance with lower latency across image classification and language modeling tasks.
Yao Teng, Yue Wu, Han Shi +5 authors
Diffusion Mamba combines the efficiency of sequence models with diffusion models to achieve high-resolution image synthesis at lower computational cost.
Xingyu Miao, Haoran Duan, Varun Ojha +4 authors
TSM, a novel method reducing accumulated error in DDIM inversion, enhances model stability and achieves high visual quality and performance in 3D generation with gradient clipping.
Fei Zhao, Taotian Pang, Chunhui Li +4 authors
AlignGPT addresses alignment inconsistencies in multimodal large language models by dynamically assigning and combining varying alignment capabilities during pre-training and instruction tuning phases.
Shiqi Yang, Zhi Zhong, Mengjie Zhao +4 authors
A simple generative transformer in Vector-Quantized GAN space achieves excellent results for image2audio and audio2image generation without the need for large models or extensive training.
Yilun Chen, Shuai Yang, Haifeng Huang +5 authors
Grounded 3D-LLM integrates 3D vision tasks into a unified generative framework using 3D large multi-modal models and scene referent tokens, demonstrating superior performance across various 3D benchmarks.
Yangjun Ruan, Chris J. Maddison, Tatsunori Hashimoto
An observational approach to scaling laws in language models can predict complex phenomena and agent performance using capability space and efficiency in converting training compute to capabilities.
Andrew Marmon, Grant Schindler, José Lezama +3 authors
Generative video models are enhanced with 3D camera motion conditioning to control and accurately simulate camera movements during video generation.
Hanwen Jiang, Arjun Karpur, Bingyi Cao +2 authors
OmniGlue, a learnable image matcher, enhances generalization through a vision foundation model and a keypoint position-guided attention mechanism, achieving superior performance on unseen image domains.
Cusuh Ham, Matthew Fisher, James Hays +4 authors
Personalized residuals and localized attention-guided sampling enhance concept-driven text-to-image generation by efficiently capturing concept identity and leveraging existing diffusion model priors.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号