Explore · 每周精选
发现最受关注的研究论文,追踪研究趋势,订阅感兴趣的期刊与关键词。
Shuming Ma, Hongyu Wang, Lingxiao Ma +7 authors
A 1-bit LLM variant, BitNet b1.58, achieves comparable performance to full-precision models with reduced computational costs and introduces new scaling laws and hardware design opportunities.
49 篇论文 · 按点赞排序
Linrui Tian, Qi Wang, Bang Zhang +1 authors
EMO, a direct audio-to-video synthesis framework, enhances realism and expressiveness in talking head video generation by capturing nuanced audio-facial relationships without intermediate models.
Anton Lozhkov, Raymond Li, Loubna Ben Allal +63 authors
StarCoder2, a large language model for code developed through a collaboration with Software Heritage, outperforms other models of similar size on various benchmarks and matches or outperforms larger models in specific areas.
Zechun Liu, Changsheng Zhao, Forrest Iandola +9 authors
MobileLLM leverages deep, thin architectures and advanced techniques to achieve high accuracy in sub-billion parameter models suitable for mobile devices.
Yixin Liu, Kai Zhang, Yuan Li +9 authors
A comprehensive review of Sora, a text-to-video generative AI model, including its underlying technologies, applications, challenges, and future directions in video generation and human-AI interaction.
Jake Bruce, Michael Dennis, Ashley Edwards +22 authors
Genie, a 11B parameter unsupervised generative model, creates action-controllable virtual worlds from unlabelled videos using spatiotemporal tokenization and autoregressive dynamics, enabling agent training from unseen video behaviors.
Soham De, Samuel L. Smith, Anushan Fernando +14 authors
Hawk and Griffin, models incorporating gated linear recurrences, achieve high performance with lower computational resources and better hardware efficiency compared to existing architectures.
Ruibin Yuan, Hanfeng Lin, Yi Wang +32 authors
ChatMusician, an open-source LLM with intrinsic musical abilities, outperforms GPT-4 in music generation and understanding tasks using text-compatible music notation.
Shangda Wu, Xu Tan, Zili Wang +3 authors
bGPT, a model using next byte prediction, simulates various digital processes with high accuracy, matching specialized models in performance and offering insights into algorithm and hardware behavior.
Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings +24 authors
Nemotron-4 15B, a large multilingual language model, excels in English, multilingual, and coding tasks, demonstrating superior performance in multilingual capabilities compared to larger and specialized models.
Fanqi Wan, Ziyi Yang, Longguang Zhong +3 authors
FuseChat extends the FuseLLM framework for knowledge fusion of chat LLMs through lightweight fine-tuning and parameter merging, achieving superior performance across various domains.
Ziheng Jiang, Haibin Lin, Yinmin Zhong +29 authors
MegaScale is a production system designed to train large language models at scale using more than 10,000 GPUs, focusing on efficiency, stability, and fault tolerance through advanced observability and diagnostic tools.
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace +8 authors
A large-scale dataset of video-text pairs, automatically curated and annotated using multimodal models, significantly improves performance in video captioning, retrieval, and generation tasks.
Ming Zhong, Yelong Shen, Shuohang Wang +6 authors
Two training-free methods, LoRA Switch and LoRA Composite, enhance multi-LoRA composition in text-to-image models, leading to improved image synthesis performance.
Ilija Radosavovic, Bike Zhang, Baifeng Shi +5 authors
A causal transformer for next token prediction of sensorimotor trajectories enables zero-shot humanoid walking and generalizes to unseen commands.
Sohee Yang, Elena Gribovskaya, Nora Kassner +2 authors
LLMs exhibit latent multi-hop reasoning for certain complex prompts, with strong evidence of the first reasoning hop and moderate evidence of the second hop, showing scaling trends with model size.
Alex Zhuang, Ge Zhang, Tianyu Zheng +7 authors
A new series of models called StructLM improves LLMs' interpretation of structured data by surpassing existing models on 14 datasets and achieving SoTA on 7 tasks, indicating that structured knowledge grounding is still challenging.
Biao Zhang, Zhongtao Liu, Colin Cherry +1 authors
Systematic experiments reveal scaling laws for large language model finetuning, showing that model size and finetuning data size are critical factors, while parameter-efficient tuning methods generally provide limited benefits.
Raghav Kapoor, Yash Parag Butala, Melisa Russak +4 authors
A dataset named OmniACT is introduced to evaluate agents' capability to generate executable scripts for various computer tasks, highlighting the challenge for current models in achieving human-level proficiency.
Arindam Mitra, Hamed Khanpour, Corby Rosset +1 authors
Orca-Math, a 7-billion-parameter SLM based on Mistral-7B, achieves 86.81% on GSM8K without external tools, employing high-quality synthetic data and iterative preference learning.
Huaxiaoyue Wang, Kushal Kedia, Juntao Ren +14 authors
MOSAIC, a modular home robot architecture, collaborates with humans to perform complex cooking tasks using pre-trained models and specific modules, achieving 68.3% success in 60 collaborative trials with various recipes.
Omkar Thawakar, Ashmal Vayani, Salman Khan +6 authors
MobiLlama is a 0.5 billion parameter small language model designed for resource-constrained devices using parameter sharing to balance accuracy and efficiency.
Shyam Marjit, Harshit Singh, Nityanand Mathur +3 authors
DiffuseKronA, a Kronecker product-based adaptation module, reduces parameter counts and enhances quality in text-to-image generative models compared to LoRA and DreamBooth.
Chenxin An, Fei Huang, Jun Zhang +4 authors
Dual Chunk Attention (DCA) enhances Llama2 70B to handle over 100k tokens without fine-tuning by decomposing attention into chunk-based modules, achieving performance comparable to finetuned models.
Tom Sander, Pierre Fernandez, Alain Durmus +2 authors
Watermarked training data can be detected with high confidence in LLMs, making it easier to identify if watermarked outputs were used for fine-tuning compared to conventional methods.
Zhuofeng Wu, He Bai, Aonan Zhang +4 authors
A strategy breaking reasoning tasks into decomposition and solving phases allows distillation of the former for efficient inference and generalization.
Muyang Li, Tianle Cai, Jiaxin Cao +7 authors
DistriFusion leverages parallelism and displaced patch parallelism to significantly speed up diffusion models like Stable Diffusion XL without compromising image quality.
Sherry Yang, Jacob Walker, Jack Parker-Holder +5 authors
Video generation can serve as a unified interface for diverse tasks and advance real-world AI applications through techniques like in-context learning and reinforcement learning, with opportunities in robotics, self-driving, and science.
Mart van Baalen, Andrey Kuzmin, Markus Nagel +5 authors
Increasing quantization dimensionality with the GPTVQ method enhances the size-accuracy trade-off for large language models by interleaving quantization and using efficient compression techniques.
Chun-Hsiao Yeh, Ta-Ying Cheng, He-Yen Hsieh +6 authors
Gen4Gen addresses data and metric challenges in multi-concept personalization for text-to-image diffusion models, enhancing image generation quality through improved data creation and empirical prompting strategies.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号