Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
DeepSeek-AI, Daya Guo, Dejian Yang +197 authors
DeepSeek-R1-Zero and DeepSeek-R1 utilize reinforcement learning and multi-stage training to enhance reasoning capabilities, with DeepSeek-R1 achieving performance comparable to OpenAI-o1-1217.
Jian Yang, Xianglong Liu, Weifeng Lv +68 authors
A comprehensive guide to code LLMs, covering their lifecycle from data curation to deployment, including techniques, trade-offs, and research-practice gaps.
Jiawei Zhao, Zhenyu Zhang, Beidi Chen +3 authors
Gradient Low-Rank Projection (GaLore) improves memory efficiency for training large language models without sacrificing performance, enabling pre-training of 7B models on consumer GPUs.
Yaowei Zheng, Richong Zhang, Junhao Zhang +2 authors
LlamaFactory is a unified framework enabling efficient fine-tuning of large language models across various tasks using a web-based user interface.
Aleksandr Nesterov, Andrey Sakhovskiy, Ivan Sviridov +5 authors
Experiments on a new Russian-language ICD coding dataset using models like BERT, LLaMA with LoRA, and RAG show significant accuracy improvements in automated clinical coding compared to manual annotations.
Mingqi Wu, Zhihao Zhang, Qiaole Dong +9 authors
Research on enhancing reasoning capabilities in large language models using reinforcement learning reveals that accurate reward signals are crucial for performance improvement, and current benchmarks may be unreliable due to data contamination.
Bowen Peng, Jeffrey Quesnelle, Honglu Fan +1 authors
YaRN extends the context window of transformer-based language models like LLaMA with improved efficiency and performance.
Fanxu Meng, Zengwei Yao, Muhan Zhang
TransMLA converts GQA-based models to MLA to reduce KV cache size and improve efficiency without sacrificing expressiveness.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman +1 authors
QLoRA enables efficient finetuning of large language models using 4-bit quantization and Low Rank Adapters, achieving high performance with reduced memory usage.
Zhengzhong Liu, Aurick Qiao, Willie Neiswanger +25 authors
LLM360 initiative promotes full transparency and reproducibility in LLM training by open-sourcing training code, data, model checkpoints, and intermediate results.
Chengyue Wu, Yukang Gan, Yixiao Ge +5 authors
A new post-pretraining method using expanded Transformer blocks for Large Language Models improves knowledge without catastrophic forgetting, yielding LLaMA Pro-8.3B that excels in general tasks, programming, and mathematics.
Jun Zhao, Zhihao Zhang, Qi Zhang +2 authors
A comprehensive study on transferring English-dominant LLMs to non-English languages using LLaMA demonstrates that comparable performance can be achieved with minimal pretraining data across various language capabilities and response quality metrics.
Yuhui Xu, Lingxi Xie, Xiaotao Gu +6 authors
QA-LoRA algorithm reduces computational requirements for large language models by quantizing weights during fine-tuning and integrating auxiliary weights post-training.
Wei Huang, Xudong Ma, Haotong Qin +7 authors
LLaMA3 performance degrades significantly when quantized to low bit-width, highlighting challenges in low-bit quantization for large language models.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng +10 authors
Using strong large language models as judges for evaluating other LLM-based chat assistants achieves high agreement with human preferences, offering a scalable and explainable solution compared to traditional benchmarks.
Xian Li, Ping Yu, Chunting Zhou +5 authors
A scalable instruction-following language model is built using auto-labelling and iterative self-augmentation and self-curation, outperforming other LLaMa-based models on Alpaca.
Weiquan Huang, Aoqi Wu, Yifan Yang +8 authors
LLM2CLIP leverages large language models to enhance CLIP's multimodal representation learning, enabling better handling of complex image captions by fine-tuning the LLM and using contrastive learning.
Chunting Zhou, Pengfei Liu, Puxin Xu +12 authors
A 65B parameter LLaMa language model trained with minimal instructional data matches or outperforms models with extensive human preference modeling in most cases, indicating that pretraining is predominantly responsible for knowledge acquisition.
Deepanway Ghosal, Yew Ken Chia, Navonil Majumder +1 authors
Fine-tuning VICUNA on a customized instruction dataset called FLANMINI enhances its problem-solving skills, leading to improved performance across various benchmarks.
Tiedong Liu, Bryan Kian Hsiang Low
A fine-tuned LLaMA model, Goat, outperforms GPT-4 in arithmetic tasks through supervised learning and task decomposition.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号