Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar +3 authors
Orca, a 13-billion parameter model, enhances small models by imitating the reasoning process from large foundation models using rich signals and diverse data, surpassing existing models in complex reasoning benchmarks.
Wei Chen, Zhiyuan Li, Zhen Guo +1 authors
An on-device Planner-Action framework, utilizing Phi-3 Mini for planning and Octopus for execution, employs model fine-tuning and multi-LoRA training to enhance efficiency and multi-domain handling on resource-constrained devices.
Ronen Eldan, Yuanzhi Li
TinyStories, a synthetic dataset of simplified stories, enables small or simple language models to produce fluent and coherent text with reasoning capabilities using an evaluation framework based on human-like grading.
Yuri Kuratov, Aydar Bulatov, Petr Anokhin +3 authors
Recursively augmented GPT-2 fine-tuning significantly extends the processing capability of generative models to handle long document sequences beyond $10^6$ elements.
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman +4 authors
RULER is a comprehensive benchmark that extends beyond simple retrieval tasks to evaluate long-context language models' performance on diverse tasks and context lengths.
Zhiyuan Hu, Yuliang Liu, Jinman Zhao +8 authors
LongRecipe extends the context window of large language models efficiently with reduced resources and preserved performance, achieving a 16-fold increase from 8k to 128k context size.
Weiquan Huang, Aoqi Wu, Yifan Yang +8 authors
LLM2CLIP leverages large language models to enhance CLIP's multimodal representation learning, enabling better handling of complex image captions by fine-tuning the LLM and using contrastive learning.
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran +3 authors
A new math instruction tuning dataset, OpenMathInstruct-1, leverages the Mixtral model to achieve competitive performance on math reasoning benchmarks using open-source LLMs.
Jingkang Yang, Yuhao Dong, Shuai Liu +8 authors
Octopus, a novel vision-language model, integrates multimodal perception and reasoning to perform diverse tasks using GPT-4 and Reinforcement Learning with Environmental Feedback in an experimental environment called OctoVerse.
Jian Xie, Kai Zhang, Jiangjie Chen +5 authors
TravelPlanner is a new benchmark for evaluating the planning capabilities of language agents in complex, real-world travel planning scenarios, demonstrating that current models still struggle with information gathering and constraint management.
David Rein, Betty Li Hou, Asa Cooper Stickland +5 authors
A dataset of extremely difficult multiple-choice questions challenges both experts and AI systems, facilitating the development of scalable oversight methods for AI-generated knowledge.
Zheng Cai, Maosong Cao, Haojiong Chen +97 authors
InternLM2 is an open-source LLM that outperforms predecessors through innovative pre-training and optimization techniques, including Supervised Fine-Tuning and Conditional Online Reinforcement Learning from Human Feedback.
Lianghui Zhu, Xinggang Wang, Xinlong Wang
Large Language Models fine-tuned as scalable judges (JudgeLM) achieve state-of-the-art performance in evaluating open-ended benchmarks through a comprehensive dataset and benchmark, enhancing judgment efficiency and accuracy.
Griffin Adams, Alexander Fabbri, Faisal Ladhak +2 authors
CoD prompt in GPT-4 improves summary density, informativeness, and human preference over vanilla prompts, with tradeoffs between readability and detail.
Tianlu Wang, Ping Yu, Xiaoqing Ellen Tan +7 authors
Shepherd, a small language model tuned for critique, outperforms or ties with larger models like ChatGPT in refining language model outputs using a high-quality feedback dataset.
Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner +9 authors
Naive finetuning of strong pretrained models with weak supervision can improve performance but is insufficient to match full model capabilities, highlighting challenges for aligning superhuman models.
Zhiqing Sun, Sheng Shen, Shengcao Cao +9 authors
The paper presents Factually Augmented RLHF to address multimodal misalignment in large multimodal models, significantly improving hallucination reduction and overall performance on vision-language tasks compared to existing methods.
Mirac Suzgun, Adam Tauman Kalai
Meta-prompting enhances language model performance by organizing and integrating multiple expert instances to handle complex tasks, demonstrating superiority over traditional methods in zero-shot settings.
Ke Wang, Houxing Ren, Aojun Zhou +7 authors
A method is presented for fine-tuning language models to solve math problems using code, achieving state-of-the-art results on MATH and GSM8K datasets.
Shengnan An, Zexiong Ma, Zeqi Lin +3 authors
LeMa, a learning-from-mistakes approach, enhances LLMs' mathematical reasoning by learning from inaccurate reasoning paths corrected by GPT-4, surpassing SOTA performance on math problems.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号