Trends · 研究趋势
数据来自 Hugging Face 论文的 AI 提取关键词,按月统计研究方向的增长与热度。
Yiyou Sun, Xinyang Han, Weichen Zhang +305 authors
Agents' Last Exam (ALE) is a benchmark for evaluating AI agents on long-term, economically valuable real-world tasks across 13 industry clusters with 1K+ tasks, revealing significant gaps between benchmark performance and practical deployment.
Alexey Dontsov, Dmitrii Korzh, Alexey Zhavoronkin +6 authors
CLEAR benchmark evaluates multimodal unlearning methods across textual and visual data, highlighting challenges and demonstrating the effectiveness of $\ell_1$ regularization on LoRA weights in mitigating catastrophic forgetting.
SAM 3D Team, Xingyu Chen, Fu-Jen Chu +20 authors
SAM 3D is a generative model that reconstructs 3D objects from single images using a multi-stage training framework that includes synthetic pretraining and real-world alignment, achieving high performance in human preference tests.
Ryan Wong, Jiawei Wang, Junjie Zhao +10 authors
WideSearch is a new benchmark evaluating the reliability of automated search agents in large-scale information collection tasks, revealing significant deficiencies in current systems.
Mingyi Deng, Lijun Huang, Yani Fan +22 authors
InteractComp evaluates search agents' ability to recognize and resolve query ambiguity through interaction, revealing significant gaps in current models' capabilities.
Xueqing Peng, Lingfei Qian, Yan Wang +41 authors
MultiFinBen is a multilingual and multimodal benchmark for financial domain tasks, evaluating LLMs across modalities and linguistic settings, revealing challenges in complex cross-lingual and multimodal financial reasoning.
Junnan Liu, Hongwei Liu, Linchen Xiao +6 authors
G-Pass@k and LiveMathBench assess Large Language Models' reasoning capabilities and consistency using complex mathematical problems.
Eilam Shapira, Omer Madmon, Itamar Reinman +3 authors
A benchmark for evaluating Large Language Models in strategic, language-based games provides insights into their rationality, mimicry of human behavior, and impact of economic characteristics on performance and outcome.
Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu +1 authors
FinTral, a multimodal LLM enhanced through domain-specific pretraining, instruction fine-tuning, and RLAIF, outperforms ChatGPT-3.5 and GPT-4 in financial analysis tasks with exceptional zero-shot performance.
Keen You, Haotian Zhang, Eldon Schoop +5 authors
Ferret-UI, a multimodal large language model tailored for mobile UI screens, enhances understanding and interaction through region annotations and a comprehensive dataset of UI tasks, outperforming existing models including GPT-4V.
Renjie Pi, Jianshu Zhang, Tianyang Han +3 authors
A new framework called Personalized Visual Instruction Tuning (PVIT) enhances multimodal large language models to recognize and engage with specific individuals in images, utilizing a curated dataset and benchmarks for evaluation.
Lichao Sun, Yue Huang, Haoran Wang +64 authors
This study assesses the trustworthiness of large language models across various dimensions, including truthfulness, safety, fairness, robustness, privacy, and machine ethics, finding a positive correlation with utility and highlighting differences between proprietary and open-source models.
Ziyu Liu, Tao Chu, Yuhang Zang +8 authors
A new benchmark (MMDU) and instruction tuning dataset (MMDU-45k) are introduced to evaluate and enhance large vision-language models' performance in multi-turn and multi-image conversations, highlighting the need for improved conversational instruction tuning data.
Fanqing Meng, Jin Wang, Chuanhao Li +9 authors
The Multimodal Multi-image Understanding (MMIU) benchmark assesses LVLMs across various multi-image tasks, highlighting challenges in spatial understanding.
Maosong Cao, Alexander Lam, Haodong Duan +3 authors
CompassJudger-1 is an open-source, all-in-one judge LLM that evaluates LLMs through unitary scoring, model comparisons, critiques, and various tasks, supported by the JudgerBench benchmark.
Jiarui Zhang, Ollie Liu, Tianyu Yu +2 authors
A benchmark called Geoperception is introduced to evaluate MLLMs' geometric description accuracy, leading to the development of Euclid, a model optimized for low-level geometric perception using synthetic data and a data curriculum.
Junyan Ye, Baichuan Zhou, Zilong Huang +12 authors
LOKI is a benchmark for evaluating the synthetic data detection capabilities of large multimodal models across various modalities.
Tianbao Xie, Danyang Zhang, Jixuan Chen +14 authors
OSWorld provides a scalable, real computer environment for evaluating multimodal agents across various operating systems and tasks, highlighting significant gaps in current LLM/VLM capabilities.
Frank F. Xu, Yufan Song, Boxuan Li +18 authors
A benchmark platform evaluates AI agents' autonomy in performing professional tasks in a simulated workplace environment, showing that a significant portion of simpler tasks can be automated but complex, long-term tasks remain challenging.
Tiannan Wang, Jiamin Chen, Qingrui Jia +43 authors
Weaver, a family of specialized large language models, achieves superior writing capabilities through pre-training and fine-tuning methods, surpasses GPT-4 in various writing tasks, and supports retrieval-augmented generation and tool usage.
北京市昌平区探索星信息技术及软件开发工作室
京ICP备2026059466号