TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning
  for Video LLMs

Yunheng Li; Jing Cheng; Shaoyong Jia; Hangyi Kuang; Shaohui Jiao; Qibin Hou; Ming-Ming Cheng

返回文献探索

Paper · arXiv 2509.18056

TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs

Yunheng Li, Jing Cheng, Shaoyong Jia, Hangyi Kuang, Shaohui Jiao, Qibin Hou, Ming-Ming Cheng

27 upvotesSeptember 22, 2025arXiv 预印本

AI 摘要

TempSamp-R1, a reinforcement fine-tuning framework, enhances multimodal large language models for video temporal grounding by using off-policy supervision and a hybrid Chain-of-Thought training paradigm, achieving state-of-the-art performance on benchmark datasets.

reinforcement fine-tuningmultimodal large language modelsvideo temporal groundingGroup Relative Policy Optimizationon-policy samplingoff-policy supervisionnon-linear soft advantage computationChain-of-Thought trainingfew-shot generalization

Abstract

This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework designed to improve the effectiveness of adapting multimodal large language models (MLLMs) to video temporal grounding tasks. We reveal that existing reinforcement learning methods, such as Group Relative Policy Optimization (GRPO), rely on on-policy sampling for policy updates. However, in tasks with large temporal search spaces, this strategy becomes both inefficient and limited in performance, as it often fails to identify temporally accurate solutions. To address this limitation, TempSamp-R1 leverages ground-truth annotations as off-policy supervision to provide temporally precise guidance, effectively compensating for the sparsity and misalignment in on-policy solutions. To further stabilize training and reduce variance in reward-based updates, TempSamp-R1 provides a non-linear soft advantage computation method that dynamically reshapes the reward feedback via an asymmetric transformation. By employing a hybrid Chain-of-Thought (CoT) training paradigm, TempSamp-R1 optimizes a single unified model to support both CoT and non-CoT inference modes, enabling efficient handling of queries with varying reasoning complexity. Experimental results demonstrate that TempSamp-R1 outperforms GRPO-based baselines, establishing new state-of-the-art performance on benchmark datasets: Charades-STA (R1@0.7: 52.9%, +2.7%), ActivityNet Captions (R1@0.5: 56.0%, +5.3%), and QVHighlights (mAP: 30.0%, +3.0%). Moreover, TempSamp-R1 shows robust few-shot generalization capabilities under limited data. Code: https://github.com/HVision-NKU/TempSamp-R1