Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search

Jonathan Light; Min Cai; Weiqin Chen; Guanzhi Wang; Xiusi Chen; Wei Cheng; Yisong Yue; Ziniu Hu

返回文献探索

Paper · arXiv 2408.10635

Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search

Jonathan Light, Min Cai, Weiqin Chen, Guanzhi Wang, Xiusi Chen, Wei Cheng, Yisong Yue, Ziniu Hu

14 upvotesAugust 20, 2024arXiv 预印本

AI 摘要

Strategist uses LLMs and self-play simulations to enhance agents' strategic skills in multi-agent games, outperforming traditional RL and other LLM-based methods.

LLMsself-improvementself-play simulationsMonte Carlo tree searchLLM-based reflectionhigh-level strategic skillsaction planningdialogue generationGame of Pure StrategyThe Resistance: Avalon

Abstract

In this paper, we propose a new method Strategist that utilizes LLMs to acquire new skills for playing multi-agent games through a self-improvement process. Our method gathers quality feedback through self-play simulations with Monte Carlo tree search and LLM-based reflection, which can then be used to learn high-level strategic skills such as how to evaluate states that guide the low-level execution.We showcase how our method can be used in both action planning and dialogue generation in the context of games, achieving good performance on both tasks. Specifically, we demonstrate that our method can help train agents with better performance than both traditional reinforcement learning-based approaches and other LLM-based skill learning approaches in games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.