A Real-World WebAgent with Planning, Long Context Understanding, and
  Program Synthesis

Izzeddin Gur; Hiroki Furuta; Austin Huang; Mustafa Safdari; Yutaka Matsuo; Douglas Eck; Aleksandra Faust

返回文献探索

Paper · arXiv 2307.12856

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra Faust

37 upvotesJuly 24, 2023arXiv 预印本

AI 摘要

WebAgent uses Flan-U-PaLM and HTML-T5 to improve task success in real-world web navigation through decomposed instructions, HTML summarization, and Python code generation.

pre-trained large language modelsLLMsautonomous web navigationopen domainnesscontext lengthinductive biasHTMLWebAgentcanonical sub-instructionstask-relevant snippetsPython programsgrounded code generationlocal attention mechanismsglobal attention mechanismslong-span denoising objectivesMiniWoB web navigation benchmarktask planning evaluation

Abstract

Pre-trained large language models (LLMs) have recently achieved better generalization and sample efficiency in autonomous web navigation. However, the performance on real-world websites has still suffered from (1) open domainness, (2) limited context length, and (3) lack of inductive bias on HTML. We introduce WebAgent, an LLM-driven agent that can complete the tasks on real websites following natural language instructions. WebAgent plans ahead by decomposing instructions into canonical sub-instructions, summarizes long HTML documents into task-relevant snippets, and acts on websites via generated Python programs from those. We design WebAgent with Flan-U-PaLM, for grounded code generation, and HTML-T5, new pre-trained LLMs for long HTML documents using local and global attention mechanisms and a mixture of long-span denoising objectives, for planning and summarization. We empirically demonstrate that our recipe improves the success on a real website by over 50%, and that HTML-T5 is the best model to solve HTML-based tasks; achieving 14.9% higher success rate than prior SoTA on the MiniWoB web navigation benchmark and better accuracy on offline task planning evaluation.