agent-Specialization/docs/papers/翻译前_英文原文.txt

【翻译前 - 英文原文】
ReAct: Synergizing Reasoning and Acting in Language Models

================================================================================
1 ABSTRACT
================================================================================
While large language models (LLMs) have demonstrated impressive performance across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with and gather additional information from external sources such as knowledge bases or environments.

================================================================================
2 INTRODUCTION
================================================================================
A unique feature of human intelligence is the ability to seamlessly combine task-oriented actions with verbal reasoning (or inner speech, Alderson-Day & Fernyhough, 2015), which has been theorized to play an important role in human cognition for enabling self-regulation or strategization (Vygotsky, 1987; Luria, 1965; Fernyhough, 2010) and maintaining a working memory (Baddeley, 1992). Consider the example of cooking up a dish in the kitchen. Between any two specific actions, we may reason in language in order to track progress ("now that everything is cut, I should heat up the pot of water"), to handle exceptions or adjust the plan according to the situation ("I don't have salt, so let me use soy sauce and pepper instead"), and to realize when external information is needed ("how do I prepare dough? Let me search on the Internet").

Recent results have hinted at the possibility of combining verbal reasoning with interactive decision making in autonomous systems. On one hand, properly prompted large language models (LLMs) have demonstrated emergent reasoning capabilities to solve complex tasks such as math problems and symbolic reasoning. On the other hand, recent work on LLM-based autonomous agents has enabled decision making in interactive environments such as the web and embodied environments. In this work, we propose ReAct, a general paradigm to combine reasoning and acting with language models for solving diverse language reasoning and decision making tasks.

================================================================================
3 METHOD
================================================================================
ReAct Prompting. For HotpotQA and Fever, we randomly select 6 and 3 cases from the training set and manually compose ReAct-format trajectories to use as few-shot exemplars in the prompts. Similar to chain-of-thought, ReAct is a prompting method that coordinates reasoning and actions to solve tasks. The key difference is that the ReAct trajectories contain explicit thoughts that not only plan and decompose goals but also incorporate reasoning about the current state, apply commonsense, and track progress.

The format is as follows: Thought 1 -> Action 1 -> Observation 1 -> ... -> Thought N -> Action N. Each thought reasons about the current situation and the next action to take, while each action can be one of the following types: (1) Search[entity], which returns the first k sentences from the corresponding Wikipedia entity page; (2) Lookup[string], which returns the next sentence containing the string in the current passage, simulating the Ctrl+F command on the browser.

================================================================================
4 RELATED WORK
================================================================================
The most well-known work of using LLMs for reasoning is Chain-of-Thought (CoT) (Wei et al., 2022), which reveals the ability of LLMs to formulate reasoning steps via few-shot prompting. Subsequent work includes self-consistency (Wang et al., 2022b) for majority voting among multiple reasoning paths and tree-of-thoughts (Yao et al., 2021) for exploring tree search-based reasoning patterns.

On the language model for action side, WebGPT (Nakano et al., 2021) uses LLMs to navigate web pages via searching and browsing. SayCan (Ahn et al., 2022) and PaLM-SayCan (Driess et al., 2022) use LLMs for robotic action planning. Inner Monologue (Huang et al., 2022b) uses "inner speech" in embodied environments but combines hard-coded language in pre-defined high-level action primitives. In contrast, ReAct interleaves free-form language reasoning and action in a general manner.

================================================================================
5 CONCLUSIONS
================================================================================
We propose ReAct, a simple yet effective method for synergizing reasoning and acting in large language models. Through a diverse set of tasks, we show that ReAct can produce interpretable, high-performing task-solving trajectories, outperforming both action-only and reasoning-only baselines. The interleaved reasoning traces allow the model to dynamically plan, track, and update action plans, while also incorporating reasoning about the external environment.

We believe that ReAct provides a promising direction for building more capable and trustworthy LLM agents. Future work includes exploring ReAct on more diverse tasks, investigating the optimal ways to prompt and finetune LLMs with interleaved reasoning and actions, and applying ReAct to other modalities beyond text.