Exploring LLM Agents: What surprises does the ReAct framework hide?

As an emerging and promising field, LLM Agents are gradually changing the way people interact with technology. This article will explore LLM Agents in depth, focusing on the ReAct framework, analyzing its principles, advantages, and practical applications.

1. Evolution of AI Agents

The concept of “Agents” is not new in the field of artificial intelligence. From the early days of simple voice-activated toys to today’s powerful digital assistants, AI Agents have come a long way.

One of the earliest examples was the 1916 Radio Rex, a voice-activated toy that responded when you called its name, and although simple in function, it pioneered the idea of ​​machines acting on command. In 1952, Bell Labs introduced the Audrey, which could recognize spoken digits, laying the foundation for speech recognition technology.

In the 1960s, Joseph Weizenbaum of MIT created ELIZA, which simulated conversations through simple pattern matching. Although ELIZA's responses were not based on real understanding, they sparked people's thinking about machine intelligent interaction and produced the famous ELIZA effect. In the 1970s, Harpy, developed by Carnegie Mellon University, was an early speech recognition system that used language models to reduce errors and could understand about 1,000 words.

By the 1980s, IBM's Tangora used statistical methods to recognize up to 20,000 words, laying the foundation for modern speech-to-text software. In the 1990s, Dragon NaturallySpeaking became one of the first commercial speech recognition systems, capable of transcribing speech to text in real time, changing the way people interact with computers. That same year, the IBM Simon smartphone combined basic speech recognition and text messaging capabilities, heralding the arrival of multi-functional digital assistants.

In 2011, Apple launched Siri, which uses natural language processing to understand user requests, perform various tasks, and conduct basic conversations. In 2014, Amazon's Alexa was launched and quickly integrated into the smart home ecosystem, not only answering questions, but also managing home devices, placing shopping orders, and providing entertainment. These systems can all be considered "Agents", and some of them, such as Siri and Alexa, which use artificial intelligence, are "AI Agents".

2. The rise of LLM Agents

With the development of LLM, LLM Agents came into being. LLM Agents are systems that can use LLM to reason about problems, formulate solutions, and execute plans with the help of a series of tools, memory, and retrieval. It mainly consists of four key components: retrieval, reasoning, memory, and tool use.

The retrieval function enables the agent to obtain real-time data and ensure the accuracy of information. For example, in the financial field, the agent can obtain the latest market trend data and provide timely advice to investors. In terms of reasoning, advanced prompting techniques such as thought chaining are used to guide the agent to make decisions. Taking the medical field as an example, the agent can reason based on the patient's symptoms and medical knowledge to provide preliminary diagnostic suggestions. Memory includes short-term memory (recent conversation history), long-term memory (structured information storage) and specific task recall, enabling the agent to adapt to different scenarios and provide personalized services. Tool use allows the agent to interact with external systems to obtain data or perform operations. For example, in e-commerce scenarios, the agent can call the inventory system to check whether the product is in stock.

3. ReAct Framework Analysis

3.1 Overview of the ReAct Framework

ReAct, or Reasoning and Acting, was proposed in the paper “ReAct: Synergizing Reasoning and Acting in Language Models” in March 2023. It aims to make LLMs think and act more like humans when solving tasks.

The ReAct framework consists of three main steps: reasoning, action, and observation. When receiving a problem, the agent first performs reasoning to generate ideas, plans, or strategies to solve the problem. Then, it takes actions based on the reasoning results, such as searching for information or interacting with the environment. Finally, the agent observes the new information generated by the action and uses this information to further reason and act until the final answer is obtained.

Take the question “What is the population of the capital of France?” for example. ReAct Agent will first infer that the question needs to be split into two parts: find the capital of France, and then query the population of the capital. Next, it will perform the action of searching for “capital of France” and observe the search results to find out that the capital of France is Paris. After that, it will infer again that it needs to find the population of Paris and take the action of searching for “population of Paris”.

3.2 Advantages of the ReAct Framework

The ReAct framework has significant advantages. First, it is a synergistic combination that combines reasoning and action to play a synergistic role. Reasoning guides action, and action provides a basis for further reasoning, forming an efficient cycle. Second, the ReAct framework improves interpretability and credibility. The task solution traces it generates are more interpretable than baselines without reasoning traces, allowing people to distinguish between model internal knowledge and external environmental information. Finally, the ReAct framework makes the problem-solving process more factual and reliable. With the help of external knowledge bases, agents can obtain more accurate information and thus come up with more reliable answers.

4. Building a text-to-SQL Agent based on the ReAct framework

4.1 Database Construction

In order to build a text-to-SQL Agent, you first need to build a sample database. Here we use the SQLite database and create two tables: the "companies" table is used to store company details, including id, title and description; the "customers" table is used to store customer information, including id, name, revenue and a foreign key company_id to associate with the "companies" table.

import sqlite3def create_sample_database(): """ Create a simple SQLite database with a 'companies' table and a 'customers' table. Each company has an id, title, and description. Each customer has an id, name, revenue, and a foreign key company_id referencing the companies table. This function resets the database for a clean slate. """ conn = sqlite3.connect("sample.db") cur = conn.cursor() # Enable foreign key support in SQLite. cur.execute("PRAGMA foreign_keys = ON;") # Create the companies table. cur.execute(""" CREATE TABLE companies ( id INTEGER PRIMARY KEY, title TEXT NOT NULL, description TEXT ) """) # Create the customers table with a foreign key to companies. cur.execute(""" CREATE TABLE customers ( id INTEGER PRIMARY KEY, name TEXT NOT NULL, revenue REAL NOT NULL,            company_id INTEGER, FOREIGN KEY (company_id) REFERENCES companies(id) ON DELETE SET NULL ) """) # Insert sample data into companies. companies_data = [ (1, 'Acme Corp', 'Leading provider of innovative solutions'), (2, 'Beta Inc', 'Specializes in data analytics and insights'), (3, 'Gamma LLC', 'Pioneer in cloud computing services') ] cur.executemany("INSERT INTO companies (id, title, description) VALUES (?, ?, ?)", companies_data) # Insert sample data into customers. customers_data = [ (1, 'Alice', 1500.0, 1), (2, 'Bob', 2400.0, 2), (3, 'Charlie', 1800.0, 1), (4, 'Diana', 3200.0, 3), (5, 'Ethan', 2100.0, 2) ] cur.executemany("INSERT INTO customers (id, name, revenue, company_id) VALUES (?, ?, ?, ?)", customers_data) conn.commit() return conn

4.2 System prompt settings

Next, you need to set up system hints for the LLM to provide it with the database schema. System hints tell the LLM about the tables and their columns so that it can generate valid SQL queries.

system_prompt = f"""You are a helpful assistant that translates natural language queries into SQL queries for the following SQLite database schema.Table "companies": - id: INTEGER PRIMARY KEY - title: TEXT NOT NULL - description: TEXTTable "customers": - id: INTEGER PRIMARY KEY - name: TEXT NOT NULL - revenue: REAL NULL - company_id: INTEGER (Foreign key referencing companies(id))Instructions: 1. When you receive a text query that is relevant to this schema, generate the corresponding SQL query. Return only the SQL query with no additional commentary or characters like ```` or quotes eg SELECT * FROM COMPANIES. 2. If you receive a text query that includes an error message, generate a corrected SQL query that fixes the error, again returning only the SQL query. 3. If the query is irrelevant, do not output any extra text."""

4.3 ReAct Logic Implementation

The core logic of the Agent revolves around the ReAct loop: using LLM to generate SQL queries from natural language requests, executing the queries, and if errors occur, feeding error messages back to the LLM so that it can adjust and iterate until the query runs successfully.

import timedef test(): nl_request = input("Enter natural language query: ").strip() if not nl_request: print("No natural language query provided.") return conn = sqlite3.connect("sample.db") max_iterations = 5 iteration = 0 feedback = None final_result = None while iteration < max_iterations: iteration += 1 print(f"--- Iteration {iteration} ---") sql_query = generate_sql_query(nl_request, feedback) if not sql_query: print("No corresponding SQL query for this query.") break result, error = execute_query(conn, sql_query) if error: print(f"SQL execution error: {error}") feedback = f"The query resulted in the following error: {error}. Please adjust the SQL query accordingly." time.sleep(1) else: if not result: print("No results found.") else: print("Query executed successfully. Results:") for row in result: print(row) final_result = result break if final_result is None: print("Failed to generate a valid SQL query after several iterations.") conn.close()

In this simple example, since the database is relatively simple, the Agent can usually generate the correct SQL query in one go. However, in more complex database scenarios, the ReAct Agent may need multiple iterations to optimize the query to ensure that the final SQL accurately matches the user's request.

5. Application expansion of ReAct framework

In addition to text-to-SQL conversion, the ReAct framework also has broad application prospects in other fields. In the field of intelligent customer service, ReAct Agent can better understand users' complex problems, call the knowledge base, query relevant data through reasoning and action, and provide users with more accurate and comprehensive answers. In terms of intelligent writing assistance, it can infer appropriate writing ideas based on the author's writing intentions, and then call various tools, such as grammar checking tools, material libraries, etc., to help authors improve the quality of their writing.

In the field of data analysis, ReAct Agent can infer the required data and analysis methods based on the analysis requirements put forward by the user, and then call the corresponding data processing tools and algorithms to generate analysis reports. For example, when a user wants to analyze the sales trend of a product in different regions, the Agent can first infer that it needs to obtain relevant data from the sales database, and then use data analysis tools to process and visualize the data, and finally present clear analysis results to the user.

This article details the development of LLM Agents, focusing on the ReAct framework. From the early prototypes of AI Agents to the rise of LLM Agents, and then to the in-depth analysis and practical application of the ReAct framework, we have seen the continuous progress of artificial intelligence in terms of interaction and problem-solving capabilities. The ReAct framework provides a powerful solution for the development of LLM Agents by combining reasoning and action, showing significant advantages and broad application potential.