The concept, degree of autonomy and level of abstraction of AI Agent

Everyone who works in AI knows that if you are not working on an agent, you would be embarrassed to say hello to people when you go out.

But what exactly is an agent? I'm afraid even experts may not be able to explain it clearly. It's really no one's fault, because the concept has such a broad meaning.

This article focuses on three things:

• What is the latest definition of Agent in the industry? Is there a consensus?
• What is the essence of the differences in the degree of autonomy among different types of agents?
• In order to design the Agent well, what abstractions need to be made?

What exactly is an Agent?

When people in the AI ​​community discuss Agents, they are usually quite entangled. A common scenario is that you have to restate the "Agent" you are currently talking about from time to time, what it specifically refers to, in order to ensure that the following discussion is a rigorous process. The reason for this awkward situation is that the word Agent has different meanings in different contexts.

In the field of AI, the origin of the word Agent is very early, and it may have appeared in academic discussions in the 1950s. So far, the more well-known academic origin comes from the reinforcement learning theory. Agent in reinforcement learning is a precise theoretical concept. It refers to an entity that can interact with the environment, has clear goals, can learn from experience, and can actively perform new actions to explore the unknown. What is mentioned here is that it can learn from experience and actively perform new actions. In fact, it is the trade-off between Exploitation and Exploration that we often encounter in machine learning.

However, the Agent concept that everyone is talking about in the current AI implementation scenario in the enterprise and in the AI ​​investment circle should be said to be a new concept developed with the emergence of LLM. Generally speaking, Agent refers to an autonomous intelligent system based on LLM technology. If you think about it carefully, this Agent is not the same concept as the Agent in the previous reinforcement learning. The Agent here emphasizes that it can complete some complex tasks in the real world (in an autonomous way); while the reinforcement learning Agent mainly emphasizes how it learns from the interaction with the environment. However, many people hope that the Agent system has the ability to "evolve autonomously", which seems to be a bit of a misappropriation of the concept of reinforcement learning Agent.

What is more troublesome is that there are still at least two interpretations of the term Agent in the current AI industry: one is a general concept, and the other is a specific concept.

• Generally speaking: Any AI system that can complete a task and is built using LLM can be called an Agent. Based on this understanding, our common chatbots, the workflows built using dify and n8n, and fully autonomous planning systems (Autonomous Agent) are all called Agents.
• Specifically refers to: a completely autonomous system. In terms of implementation, it is generally a specialized AI system driven by an agent loop. In each loop iteration, it uses LLM dynamic decision-making to automatically call appropriate tools and access appropriate memories. Each time it passes through a loop, it moves one step closer to the mission goal. Deep Research products or technologies launched by many companies are generally this type of agent. Here are two more examples of autonomous agents:
• LlamaIndex is a specialized implementation of autonomous agents [1].
• The last Autonomous Agent model is mentioned in Anthropic’s famous “Building effective agents” blog [2]. It is shown in the following figure:
  
  

Given the confusion surrounding the concept of Agent, many AI technology experts with quick thinking have begun to clarify the concept. Worthy of mention are a blog post by Anthropic[2], a blog post by the founder of LangChain[3], an official document by LlamaIndex[4], and an interview with Andrew Ng[5]. Based on these comments or materials, the industry seems to have gradually reached the following consensus on the concept of Agent:

• There is no “black and white” distinction between Workflow and Autonomous Agent. These systems differ only in the degree of autonomy. AgenticSystemTo replace the original broad and ambiguous concept of Agent. Regarding the term Agentic System, the founder of LangChain has the clearest statement, as follows[3]:

In practice, we see that most “agentic systems” are a combination of workflows and agents. This is why I actually hate talking about whether something is an agent, but prefer talking about how agentic a system is.

(Translation: In practice, we find that most "agentic systems" are a combination of workflow and agent. This is why I don't like to discuss whether something is an agent, but prefer to discuss how "agentic" a system is)

What is the nature of the degree of autonomy of an agent?

Let’s now take a closer look at some of the agent design patterns from Anthropic’s “Building effective agents” blog [2].

The above diagram shows prompt chaining. The model calls are executed sequentially. The entire workflow is completed in several steps. What to do in the first step and what to do in the second step are all determined in advance.

This diagram shows the Parallelization design pattern. The difference from the previous pattern is that Prompt Chaining is executed serially, while Parallelization means that several LLM calls are executed in parallel. However, regardless of parallel or serial, the execution path is determined in advance.

The pattern expressed in this diagram is called Routing, which means that only one of the three LLM Call nodes will be selected for execution. It is significantly different from the previous two patterns: the execution paths of Prompt Chaining and Parallelization are determined in advance, while the execution path of Routing can only be determined after the input data arrives. Analogous to programming languages, Routing is similar to if conditional judgment.

This diagram shows the Autonomous Agent mode (shown at the beginning of this article). This mode is more autonomous. In the previous Routing mode, although the specific execution path is dynamically determined based on the input data, the three optional execution paths are at least determined in advance. In the Autonomous Agent mode, the specific execution path cannot be determined in advance. The system usually executes multiple rounds/steps, and it is impossible to determine in advance what action will be executed in each step, and it is also impossible to determine in advance how many steps will be executed in total.

The above Agent design patterns are not all possible cases, but they basically show an Agent (or more precisely, an AgenticSystemThe essence of different degrees of autonomy is when the execution path of the system orchestration is decided :

• Static orchestration : Similar to Prompt Chaining and Parallelization, the execution path is completely determined in advance. It is like a designer gives you a very detailed "drawing", and you just need to execute it according to the predetermined "drawing".
• Dynamic program arrangement : Similar to the Routing mode. The possible execution logic and possible execution path are determined in advance, but the specific execution path can only be determined dynamically based on the input data. This is like Zhuge Liang giving you several "treasure bags" and telling you which "treasure bag" to open if you encounter any trouble. In short, it is equivalent to thinking of measures to deal with various possible situations in advance.
• Autonomous orchestration : Similar to the Autonomous Agent mode. It is impossible to imagine all possible situations in advance, and the execution path also needs to be determined according to the execution dynamics. Generally speaking, the Autonomous Agent may write programs on the spot to solve the problem according to the actual situation. This is like "a general is not always subject to the orders of the king when he is away from home." A general leading troops outside can only act expediently when encountering an emergency.

In short, from Prompt Chaining and Parallelization to Routing and Autonomous Agent, the system's autonomy is getting higher and higher, and the human engineer's precise control over the system is gradually weakening, so the system's behavior is becoming more and more dynamic. Of course, generally speaking, the complexity of the problems it can handle is also increasing.

Designing Agent Abstraction Levels

To design an agent well, it is not enough to just understand model algorithm technology or engineering programming technology. Designing an agent is a system engineering that is abstracted at multiple levels.

At the top level, we need to find business scenario adaptation. On the one hand, we need to have a deep understanding of the business process and know where the business value is the greatest; on the other hand, we also need to understand the boundaries of AI technology based on LLM and what problems it is good at solving. For example, the sales field is a large field, involving complex interactive processes with various types of customers, agents, and platforms. One of the typical AI landing scenarios is the automation of outreach to new customers: customers who are interested in their own products may have left their contact information through the website, conference booth or other channels, and the next step is generally to send emails to customers. If this email just introduces their own products in a stereotyped way, the conversion rate will not be high. And writing personalized emails for customers is exactly what generative AI is good at. Finding scenarios suitable for AI efficiency improvement in familiar fields and completing the first adaptation of technology and business may require close cooperation between business experts and technical experts.

The next level to consider is what kind of product form the Agent will present to the target users. Is a chatbot enough? Or is it enough to add simple human-in-the-loop interactions on the basis of automation? Or for more complex scenarios, like the IVERS model mentioned in the article "The Division and Integration of Generative AI and Traditional Software", more complex instructions, visual feedback, and subsequent multiple rounds of interactions are required between the product and the user? For example, in the field of AI Coding, Cursor provides us with a good Agent product example. The design process at this level requires the participation of professional AI product managers.

Next is the decomposition of agent granularity. This is still at the level of business process abstraction and has not really reached the AI ​​technology level. Can the entire business process that needs to be automated be divided into smaller, more professional, and more targeted sub-processes? Each sub-process can be implemented as an agent separately, and then the overall business goal can be achieved through multi-agent collaboration. For the decomposition of sub-processes, the clearer the boundaries and the clearer the goals, the higher the accuracy of the implemented AI Agent. Let's look at the previous sales outreach scenario again. Automatically writing personalized marketing emails can be divided into at least two steps: the first step is to search for more information in the company's internal intelligence database + public Internet based on the clues left by the customer, and summarize the information to form a description of the customer profile (which may include the customer's background, core business areas, current business development directions, etc.). The second step is to write personalized email content based on the customer profile information formed in the previous step and combined with its own products. Of course, a complete automation product may also automatically complete the last step and send emails in batches. The design work at this level is actually very important, and it is in line with the traditional BPM (Business Process Management) modeling and SOP (Standard Operating Procedure) sorting. If compared with the traditional software development model, this work belongs to the craft of DDD (Domain-Driven Design).

Further down, we come to the Agent design patterns described in Anthropic’s blog “Building effective agents”[2]. This level of abstraction no longer involves business concepts. Prompt Chaining, Routing, Parallelization, Orchestrator-workers, Evaluator-optimizer, Autonomous Agent, all of these are only one step away from technical implementation. But this level of abstraction is still only at the logical level.

Further down are the application layer code, framework code, and LLM model, which are the real AI technical aspects.

Abstract ability is the crown of human advanced thinking. It is precisely because the current LLM cannot completely replicate these human thinking processes that the design of AI Agents requires human experts and engineers to undertake the above abstract work. These tasks are indeed somewhat complicated, and some "experts" who are already confused in their hearts naturally cannot understand so many levels of abstraction. Therefore, the vague concept of the term Agent and the lack of consensus among people on Agent are easier to understand.