A thorough understanding of the tool calling of the agent based on Function Calling

Preface

AI agents are software or hardware entities that have a certain degree of autonomy, can perceive the environment, and perform specific tasks through intelligent decision-making. It combines artificial intelligence technologies (such as machine learning, natural language processing, computer vision, etc.) and can achieve goals independently or collaboratively. Function Calling based on the Large Language Model (LLM) enables agents to use tools effectively and interact with external APIs.

Not all LLM models support Function Calling. Models that support Function Calling (such as gpt-4, qwen-plus, etc.) can detect when a function needs to be called and output the function name and required parameters of the called function in JSON format.

Function Calling improves output stability and simplifies the complexity of prompt engineering. For models that do not support Function Calling, ReACT's relatively complex prompt engineering can be used to require the model to return a response in a specific format to distinguish different stages (thinking, action, observation).

Function Calling has two main uses:

• Obtaining data: For example, retrieving content from a knowledge base based on keywords, obtaining business data through a specific API interface
• Execute actions: For example, modify business status data through API interfaces and execute scheduled business operations

This article contains the following content:

• Detailed introduction to the Function Calling tool calling process and the interactive messages involved
• Manually write Agent code to implement Function Calling tool call

Function Calling tool calling process and interactive messages

Let's take the weather query of Beijing and Guangzhou as an example. LLM adopts Tongyi Qianwenqwen-plusThe process of querying weather is as follows:

1. Initiate a query request

When initiating a query to LLM, the messages list contains only one message (role is user, content is the user query content). In addition, the tools definition is also required.

The tools definition contains the following:

• name: function name
• description: function description
• parameters: parameter definition

In this example, the function is definedget_weather(location).

We use curl to initiate a POST request. The JSON structure of the body can be found at https://platform.openai.com/docs/api-reference/chat/create

#!/bin/bash

export  OPENAI_API_BASE = "https://dashscope.aliyuncs.com/compatible-mode/v1"
export  OPENAI_API_KEY = "sk-xxx" # Replace with your key

curl  ${OPENAI_API_BASE} /chat/completions  \
-H "Content-Type: application/json"  \
-H "Authorization: Bearer  $OPENAI_API_KEY "  \
-d '{
  "model": "qwen-plus",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather like in Beijing and Guangzhou?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "location"
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}'

2. LLM returns tool_calls to get Beijing weather

After reasoning, LLM found that it needed to call a function to obtain Beijing's weather, and the reply message included tool_calls information.

In this case, you need to call the functionget_weather, the parameter name islocation, the parameter value isBeijing.

The complete JSON response is as follows:

{
  "choices": [
    {
      "message": {
        "content": "",
        "role": "assistant",
        "tool_calls": [
          {
            "index": 0,
            "id": "call_3ee91e7e0e0b420d811165",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"Beijing\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls",
      "index": 0,
      "logprobs": null
    }
  ],
  "object": "chat.completion",
  "usage": {
    "prompt_tokens": 166,
    "completion_tokens": 17,
    "total_tokens": 183,
    "prompt_tokens_details": {
      "cached_tokens": 0
    }
  },
  "created": 1745131660,
  "system_fingerprint": null,
  "model": "qwen-plus",
  "id": "chatcmpl-7c4fc4c8-92fa-90cc-aaf6-f673d7ab4220"
}

3. Processing function call to obtain Beijing weather

Parse and process LLM's tool_calls to obtain the function name and parameter list, and call the corresponding API interface to obtain the result.

For example: throughhttp://weather.cma.cn/api/now/54511Weather conditions in Beijing are available.

The complete JSON response is as follows:

{
  "msg": "success",
  "code": 0,
  "data": {
    "location": {
      "id": "54511",
      "name": "Beijing",
      "path": "China, Beijing, Beijing"
    },
    "now": {
      "precipitation": 0.0,
      "temperature": 24.3,
      "pressure": 1007.0,
      "humidity": 35.0,
      "windDirection": "Southwest wind",
      "windDirectionDegree": 207.0,
      "windSpeed": 2.7,
      "windScale": "Breeze"
    },
    "alarm": [],
    "lastUpdate": "2025/04/20 14:25"
  }
}

4. Send context information and function call results to LLM

The message list sent to LLM contains 3 messages:

• The first role isuser, is the user's input
• The second role isassistant, is the tool_calls response of LLMget_weather('Beijing')
• The third role istool, is the tool callget_weather('Beijing')Results

#!/bin/bash

export  OPENAI_API_BASE = "https://dashscope.aliyuncs.com/compatible-mode/v1"
export  OPENAI_API_KEY = "sk-xxx" # Replace with your key

curl  ${OPENAI_API_BASE} /chat/completions  \
-H "Content-Type: application/json"  \
-H "Authorization: Bearer  $OPENAI_API_KEY "  \
-d '{
  "model": "qwen-plus",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather like in Beijing and Guangzhou?"
    },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_3ee91e7e0e0b420d811165",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"Beijing\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "content": "{\"msg\":\"success\",\"code\":0,\"data\":{\"location\":{\"id\":\"54511\",\"name\":\"北京\",\"path\":\"北京, 北京\"},\"now\":{\"precipitation\":0.0,\"temperature\":24.3,\"pressure\":1007.0,\"humidity\":35.0,\"windDirection\":\"Southwest wind\",\"windDirectionDegree\":207.0,\"windSpeed\":2.7,\"windScale\":\"Breeze\"},\"alarm\":[],\"lastUpdate\":\"2025/04/20 14:25\"}}",
      "tool_call_id": "call_3ee91e7e0e0b420d811165"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "location"
            }
          },
          "required": [
            "location"
          ]
        }
      }
    }
  ],
  "tool_choice": "auto"
}'

5. LLM returns to tool_calls to obtain Guangzhou weather

After reasoning, LLM found that it needed to call a function to obtain the weather in Guangzhou, and the reply message included tool_calls information.

In this case, you need to call the functionget_weather, the parameter name islocation, the parameter value isGuangzhou.

The complete JSON response is as follows:

{
  "choices": [
    {
      "message": {
        "content": "",
        "role": "assistant",
        "tool_calls": [
          {
            "index": 0,
            "id": "call_4a920a1bb9d54f8894c1ac",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"Guangzhou\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls",
      "index": 0,
      "logprobs": null
    }
  ],
  "object": "chat.completion",
  "usage": {
    "prompt_tokens": 312,
    "completion_tokens": 19,
    "total_tokens": 331,
    "prompt_tokens_details": {
      "cached_tokens": 0
    }
  },
  "created": 1745132731,
  "system_fingerprint": null,
  "model": "qwen-plus",
  "id": "chatcmpl-5e002b5b-7220-927e-9637-554355f80658"
}

6. Processing function call to obtain Guangzhou weather

Parse and process LLM's tool_calls to obtain the function name and parameter list, and call the corresponding API interface to obtain the result.

For example: throughhttp://weather.cma.cn/api/now/59287Weather conditions in Guangzhou are available.

The complete JSON response is as follows:

{
  "msg": "success",
  "code": 0,
  "data": {
    "location": {
      "id": "59287",
      "name": "Guangzhou",
      "path": "China, Guangdong, Guangzhou"
    },
    "now": {
      "precipitation": 0.0,
      "temperature": 30.1,
      "pressure": 1002.0,
      "humidity": 64.0,
      "windDirection": "Southeast wind",
      "windDirectionDegree": 167.0,
      "windSpeed": 2.4,
      "windScale": "Breeze"
    },
    "alarm": [],
    "lastUpdate": "2025/04/20 14:25"
  }
}

7. Send context information and function call results to LLM

The message list sent to LLM contains 5 messages:

• The first role isuser, is the user's input
• The second role isassistant, is the tool_calls response of LLMget_weather('Beijing')
• The third role istool, is the tool callget_weather('Beijing')Results
• The fourth role isassistant, is the tool_calls response of LLMget_weather('Guangzhou')
• Article 5 role istool, is the tool callget_weather('Guangzhou')Results

#!/bin/bash

export  OPENAI_API_BASE = "https://dashscope.aliyuncs.com/compatible-mode/v1"
export  OPENAI_API_KEY = "sk-xxx" # Replace with your key

curl  ${OPENAI_API_BASE} /chat/completions  \
-H "Content-Type: application/json"  \
-H "Authorization: Bearer  $OPENAI_API_KEY "  \
-d '{
  "model": "qwen-plus",
  "messages": [
    {
      "role": "user",
      "content": "What's the weather like in Beijing and Guangzhou?"
    },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_3ee91e7e0e0b420d811165",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"Beijing\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "content": "{\"msg\":\"success\",\"code\":0,\"data\":{\"location\":{\"id\":\"54511\",\"name\":\"北京\",\"path\":\"北京, 北京\"},\"now\":{\"precipitation\":0.0,\"temperature\":24.3,\"pressure\":1007.0,\"humidity\":35.0,\"windDirection\":\"Southwest wind\",\"windDirectionDegree\":207.0,\"windSpeed\":2.7,\"windScale\":\"Breeze\"},\"alarm\":[],\"lastUpdate\":\"2025/04/20 14:25\"}}",
      "tool_call_id": "call_3ee91e7e0e0b420d811165"
    },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_4a920a1bb9d54f8894c1ac",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"Guangzhou\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "content": "{\"msg\":\"success\",\"code\":0,\"data\":{\"location\":{\"id\":\"59287\",\"name\":\"Guangzhou\",\"path\":\"Guangzhou, Guangdong, China\"},\"now\":{\"precipitation\":0.0,\"temperature\":30.1,\"pressure\":1002.0,\"humidity\":64.0,\"windDirection\":\"Southeast Wind\",\"windDirectionDegree\":167.0,\"windSpeed\":2.4,\"windScale\":\"Breeze\"},\"alarm\":[],\"lastUpdate\":\"2025/04/20 14:25\"}}",
      "tool_call_id": "call_4a920a1bb9d54f8894c1ac"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "location"
            }
          },
          "required": [
            "location"
          ]
        }
      }
    }
  ],
  "tool_choice": "auto"
}'

8. LLM generates final response

LLM generates the final response:

The current weather conditions in Beijing are as follows:
- Temperature: 24.3℃
- Humidity: 35%
- Wind direction: Southwest
- Wind speed: Light breeze

The current weather conditions in Guangzhou are as follows:
- Temperature: 30.1℃
- Humidity: 64%
- Wind direction: Southeast
- Wind speed: Light breeze 

The above information is from the latest update, I hope it will be helpful to you!

The complete JSON response is as follows:

{
  "choices": [
    {
      "message": {
        "content": "The current weather conditions in Beijing are as follows:\n- Temperature: 24.3℃\n- Humidity: 35%\n- Wind direction: Southwest wind\n- Wind speed: Breeze\n\nThe current weather conditions in Guangzhou are as follows:\n- Temperature: 30.1℃\n- Humidity: 64%\n- Wind direction: Southeast wind\n- Wind speed: Breeze\n\nThe above information is from the latest update, I hope it will be helpful to you!",
        "role": "assistant"
      },
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null
    }
  ],
  "object": "chat.completion",
  "usage": {
    "prompt_tokens": 460,
    "completion_tokens": 105,
    "total_tokens": 565,
    "prompt_tokens_details": {
      "cached_tokens": 0
    }
  },
  "created": 1745133460,
  "system_fingerprint": null,
  "model": "qwen-plus",
  "id": "chatcmpl-fd1edc89-3ddb-9e27-9029-d2be2c81f3c1"
}

Manually write Agent code to implement Function Calling tool call

1. Create a Python environment

uv init agent
cd  agent
uv venv
.venv \ Scripts \ activate

uv  add  openai requests python-dotenv

2. Set up your API Key

Create .env, the content of .env is as follows (note to change OPENAI_API_KEY to your key)

OPENAI_API_KEY=your_api_key_here
OPENAI_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1

Add .env to .gitignore

3. Implement Agent code

The main code logic of the agent implemented based on the OpenAI SDK is: within the allowed number of iterations, loop processing, initiate chat completions until there are no tool_calls, the iteration ends, and the results are output.

pseudocode:

maxIter = 5 # Maximum number of iterations
for iterSeq in range(1, maxIter+1):
    Construct a chat completion request (with tools list and tool_choice)
        The number of iterations reaches the maximum value, and tool_choice is set to none (the tool is no longer called)
        Otherwise tool_choice is set to auto (calling the tool as needed)
    Get chat completion results
    If the chat completion result has tool_calls
        Parse and call the corresponding function
        Add message to message list and continue iterating
    Otherwise, it means that there is no need to call the tool, the iteration ends, and the results are output

The complete main.py code is as follows:

import json
import os
import requests
import urllib.parse
from typing import Iterable
from openai import OpenAI
from openai.types.chat.chat_completion_message_param import ChatCompletionMessageParam
from openai.types.chat.chat_completion_message_tool_call import (
    ChatCompletionMessageToolCall,
)
from openai.types.chat.chat_completion_user_message_param import (
    ChatCompletionUserMessageParam,
)
from openai.types.chat.chat_completion_tool_message_param import (
    ChatCompletionToolMessageParam,
)
from openai.types.chat.chat_completion_assistant_message_param import (
    ChatCompletionAssistantMessageParam,
)

# Load environment variables
from dotenv import load_dotenv
load_dotenv()

api_key = os.getenv("OPENAI_API_KEY")
base_url = os.getenv("OPENAI_API_BASE")
model = "qwen-plus"
client = OpenAI(api_key=api_key, base_url=base_url)

# Tool definition
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "location"}
                },
                "required": ["location"],
            },
        },
    }
]

# Get the weather
def get_weather(location: str) -> str:
    url = "http://weather.cma.cn/api/autocomplete?q=" + urllib.parse.quote(location)
    response = requests.get(url)
    data = response.json()
    if data["code"] != 0:
        return "No information found for this location"
    location_code = ""
    for item in data["data"]:
        str_array = item.split("|")
        if (
            str_array[1] == location
            or str_array[1] + "city" == location
            or str_array[2] == location
        ):
            location_code = str_array[0]
            break
    if location_code == "":
        return "No information found for this location"
    url = f"http://weather.cma.cn/api/now/{location_code}"
    return requests.get(url).text

# Implement tool calls
def invoke_tool(
    tool_call: ChatCompletionMessageToolCall,
) -> ChatCompletionToolMessageParam:
    result = ChatCompletionToolMessageParam(role="tool", tool_call_id=tool_call.id)
    if tool_call.function.name == "get_weather":
        args = json.loads(tool_call.function.arguments)
        result["content"] = get_weather(args["location"])
    else:
        result["content"] = "Function is not defined"
    return result

def main():
    query = "What's the weather like in Beijing and Guangzhou?"
    messages: Iterable[ChatCompletionMessageParam] = list()
    messages.append(ChatCompletionUserMessageParam(role="user", content=query))
    maxIter = 5 # Maximum number of iterations
    for iterSeq in range(1, maxIter+1):
        print(f">> iterSeq:{iterSeq}")
        print(f">>> messages: {messages}")
        # When the number of iterations reaches the maximum, the tool is no longer called
        toolChoice = "auto" if iterSeq < maxIter else "none"
        # Make a request to LLM
        chat_completion = client.chat.completions.create(
            messages=messages,
            model=model,
            tools=tools,
            tool_choice=toolChoice
        )
        tool_calls = chat_completion.choices[0].message.tool_calls
        content = chat_completion.choices[0].message.content
        if isinstance(tool_calls, list):
            # LLM's response information contains tool_calls information
            messages.append(
                ChatCompletionAssistantMessageParam(
                    role="assistant", tool_calls=tool_calls, content=""
                )
            )
            for tool_call in tool_calls:
                print(f">>> tool_call: {tool_call}")
                result = invoke_tool(tool_call)
                print(f">>> tool_call result: {result}")
                messages.append(result)
        else:
            # The response information of LLM does not contain tool_calls information. The iteration ends and the response text is obtained.
            print(f">>> final result: \n{content}")
            return
main()

Run the code:uv run .\main.py

The output log is as follows:

>> iterSeq:1
>>> messages: [{'role': 'user', 'content': 'What's the weather like in Beijing and Guangzhou'}]
>>> tool_call: ChatCompletionMessageToolCall(id='call_db29421754a8447590d99d', function=Function(arguments='{"location": "Beijing"}', name='get_weather'), type='function', index=0)
>>> tool_call result: {'role': 'tool', 'tool_call_id': 'call_db29421754a8447590d99d', 'content': '{"msg":"success","code":0,"data":{"location":{"id":"54511","name":"北京","path":"北京, China, 北京"},"now":{"precipitation":0.0,"temperature":24.5,"pressure":1006.0,"humidity":34.0,"windDirection":"Southwest wind","windDirectionDegree":191.0,"windSpeed":2.8,"windScale":"Breeze"},"alarm":[],"lastUpdate":"2025/04/20 15:35"}}'}
>> iterSeq:2
>>> messages: [{'role': 'user', 'content': 'How is the weather in Beijing and Guangzhou'}, {'role': 'assistant', 'tool_calls': [ChatCompletionMessageToolCall(id='call_db29421754a8447590d99d', function=Function(arguments='{"location": "北京"}', name='get_weather'), type='function', index=0)], 'content': ''}, {'role': 'tool', 'tool_call_id': 'call_db29421754a8447590d99d', 'content': '{"msg":"success","code":0,"data":{"location":{"id":"54511","name":"北京","path":"中国, 北京, Beijing"},"now":{"precipitation":0.0,"temperature":24.5,"pressure":1006.0,"humidity":34.0,"windDirection":"Southwest wind","windDirectionDegree":191.0,"windSpeed":2.8,"windScale":"Breeze"},"alarm":[],"lastUpdate":"2025/04/20 15:35"}}'}]
>>> tool_call: ChatCompletionMessageToolCall(id='call_ae1c03437392444c869cbf', function=Function(arguments='{"location": "Guangzhou"}', name='get_weather'), type='function', index=0)
>>> tool_call result: {'role': 'tool', 'tool_call_id': 'call_ae1c03437392444c869cbf', 'content': '{"msg":"success","code":0,"data":{"location":{"id":"59287","name":"Guangzhou","path":"Guangzhou, Guangdong, China"},"now":{"precipitation":0.0,"temperature":30.4,"pressure":1001.0,"humidity":64.0,"windDirection":"Southeast wind","windDirectionDegree":165.0,"windSpeed":2.2,"windScale":"Breeze"},"alarm":[],"lastUpdate":"2025/04/20 15:35"}}'}
>> iterSeq:3
>>> messages: [{'role': 'user', 'content': 'How is the weather in Beijing and Guangzhou'}, {'role': 'assistant', 'tool_calls': [ChatCompletionMessageToolCall(id='call_db29421754a8447590d99d', function=Function(arguments='{"location": "北京"}', name='get_weather'), type='function', index=0)], 'content': ''}, {'role': 'tool', 'tool_call_id': 'call_db29421754a8447590d99d', 'content': '{"msg":"success","code":0,"data":{"location":{"id":"54511","name":"北京","path":"中国, 北京, Beijing"},"now":{"precipitation":0.0,"temperature":24.5,"pressure":1006.0,"humidity":34.0,"windDirection":" Southwest wind","windDirectionDegree":191.0,"windSpeed":2.8,"windScale":"Breeze"},"alarm":[],"lastUpdate":"2025/04/20 15:35"}}'}, {'role': 'assistant', 'tool_calls': [ChatCompletionMessageToolCall(id='call_ae1c03437392444c869cbf', function=Function(arguments='{"location": "Guangzhou"}', name='get_weather'), type='function', index=0)], 'content': ''}, {'role': 'tool', 'tool_call_id': 'call_ae1c03437392444c869cbf', 'content': '{"msg":"success","code":0,"data":{"location":{"id":"59287","name":"Guangzhou","path":"Guangzhou, Guangdong, China"},"now":{"precipitation":0.0,"temperature":30.4,"pressure":1001.0,"humidity":64.0,"windDirection":"Southeast wind","windDirectionDegree":165.0,"windSpeed":2.2,"windScale":"Breeze"},"alarm":[],"lastUpdate":"2025/04/20 15:35"}}'}]
>>> final result: 
The current weather conditions in Beijing are as follows:
- Temperature: 24.5°C
- Humidity: 34%
- Wind direction: Southwest
- Wind speed: light breeze (2.8 m/s)
- Last updated: 2025/04/20 15:35

The current weather conditions in Guangzhou are as follows:
- Temperature: 30.4°C
- Humidity: 64%
- Wind direction: Southeast
- Wind speed: light breeze (2.2 m/s)
- Last updated: 2025/04/20 15:35

END