Higress.ai site is newly released, easily unlocking new AI capabilities and opening up global services!

Written by

Audrey Miles

Updated on:July-10th-2025

With the rapid development of big model technology, the engineering of AI applications has put forward many requirements for the underlying infrastructure, especially in terms of security, efficiency, performance, etc. Among them, the AI gateway is one of the most important AI infrastructure components.

The AI Gateway is a deep evolution of the traditional API Gateway for large model scenarios. While ensuring the basic gateway capabilities, it has also made special enhancements to the characteristics of AI services:

Adaptation to scenario characteristics: Specially optimizes transmission requirements such as long connections, high concurrency, and large bandwidth, and adapts to the high latency characteristics of large model services.
Intelligent traffic management: supports multi-model dynamic routing, intelligent load balancing, API Key rotation scheduling, and semantic request caching.
Security and compliance assurance: built-in content security filtering, token quota management, multi-level current limiting and circuit breaking, and other security capabilities.
Fine-grained cost control: Provides operation and maintenance tools such as call audit analysis, grayscale traffic distribution, and automatic retry of failed requests.

By unifying the access layer protocol, the AI Gateway helps developers achieve efficient integration and management of multi-source AI services, reducing access and operation and maintenance costs in complex scenarios. For a more comprehensive description of the core capabilities and usage scenarios of the AI Gateway, please refer to the following two articles:

The core capabilities of the AI Gateway are still in the early stages of definition, but they are inseparable from the core of rapid integration of AI Agent and LLM API.

These eight usage scenarios are the most frequently used ones that we have summarized in the process of serving open source and commercial users. As the capabilities of the AI gateway are expanded and enhanced, the usage scenarios are also gradually enriched.

The Higress open source site has added a sub-site specifically for AI scenarios on the original main site, and provides a Chinese version and an international version (Beta). The international version is used to serve global developers.

In addition to providing common best practices (article format)/community/enterprise version/GitHub/documentation and other functions, Higress.ai has specially designed a [scenario experience] for quickly experiencing the AI gateway , and provides two ways of experience: open source experience and cloud experience. At the same time, we will launch the latest capabilities of the Higress AI gateway on this site . For example, we are about to launch the AI Guideline prompt function, and developers can quickly convert Nginx/Kong's Lua plug-in into a Higress Wasm plug-in based on AI programming tools such as Tongyi Lingma/Cursor.

After Higress.ai goes online, you may be concerned about the following issues:

Higress.ai and Higress.cn

What's the difference?

Cloud Native

Higress is a cloud-native API gateway. Its core is based on Istio and Envoy. It integrates traffic gateway, microservice gateway, security gateway and AI gateway into one. Wasm plug-ins can be written in Go/Rust/JS, etc. It provides dozens of ready-made general plug-ins and a ready-to-use console.

Higress.cn is the main site of Higress. As the official technical portal and one-stop resource platform of Higress, it focuses on providing developers with core capability demonstrations, open source ecosystem support and best practices for enterprise users related to the Higress technology stack.

Among them, AI gateway is a key component of modern AI infrastructure, and its technological evolution is deeply coupled with the ecological development of large language models. In the LLM technology stack, the continuous emergence of new technologies such as retrieval enhancement generation (RAG), agent, and MCP protocol has opened up multi-dimensional technological evolution directions for AI gateways in terms of protocol optimization, traffic management, model scheduling, and other dimensions.

In order to better show developers the richness of AI gateway content, Higress.ai came into being, aiming to provide an independent channel for experiencing AI gateway and display for typical AI application scenarios such as Agent development framework integration and LLM API governance. At the same time, Higress.ai will also showcase Higress's exploration of AI gateways, and work with AI developers to define the technical direction of the next generation of AI native gateways. In addition, Higress.ai will serve as our starting point for serving global AI developers.

It should be noted that the AI gateway is not a new form independent of the API gateway. It is essentially also an API gateway. The difference is that it has been specially extended to meet the new requirements of AI scenarios. It is both a continuation of the API gateway and an evolution of the API gateway.

Will Higress only be an AI gateway in the future?

Cloud Native

Of course not.

In the AI era, both Agents and large models have put forward more requirements on the access layer to avoid the "burden" of services. This has brought a historical development opportunity to AI gateways.

As early as last June when we released v1.4, we open-sourced many AI gateway capabilities. This was not a sudden idea after the accelerated development of large models during the Spring Festival. Further reading: Open-source AI gateway capabilities in June last year .

We believe that AI workloads and classic workloads will continue to merge to unleash the unlimited capabilities of AI and form unified management at the access layer. Therefore, Higress continues to focus on traffic gateways, microservice gateways, and security gateways to improve capabilities and experience.

At the traffic gateway level, Higress can be used as the Ingress entry gateway for the K8s cluster, and is compatible with a large number of K8s Nginx Ingress annotations, allowing for quick and smooth migration from K8s Nginx Ingress to Higress.

At the microservice gateway level, Higress can connect to various types of registry centers to discover service configuration routes, such as Nacos, ZooKeeper, Consul, Eureka, etc., and deeply integrates microservice technology stacks such as Dubbo, Nacos, and Sentinel. Compared with traditional Java-based microservice gateways, Higress based on the Envoy C++ gateway kernel can show better performance, significantly reduce resource utilization, and reduce costs.

At the security gateway level, Higress provides WAF capabilities and supports multiple authentication strategies, such as key-auth, hmac-auth, jwt-auth, basic-auth, OIDC, etc.

Higress's traffic gateway, microservice gateway, security gateway, and AI gateway all provide business-enhanced cloud services. The cloud service product on Alibaba Cloud is [API Gateway].