DEV Community: Apache APISIX

Release Apache APISIX Ingress Controller 2.0

Yilia — Mon, 22 Dec 2025 06:50:22 +0000

Apache APISIX Ingress Controller 2.0 is officially released. It delivers comprehensive Gateway API support, flexible multi-data-plane deployment, and etcd-free operation for robust, scalable Kubernetes traffic management.

Built on the high-performance API gateway Apache APISIX, APISIX Ingress Controller has undergone multiple iterations and validations, and is now capable of handling large-scale traffic management demands. The Apache APISIX community is pleased to announce the official release of APISIX Ingress Controller 2.0. This release delivers substantial enhancements across three foundational pillars—comprehensive compatibility, adaptable architecture, and enterprise-grade stability—empowering users to migrate their technology stacks smoothly and reliably.

Highlights of Apache APISIX Ingress Controller 2.0

Support Gateway API

This release achieves a significant milestone in Gateway API coverage with the addition of TCPRoute, UDPRoute, GRPCRoute, and TLSRoute. These extensions provide native, protocol-aware routing for a wide range of traffic types—from traditional HTTP and TCP/UDP to modern gRPC and TLS passthrough/termination. This unified support allows organizations to manage diverse ingress requirements within a consistent, future-ready configuration model, simplifying multi-protocol deployment and easing the transition to full Gateway API adoption.

Introduce Gateway API Extensions

Building upon adherence to the Gateway API design principles, APISIX Ingress Controller 2.0 introduces a set of API extensions under apisix.apache.org/v1alpha1 based on the Gateway API. These extensions provide additional capabilities not currently directly covered by the standard Gateway API, while maintaining the core semantics and usage patterns of the standard resources. They are designed to meet more complex and diverse real-world usage scenarios.

GatewayProxy: It defines the connection between the APISIX Ingress Controller and the APISIX, including auth, endpoints, and global plugins. It is referenced via parametersRef in Gateway, GatewayClass, or IngressClass resources.
BackendTrafficPolicy: It is for fine-grained traffic management of backend services, including load balancing, timeouts, retries, and host header handling in the APISIX Ingress Controller.
Consumer: It defines API consumers and their credentials, enabling authentication and plugin configuration for controlling access to API endpoints.
PluginConfig: It defines reusable plugin configurations that can be referenced by other resources like HTTPRoute, enabling separation of routing logic and plugin settings for better reusability and manageability.
HTTPRoutePolicy: It configures advanced traffic management and routing policies for HTTPRoute or Ingress resources, enhancing functionality without modifying the original resources.

These extensions offer a standardized, vendor-supported path to leverage advanced APISIX features directly within the Gateway API ecosystem.

Support APISIX Standalone API-Driven Mode

APISIX Ingress Controller 2.0 offers a lightweight, etcd-free deployment option through its Standalone API-Driven Mode.

This deployment paradigm stores routing configurations entirely in memory rather than in a configuration file. Updates are performed through a dedicated Standalone Admin API, which replaces the full configuration in a single operation and takes effect immediately via hot reloading, without requiring a restart.

This mode is designed specifically for the APISIX Ingress Controller and is primarily intended for integration with ADC (API Declarative CLI).

Support Multi-Data-Plane Deployment Mode

This release introduces flexible deployment options supporting multiple data plane modes, enabling a single ingress controller to manage several independent APISIX instances. This approach is ideal for environments requiring strict isolation—such as multi-tenancy, staging vs. production, or region-based routing—while maintaining centralized control.

Admin API Mode

In the traditional deployment approach, APISIX uses etcd as its configuration center, allowing administrators to dynamically manage routes, upstreams, and other resources through RESTful APIs. It supports distributed cluster deployments with real-time configuration synchronization.

Standalone Mode

APISIX can also run independently without relying on etcd, which is especially well‑suited for Kubernetes and single‑node deployments. It stores configurations in memory and manages them through the dedicated /apisix/admin/configs endpoint.

This mode is particularly suitable for Kubernetes environments and single-node deployments, where the API-driven memory management approach combines the convenience of traditional Admin API with the simplicity of Standalone mode.

This multi-mode strategy empowers organizations to tailor their ingress architecture to diverse requirements without sacrificing manageability or control.

Conclusion

Apache APISIX Ingress Controller 2.0 represents a significant evolution in Kubernetes ingress management, delivering a robust platform built for the complexity of modern, multi-protocol applications. By uniting comprehensive Gateway API support, extensible configuration through official API extensions, a lightweight standalone deployment mode, and versatile multi-data-plane management, this release provides a cohesive and powerful foundation for dynamic cloud environments.

Whether you are standardizing ingress across diverse workloads, seeking greater architectural flexibility, or requiring enterprise-grade stability at scale, APISIX Ingress Controller 2.0 offers a forward-looking solution that simplifies operations without compromising capability. It stands as a testament to the community-driven innovation within the Apache APISIX ecosystem, designed to meet today's demands while adapting to tomorrow's challenges.

For a complete list of features and changes, please refer to the Release Changelog.

Load Balancing AI/ML API with Apache APISIX

Yilia — Thu, 31 Jul 2025 09:13:52 +0000

This blog provides a step-by-step guide to configure Apache APISIX for AI traffic splitting and load balancing between API versions, covering security setup, canary testing, and deployment monitoring.

Overview

AI/ML API is a one-stop, OpenAI-compatible endpoint that is trusted by 150,000+ developers to 300+ state-of-the-art models—chat, vision, image/video/music generation, embeddings, OCR, and more—from Google, Meta, OpenAI, Anthropic, Mistral, and others.

Apache APISIX is a dynamic, real-time, high-performance API Gateway. APISIX API Gateway provides rich traffic management features and can serve as an AI Gateway through its flexible plugin system.

Modern AI workloads often require smooth version migrations, A/B testing, and rolling updates. This guide shows you how to:

Install Apache APISIX with Docker quickstart.
Secure the Admin API with keys and IP whitelisting.
Define separate routes for API versions v1 and v2.
Implement weighted traffic splitting (50/50) via the traffic-split plugin.
Verify the newly created split endpoint functionality.
Load test and monitor distribution accuracy.

To perform authenticated requests, you'll need an AI/ML API key. You can get one at https://aimlapi.com/app/keys/ and use it as a Bearer token in your Authorization headers.

Quickstart Installation

# 1. Download and run the quickstart script (includes etcd + APISIX)
curl -sL https://run.api7.ai/apisix/quickstart | sh

# 2. Confirm APISIX is up and running
curl -I http://127.0.0.1:9080 | grep Server
# ➜ Server: APISIX/3.13.0

Tip: If you encounter port conflicts, adjust Docker host networking or map to different ports in the quickstart script.

Secure the Admin API

By default, quickstart bypasses Admin API authentication. For any non-development environment, enforce security:

1. Set an Admin Key

Edit conf/config.yaml inside the APISIX container or local install directory, replacing the example key with your own API key obtained from the link above:

apisix:
  enable_admin: true            # Enable Admin API
  admin_key_required: true      # Reject unauthenticated Admin requests
  admin_key:
    - name: admin
      key: YOUR_ADMIN_KEY_HERE  # Generated admin key - you can replace this with a secure key as you wish
      role: admin

Security Best Practice: Use at least 32 characters, mix letters/numbers/symbols, and rotate keys quarterly.

2. Whitelist Management IPs (allow_admin)

Add your management or local networks under the admin: section:

admin:
  allow_admin:
    - 127.0.0.0/24   # Localhost & host network
    - 0.0.0.0/0      # Allow all (temporary/testing only)

Warning: 0.0.0.0/0 opens Admin API to the world! Lock this down to specific subnets in production.

3. Restart APISIX

docker restart apisix-quickstart

Check Logs: docker logs apisix-quickstart --tail 50 to ensure no errors about admin authentication.

Define Basic Routes for v1 and v2

Before splitting traffic, ensure each version works individually.

1. Route for v1

curl -i http://127.0.0.1:9180/apisix/admin/routes/test-v1 \
  -X PUT \
  -H "X-API-KEY: YOUR_ADMIN_KEY_HERE" \
  -d '{
    "uri": "/test/v1",
    "upstream": {
      "type": "roundrobin",
      "nodes": {"api.aimlapi.com:443": 1},
      "scheme": "https",
      "pass_host": "node"
    }
  }'

Tip: Use id fields if you want to manage or delete routes easily later.

2. Route for v2

curl -i http://127.0.0.1:9180/apisix/admin/routes/test-v2 \
  -X PUT \
  -H "X-API-KEY: YOUR_ADMIN_KEY_HERE" \
  -d '{
    "uri": "/test/v2",
    "upstream": {
      "type": "roundrobin",
      "nodes": {"api.aimlapi.com:443": 1},
      "scheme": "https",
      "pass_host": "node"
    }
  }'

Implement Traffic Splitting (50/50)

Use the traffic-split plugin for controlled distribution between v1 and v2. In the admin request below, replace YOUR_ADMIN_KEY_HERE with your actual key.

curl -i http://127.0.0.1:9180/apisix/admin/routes/aimlapi-split \
  -X PUT \
  -H "X-API-KEY: YOUR_ADMIN_KEY_HERE" \
  -d '{
    "id": "aimlapi-split",
    "uri": "/chat/completions",
    "upstream": {
      "type": "roundrobin",
      "nodes": {"api.aimlapi.com:443": 1},
      "scheme": "https",
      "pass_host": "node"
    },
    "plugins": {
      "traffic-split": {
        "rules": [
          {
            "weight": 50,
            "upstream": {"type":"roundrobin","nodes":{"api.aimlapi.com:443":1},"scheme":"https","pass_host":"node"},
            "rewrite": {"uri":"/v1/chat/completions"}
          },
          {
            "weight": 50,
            "upstream": {"type":"roundrobin","nodes":{"api.aimlapi.com:443":1},"scheme":"https","pass_host":"node"},
            "rewrite": {"uri":"/v2/chat/completions"}
          }
        ]
      }
    }
  }'

Tip: Adjust the weight values to shift traffic ratios (e.g., 80/20 for canary).

Note: rewrite must match the internal API path exactly.

Verify Split Endpoint Functionality

Test the /chat/completions endpoint you just created. Replace <AIML_API_KEY> with the key obtained earlier and use it as a Bearer token:

curl -v -X POST http://127.0.0.1:9080/chat/completions \
  -H "Authorization: Bearer <AIML_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4","messages":[{"role":"user","content":"ping"}]}'

Expected Output:

{"content":"Pong! How can I assist you today?"}

Tip: Use -v for verbose output to troubleshoot headers or TLS issues.

Load Test & Distribution Validation

After configuring the split route, use the following commands to validate distribution. Replace <AIML_API_KEY> with your Bearer token.

# 1. Send 100 test requests
time seq 100 | xargs -I {} curl -s -o /dev/null -X POST http://127.0.0.1:9080/chat/completions \
  -H "Authorization: Bearer <AIML_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4","messages":[{"role":"user","content":"ping"}]}'
# 2. Check APISIX logs for upstream hits (replace IPs with actual resolved IPs)
echo "v1 hits: $(docker logs apisix-quickstart --since 5m | grep -c '188.114.97.3:443')"
echo "v2 hits: $(docker logs apisix-quickstart --since 5m | grep -c '188.114.96.3:443')"

Expected: Approximately 50 requests to each upstream.

Tip: Use Prometheus or OpenTelemetry plugins for real‑time metrics instead of manual log parsing.

Best Practices & Next Steps

Rate Limiting & Quotas: Add limit-count plugin to protect your upstream from spikes.
Authentication: Layer on the key-auth plugin for consumer management.
Circuit Breaker: Prevent cascading failures with the api-breaker plugin.
Observability: Integrate Prometheus, Skywalking, or Loki for dashboards and alerts.
Infrastructure as Code: Consider managing APISIX config via Kubernetes CRDs or ADC for reproducibility.

References

Announcing APISIX Integration with AI/ML API

Yilia — Wed, 30 Jul 2025 08:56:39 +0000

We're thrilled to announce that AI/ML API has become a supported provider to the ai-proxy, ai-proxy-multi, and ai-request-rewrite plugins in Apache APISIX. All the AI/ML APIs will be supported in the next APISIX version.

Introduction

AI/ML API is a single endpoint that gives you access to more than 300 ready-to-use AI models—large language models, embeddings, image and audio tools—through one standard REST interface. It is used by over 150,000 developers and organizations as a centralized LLM API gateway.

We're thrilled to announce that AI/ML API has become a supported provider to the ai-proxy, ai-proxy-multi, and ai-request-rewrite plugins in Apache APISIX.

AI/ML API provides a unified OpenAI-compatible API with access to 300+ LLMs such as GPT-4, Claude, Gemini, DeepSeek, and others. This integration bridges the gap between your API infrastructure and leading AI services, enabling you to deploy intelligent features—like chatbots, real-time translations, and data analysis—faster than ever.

Proxy to OpenAI via AI/ML API

Prerequisites

Install APISIX.
Generate your API key on AI/ML API dashboard.

Configure the Route

Create a route and configure the ai-proxy plugin as such:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-proxy-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-proxy": {
        "provider": "aimlapi",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'" # Generated openai key from AI/ML API dashboard
          }
        },
        "options":{
          "model": "gpt-4"
        }
      }
    }
  }'

Test the Integration

Send a POST request to the route with a system prompt and a sample user question in the request body:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -H "Host: api.openai.com" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a mathematician" },
      { "role": "user", "content": "What is 1+1?" }
    ]
  }'

Verify Response

You should receive a response similar to the following:

{
  ...,
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "logprobs": null,
      "message": {
        "role": "assistant",
        "content": "1 + 1 equals 2.",
        "refusal": null,
        "annotations": []
      }
    }
  ],
  "created": 1753845968,
  "model": "gpt-4-0613",
  "usage": {
    "prompt_tokens": 1449,
    "completion_tokens": 1008,
    "total_tokens": 2457
  ...
}

Core Use Cases

1.Unified AI Service Management

Multi-Model Proxy and Load Balancing: Replace hardcoded vendor endpoints with a single APISIX interface, dynamically routing requests to models from OpenAI, Claude, DeepSeek, Gemini, Mistral, etc., based on cost, latency, or performance needs.
Vendor-Agnostic Workflows: Seamlessly switch between models (e.g., GPT-4 for creative tasks, Claude for document analysis) without code changes.

2.Cost-Optimized Token Governance

Token-Based Budget Enforcement: Set per-team/monthly spending limits; auto-throttle requests when thresholds are exceeded.
Caching & Fallbacks: Cache frequent LLM responses (e.g., FAQ answers) or reroute to cheaper models during provider outages.

3.Real-Time AI Application Scaling

Chatbots & Virtual Agents: Power low-latency conversational interfaces with streaming support for token-by-token responses.
Data Enrichment Pipelines: Augment APIs with AI—e.g., auto-summarize user reviews or translate product descriptions on-the-fly.

4.Hybrid/Multi-Cloud AI Deployment

Unified Control Plane: Manage on-prem LLMs (e.g., Llama 3) alongside cloud APIs (OpenAI, Azure) with consistent policy enforcement.
High Availability & Fault Tolerance: Built-in health-checks, automatic retries and failover; if one LLM fails, traffic is rerouted within seconds to keep services alive.

5.Enterprise AI Security & Compliance

Data Security and Compliance: Prompt Guard, content moderation, PII redaction, and full audit logs in a single place.
One Auth Layer for 300+ LLMs: Unified authentication (JWT/OAuth2/OIDC) and authorization for 300+ LLM keys and policies.

Conclusion

With AI/ML API now natively supported in Apache APISIX, you no longer have to choose between speed, security, or scale—you get all three.

One line of YAML turns your gateway into a 300-model AI powerhouse.
Zero code changes let you hot-swap GPT-4 for Claude, or route 10 % of traffic to a cheaper model for instant cost savings.
Built-in guardrails (PII redaction, token budgets, content moderation) keep compliance teams happy while your product team ships faster.

More Resources

Related APISIX AI Plugins
AI/ML API Community

From stdio to HTTP SSE: Host Your MCP Server with APISIX API Gateway

Yilia — Mon, 21 Apr 2025 10:24:20 +0000

Introduction

In contemporary API infrastructure, HTTP protocols and streaming communications (like SSE, WebSocket) have become mainstream for building real-time, interactive applications. Over the past few months, the Model Context Protocol (MCP) has gained popularity. However, most MCP Servers are implemented via stdio for local environments and cannot be invoked by external services and developers.

To bridge these services with modern API architectures, Apache APISIX has introduced the mcp-bridge plugin. It seamlessly converts stdio-based MCP services into HTTP SSE streaming interfaces and manages them through an API gateway for routing and traffic management.

Model Context Protocol (MCP) Overview

MCP is an open protocol that standardizes how AI applications provide context information to large language models (LLMs). It allows developers to switch between different LLM providers while ensuring data security and facilitating integration with local or remote data sources. Supporting a client-server architecture, MCP servers expose specific functionalities that are accessible to clients via these servers.

What Is the `mcp-bridge` Plugin?

The Apache APISIX mcp-bridge plugin launches a subprocess to manage the MCP Server, takes over its stdio channel, transforms client HTTP SSE requests into MCP protocol calls, and pushes responses back to the client via SSE.

Key features:

📡 Wraps MCP RPC calls into SSE message streams
🔄 Manages subprocess stdio lifecycle with queued RPC scheduling
🗂️ Lightweight MCP session management (including session ID, ping keep-alive, and queuing)
🧰 Supports session sharing across multiple workers for stability in APISIX multi-worker environments

How It Works and Architecture Diagram

Below is a sequence diagram illustrating the working mechanism of the mcp-bridge plugin, helping you to understand the data flow from stdio to SSE:

✅ Highlights:

APISIX manages SSE long-lived connections
The mcp-bridge plugin handles subprocesses, stdio, and scheduling queues
Clients receive real-time subprocess outputs, forming streaming SSE responses

Application Scenarios and Benefits

✅ Typical Application Scenarios

🛠️ Integrating existing MCP/stdio services with web platforms
🖥️ Cross-language and cross-platform subprocess service management

✅ Benefits

🌐 Modernization: Instantly transform stdio services into HTTP SSE APIs
🕹️ Managed: Unified management of subprocess launch and IO lifecycle
📈 Scalability: Session sharing in multi-worker environments for large-scale deployment support
🔄 Traffic Control Integration: Seamless API management system integration with APISIX traffic control, authentication, and rate-limiting plugins

Authentication and Rate Limiting with Apache APISIX Plugins

Apache APISIX provides robust authentication plugins (like OAuth 2.0, JWT, and OIDC) and rate-limiting plugins (such as rate limiting and circuit breakers). These enhance the mcp-bridge plugin, ensuring secure authentication and traffic control for connected MCP services.

Authentication Plugins

Support for OAuth 2.0, JWT, and OIDC plugins to protect APIs and MCP services.
Automatic client identity verification during API gateway requests to prevent unauthorized access.

Rate-Limiting Plugins

Rate Limiting: Restricts each client's request rate to prevent system overload.
Circuit Breaker: Automatically switches or returns errors to avoid system crashes during high traffic or failures.

Adding Authentication and Rate Limiting to MCP Servers

By integrating authentication and rate-limiting plugins with the mcp-bridge plugin, you can enhance API security and ensure system stability in high-concurrency environments.

Roadmap

The current version is a prototype. Future enhancements include:

Currently, MCP sessions are not shared across multiple APISIX instances. For multi-node APISIX clusters, proper session persistence configuration on the front-end load balancer is essential to ensure requests from the same client always go to the same APISIX instance.
The current MCP SSE connection is loop-driven. While the loop doesn't consume many resources (stdio read/write will be synchronous non-blocking calls), it's not efficient. We plan to connect to a message queue for an event-driven, scalable cluster approach.
The MCP session management module is just a prototype. We intend to abstract an MCP proxy server module to support launching MCP servers within APISIX for advanced scenarios. This proxy server module will be event-driven rather than loop-driven.

Summary

The Apache APISIX mcp-bridge plugin significantly simplifies the integration of Model Context Protocol (MCP) services with the HTTP API world. It offers a modern streaming interface management approach for traditional services.

APISIX-MCP: Embracing Intelligent API Management with AI + MCP

Yilia — Wed, 02 Apr 2025 03:03:57 +0000

This article introduces the MCP protocol and its application in APISIX-MCP. APISIX-MCP simplifies API management through natural language interaction, supporting the creation, updating, and deletion of resources.

Preface

With the explosive growth of large-scale AI model applications, many traditional systems are eager to integrate AI capabilities quickly. However, the current landscape of AI tools lacks unified standards, resulting in severe fragmentation. Different models vary in capability and integration methods, creating significant challenges for traditional applications during adoption.

Against this backdrop, in late 2024, Anthropic—the company behind the renowned Claude model—introduced the Model Context Protocol (MCP). MCP positions itself as the "USB-C interface" for AI applications. Just as USB-C standardizes connections for peripherals and accessories, MCP provides a standardized approach for AI models to connect with diverse data sources and tools.

Numerous services and applications have already adopted MCP. For example:

GitHub-MCP enables natural language code submissions and PR creation.
Figma MCP allows AI to generate UI designs directly.
With Browser-tools-MCP, tools like Cursor can debug code by interacting with DOM elements and console logs.

The official MCP repository includes implementations for Google Drive, Slack, Git, and various databases. As an open standard, MCP has gained widespread recognition in the AI community, attracting third-party developers who contribute hundreds of new MCP services daily. Anthropic, as the founder, actively drives MCP’s evolution by refining the protocol and educating developers.

About APISIX-MCP

The rise of MCP offers traditional applications a new technical pathway. Leveraging MCP’s standardized integration capabilities, we developed APISIX-MCP, which bridges large language models with Apache APISIX’s Admin API through natural language interaction. The current implementation supports the following operations:

General Operations

get_resource: Retrieve resources by type (routes, services, upstreams, etc.).
delete_resource: Delete resources by ID.

API Resource Management

create_route/update_route/delete_route: Manage routes.
create_service/update_service/delete_service: Manage services.
create_upstream/update_upstream/delete_upstream: Manage upstreams.
create_ssl/update_ssl/delete_ssl: Manage SSL certificates.
create_or_update_proto: Manage Protobuf definitions.
create_or_update_stream_route: Manage stream routes.

Plugin Operations

get_all_plugin_names: List all available plugins.
get_plugin_info/get_plugins_by_type/get_plugin_schema: Fetch plugin configurations.
create_plugin_config/update_plugin_config: Manage plugin configurations.
create_global_rule/update_global_rule: Manage global plugin rules.
get_plugin_metadata/create_or_update_plugin_metadata/delete_plugin_metadata: Manage plugin metadata.

Security Configuration

get_secret_by_id/create_secret/update_secret: Manage secrets.
create_or_update_consumer/delete_consumer: Manage consumers.
get_credential/create_or_update_credential/delete_credential: Manage consumer credentials.
create_consumer_group/delete_consumer_group: Manage consumer groups.

How to Use APISIX-MCP

APISIX-MCP is now open-sourced and available on npm and GitHub. It can be configured via any MCP-compatible AI client, such as Claude Desktop, Cursor, or the Cline plugin for VSCode.

Below is a step-by-step guide using Cursor:

Open Cursor, click the settings icon, and navigate to the settings page.

Click "Add new global MCP server" to edit the mcp.json configuration file:

   {
     "mcpServers": {
       "apisix-mcp": {
         "command": "npx",
         "args": ["-y", "apisix-mcp"],
         "env": {
           "APISIX_SERVER_HOST": "your-apisix-server-host",
           "APISIX_ADMIN_API_PORT": "your-apisix-admin-api-port",
           "APISIX_ADMIN_API_PREFIX": "your-apisix-admin-api-prefix",
           "APISIX_ADMIN_KEY": "your-apisix-api-key"
         }
       }
     }
   }

In the mcpServers field of the configuration file, add a service apisix-mcp, which can be changed. Then configure the commands for running the MCP service.

command: npx (Node.js package executor).
args: -y (auto-install dependencies) and apisix-mcp (package name).
env: Customize APISIX connection settings (defaults below):

In the env field, you can specify the APISIX service access address, Admin API port, prefix, and authentication key. These environment variables have default values, so if you start APISIX without any custom configuration, you can omit the env field entirely. The default values for each variable are as follows:

| Variable | Description | Default Value |

|---------------------------|--------------------------------------|-----------------------------|

| APISIX_SERVER_HOST | APISIX server host | http://127.0.0.1 |

| APISIX_ADMIN_API_PORT | Admin API port | 9180 |

| APISIX_ADMIN_API_PREFIX | Admin API prefix | /apisix/admin |

| APISIX_ADMIN_KEY | Admin API authentication key | edd1c9f034335f136f87ad84b625c8f1 |

Upon successful configuration, the MCP Servers list will show a green indicator for apisix-mcp, along with available tools.

Note: If setup fails, refer to the APISIX-MCP GitHub documentation for manual builds.

In the chat panel, select Agent mode and choose a model (e.g., Claude Sonnet 3.5/3.7 or GPT-4o).

Next, we can enter relevant operational commands to verify if the MCP service is functioning correctly. Following the workflow in APISIX's Getting Started documentation, we input the following into the dialog box and send the message:

"Help me create a route with path /api for accessing https://httpbin.org upstream, with CORS and rate-limiting plugins. Print the route details after configuration."

Next, in Cursor, you will see a process similar to the MCP tool invocation demonstrated in the video below. Due to the inherent randomness of large AI model responses, the exact operations performed may vary from the example shown.

Here, the auto-execution mode (YOLO Mode) is enabled, allowing Cursor to automatically invoke all tools in the MCP server. From the video, we can observe the AI performing the following operations based on our requirements:

Analyzing the plugins we need to configure, then calling get_plugins_list to retrieve all plugin names
Invoking get_plugin_schema to examine detailed configuration information for different plugins
Calling create_route to create the route
Using update_route to add the previously queried plugin configurations to the route
Executing get_route to verify whether the route was successfully configured and if the configuration is correct

The resulting route configuration includes:

Route ID: httpbin
Path: /api/*
Methods: GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS

CORS Plugin:

allow_origins: *
allow_methods: *
allow_headers: *
expose_headers: X-Custom-Header
max_age: 3600
allow_credential: false

limit-count Plugin:

count: 100
time_window: 60
key: remote_addr
rejected_code: 429
policy: local

Upstream:

type: roundrobin (load balancing strategy using round-robin)  
upstream node: httpbin.org:443 (backend service address)

Advantages of AI-Driven Operations

In the above process, we accomplished the creation of a route configured with CORS and rate-limiting through just one round of natural language interaction with AI. Compared to manual route configuration, leveraging AI offers several distinct advantages:

Reduced Cognitive Load: Eliminates manual documentation lookup and parameter memorization.
Automated Workflows: AI decomposes tasks (e.g., plugin setup → route creation) without human intervention.
Closed-Loop Validation: Auto-verification ensures correctness.
Iterative Optimization: Continuous dialogue refines configurations.

This interaction model transforms complex configuration processes into natural conversational experiences while maintaining accuracy and verifiability. These capabilities are achieved through the MCP protocol's semantic parsing of requirements, intelligent tool invocation, and final execution via Admin API.

It's important to note that APISIX-MCP isn't designed to completely replace manual configuration, but rather to optimize efficiency for high-frequency operations. Its value shines particularly in configuration debugging and rapid validation scenarios, creating effective complementarity with traditional management approaches. As the MCP ecosystem continues to evolve, we can anticipate deeper integration of such tools in API management, promising more sophisticated capabilities.

Conclusion

MCP enables intelligent operations for complex API systems. APISIX-MCP lowers the barrier to Apache APISIX adoption, with future plans for AI-traffic-specific plugins. The fusion of AI and API management promises smarter, more efficient infrastructure governance.

What Is an AI Gateway: Differences from API Gateway

Yilia — Fri, 28 Mar 2025 03:26:32 +0000

"The future isn't AI gateways—it's API gateways that speak AI."_ This blog explores AI gateways, their differences from API gateways, and why evolved solutions like Apache APISIX AI Gateway are shaping the future.

What Is an AI Gateway? Why Did It Arise in the AI Era?

The AI era has ushered in unprecedented complexity in deploying and managing artificial intelligence (AI) models. Organizations now juggle multiple models—from computer vision to large language models (LLMs)—across diverse environments (cloud, edge, hybrid). Traditional API gateways, designed for general-purpose data traffic, often fall short in addressing the unique challenges posed by AI workloads. This is where AI gateways emerge as critical middleware, acting as a unified control plane for routing, securing, and optimizing AI workloads.

The Rise of AI Gateways

The proliferation of generative AI and LLMs (Large Language Models) has introduced unique challenges:

Token Consumption: LLMs process requests in tokens, requiring granular tracking for cost and performance optimization.
Stream-Type Requests: AI agents often generate real-time, streaming responses (e.g., ChatGPT's incremental output), demanding low-latency handling.
Tool Integration: AI systems increasingly rely on external data sources and APIs (e.g., retrieving live weather data or CRM records).

According to a 2023 Gartner report, over 75% of enterprises now use AI models in production, driving demand for specialized infrastructure. Traditional API gateways, designed for RESTful APIs and static request-response cycles, struggle with these AI-specific demands. Enter the AI gateway—a purpose-built solution to manage AI-native traffic.

AI Agents vs. Traditional Devices: Why Stream-Type Requests Demand Specialized Handling

AI agents (e.g., chatbots, coding assistants) generate fundamentally different traffic patterns than traditional clients:

Metric	Traditional API Requests	AI Agent Requests
Request Type	Synchronous (HTTP GET/POST)	Asynchronous, streaming (SSE)
Latency	Milliseconds	Seconds-minutes (for chunks)
Billing	Per API call	Per token or compute time
Failure Modes	Timeouts, HTTP errors	Partial completions, hallucinations

The Stream-Type Challenge

When an AI agent requests a poem generated by GPT-4, the response is streamed incrementally. Traditional API gateways, built for atomic requests, struggle with:

Partial Responses: Aggregating chunks into a coherent audit log.
Token Accounting: Accurately counting tokens across streaming chunks.
Real-Time Observability: Monitoring latency per token or detecting drift in response quality.

Many purpose-built AI gateways lack distributed tracing, forcing engineers to cobble together metrics. In contrast, API gateways like Apache APISIX provide built-in integrations with Prometheus and Grafana, enabling token-level dashboards.

Two Types of AI Gateways: Purpose-Built vs. API Gateway Evolutions

Today's AI gateways fall into two categories:

Specific Purpose-Built AI Gateways

These are built from the ground up to address AI use cases. Startups like PromptLayer and LangChain offer solutions focused on:

Token-Based Rate Limiting: Enforcing usage quotas based on tokens instead of API calls.
Prompt Engineering Tools: Allowing developers to test and optimize prompts.
AI-Specific Analytics: Tracking metrics like response hallucination rates or token costs.

Example: OpenAI's API uses token-based pricing ($0.06 per 1K tokens for GPT-4), requiring gateways to meter usage precisely. A dedicated AI gateway might integrate token counters directly into its throttling logic.

However, these gateways often lack the observability and scalability of mature API management platforms. For instance, measuring token consumption across distributed microservices can lead to inaccuracies if the gateway lacks distributed tracing capabilities.

Evolved AI Gateways from API Gateways

Established API gateways like Kong, Apache APISIX, and AWS API Gateway are adapting to AI workloads by adding:

Streaming Support: Handling Server-Sent Events (SSE) and WebSockets for real-time AI responses.
Token-Aware Plugins: Extending rate-limiting plugins to track tokens.
LLM Orchestration: Managing multiple AI models (e.g., routing requests to cost-effective models like Mistral-7B for simple tasks).

Mature API gateways leverage decades of experience in security (OAuth, JWT), scalability (load balancing), and monetization—features often missing in AI-first solutions.

Why Evolved AI Gateways Are Winning Long-Term

While purpose-built AI gateways excel in niche scenarios, evolved API gateways are becoming the default choice for three reasons:

Cost Efficiency: Maintaining separate gateways for AI and non-AI traffic doubles operational overhead. Converged systems reduce costs by 30–50% (Gartner, 2023).
Flexibility: Enterprises can't predict which AI models will dominate. Platforms like Apache APISIX allow seamless integration of new LLMs without rearchitecting.
Future-Proofing: As AI becomes embedded in all apps (e.g., AI-powered search in e-commerce), gateways must handle hybrid workloads.

Model Context Protocol (MCP): Bridging AI Assistants and External Tools

To connect AI agents with external data and APIs, the Model Context Protocol (MCP) has emerged as a standardized framework. MCP defines how AI models request and consume external resources, such as:

Data Sources: SQL databases, vector stores (e.g., Pinecone).
APIs: CRM systems, payment gateways.
Tools: Code interpreters, and image generators.

How MCP Works

Context Injection: An AI assistant sends a request with a context header specifying required tools (MCP-Context: weather_api, crm).
Gateway Routing: The AI gateway validates permissions, injects API keys, and routes the request to relevant services.
Response Synthesis: The gateway aggregates API responses (e.g., weather data + CRM contacts) and feeds them back to the AI model.

Example: A user asks, "Email our top client in NYC about today's weather." The AI gateway uses MCP to:

Fetch the top client from Salesforce.
Retrieve NYC weather from OpenWeatherMap.
Pass this context to GPT-4 to draft the email.

Benefits of MCP

Security: Centralized policy enforcement (e.g., masking PII in CRM responses).
Cost Control: Caching frequent data requests (e.g., product catalogs).
Interoperability: Standardizing AI-to-API communication across vendors.

Future of AI Gateways: Convergence with API Monetization

As AI adoption matures, two trends will shape AI gateways:

Trend 1: The Decline of Standalone AI Gateways

Niche AI gateways will struggle to compete with evolved API gateways that offer:

Unified Governance: One platform for REST, GraphQL, and AI APIs.
Monetization Models: Token-based billing, subscription tiers.
Enterprise Features: Role-based access control (RBAC), audit logging.

Under such a trend, AI traffic will flow through traditional API gateways enhanced with AI capabilities.

Trend 2: API Gateways as AI Orchestrators

Future API gateways will act as AI orchestrators, handling:

Model Routing: Directing requests to optimal models based on cost, latency, or accuracy.
Hybrid Workflows: Blending AI and non-AI services (e.g., validating a GPT-4 response against a database).
Token Analytics: Real-time dashboards showing token spend by team or project.

The Bottom Line

In the future, the line between "AI gateway" and "API gateway" will blur. But the unchangeable fact is APIs are the basics of API gateways and AI gateways. Companies that adopt AI-ready API gateways today will gain a strategic edge in scalability, cost control, and innovation.

Conclusion: Embracing AI-API Convergence

AI gateways are not a replacement but an evolution of API gateways. While purpose-built solutions address immediate LLM challenges, their limitations in observability and scalability make them transitional. Established API gateways—enhanced with streaming support, token-aware plugins, and MCP—are poised to dominate.

Solutions like Apache APISIX AI Gateway exemplify this shift, blending AI-native features with battle-tested API management. As AI permeates every app, enterprises must choose platforms that scale beyond siloed use cases. The winners? Adaptable, extensible tools that speak both API and AI.

Comprehensive Overview of APISIX AI Gateway Features

Yilia — Mon, 24 Feb 2025 07:46:15 +0000

This article will provide an in-depth look at the AI gateway features of the current and upcoming versions of APISIX. As a multifunctional API and AI gateway, Apache APISIX offers efficient and secure LLM API calls for AI applications.

Introduction: The Rise of AI Agents and the Evolution of AI Gateway

In recent years, AI agents such as AutoGPT, Chatbots, and AI Assistants have seen rapid development. These applications rely heavily on API calls to large language models (LLMs), which has brought about challenges considering high concurrency, cost control, and security.

Traditional API gateways primarily serve Web APIs and microservices and are not optimized for the unique needs of AI applications. This has led to the emergence of the concept of AI gateway. An AI gateway needs to provide enhanced capabilities in the following areas:

Multi-LLM Proxy: Support for multiple LLM providers to avoid vendor lock-in.
Token Rate Limiting: Prevent API abuse and optimize cost management.
Security Protection: Including prompt filtering and content moderation to ensure compliance of AI applications.
Smart Traffic Management: Dynamically adjust LLM weights based on cost, latency, and stability.

Apache APISIX is not only an API gateway but also an AI gateway through its plugins, helping AI applications call LLM APIs more efficiently and securely.

LLM Proxy: Efficient Management of Multiple LLM Backends

AI applications typically do not rely on a single LLM provider but need to dynamically select the best model based on requirements. For example:

Using OpenAI GPT-4 for general text generation and Claude for legal document processing.
Switching between Mistral and Gemini to optimize cost and throughput.

Apache APISIX's LLM Proxy offers the following capabilities:

✅ Support for Multiple LLM Providers: Including OpenAI, DeepSeek, Claude, Mistral, Gemini, etc., to avoid vendor lock-in.

✅ LLM Weight and Priority Management: Adjust traffic distribution based on business needs.

✅ Multi-LLM Load Balancing: Dynamically adjust LLM weights based on latency, cost, and stability.

✅ Retry and Fallback Mechanisms: Ensure business continuity if an LLM API fails.

✅ Load Balancing Across Different Providers of the Same LLM:

For example:

Privately deployed DeepSeek.
Official DeepSeek API.
DeepSeek API from Volcano Engine

Users can flexibly allocate traffic weights among different DeepSeek providers based on latency, stability, and price to achieve the best calling strategy.

These capabilities enable AI applications to adapt flexibly to different LLMs, improve reliability, and reduce API calling costs.

AI Security Protection: Ensuring Safe and Compliant Use of AI

AI APIs may involve sensitive data, misleading information, and potential misuse. Therefore, an AI gateway needs to provide security at multiple levels.

The AI security capabilities provided by Apache APISIX include:

✅ AI RAG (Retrieval-Augmented Generation): Supports enterprise-owned knowledge bases to reduce LLM hallucinations and improve output reliability.

✅ Prompt Guard: Automatically intercepts sensitive, illegal, and inappropriate prompts to prevent malicious use by users.

✅ Prompt Decorator: Automatically adds content before and after user input to enhance the quality of LLM-generated content.

✅ Prompt Template: Makes it easier for users to reuse standardized prompts and improve interaction experience.

✅ Response Filtering & Moderation: Intercepts sensitive or non-compliant AI-generated content.

✅ Logging & Auditing: Provides complete API request logs for compliance audits.

These security measures ensure that AI applications meet enterprise-level security requirements and avoid compliance risks due to misleading AI content.

Token Observability and Management: Preventing High Bills Due to API Abuse

Calling LLM APIs consumes tokens, and API abuse can lead to significant costs. Apache APISIX provides fine-grained token monitoring and management mechanisms.

The token management capabilities of Apache APISIX include:

✅ Token Rate Limiting by Route/Service/Consumer/Consumer Group/Custom Dimension

✅ Support for Multiple Rate Limiting Modes:

Single-machine vs. cluster rate limiting to accommodate different scales of AI API services.
Fixed time window vs. sliding time window to flexibly control API rates.

✅ Different Rate Limiting Policies for Different LLMs: Prevent cost overruns.

Through Apache APISIX, enterprises can achieve fine-grained management of token resources and prevent high bills due to API abuse.

Smart Routing: Dynamic Traffic Management for AI APIs

During AI API calls, different tasks may require different LLMs. For example:

Code generation requests → sent to GPT-4 or DeepSeek.
Long-form summarization tasks → sent to Claude.
General conversations → sent to GPT-3.5 or Gemini.

The smart routing capabilities of Apache APISIX include:

✅ Context-Aware Routing Based on Request Content:

Select the optimal LLM based on prompt type.
Allocate different models (GPT-4 Turbo vs. GPT-3.5) based on user level (paid vs. free users).

✅ Response Caching: Reduce redundant API calls and improve response speed.

These capabilities help AI APIs run more efficiently, reduce API latency, and increase throughput.

Conclusion

With the rapid development of AI technology, API gateways also need to evolve to meet the unique needs of AI applications. Apache APISIX, with its LLM Proxy, token rate limiting, security protection, and smart routing features, has become the best choice for an AI gateway.

The core advantages of Apache APISIX compared to traditional API gateways are:

🚀 Support for Multiple LLM Providers: Avoid vendor lock-in.

⚡️ Smart Traffic Scheduling: Dynamic load balancing to improve API reliability.

🔒 Built-in Security Capabilities: Including prompt protection and content moderation to ensure secure and compliant AI APIs.

💰 Token Rate Limiting: Prevent high bills due to API abuse.

📊 High-performance Architecture: Meet the high concurrency needs of AI applications.

If you are building AI-related applications and want to have both a powerful API gateway and AI gateway, give Apache APISIX a try!

Cloud vs Open Source vs Commercial API Gateways: Which One Fits Your Needs?

Yilia — Mon, 17 Feb 2025 09:39:13 +0000

Introduction

API gateways have become essential components in modern cloud architectures. They provide security, traffic management, observability, and service orchestration—critical for handling APIs at scale. However, with multiple API gateway solutions available, choosing the right one can be challenging.

Broadly, API gateways fall into three categories:

Cloud API Gateways (e.g., Amazon API Gateway, Google Apigee)
Open Source API Gateways (e.g., Apache APISIX, Kong Gateway, Tyk)
Commercial API Gateways (e.g., MuleSoft, Boomi)

Each option has its advantages and trade-offs. This article provides a deep dive into their differences, hidden risks, and a strategic recommendation for companies looking to scale API usage and adopt hybrid cloud architectures.

Cloud API Gateways: Convenience vs. Lock-in

Pros:

✅ Fully managed, reducing operational burden

✅ Deep integration with cloud provider services (IAM, logging, monitoring)

✅ High availability and auto-scaling out of the box

Cons:

❌ Vendor Lock-in: API definitions, policies, and configurations are tied to the cloud provider

❌ No Customization: Cloud API gateways are closed-source, limiting the ability to add custom plugins or functionality

❌ No Hybrid Cloud Support: Cloud-managed API gateways cannot be deployed on-premise or across multi-cloud environments

Use Case:

Cloud API gateways are ideal for startups and small teams that need a quick, managed solution without worrying about infrastructure maintenance. However, as API traffic grows or hybrid cloud requirements arise, their limitations become apparent.

🔹 Example: Amazon API Gateway is a popular choice for cloud-native applications but lacks flexibility for on-premises deployments.

Open Source API Gateways: Control and Flexibility

Pros:

✅ Fully customizable, allowing teams to extend functionality as needed

✅ No licensing fees, reducing costs in the long run

✅ Can be deployed anywhere—on-prem, multi-cloud, hybrid cloud

Cons:

❌ Operational Overhead: Requires self-hosting, upgrades, and security management

❌ Governance Risks: Some open-source projects are controlled by a single vendor, which may later change licensing terms (e.g., Redis, ELK Stack)

❌ Scalability Complexity: Running an open-source API gateway at enterprise scale requires expertise in deployment and maintenance

Key Consideration: Choosing a Foundation-Owned Open Source Project

Some open-source API gateways are vendor-controlled, meaning the company behind them can change licensing terms. For example, Redis and Elasticsearch modified their open-source licenses to prevent cloud providers from offering managed services.

To avoid future licensing risks, it's safer to choose an API gateway governed by a neutral open-source foundation, such as:

Commercial API Gateways: Enterprise Features with Pricing Risks

Pros:

✅ Enterprise-grade security

✅ SLA-backed support with 24/7 assistance

✅ Monetization and API analytics for businesses offering APIs as a service

Cons:

❌ High Licensing Costs: Typically charged per API call, which can become expensive

❌ Potential Pricing Changes: Many vendors modify pricing models over time, significantly increasing costs (e.g., Apigee, Kong Enterprise)

❌ Limited Deployment Flexibility: Some solutions require vendor-managed infrastructure, reducing control

Use Case:

Best suited for large enterprises with strict security, compliance, and SLA requirements. However, pricing unpredictability is a concern—many companies have faced sudden cost increases.

Strategic Recommendation: A Hybrid Approach for Growth

If your API traffic is growing rapidly and hybrid cloud is part of your strategy, the best approach is:

1. Start with an Open Source API Gateway

Avoid vendor lock-in
Maintain control over deployment and customization
Lower costs by eliminating per-call fees

2. Upgrade to a Commercial Version When Needed

When security requirements increase
When managing multi-cluster deployments
When enterprise support and SLAs become necessary

By following this approach, you get the best of both worlds: flexibility, cost control, and enterprise-grade features when needed.

🔹 Example Strategy: A company starts with Apache APISIX (open source) and later upgrades to API7 Enterprise when requiring advanced security and SLA support.

FAQ: Common Questions About API Gateway Selection

1. Why not start with a cloud API gateway and switch later?

Switching API gateways is complex due to differences in configurations, rate-limiting rules, authentication methods, and monitoring setups. If hybrid cloud or multi-cloud is part of your strategy, starting with an open-source gateway ensures long-term flexibility.

2. How do I avoid open-source licensing risks?

Choose projects governed by neutral software foundations (e.g., Apache Software Foundation, CNCF). Avoid projects fully controlled by a single company, as they may change their licensing model.

3. What is the best API gateway for hybrid cloud?

Open-source gateways like Apache APISIX and Envoy Proxy offer full deployment flexibility, making them ideal for hybrid and multi-cloud architectures.

Conclusion: Making the Right Choice

Choosing the right API gateway depends on your scalability, budget, and cloud strategy:

Criteria	Cloud API Gateway	Open Source API Gateway	Commercial API Gateway
Cost	High (Pay per API call)	Low (Free)	High (License fees)
Customization	Limited	Full control	Limited
Deployment Flexibility	Cloud-only	Anywhere (Hybrid, Multi-cloud)	Varies
Enterprise Features	Basic	Requires customization	Advanced (Security, Compliance)
Support	Cloud provider support	Community-driven	SLA-backed

If API traffic is growing rapidly and hybrid cloud adoption is planned, a hybrid approach—starting with open source and upgrading to a commercial enterprise solution when needed—offers the best flexibility and cost efficiency.

Announcing Integration between Apache APISIX and open-appsec WAF

Yilia — Wed, 23 Oct 2024 02:56:13 +0000

Introduction

open-appsec WAF is excited to announce a new integration with the open-source API gateway Apache APISIX.

This new collaboration between the open-appsec and API7 teams now allows users to protect their web APIs and other web services exposed by Apache APISIX against unknown and known attack types effectively based on open-appsec's advanced machine-learning-based technology and also adds several more enhanced security capabilities.

About Apache APISIX

Apache APISIX is a modern, flexible, and high-performance open-source API gateway solution designed to handle various use cases in microservices and cloud-native architectures. Its primary purpose is to facilitate API management by serving as a gateway for managing, securing, and optimizing API traffic between clients and backend services.

Further use cases for APISIX as an API gateway include load balancing, rate limiting, authentication, and authorization. It provides comprehensive features such as traffic control, dynamic upstream, and plugin extensibility, enabling developers to customize and extend functionality according to their specific needs.

About open-appsec WAF

open-appsec WAF provides automatic, preemptive threat prevention and integrates with various types of reverse proxies like NGINX as well as API gateways like APISIX. It is machine-learning-based, meaning it doesn't require signatures (or updates) at all. This enables it to provide automatic, state-of-the-art threat prevention even for true zero-day attacks while significantly reducing both administrative effort and the amount of false positives.

In addition, open-appsec provides many additional security layers such as AntiBot, rate limiting, schema enforcement, snort signature support, custom rules/exceptions, and more. open-appsec can be managed centrally using a Web UI provided as a SaaS service and also locally using a declarative configuration file.

Website: www.openappsec.io
Github: github.com/openappsec
Docs: docs.openappsec.io
Playgrounds: www.openappsec.io/playground

Integrating Apache APISIX with open-appsec

With this new integration, APISIX users will now have access to open-appsec WAF as an integrated, state-of-the-art machine-learning-based WAF solution for the protection of their web APIs and web applications.

They can now use e.g. open-appsec's free and open-source "Community Edition" to get effective, AI-based protection against known but also unknown attacks for everything exposed by their APISIX API gateway, while at the same time reducing the amount of false positives significantly unburdening the administrator from tedious tasks such as creating exceptions, updating traditional signature-based policies and more.

This integration will be available for all common platforms: Linux, Docker, and Kubernetes.

Linux

For Linux "embedded" deployments of APISIX, an open-appsec installer will add an "open-appsec attachment" module to the existing APISIX installation and also install the "open-appsec agent" alongside it, which will receive the traffic from the attachment, inspect it, and return the concluded action to block or allow the traffic back to the APISIX respectively the open-appsec attachment integrated with it.

Here's a simple architecture schematic for Linux deployment.

Docker

For Docker-based deployments of APISIX with open-appsec WAF, there is a special APISIX container image available, to which the open-appsec attachment was already added and also an enhanced docker-compose file, which deploys both, the APISIX gateway container as well as an open-appsec Agent that does the security inspection and returns the concluded decisions to the APISIX gateway to allow or block traffic.

Here's a simple architecture schematic for deployment on Docker.

Kubernetes

For Kubernetes based-deployments of APISIX integrated with open-appsec, there's a Helm chart available, which is based on the official APISIX Helm chart and further enhanced to also include the open-appsec attachment in the APISIX gateway container and also deploys the open-appsec agent. Further, you will have the option to configure open-appsec in a declarative "DevOps-style" way using custom resources in K8s as an alternative to using the open-appsec central management Web UI.

Here's a simple architecture schematic for deployment on Kubernetes.

Adding open-appsec WAF to APISIX on Linux

To install open-appsec on a Linux system with APISIX installed, please follow these steps:

1. Prerequisites

The Linux platform must be Ubuntu 22.04.
Make sure to have APISIX installed.

You can find the list of supported APISIX versions here: https://downloads.openappsec.io/packages/supported-apisix.txt.

If you don't have APISIX installed yet, you can use the following commands to perform an APISIX installation in "traditional mode". By running these commands you will first install the etcd database for APISIX, then add the required repos before installing and starting APISIX.

Install etcd Database

ETCD_VERSION='3.5.4'
wget https://github.com/etcd-io/etcd/releases/download/v${ETCD_VERSION}/etcd-v${ETCD_VERSION}-linux-amd64.tar.gz
tar -xvf etcd-v${ETCD_VERSION}-linux-amd64.tar.gz && cd etcd-v${ETCD_VERSION}-linux-amd64
cp -a etcd etcdctl /usr/bin/
nohup etcd >/tmp/etcd.log 2>&1 &
etcd

Add and Update Package Repositories

apt install gnupg
echo "deb http://openresty.org/package/debian bullseye openresty" | tee /etc/apt/sources.list.d/openresty.list
wget -O - https://openresty.org/package/pubkey.gpg | apt-key add -
wget -O - http://repos.apiseven.com/pubkey.gpg | apt-key add -
echo "deb http://repos.apiseven.com/packages/debian bullseye main" | tee /etc/apt/sources.list.d/apisix.list
apt update

Install, Initiate, and Start APISIX

apt install apisix=3.9.1-0
apisix init
apisix start

2. Download the open-appsec Installer

wget https://downloads.openappsec.io/open-appsec-install && chmod +x open-appsec-install

3. Install open-appsec

Install open-appsec to integrate with the existing APISIX installation.

Note that the --prevent flag will install open-appsec with a default policy already set to prevent mode.

./open-appsec-install --auto --prevent

4. Get and Store APISIX Admin Key

Get the APISIX admin key from the APISIX config.yaml configuration file and store it in the APISIX_KEY env variable.

export APISIX_KEY=$(awk '/key:/{ if ($2 ~ /^edd1/) print $2 }' /usr/local/apisix/conf/config.yaml )

5. Configure Route to Expose Services

Configure an example route in the APISIX gateway to expose an external web service or web API. In this example, we use httpbin.org as the example backend.

curl http://127.0.0.1:9180/apisix/admin/routes/100 -H "X-API-KEY:$APISIX_KEY" -X PUT -d '{
    "methods": [
      "GET"
    ],
    "uri": "/anything",
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

6. Validate

Let's see if this route works by accessing it.

curl -s -v -G --data-urlencode email=user@domain.abc http://localhost:9080/anything

7. Simulate an SQL Injection Attack

Now let's try to simulate an SQL injection attack (see 'OR '1'='1' in the below HTTP request) against the httpbin.org service exposed by the APISIX gateway which is now protected by the open-appsec WAF.

curl -s -v -G --data-urlencode email=user@domain.abc' OR '1'='1 http://localhost:9080/anything

This simulated attack now gets blocked successfully by open-appsec's contextual machine-learning WAF engine.

8. Review Log Files

Check out the corresponding log files showing the "prevent" for the HTTP request with the simulated attack that we just sent.

tail -f /var/log/nano_agent/cp-nano-http-transaction-handler.log*| grep -i user@domain.abc

Alternatively you can use the open-appsec-ctl tool:

open-appsec-ctl --view-logs | grep -i user@domain.abc

9. Connect to open-appsec for Central Management (Optional)

Optionally you can connect your deployment now to https://my.openappsec.io for centrally managing open-appsec with an easy-to-use Web UI, monitoring security events and more, see section How to Manage Your open-appsec WAF Deployment Centrally? further below for more information.

Congratulations! You successfully added open-appsec WAF to your existing APISIX installation and verified that your web services exposed by the APISIX gateway are now protected against web attacks.

Deploying APISIX with open-appsec WAF on Containerized Platforms (Docker)

To install APISIX integrated with open-appsec on Docker, you can follow the steps shown below.

Opposite to the above example here we are deploying APISIX in "standalone mode", which means it's declaratively configured using a docker volume mount with a yaml file that holds the configurations and therefore won't require an etcd database deployment.

Note that APISIX supports both, traditional as well as standalone modes in all deployment types (Linux, Docker, …)

1. Prerequisite

Make sure to have a Linux platform with both Docker and docker-compose tools installed.

2. Create a Folder for open-appsec

Within the directory that you want to use for the deployment, create a folder appsec-localconfig which will hold the appsec declarative configuration file:

mkdir ./appsec-localconfig

3. Download the open-appsec File into the Folder

Download the initial declarative configuration file for open-appsec into that folder.

wget https://raw.githubusercontent.com/openappsec/openappsec/main/config/linux/latest/prevent/local_policy.yaml -O appsec-localconfig/local_policy.yaml

Note that this example declarative configuration file is already set to prevent attacks.

4. Create a Folder for APISIX

Create another folder apisix-localconfig which will hold the declarative configuration file for APISIX: mkdir ./apisix-localconfig.

5. Download APISIX File into the Folder

Let's download a simple declarative configuration file also for APISIX so we can verify open-appsec protection after the deployment.

wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/apisix/apisix-example-config/apisix-standalone.yaml -O ./apisix-localconfig/apisix-standalone.yaml

6. Create a `docer-compose.yaml` File

Create a docker-compose.yaml file with the content below, which can be downloaded as follows:

wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/apisix/docker-compose.yaml

version: "3"

services:
  apisix:
    container_name: apisix
    image: "ghcr.io/openappsec/apisix-attachment:latest"
    ipc: service:appsec-agent
    restart: always
    volumes:
      - ./apisix-localconfig/apisix-standalone.yaml:/usr/local/apisix/conf/apisix.yaml:ro
    environment:
      - APISIX_STAND_ALONE=true
    ports:
      - "9180:9180/tcp"
      - "9080:9080/tcp"
      - "9091:9091/tcp"
      - "9443:9443/tcp"

  appsec-agent:
    container_name: appsec-agent
    image: 'ghcr.io/openappsec/agent:latest'
    ipc: shareable
    restart: unless-stopped
    environment:
      # adjust with your own email below
      - user_email=user@email.com
      - registered_server="APISIX Server"
    volumes:
      - ./appsec-config:/etc/cp/conf
      - ./appsec-data:/etc/cp/data
      - ./appsec-logs:/var/log/nano_agent
      - ./appsec-localconfig:/ext/appsec
    command: /cp-nano-agent

7. Update Your Email Address (Optional)

Edit the docker-compose.yaml file and replace "user@email.com" with your own email address, so we can provide assistance in case of any issues with the specific deployment in the future and provide information proactively regarding open-appsec.

This is an optional parameter and can be removed. If we send automatic emails, there will also be an opt-out option included for receiving similar communication in the future.

8. Start All Containers

Run docker-compose up to start the deployment of all relevant containers:

docker-compose up -d

9. Check Container Status

Check if the apisix-attachment and the appsec-agent containers are up and running.

docker ps

10. Validate the Standalone Configuration

Let's see if the standalone configuration works by accessing it.

curl -s -v -G --data-urlencode email=user@domain.abc http://localhost:9080/anything

11. Simulate an SQL Injection Attack

Now let's try to simulate an SQL injection attack against the httpin.org service exposed by the APISIX gateway container which is now protected by open-appsec.

curl -s -v -G --data-urlencode email=user@domain.abc' OR '1'='1 http://localhost:9080/anything

12. Connect to open-appsec for Central Management (Optional)

Optionally you can connect your deployment now to https://my.openappsec.io for centrally managing open-appsec with an easy-to-use Web UI, monitoring security events, and more, see section How to Manage Your open-appsec WAF Deployment Centrally? further below for more information.

Deploying APISIX with open-appsec WAF on Kubernetes Using Helm

1. Prerequisite

Make sure the Kubernetes platform and Helm tool are available.

2. Download open-appsec

Download the open-appsec for the APISIX Helm chart here.

wget https://downloads.openappsec.io/packages/helm-charts/apisix/open-appsec-k8s-apisix-latest.tgz

3. Install Helm Chart

This example helm install command is installing the open-appsec for the APISIX Helm chart which is based on an extended version of the official APISIX Helm chart.

It will deploy the APISIX gateway as the APISIX Ingress Controller, as well as open-appsec WAF integrated with it. It also offers an additional configuration option specifically for open-appsec WAF (see values.yaml inside the Helm chart and open-appsec docs).

After deployment, you can assign your K8s ingress resources to the APISIX gateway by configuring them to use the following ingress class: appsec-apisix.

helm install open-appsec-k8s-apisix-latest.tgz \
--name-template=appsec-apisix \
--set rbac.create=true \
--set appsec.mode=standalone \
--set service.type=LoadBalancer \
--set appsec.persistence.enabled=false \
--set ingress-controller.enabled=true \
--set ingress-controller.config.ingressClass=appsec-apisix \
--set appsec.userEmail=”<your-email-address>” \
--set appsec.agentToken= \
--create-namespace \
-n appsec-apisix

Replace in the Helm install command above with your own email address, so we can send you news and updates related to open-appsec and better support you with your deployment if needed! You can unsubscribe at any time or alternatively, just remove that line if you prefer not to provide your email.

4. Validate

Validate that pods were properly deployed and are in a ready state:

kubectl get pods -n appsec-apisix

5. Create an open-appsec Policy Resource

Run the following command to create the "open-appsec-best-practice-policy" in K8s.

Note that this example policy is already pre-configured to prevent attacks.

kubectl apply -f https://raw.githubusercontent.com/openappsec/openappsec/main/config/k8s/v1beta1/open-appsec-k8s-prevent-config-v1beta1.yaml

You can also create your own custom policy, here you find all the details.

6. Fetch the Target Resource Name

Find out the name of the relevant ingress resource which you want to protect:

kubectl get ing -A

7. Edit the Ingress Resource

kubectl edit ing/<ingress name> -n <ingress namespace>

8. Change the ingressClassname

Change the ingressClassname to use open-appsec:

spec: ingressClassName: appsec-apisix

9. Add Annotation to the Ingress Resource

Add this annotation to the ingress resource to activate open-appsec for this ingress by specifying the desired open-appsec policy custom resource.

openappsec.io/policy: open-appsec-best-practice-policy

10. Validate the Standalone Configuration

Let's see if the standalone configuration works by accessing it:

curl -s -v -G --data-urlencode email=user@domain.abc http://[YOUR-INGRESS-HOSTNAME]

11. Simulate an SQL Injection Attack

Now let's try to simulate an SQL injection attack against the httpin.org service exposed by the APISIX gateway container which is now protected by open-appsec.

curl -s -v -G --data-urlencode email=user@domain.abc' OR '1'='1 http://[YOUR-INGRESS-HOSTNAME]

Replace [YOUR-INGRESS-HOSTNAME] in the command above with the hostname you set in the ingress resource which you protected with open-appsec WAF in the earlier steps, also change "http" to "https" if required.

Congratulations! You successfully deployed APISIX integrated with open-appsec WAF and verified that your web services exposed by the APISIX Gateway are now protected against attacks.

How to Manage Your open-appsec WAF Deployment Centrally?

If you like you can also manage your open-appsec WAF deployment (integrated with APISIX) centrally using the open-appsec Web UI (SaaS Service) available at https://my.openappsec.io, by connecting the open-appsec agent to a deployment profile in the central Web UI.

You can alternatively continue to manage your deployment locally but still connect to a central WebUI profile in "Declarative mode" so that you will be able to see the local configuration (read-only) in the Web UI.

Alongside the configuration of open-appsec the Web UI allows you to also see much more information like the status of deployed open-appsec agents, security logs, dashboards and more.

For instructions on how to connect your deployment to the central Web UI see the open-appsec docs available at https://docs.openappsec.io.

Below you find some screenshots of the Web UI.

Summary

In this blog we explained how open-appsec can integrate with Apache APISIX on all of the following: regular Linux-based deployments, containerized deployments (Docker), and also Kubernetes environments.

Following the deployment steps for APISIX with open-appsec WAF, we simulated SQL injection attacks, which were effectively prevented by open-appsec's machine learning-based WAF technology.

Additionally, it was explained, what the benefits are of connecting to open-appsec central WebUI for managing, monitoring, log analysis, and reporting.

We hope these new integrations will prove very useful to enhance the security of your APISIX API gateway and its exposed web APIs and web applications with open-appsec machine learning-based WAF.

Welcome you to contact us if you have any feedback, or questions or might face some technical challenge that you want us to assist with. You can reach the open-appsec team via the chat on https://www.openappsec.io or via email to: info@openappsec.io.

Additional Resources

open-appsec WAF

Website: https://www.openappsec.io

GitHub: https://github.com/openappsec

Docs: https://docs.openappsec.io

Playgrounds: https://www.openappsec.io/playground

Blogs: https://www.openappsec.io/blogs

Apache APISIX

Website: https://apisix.apache.org

GitHub: https://github.com/apache/apisix

Docs: https://apisix.apache.org/docs/

Blogs: https://apisix.apache.org/blog/

DRY your Apache APISIX config

Nicolas Fränkel — Thu, 05 Sep 2024 09:02:00 +0000

DRY is an important principle in software development. This post will show you how to apply it to Apache APISIX configuration.

The DRY principle

"Don't repeat yourself" (DRY) is a principle of software development aimed at reducing repetition of information which is likely to change, replacing it with abstractions that are less likely to change, or using data normalization which avoids redundancy in the first place.

-- Wikipedia - Don't repeat yourself

The main idea behind DRY is that if you repeat yourself and the information changes, then you must update the changed information in multiple places. It's not only extra effort; there's a chance you'll forget about it and have different information in different places. DRY shines in bug fixing.

Imagine a code snippet containing a bug. Imagine now that you have duplicated the snippet in two different places. Now, you must fix the bug in these two places, and that's the easy part; the hard being to know about the duplication in the first place. There's a high chance that the person duplicating and the one fixing are different. If the snippet had been refactored to be shareable and called from the two places instead, you only need to fix the bug in this one place.

Most people associate DRY with code. However, it could be more limiting and contrary to the original idea.

The principle has been formulated by Andy Hunt and Dave Thomas in their book The Pragmatic Programmer. They apply it quite broadly to include database schemas, test plans, the build system, even documentation.

-- Wikipedia - Don't repeat yourself

Sound configuration systems allow DRY or even encourage it.

DRY in Apache APISIX

Apache APISIX offers DRY configuration in two places.

DRY upstreams

In an e-commerce context, your beginner journey to define a route on Apache APISIX probably starts like the following:

routes:
  - id: 1
    name: Catalog
    uri: /products*
    upstream:
      nodes:
        "catalog:8080": 1

If you're familiar with APISIX, we defined a route to the catalogue under the /products URI. However, there's an issue: you probably want would-be customers to browse the catalogue but want to prevent people from creating, deleting, or updating products. Yet, the route matches every HTTP method by default.

We should allow only authenticated users to manage the catalogue so everybody can freely browse it. To implement this approach, we need to split the route in two:

routes:
  - id: 1
    name: Read the catalogue
    methods: [ "GET", "HEAD" ]                         #1
    uri: /products*
    upstream:                                          #2
      nodes:
        "catalog:8080": 1
  - id: 1
    name: Read the catalogue
    methods: [ "PUT", "POST", "PATCH", "DELETE" ]      #3
    uri: /products*
    plugins:
      key-auth: ~                                      #4
    upstream:                                          #2
      nodes:
        "catalog:8080": 1

Match browsing
Duplicated upstream!
Match managing
Only authenticated consumers can use this route; key-auth is the simplest plugin for this

We fixed the security issue in the simplest way possible: by copy-pasting. By doing so, we duplicated the upstream section. If we need to change the topology, e.g., by adding or removing nodes, we must do it in two places. It defeats the DRY principle.

In real-world scenarios, especially when they involve containers, you wouldn't implement the upstream by listing nodes. You should instead implement a dynamic service discovery to accommodate topology changes. However, the point still stands when you need to change the service discovery configuration or implementation. Hence, my point applies equally to nodes and service discovery.

Along with the Route abstraction, APISIX offers an Upstream abstraction to implement DRY. We can rewrite the above snippet like this:

upstreams:
  - id: 1                                              #1
    name: Catalog
    nodes:
      "catalog:8080": 1

routes:
  - id: 1
    name: Read the catalogue
    methods: [ "GET", "HEAD" ]
    uri: /products*
    upstream_id: 1                                     #2
  - id: 1
    name: Read the catalogue
    methods: [ "PUT", "POST", "PATCH", "DELETE" ]
    uri: /products*
    upstream_id: 1                                     #2
    plugins:
      key-auth: ~

Define an upstream with ID 1
Reference it in the route

If anything happens in the topology, we must update the change only in the single Upstream.

Note that defining the upstream embedded and referencing it with upstream_id are mutually exclusive.

DRY plugin configuration

Another area where APISIX can help you DRY your configuration with the Plugin abstraction. APISIX implements most features, if not all, through plugins

Let's implement path-based versioning on our API. We need to rewrite the URL before we forward it.

routes:
  - id: 1
    name: Read the catalogue
    methods: [ "GET", "HEAD" ]
    uri: /v1/products*
    upstream_id: 1
    plugins:
      proxy-rewrite:
        regex_uri: [ "/v1(.*)", "$1" ]                 #1
  - id: 1
    name: Read the catalogue
    methods: [ "PUT", "POST", "PATCH", "DELETE" ]
    uri: /v1/products*
    upstream_id: 1
    plugins:
      proxy-rewrite:
        regex_uri: [ "/v1(.*)", "$1" ]                 #1

Remove the /v1 prefix before forwarding

Like with upstream above, the plugins section is duplicated. We can also factor the plugin configuration in a dedicated Plugin Config object. The following snippet has the same effect as the one above:

plugin_configs:
  - id: 1                                              #1
    plugins:
      proxy-rewrite:
        regex_uri: [ "/v1(.*)", "$1" ]

routes:
  - id: 1
    name: Read the catalogue
    methods: [ "GET", "HEAD" ]
    uri: /v1/products*
    upstream_id: 1
    plugin_config_id: 1                                #2
  - id: 1
    name: Read the catalogue
    methods: [ "PUT", "POST", "PATCH", "DELETE" ]
    uri: /v1/products*
    upstream_id: 1
    plugin_config_id: 1                                #2

Factor the plugin configuration in a dedicated object
Reference it

Astute readers might have noticed that I'm missing part of the configuration: the auth-key mysteriously disappeared! Indeed, I removed it for the sake of clarity.

Unlike upstream and upstream_id, plugins and plugin_config_id are not mutually exclusive. We can fix the issue by just adding the missing plugin:

routes:
  - id: 1
    name: Read the catalogue
    methods: [ "GET", "HEAD" ]
    uri: /v1/products*
    upstream_id: 1
    plugin_config_id: 1
  - id: 1
    name: Read the catalogue
    methods: [ "PUT", "POST", "PATCH", "DELETE" ]
    uri: /v1/products*
    upstream_id: 1
    plugin_config_id: 1
    plugins:
      key-auth: ~                                      #1

Fix it!

This way, you can move the shared configuration to a plugin_config object and keep a specific one to the place it applies to. But what if the same plugin with different configurations is used in the plugin_config and directly in the route? The documentation is pretty clear about it:

Consumer > Consumer Group > Route > Plugin Config > Service

In short, the plugin configuration in a route overrules the configuration in the plugin_config_id. It also allows us to provide the apikey variable for the key-auth plugin in a consumer and only set it in a route. APISIX will find and use the key for each consumer!

Conclusion

DRY is not only about code; it's about data management in general. Configuration is data and thus falls under this general umbrella.

APISIX offers two DRY options: one for upstream - upstream_id, and one for plugin - plugin_config_id. Upstreams are exclusive; plugins allow for overruling.

Both mechanisms should help you toward DRYing your configuration and make it more maintainable in the long run.

To go further:

Originally published at A Java Geek on September 1^st, 2024

When (not) to write an Apache APISIX plugin

Nicolas Fränkel — Thu, 29 Aug 2024 09:02:00 +0000

When I introduce Apache APISIX in my talks, I mention the massive number of existing plugins, and that each of them implements a specific feature. One of the key features of Apache APISIX is its flexibility. If a feature is missing, you can create your own plugin in Lua or a language compiled into Wasm, showcasing the platform's adaptability to your specific needs. In this post, I aim to provide practical alternatives to writing a custom plugin, offering solutions you can quickly implement in your projects.

Cons of writing a plugin

Before describing alternatives, let me explain the issues of writing a plugin.

The biggest argument against writing a plugin is quite generic. You write code: suddenly, you need to take care of it. It includes fixing bugs, updating dependencies, keeping the code synchronized with APISIX's latest version, etc.

As I mentioned above, APISIX comes with a list of out-of-the-box plugins. A huge majority of them are enabled in the default configuration. However, if you want to add a plugin to the list, you must add all required plugins individually, as your configuration replaces the default one; this is the case with a custom plugin.

Custom plugins require you to configure APISIX with the path to the plugin(s) folder:

apisix:
  extra_lua_path: /opt/?.lua

Moreover, some plugins may require additional configuration. For example, in my previous version of Evolving your APIs, I set a custom nginx snippet to add a Lua shared dictionary to use it in the code's plugin:

nginx_config:
  http:
    custom_lua_shared_dict:
      plugin-unauth-limit: 100m

Finally, writing a custom plugin requires a fairly advanced understanding of Apache APISIX and its inner workings. This knowledge is a good idea, but it's not great to make it a requirement.

The `vars` and `filter_func` parameters

In my earlier blog post Free tier API with Apache APISIX, I implemented an API-free tier with the help of the vars parameter. As a reminder, vars is an additional matching condition on your route besides the usual ones: URI, HTTP method, and host.

In the mentioned post, I used vars to add a match on an HTTP header.

routes:
  - uri: /get
    upstream_id: 1
    vars: [[ "http_apikey", "~~", ".*"]]                      #1

Match only if the request has an HTTP header named apikey

However, the vars parameter has its limitations, particularly in its support of a limited range of operators, which may restrict its use in more complex scenarios. Here it is for convenience:

Operator	Description	Example
`==`	equal	`["arg_version", "==", "v2"]`
`~=`	not equal	`["arg_version", "~=", "v2"]`
`>`	greater than	`["arg_ttl", ">", 3600]`
`>=`	greater than or equal to	`["arg_ttl", ">=", 3600]`
`<`	less than	`["arg_ttl", "<", 3600]`
`⇐`	less than or equal to	`["arg_ttl", "⇐", 3600]`
`~~`	match RegEx	`["arg_env", "~~", "[Dd]ev"]`
`~*`	match RegEx (case-insensitive)	`["arg_env", "~~", "dev"]`
`in`	exist on the right-hand side	`["arg_version", "in", ["v1","v2"]]`
`has`	contain item on the right-hand side	`["graphql_root_fields", "has", "owner"]`
`!`	reverse the adjacent operator	`["arg_env", "!", "~~", "[Dd]ev"]`
`ipmatch`	match an IP address	`["remote_addr", "ipmatch", ["192.168.102.40", "192.168.3.0/24"]]`

Note that the DSL also supports boolean operators.

Imagine that the need goes beyond what we can express with the DSL. It's time to break our bounds and leverage the full power of Lua.

With filter_func, we can write a dedicated Lua function:

It accepts a vars arg, allowing you to access APISIX built-in variables, including nginx variables, e.g., HTTP headers.
It must return a boolean value. As for vars, APSIX uses the value to decide whether the route matches or not.

The `serverless` plugin

The serverless plugin actually consists of two plugins: serverless-pre-function and serverless-pre-function. As their name implies, the former executes before any other plugin in that phase and the latter after any other plugin in that phase. Note that it's because of their respective default priority. While it's technically possible to override the priority, common sense should prevent you from ever thinking about doing so.

With serverless, you configure two parameters:

The phase in which APISIX executes it
A sequential array of Lua functions

A widespread use case with serverless is to log input and output data.

routes:
  - uri: /get
    upstream_id: 1
    plugins:
      serverless-pre-function:
        phase: rewrite                                             #1
        functions:
          - >
            return function(conf, ctx)
              local core = require("apisix.core")
              core.log.warn("conf: ", core.json.encode(conf))      #2
              core.log.warn("ctx : ", core.json.encode(ctx, true)) #3
            end
      serverless-post-function:
        phase: log                                                 #4
        functions:
          - >
            return function(conf, ctx)
              local core = require("apisix.core")
              core.log.warn("ctx : ", core.json.encode(ctx, true)) #5
            end

Execute at the start of the rewrite phase
Serialize the configuration to JSON and write it in the log. We use the warn level because it's the default one
Serialize the context to JSON and write it in the log
Execute at the start of the log phase
Serialize the context to JSON and write it in the log again. The context will probably have changed between the two phases

The APISIX model only allows a unique plugin per route. It's a limitation of this approach: while you can have multiple functions per phase, you can't span more than two phases, one for pre and one for post.

The `script` parameter

I must admit that I learned about script when researching for this post. With script, you can write Lua code directly in your config without needing a full-fledged plugin! script comes with a huge limitation, though: it's exclusive with plugins.

Scripts and Plugins are mutually exclusive, and a Script is executed before a Plugin. This means that after configuring a Script, the Plugin configured on the Route will not be executed.

I believe that, at this point, you'd better write a plugin instead.

The `_meta.filter` parameter

So far, our scope has been the route (or the service if you prefer the latter). However, an alternative is to execute a plugin conditionally. For example, imagine a route configured with the limit-count plugin to rate limit the number of requests. We want to test the infrastructure in a stress test. Instead of creating our own plugin, we can bypass the plugin if a specific header is present.

The filter syntax is the same as the vars syntax.

routes:
  - uri: /get
    upstream_id: 1
    plugins:
      limit-count:                                                              #1
        count: 1
        time_window: 60
        rejected_code: 429
        _meta:
          filter: [["http_Secret-Header", "~=", "MySuperDuperSecretBypassKey"]] #2

Configure the limit-count plugin
Execute it only if the HTTP header has a different value

Summary

Writing a custom plugin entails lots of downsides. I showed a couple of other alternatives in this post:

Alternative	Scope	Feature	Comments
`vars`	`route`	Additional criterion to match a route	Simple DSL with a couple of comparison operators and boolean operators
`filter_func`	`route`	Additional criterion to match a route	Full-fledged Lua function Can access APISIX and nginx variables No access to the context
`script`	`route`	Everything a plugin can do	Exclusive with `plugins` Full access to the context
`_meta.filter`	`plugin`	Execute a plugin conditionally	Simple DSL with a couple of comparison operators and boolean operators

Before writing a plugin, I suggest you design your feature using one of the above alternatives (but script).

To go further:

Originally published at A Java Geek on August 25^th, 2024

Free tier API with Apache APISIX

Nicolas Fränkel — Thu, 01 Aug 2024 09:02:00 +0000

Lots of service providers offer a free tier of their service. The idea is to let you kick their service's tires freely. If you need to go above the free tier at any point, you'll likely stay on the service and pay. In this day and age, most services are online and accessible via an API. Today, we will implement a free tier with Apache APISIX.

A naive approach

I implemented a free tier in my post Evolving your RESTful APIs, a step-by-step approach, albeit in a very naive way. I copy-pasted the limit-count plugin and added my required logic.

function _M.access(conf, ctx)
    core.log.info("ver: ", ctx.conf_version)

    -- no limit if the request is authenticated
    local key = core.request.header(ctx, conf.header)                               #1
    if key then
        local consumer_conf = consumer_mod.plugin("key-auth")                       #2
        if consumer_conf then
            local consumers = lrucache("consumers_key", consumer_conf.conf_version, #3
                    create_consume_cache, consumer_conf)
            local consumer = consumers[key]                                         #4
            if consumer then                                                        #5
                return
            end
        end
    end
-- rest of the logic

Get the configured request header value
Get the consumer's key-auth configuration
Get consumers
Get the consumer with the passed API key if they exist
If they exist, bypass the rate limiting logic

The downside of this approach is that the code is now my own. It has evolved since I copied it, and I'm stuck with the version I copied. We can do better, with the help of the vars parameter on routes.

APISIX route matching

APISIX delegates its matching rule to a router.
Standard matching parameters include:

The URI
The HTTP method. By default, all methods match.
The host
The remote address

Most users do match on the URI; a small minority use HTTP methods and the host. However, they are not the only matching parameters. Knowing the rest will bring you into the world of advanced users of APISIX.

Let's take a simple example, header-based API versioning. You'd need actually to match a specific HTTP request header. I've already described how to do it previously. In essence, vars is an additional matching criterion that allows the evaluation of APISIX and nginx variables.

routes:
  - uri: /*
    upstream_id: 1
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v1+json" ]]

The above route will match if, and only if, the HTTP Accept header is equal to vnd.ch.frankel.myservice.v1+json. You can find the complete list of supported operators in the lua-resty-expr project.

APISIX matches routes in a non-specified order by default. If URIs are disjointed, that's not an issue.

routes:
  - uri: /*
    upstream_id: 1
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v1+json" ]]
  - uri: /*
    upstream_id: 2
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v2+json" ]]

Problems arise when URIs are somehow not disjointed. For example, imagine I want to set a default route for unversioned calls.

routes:
  - uri: /*
    upstream_id: 1
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v1+json" ]]
  - uri: /*
    upstream_id: 2
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v2+json" ]]
  - uri: /*
    upstream_id: 1

We need the third route to be evaluated last. If it's evaluated first, it will match all requests, regardless of their HTTP headers. APISIX offers the priority parameter to order route evaluation. By default, a route's priority is 0. Let's use priority to implement the versioning correctly:

routes:
  - uri: /*
    upstream_id: 1
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v1+json" ]]
    priority: 10                                              #1
  - uri: /*
    upstream_id: 2
    vars: [[ "http_accept", "==", "vnd.ch.frankel.myservice.v2+json" ]]
    priority: 10                                              #1
  - uri: /*
    upstream_id: 1

Evaluated first. The order is not relevant since the URIs are disjointed.

Implementing free tier with matching rules

We are now ready to implement our free tier with matching rules.

The first route to be evaluated should be the one with authentication and no rate limit:

routes:
  - uri: /get
    upstream_id: 1
    vars: [[ "http_apikey", "~~", ".*"]]                      #1
    plugins:
      key-auth: ~                                             #2
    priority: 10                                              #3

Match if the request has an HTTP header named apikey
Authenticate the request
Evaluate first

The other route is evaluated afterward.

  - uri: /get
    upstream_id: 1
    plugins:
      limit-count:                                            #1
        count: 1
        time_window: 60
        rejected_code: 429

Rate limit this route

When you configure APISIX with the above snippets, and it receives a request to /get, it tries to match the first route only if it has an apikey request header:

If it has one:
- The key-auth plugin kicks in
- If it succeeds, APISIX forwars the request to the upstream
- If it fails, APISIX returns a 403 HTTP status code
If it has no such request header, it matches the second route with a rate limit.

Conclusion

A free tier is a must for any API service provider worth its salt. In this post, I've explained how to configure such free tier with Apache APISIX.

The complete source code for this post can be found on GitHub:

ajavageek / free-tier-apisix

To go further:

Originally published at A Java Geek on July 28^th, 2024

DEV Community: Apache APISIX

Release Apache APISIX Ingress Controller 2.0

Highlights of Apache APISIX Ingress Controller 2.0

Support Gateway API

Introduce Gateway API Extensions

Support APISIX Standalone API-Driven Mode

Support Multi-Data-Plane Deployment Mode

Admin API Mode

Standalone Mode

Conclusion

Load Balancing AI/ML API with Apache APISIX

Overview

Quickstart Installation

Secure the Admin API

1. Set an Admin Key

2. Whitelist Management IPs (allow_admin)

3. Restart APISIX

Define Basic Routes for v1 and v2

1. Route for v1

2. Route for v2

Implement Traffic Splitting (50/50)

Verify Split Endpoint Functionality

Load Test & Distribution Validation

Best Practices & Next Steps

References

Announcing APISIX Integration with AI/ML API

Introduction

Proxy to OpenAI via AI/ML API

Prerequisites

Configure the Route

Test the Integration

Verify Response

Core Use Cases

Conclusion

More Resources

From stdio to HTTP SSE: Host Your MCP Server with APISIX API Gateway

Introduction

Model Context Protocol (MCP) Overview

What Is the mcp-bridge Plugin?

How It Works and Architecture Diagram

Application Scenarios and Benefits

Authentication and Rate Limiting with Apache APISIX Plugins

Authentication Plugins

Rate-Limiting Plugins

Adding Authentication and Rate Limiting to MCP Servers

Roadmap

Summary

APISIX-MCP: Embracing Intelligent API Management with AI + MCP

Preface

About APISIX-MCP

General Operations

API Resource Management

Plugin Operations

Security Configuration

How to Use APISIX-MCP

Advantages of AI-Driven Operations

Conclusion

What Is an AI Gateway: Differences from API Gateway

What Is an AI Gateway? Why Did It Arise in the AI Era?

The Rise of AI Gateways

AI Agents vs. Traditional Devices: Why Stream-Type Requests Demand Specialized Handling

The Stream-Type Challenge

Two Types of AI Gateways: Purpose-Built vs. API Gateway Evolutions

Specific Purpose-Built AI Gateways

Evolved AI Gateways from API Gateways

Why Evolved AI Gateways Are Winning Long-Term

Model Context Protocol (MCP): Bridging AI Assistants and External Tools

How MCP Works

Benefits of MCP

Future of AI Gateways: Convergence with API Monetization

Trend 1: The Decline of Standalone AI Gateways

Trend 2: API Gateways as AI Orchestrators

The Bottom Line

Conclusion: Embracing AI-API Convergence

Comprehensive Overview of APISIX AI Gateway Features

Introduction: The Rise of AI Agents and the Evolution of AI Gateway

LLM Proxy: Efficient Management of Multiple LLM Backends

AI Security Protection: Ensuring Safe and Compliant Use of AI

Token Observability and Management: Preventing High Bills Due to API Abuse

Smart Routing: Dynamic Traffic Management for AI APIs

What Is the `mcp-bridge` Plugin?

6. Create a `docer-compose.yaml` File