DEV Community: Navendu Pottekkat

Building a Model Context Protocol (MCP) Server in Go

Navendu Pottekkat — Sun, 13 Apr 2025 06:28:52 +0000

Model Context Protocol (MCP) servers allow LLMs (MCP hosts/clients) to access prompts, resources, and tools in a standard way, allowing you to build agents and complex workflows on top of LLMs.

SDKs make building and integrating MCP clients and servers easy. While there isn't an official SDK for Go yet, the community-built mark3labs/mcp-go has been gaining a lot of popularity among Go developers—including myself.

I used this SDK today to make a real-world MCP for a real project, and it has been pretty neat so far. This article is a quick walkthrough of how I set up an MCP server using the MCP Go SDK for DiceDB, an in-memory key-value store like Redis.

Install the mcp-go Module

To use the mcp-go module, run:

go get github.com/mark3labs/mcp-go

We'll also use the dicedb-go module to talk to the database:

go get github.com/dicedb/dicedb-go

Create a New MCP Server

The entire server fits into a single main.go file, thanks to the abstractions provided by the SDK. Let's break down the important bits, starting with setting up the server:

package main

// Import ALL required modules
import (
    "context"
    "fmt"
    "net"
    "strconv"
    "strings"

    "github.com/dicedb/dicedb-go"
    "github.com/dicedb/dicedb-go/wire"
    "github.com/mark3labs/mcp-go/mcp"
    "github.com/mark3labs/mcp-go/server"
)

func main() {
    // Create a new MCP server
    s := server.NewMCPServer(
        "DiceDB MCP", // Name of the server
        "0.1.0", // Version
        // Set listChanged to false as this example
        // server does not emit notifications
        // when the list of available tool changes
        // https://modelcontextprotocol.io/specification/2024-11-05/server/tools#capabilities
        server.WithToolCapabilities(false),
    )

Here's what's happening here:

4: We import the MCP and DiceDB modules and some helpers.
19-27: We create a new MCP server instance:
- 19: "DiceDB MCP" is just a human-readable name.
- 20: "0.1.0" is the version.
- 26: server.WithToolCapabilities(false) indicates that the server won't notify clients when the list of available tools changes, which is the case for our simple server.

Define a New Tool

Now, let's add a tool. This one just pings the DiceDB server to check if it's reachable:

    pingTool := mcp.NewTool("ping", // Tool name
        // Short description of what the tool does
        mcp.WithDescription("Ping the DiceDB server to check connectivity"),
        // Add a string property to the tool schema
        // to accept the URL of the DiceDB server
        mcp.WithString("url",
            // Description of the property
            mcp.Description("The URL of the DiceDB server in format 'host:port'"),
            // Default value that compliant clients
            // can use if the value isn't explicitly set
            mcp.DefaultString("localhost:7379"),
        ),
    )

Under the hood, these definitions get translated to a JSON schema following the JSON-RPC 2.0 specification. MCP clients use this schema to discover and call the tool.

The "ping" tool has the following properties:

29: mcp.NewTool("ping", ... defines a tool named "ping" that can be invoked by MCP clients.
31: mcp.WithDescription adds a description property to the tool schema that helps both humans and AI (MCP host/client) decide which tool to use.
34: mcp.WithString adds a string property to the tool schema to obtain the URL of the DiceDB server (from the MCP host/client).
39: mcp.DefaultString isn't defined in the MCP specification, but compliant clients can use this to set a default value if none is provided.

Next, we wire in the actual logic.

Create a Handler Function

Now that we've defined the tool let's add what happens when it's invoked. In our case, we want to ping the DiceDB server to check if it's reachable:

    s.AddTool(pingTool, func(ctx context.Context, request mcp.CallToolRequest) (*mcp.CallToolResult, error) {
        // Extract the URL argument from the client request
        url := request.Params.Arguments["url"].(string)

        // Parse host and port from URL
        parts := strings.Split(url, ":")
        host := parts[0]
        port := 7379 // Default to 7379 if no port is provided

        if len(parts) > 1 {
          if p, err := strconv.Atoi(parts[1]); err == nil {
            port = p
          }
        }

        // Create a new DiceDB client
        client, err := dicedb.NewClient(host, port)
        if err != nil {
            return mcp.NewToolResultText(fmt.Sprintf("Error connecting to DiceDB: %v", err)), nil
        }

        // Send the PING command to DiceDB
        resp := client.Fire(&wire.Command{Cmd: "PING"})

        // Return the result to the MCP client
        return mcp.NewToolResultText(fmt.Sprintf("Response from DiceDB: %v", resp)), nil
    })

Here, we define a handler function that runs when the MCP client calls the tool. This is what it does:

45: Extracts the URL from the client request.
48-56: Splits the string to separate host and port. If a port isn't specified, we fall back to DiceDB's default (7379).
59-62: Initializes a new DiceDB client using the dicedb-go SDK.
65: Sends a PING command to DiceDB.
68: Returns the formatted response back to the MCP client.

See how the SDK provides neat wrappers like NewToolResultText to structure responses as required by the MCP specification.

Start the Server

All that's left is to start the server:

    if err := server.ServeStdio(s); err != nil {
        fmt.Printf("Error starting server: %v\n", err)
    }
}

This uses the stdio transport, meaning our MCP server communicates using standard input and output streams. This is the simplest way to run an MCP server locally.

The MCP specification also allows SSE or Server-Sent Events transport, which enables server-to-client streaming and uses HTTP POST for client-to-server messages. You can also create custom transports for specific needs.

Build/Install the Server

Before using the server, build the binary:

go build -o ./dist/dicedb-mcp

Or install it globally (adds it to $GOBIN):

go install

This gives you either a local or a global runnable binary that can be executed by MCP clients and hosts.

Use with MCP Hosts/Clients

To use this MCP server with host applications like Claude Desktop or Cursor, add it to their configuration file (claude_desktop_config.json or mcp.json):

{
    "mcpServers": {
        "dicedb-mcp": {
            "command": "path/to/dicedb-mcp"
        }
    }
}

Now you can prompt the LLM like:

Can you check if DiceDB is running at localhost:7379?

The LLM will detect the ping tool we registered, ask to run it, and show the result returned by the MCP server.

Note: Make sure you have DiceDB running. You can also apply this pattern to wrap other tools into MCP servers instead of DiceDB, as used in this example.

Learn More

We've built a minimal example to demonstrate how the MCP Go SDK works. It is functional but intentionally simple.

You can check out a more robust DiceDB MCP implementation (with additional tools and features) on GitHub.

To go further, I recommend these resources:

Kubernetes Gateway API v1.0: Should You Switch?

Navendu Pottekkat — Thu, 04 Jan 2024 02:20:45 +0000

It has been over a month since the Kubernetes Gateway API made its v1.0 release, signifying graduation to the generally available status for some of its key APIs.

I wrote about the Gateway API when it graduated to beta last year, but a year later, the question still remains. Should you switch to the Gateway API from the Ingress API?

My answer from last year was you shouldn't. And I had strong reasons.

The Gateway API and its implementations were still in their infancy. The Ingress API, on the other hand, was stable and covered some primary use cases that might work for most users.

For users requiring more capabilities, I suggested using the custom resources provided by the Ingress controllers by trading off portability (switching between different Ingress implementations).

With the v1.0 release, this might change. The Gateway API is much more capable now, and its 20+ implementations are catching up quickly.

So, if you are starting anew and choosing between the Ingress and the Gateway API, I suggest you pick the Gateway API if the API and the implementation you choose support all the features you want.

What's Wrong with the Ingress API?

The Ingress API works very well, but only for a small subset of common use cases. To extend its capabilities, Ingress implementations started using custom annotations.

For example, if you chose Nginx Ingress, you will use some of its dozens of annotations that are not portable if you decide to switch to another Ingress implementation like Apache APISIX.

These implementation-specific annotations are also cumbersome to manage and defeat the purpose of managing Ingress in a Kubernetes-native way.

Eventually, Ingress controller implementations started developing their own CRDs to expose more features to Kubernetes users. These CRDs are specific to the Ingress controller. But if you can sacrifice portability and stick to one Ingress controller, the CRDs are easier to work with and offer more features.

The Gateway API aims to solve this problem once and for all by providing the vendor agnosticism of the Ingress API and the flexibility of the CRDs. It is positioned very well to achieve this goal.

In the long term, the Ingress API is not expected to receive any new features, and all efforts will be made to converge with the Gateway API. So, adopting the Ingress API can cause issues when you inadvertently hit limits with its capabilities.

Obvious Benefits

Expressive, extensible, and role-oriented are the key ideas that shaped the development of the Gateway API.

Unlike the Ingress API, the Gateway API is a collection of multiple APIs (HTTPRoute, Gateway, GatewayClass, etc.), each catering to different organizational roles.

For example, the application developers need to only care about the HTTPRoute resource, where they can define rules to route traffic. They can delegate the cluster-level details to an operator who manages the cluster and ensures that it meets the developers' needs using the Gateway resource.

This role-oriented design of the API allows different people to use the cluster while maintaining control.

The Gateway API is also much more capable than the Ingress API. Features that require annotations in the Ingress API are supported out-of-the-box in the Gateway API.

An Official Extension

Although the Gateway API is an official Kubernetes API, it is implemented as a set of CRDs.

This is no different from using default Kubernetes resources. But you just have to install these CRDs like an official extension.

This allows for fast iteration compared to Kubernetes, which is slowly moving toward long-term stability.

Will It Proliferate?

As this famous XKCD comic reminds us frequently, standards tend to proliferate.

A version of this was seen in the Ingress and Gateway APIs. It usually goes like this:

A standard emerges to unify different projects/their standards (Kubernetes Ingress API).
The unified standard has limitations the implementors want to overcome (Ingress API was limited).
Implementations diverge from the standard because of these limitations (Custom CRDs, annotations).
Each implementation now has its own standard (non-portable CRDs, annotations).
A new standard emerges to unify these different standards (Kubernetes Gateway API).

It is reasonable to think that the Gateway API might not be the end game here. But I believe it has every chance of being the standard for routing in Kubernetes.

Again, I have my strong reasons.

Broad adoption is critical to prevent standard proliferation as there are fewer incentives for the implementations to work on a different standard. The Gateway API already has more than 25 implementations.

An implementation can conform to the Gateway API on different levels:

Core: All implementations are expected to conform to these.
Extended: These might only be available in some implementations but are standard APIs.
Implementation-specific: Specific to implementations but added through standard extension points.

A niche feature can move from implementation-specific to extended to core as more implementations support these features. i.e., the API allows room for custom extensions while ensuring it follows the standard.

The Service Mesh Interface (SMI) project was a similar attempt to standardize configuring service meshes in Kubernetes. However, the project received little traction after the initial involvement of the service mesh projects and slowly died out.

SMI did not support many common denominator features that users expected in a service mesh. It also did not move fast enough to support these features. Eventually, service mesh implementations fell behind in conforming to SMI (I used to work closely with SMI under the CNCF TAG Network on a project that reported SMI conformance).

These are universal reasons, but the project is now being resurrected through the Gateway API. The Gateway API for Mesh Management and Administration (GAMMA) initiative aims to extend the Gateway API to work with service meshes.

The SMI project recently merged with the GAMMA initiative, which is excellent for the Gateway API. Istio, undoubtedly the most popular service mesh, also announced that the Gateway API will be the default API to manage Istio in the future. Such adoptions prevent proliferation.

Migration Guide

The Gateway API documentation has a comprehensive guide on migrating your Ingress resources to Gateway resources. Instead of restating it, let's try using the ingress2gateway tool to convert our Ingress resources to corresponding Gateway API resources.

You can download and install the binary for your operating system directly from the releases page.

Let's take a simple Ingress resource:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin-route
spec:
  ingressClassName: apisix
  rules:
    - host: local.httpbin.org
      http:
        paths:
          - backend:
              service:
                name: httpbin
                port:
                  number: 80
            path: /
            pathType: Prefix

This will route all traffic with the provided host address to the httpbin service.

To convert it to the Gateway API resource, we can run:

ingress2gateway print --input_file=ingress.yaml

This Gateway API resource will be as shown below:

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
  name: httpbin-route
spec:
  hostnames:
  - local.httpbin.org
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: httpbin
      port: 80

Viable Alternatives

There are other viable alternatives for configuring gateways in Kubernetes.

In Apache APISIX, you can deploy it in standalone mode and define route configurations in a yaml file. You can update this yaml file through traditional workflows, and it can be pretty helpful in scenarios where managing the gateway configuration via the Kubernetes API is not required.

Implementation-specific custom CRDs are also viable alternatives if you don't plan to switch to a different solution or if your configuration is small enough to migrate easily.

In any case, the Gateway API is here to stay.

A "Tiny" APISIX Plugin

Navendu Pottekkat — Mon, 27 Nov 2023 16:00:56 +0000

A key feature of Apache APISIX is its pluggable architecture. In addition to providing 80+ Lua plugins out of the box, APISIX also supports external plugins written in other languages through plugin runners and WebAssembly (Wasm).

In this article, we will write a "tiny" Go plugin for APISIX, compile it to a Wasm binary, run it in APISIX, and learn how it all works. We will also compare the benefits and costs of using Wasm plugins, external plugins (plugin runners), and native Lua plugins.

APISIX and Wasm

APISIX supports Wasm through the WebAssembly for Proxies (proxy-wasm) specification. APISIX is a host environment that implements the specification, and developers can use the SDKs available in multiple languages to create plugins.

Using Wasm plugins in APISIX has multiple advantages:

Many programming languages compile to Wasm. This allows you to leverage the capabilities of your tech stack in APISIX plugins.
The plugins run inside APISIX and not on external plugin runners. This means you compromise less on performance while writing external plugins.
Wasm plugins run inside APISIX but in a separate VM. So even if the plugin crashes, APISIX can continue to run.
APISIX can only maintain its Wasm support without having to maintain plugin runners for multiple languages*.

* These advantages come with a set of caveats which we will look at later.

APISIX's plugin architecture below shows native Lua plugins, external plugins through plugin runners, and Wasm plugins:

A "Tiny" Go Plugin

Let's get coding! To write a Go plugin, we will use the proxy-wasm-go-sdk.

Reading through the documentation, you will understand why this plugin is called "tiny," i.e., the SDK uses the TinyGo compiler instead of the official Go compiler. You can read more about why this is the case on the SDK\'s overview page, but the TLDR version is that the Go compiler can only produce Wasm binaries that run in the browser.

For our example, we will create a plugin that adds a response header. The code below is pretty self-explanatory, and you can refer to other plugins for more implementation details:

// references:
// https://github.com/tetratelabs/proxy-wasm-go-sdk/tree/main/examples
// https://github.com/apache/apisix/blob/master/t/wasm/
package main

import (
    "github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
    "github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"

    "github.com/valyala/fastjson"
)

func main() {
    proxywasm.SetVMContext(&vmContext{})
}

// each plugin has its own VMContext.
// it is responsible for creating multiple PluginContexts for each route.
type vmContext struct {
    types.DefaultVMContext
}

// each route has its own PluginContext.
// it corresponds to one instance of the plugin.
func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext {
    return &pluginContext{}
}

type header struct {
    Name  string
    Value string
}

type pluginContext struct {
    types.DefaultPluginContext
    Headers []header
}

func (ctx *pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus {
    data, err := proxywasm.GetPluginConfiguration()
    if err != nil {
        proxywasm.LogErrorf("error reading plugin configuration: %v", err)
        return types.OnPluginStartStatusFailed
    }

    var p fastjson.Parser
    v, err := p.ParseBytes(data)
    if err != nil {
        proxywasm.LogErrorf("error decoding plugin configuration: %v", err)
        return types.OnPluginStartStatusFailed
    }
    headers := v.GetArray("headers")
    ctx.Headers = make([]header, len(headers))
    for i, hdr := range headers {
        ctx.Headers[i] = header{
            Name:  string(hdr.GetStringBytes("name")),
            Value: string(hdr.GetStringBytes("value")),
        }
    }
    return types.OnPluginStartStatusOK
}

// each HTTP request to a route has its own HTTPContext
func (ctx *pluginContext) NewHttpContext(contextID uint32) types.HttpContext {
    return &httpContext{parent: ctx}
}

type httpContext struct {
    types.DefaultHttpContext
    parent *pluginContext
}

func (ctx *httpContext) OnHttpResponseHeaders(numHeaders int, endOfStream bool) types.Action {
    plugin := ctx.parent
    for _, hdr := range plugin.Headers {
        proxywasm.ReplaceHttpResponseHeader(hdr.Name, hdr.Value)
    }

    return types.ActionContinue
}

To compile our plugin to a Wasm binary, we can run:

tinygo build -o custom_response_header.go.wasm -scheduler=none -target=wasi ./main.go

Configuring APISIX to Run the Plugin

To use the Wasm plugin, we first have to update our APISIX configuration file to add this:

wasm:
  plugins:
    - name: custom-response-header
      priority: 7000
      file: /opt/apisix/wasm/custom_response_header.go.wasm

Now we can create a route and enable this plugin:

routes:
  - uri: /*
    upstream:
      type: roundrobin
      nodes:
        "127.0.0.1:80": 1
    plugins:
      custom-response-header:
       conf: |
        {
          "headers": [{
            "name": "X-Go-Wasm", "value": "APISIX"
          }]
        }
#END

Note: The above configuration assumes that APISIX is deployed in standalone mode.

Testing the Plugin

We can send a request to the created route to test the plugin:

curl http://127.0.0.1:9080 -s --head

The response header would be as shown below:

...
X-Go-Wasm: APISIX

Wasm for the Win?

From this article, it seems like using Wasm plugins benefits both users and APISIX maintainers.

A test using our example custom-response-header function implemented through a Lua plugin, an external Go plugin runner, and a Wasm plugin show how the performance varies:

Running 30s tests with 5 threads and 50 connections using wrk.

Looking solely at this example, it might be tempting to ask why anyone would want to use plugin runners or write Lua plugins. Well, all the advantages of using Wasm comes with the following caveats:

Limited plugins: The Wasm implementation of programming languages often lacks complete support. For Go, we were limited to using the TinyGo compiler, similar to other languages.
Immature stack: Wasm and its usage outside the browser is still a relatively new concept. The proxy-wasm spec also has its limitations due to its relative novelty.
Lack of concurrency: Wasm does not have built-in concurrency support. This could be a deal breaker for typical APISIX uses cases where high performance is critical.
Better alternatives: Since APISIX can be extended using Lua plugins or plugin runners, there are always alternatives to using Wasm.

Despite these caveats, the future of Wasm in APISIX and other proxies seems promising. You can choose to hop on the Wasm bandwagon if its benefits tip the scale against these costs. But currently, APISIX plans to continue supporting all three ways of creating custom plugins for the foreseeable future.

How is Apache APISIX Fast?

Navendu Pottekkat — Wed, 13 Sep 2023 07:01:22 +0000

"High speed," "minimum latency," and "ultimate performance" are often used to characterize Apache APISIX. Even when someone asks me about APISIX, my answer always includes "high-performance cloud native API gateway."

Performance benchmarks (vs. Kong, Envoy) confirm these characteristics are indeed accurate (test yourself).

Tests run for 10 rounds with 5000 unique routes on Standard D8s v3 (8 vCPUs, 32 GiB memory).

But how does APISIX achieve this?

To answer that question, we must look at three things: etcd, hash tables, and radix trees.

In this article, we will look under the hood of APISIX and see what these are and how all of these work together to keep APISIX maintaining peak performance while handling significant traffic.

etcd as the Configuration Center

APISIX uses etcd to store and synchronize configurations.

etcd is designed to work as a key-value store for configurations of large-scale distributed systems. APISIX is intended to be distributed and highly scalable from the ground up, and using etcd over traditional databases facilitates that.

Another key indispensable feature for API gateways is to be highly available, avoiding downtime and data loss. You can efficiently achieve this by deploying multiple instances of etcd to ensure a fault-tolerant, cloud native architecture.

APISIX can read/write configurations from/to etcd with minimum latency. Changes to the configuration files are notified instantly, allowing APISIX to monitor only the etcd updates instead of polling a database frequently, which can add performance overhead.

This chart summarizes how etcd compares with other databases.

Hash Tables for IP Addresses

IP address-based allowlists/denylists are a common use case for API gateways.

To achieve high performance, APISIX stores the list of IP addresses in a hash table and uses it for matching (O(1)) than iterating through the list (O(N)).

As the number of IP addresses in the list increases, the performance impact of using hash tables for storage and matching becomes apparent.

Under the hood, APISIX uses the lua-resty-ipmatcher library to implement this functionality. The example below shows how the library is used:

local ipmatcher = require("resty.ipmatcher")
local ip = ipmatcher.new({
    "162.168.46.72",
    "17.172.224.47",
    "216.58.32.170",
})

ngx.say(ip:match("17.172.224.47")) -- true
ngx.say(ip:match("176.24.76.126")) -- false

The library uses Lua tables which are hash tables. The IP addresses are hashed and stored as indices in a table, and to search for a given IP address, you just have to index the table and test whether it is nil or not.

To search for an IP address, it first computes the hash (index) and checks its value. If it is non-empty, we have a match. This is done in constant time O(1).

Radix Trees for Routing

Please forgive me for tricking you into a data structures lesson! But hear me out; this is where it gets interesting.

A key area where APISIX optimizes performance is route matching.

APISIX matches a route with a request from its URI, HTTP methods, host, and other information (see router). And this needs to be efficient.

If you have read the previous section, an obvious answer would be to use a hash algorithm. But route matching is tricky because multiple requests can match the same route.

For example, if we have a route /api/*, then both /api/create and /api/destroy must match the route. But this is not possible with a hash algorithm.

Regular expressions can be an alternate solution. Routes can be configured in a regex, and it can match multiple requests without the need to hardcode each request.

If we take our previous example, we can use the regex /api/[A-Za-z0-9]+ to match both /api/create and /api/destroy. More complex regexes could match more complex routes.

But regex is slow! And we know APISIX is fast. So instead, APISIX uses radix trees which are compressed prefix trees (trie) that work really well for fast lookups.

Let's look at a simple example. Suppose we have the following words:

romane
romanus
romulus
rubens
ruber
rubicon
rubicundus

A prefix tree would store it like this:

The highlighted traversal shows the word "rubens."

A radix tree optimizes a prefix tree by merging child nodes if a node only has one child node. Our example trie would look like this as a radix tree:

The highlighted traversal still shows the word "rubens." But the tree looks much smaller!

When you create routes in APISIX, APISIX stores them in these trees.

APISIX can then work flawlessly because the time it takes to match a route only depends on the length of the URI in the request and is independent of the number of routes (O(K), K is the length of the key/URI).

So APISIX will be as quick as it is when matching 10 routes when you first start out and 5000 routes when you scale.

This crude example shows how APISIX can store and match routes using radix trees:

The highlighted traversal shows the route /user/* where the * represents a prefix. So a URI like /user/navendu will match this route. The example code below should give more clarity to these ideas.

APISIX uses the lua-resty-radixtree library, which wraps around rax, a radix tree implementation in C. This improves the performance compared to implementing the library in pure Lua.

The example below shows how the library is used:

local radix = require("resty.radixtree")
local rx = radix.new({
    {
        paths = { "/api/*action" },
        metadata = { "metadata /api/action" }
    },
    {
        paths = { "/user/:name" },
        metadata = { "metadata /user/name" },
        methods = { "GET" },
    },
    {
        paths = { "/admin/:name" },
        metadata = { "metadata /admin/name" },
        methods = { "GET", "POST", "PUT" },
        filter_fun = function(vars, opts)
            return vars["arg_access"] == "admin"
        end
    }
})

local opts = {
    matched = {}
}

-- matches the first route
ngx.say(rx:match("/api/create", opts)) -- metadata /api/action
ngx.say("action: ", opts.matched.action) -- action: create

ngx.say(rx:match("/api/destroy", opts)) -- metadata /api/action
ngx.say("action: ", opts.matched.action) -- action: destroy

local opts = {
    method = "GET",
    matched = {}
}

-- matches the second route
ngx.say(rx:match("/user/bobur", opts)) -- metadata /user/name
ngx.say("name: ", opts.matched.name) -- name: bobur

local opts = {
    method = "POST",
    var = ngx.var,
    matched = {}
}

-- matches the third route
-- the value for `arg_access` is obtained from `ngx.var`
ngx.say(rx:match("/admin/nicolas", opts)) -- metadata /admin/name
ngx.say("admin name: ", opts.matched.name) -- admin name: nicolas

The ability to manage a large number of routes efficiently has made APISIX the API gateway of choice for many large-scale projects.

Look under the Hood

There is only so much I can explain about the inner workings of APISIX in one article.

But the best part is that the libraries mentioned here and Apache APISIX are entirely open source, meaning you can look under the hood and modify things yourself.

And if you can improve APISIX to get that final bit of performance, you can contribute the changes back to the project and let everyone benefit from your work.

A Comprehensive Guide to API Gateways, Kubernetes Gateways, and Service Meshes

Navendu Pottekkat — Fri, 09 Jun 2023 06:18:17 +0000

Translations: Simplified Chinese.

There is still a lot of confusion about API gateways, Kubernetes gateways, and service meshes. A lot of this is because:

People often mention these technologies with the same keywords, like canary deployments, rate limiting, and service discovery.
All these technologies use reverse proxies.
Some API gateways have their own Kubernetes gateways and service meshes and vice versa.
There are a lot of articles/videos that compare the three technologies and conclude why one is better than the other.

In this article, I will try to explain these technologies and share how they fundamentally differ and cater to different use cases.

API Gateways

An API gateway sits between your client applications and your APIs. It accepts all client requests, forwards them to the required APIs, and returns the response to clients in a combined package.

It is basically a reverse proxy with a lot of capabilities.

On top of this, an API gateway can also have features like authentication, security, fine-grained traffic control, and monitoring, leaving the API developers to focus only on business needs.

There are many API gateway solutions available. Some of the popular free and open source solutions are:

Apache APISIX: A high-performance, extensible, cloud native API gateway built on top of Nginx.
Gloo Edge: An API gateway built on top of Envoy proxy.
Kong: A pluggable API gateway also built on Nginx.
Tyk: An API gateway written in Go supporting REST, GraphQL, TCP, and gRPC protocols.

Cloud platforms like GCP, AWS, and Azure also have their own proprietary API gateways.

API gateways, Kubernetes gateways, and service meshes support canary deployments—gradually rolling out a new software version to a small subset of users before making it generally available.

The example below shows how to configure a canary deployment in Apache APISIX.

You can send a request to the APISIX Admin API with the following configuration:

curl http://127.0.0.1:9180/apisix/admin/routes/1 \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "uri":"/*",
  "plugins":{
    "traffic-split":{
      "rules":[
        {
          "weighted_upstreams":[
            {
              "upstream":{
                "name":"api-v1",
                "type":"roundrobin",
                "nodes":{
                  "api-v1:8080":1
                }
              },
              "weight":95
            },
            {
              "weight":5
            }
          ]
        }
      ]
    }
  },
  "upstream":{
    "type":"roundrobin",
    "nodes":{
      "api-v2:8080":1
    }
  }
}'

APISIX will now route 95% of the traffic to the api-v1 service and 5% to the api-v2 service.

Kubernetes Gateways

Kubernetes gateways are just Kubernetes-native API gateways. i.e., you can manage these API gateways with the Kubernetes API, similar to a Kubernetes pod, service, or deployment.

In Kubernetes, your APIs are pods and services deployed in a cluster. You then use a Kubernetes gateway to direct external traffic to your cluster.

Kubernetes provides two APIs to achieve this, the Ingress API and the Gateway API.

Kubernetes Ingress API

The Ingress API was created to overcome the limitations of the default service types, NodePort and LoadBalancer, by introducing features like routing and SSL termination. It also standardized how you expose Kubernetes services to external traffic.

It has two components, the Ingress and the Ingress controller.

The Ingress Kubernetes native object defines a set of rules on how external traffic can access your services.

This example configuration shows routing traffic based on URI path with the Kubernetes Ingress object:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-routes
spec:
  ingressClassName: apisix
  rules:
    - http:
        paths:
          - backend:
              service:
                name: api-v1
                port:
                  number: 8080
            path: /v1
            pathType: Exact
          - backend:
              service:
                name: api-v2
                port:
                  number: 8080
            path: /v2
            pathType: Exact

An Ingress controller implements these rules and routes traffic to your cluster using a reverse proxy.

There are over 20 Ingress controller implementations. APISIX has an Ingress controller that wraps around APISIX API gateway to work as Kubernetes Ingress.

The APISIX Ingress controller converts the Kubernetes Ingress object to APISIX configuration.

APISIX then implements this configuration.

You can swap APISIX with any other Ingress controller, as the Ingress API is not tied to any specific implementation.

This vendor neutrality works well for simple configurations. But if you want to do complex routing like a canary deployment, you must rely on vendor-specific annotations.

The example below shows how to configure a canary deployment using Nginx Ingress. The custom annotations used here are specific to Nginx:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "5"
  name: api-canary
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: api-v2
          servicePort: 8080
        path: /

The above configuration will route 5% of the traffic to the api-v2 service.

In addition to annotations, Ingress controllers like APISIX have custom Kubernetes CRDs to overcome the limitations of the Ingress API.

The example below uses the APISIX CRD, ApisixRoute to configure a canary deployment:

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: api-canary
spec:
  http:
    - name: route
      match:
        paths:
          - /*
      backends:
        - serviceName: api-v1
          servicePort: 8080
          weight: 95
        - serviceName: api-v2
          servicePort: 8080
          weight: 5

These custom CRDs made it much easier to configure Ingress and leverage the full capabilities of the API gateway underneath but at the expense of portability.

Kubernetes Gateway API

The Gateway API is a new Kubernetes object that aims to "fix" the Ingress API.

It takes inspiration from the custom CRDs developed by Ingress controllers to add HTTP header-based matching, weighted traffic splitting, and other features that require custom proprietary annotations with the Ingress API.

The example below shows configuring a canary deployment with the Kubernetes Gateway API:

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
  name: api-canary
spec:
  rules:
  - backendRefs:
    - name: api-v1
      port: 8080
      weight: 95
    - name: api-v2
      port: 8080
      weight: 5

Any Ingress controller (that implements the Gateway API) can now implement this configuration.

The Gateway API also makes many improvements over the Ingress API, but it is still in alpha, and the Gateway API implementations are constantly breaking.

Service Meshes

API gateways and Kubernetes gateways work across application boundaries solving edge problems while abstracting your APIs.

Service Meshes solve a different challenge.

A service mesh is more concerned about inter-service communication (east-west traffic) than service-client communication (north-south traffic).

Typically, this is achieved by deploying sidecar proxies with APIs/services.

Here, the sidecar proxies handle the service-to-service communication instead of the developer having to code the networking logic to the services.

There are a lot of service meshes available. Some of the popular ones are:

Istio: By far the most popular service mesh. It is built on top of Envoy proxy, which many service meshes use.
Linkerd: A lightweight service mesh that uses linkerd2-proxy, written in Rust specifically for Linkerd.
Consul Connect: A service mesh emphasizing security and observability. It can work with either a built-in proxy or Envoy.

New service mesh offerings like Cilium offer alternatives to sidecar-based service meshes by using networking capabilities directly from the kernel through eBPF.

A typical service mesh requires 8 proxies for 8 services whereas eBPF-based service meshes like Cilium don't. Adapted from Cilium Service Mesh – Everything You Need to Know.

Service meshes also have basic ingress/egress gateways to handle north-south traffic to and from the services. Ingress gateways are the entry points of external traffic to a service mesh, and egress gateways allow services inside a mesh to access external services.

Apache APISIX also has a service mesh implementation called Amesh. It works with Istio's control plane using the xDS protocol replacing the default Envoy proxy in the sidecar.

A service mesh lets you configure canary deployments. For example, you can split the requests from one service between two versions of another service.

The example below shows configuring a canary deployment with Istio service mesh:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: api-virtual-service
spec:
  hosts:
    - api
  http:
    - route:
        - destination:
            host: api
            subset: v1
          weight: 80
        - destination:
            host: api
            subset: v2
          weight: 20

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: api-destination-rule
spec:
  host: api
  subsets:
    - name: v1
      labels:
        version: "1.0"
    - name: v2
      labels:
        version: "2.0"

These configurations are specific to Istio. To switch to a different service mesh, you must create a different but similarly vendor-dependant configuration.

The Service Mesh Interface (SMI) specification was created to solve this portability issue.

The SMI spec is a set of Kubernetes CRDs that a service mesh user can use to define applications without binding to service mesh implementations.

A standardization attempt will only work if all the projects are on board. But this did not happen with the SMI spec, and only a few projects participated actively.

More recently, the Kubernetes SIG Network has been evolving the Gateway API to support service meshes.

The GAMMA (Gateway API for Mesh Management and Administration) initiative is a dedicated group with the Gateway API project with goals to "investigate, design, and track Gateway API resources, semantics, and other artifacts related to service mesh technology and use-cases."

Gateway API is a natural next step to the Ingress API, but we must wait to see how it will work for service meshes. Istio has announced its intention to use the Gateway API as its default API for all traffic management and continues to drive the project forward.

The example below shows configuring a canary deployment in Istio with the Gateway API. The underlying idea is using parentRefs to attach to other services instead of the gateway:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: api-canary
spec:
  parentRefs:
  - kind: Service
    name: api-a
    port: 8080
  rules:
  - backendRefs:
    - name: api-b-v1
      port: 8080
      weight: 95
    - name: api-b-v2
      port: 8080
      weight: 5

There are some concerns that the GAMMA project might become skewed to serve the needs of one particular project than the larger community, which will eventually lead to other projects using their own APIs, similar to the custom CRD scenario after the Kubernetes Ingress API.

But the Gateway API project has been the best attempt at standardizing traffic management in service meshes. The SMI project also joined the GAMMA initiative with a shared vision and will help advocate for consistent implementations of the Gateway API by service mesh projects.

Other projects like Flagger and Argo Rollouts have also integrated with the Gateway API.

What Should You Use?

There is only one correct answer to this question; "it depends."

If you are developing APIs and need authentication, security, routing, or metrics, you are better off using an API gateway than building this on your own in your APIs.

If you want to do something similar in a Kubernetes environment, you should use a Kubernetes gateway instead of trying to wrangle your API gateway to work on Kubernetes. Thankfully, a lot of API gateways also work with Kubernetes-native configurations.

But sometimes, the features offered by an API gateway + Ingress controller might be an overkill for a Kubernetes environment, and you might want to switch back to simple traffic management.

Service meshes, on the other hand, solve an entirely different set of problems. They also bring their own gateways to handle north-south traffic (usually enough) but also let you use your own gateways with more features.

The convergence of the API gateway and the service mesh through the Kubernetes Gateway API should make it easier for the application developer to focus on solving problems than worry about the underlying implementation.

Projects like Apache APISIX use the same technology to build the API gateway and service mesh offerings and integrate well with these specifications, incentivizing vendor-neutral choices.

It is also likely that you will not need any of these. You may not even need microservices or a distributed architecture, but when you need them, gateways and meshes can make your lives a lot easier.

Comparing Kubernetes Gateway and Ingress APIs

Navendu Pottekkat — Wed, 01 Feb 2023 04:44:24 +0000

A couple of months ago, the new Kubernetes Gateway API graduated to beta.

Why do you need another API to handle external traffic when you have the stable Kubernetes Ingress API and dozens of implementations? What problems of the Ingress API does the new Gateway API solve? Does this mean the end of the Ingress API?

I will try to answer these questions in this article by getting hands-on with these APIs and looking at how they evolved.

Standardizing External Access to Services: The Ingress API

The Kubernetes Ingress API was created to standardize exposing services in Kubernetes to external traffic. The Ingress API overcame the limitations of the default service types, NodePort and LoadBalancer, by introducing features like routing and SSL termination.

There are over 20 implementations of Ingress controllers available. In this article, I will use Apache APISIX and its Ingress controller for examples.

You can create an Ingress resource to configure APISIX or any other Ingress implementations.

The example below shows how you can route traffic between two versions of an application with APISIX Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-routes
spec:
  ingressClassName: apisix
  rules:
    - host: local.navendu.me
      http:
        paths:
          - backend:
              service:
                name: bare-minimum-api-v1
                port:
                  number: 8080
            path: /v1
            pathType: Prefix
          - backend:
              service:
                name: bare-minimum-api-v2
                port:
                  number: 8081
            path: /v2
            pathType: Prefix

Tip: You can check out this hands-on tutorial to learn more about setting up Ingress on Kubernetes with Apache APISIX Ingress controller.

Since the Ingress API is not tied to any particular controller implementation, you can swap APISIX with any other Ingress controller, and it will work similarly.

This is okay for simple routing. But the API is limited, and if you want to use the full features provided by your Ingress controller, you are stuck with annotations.

For example, the Kubernetes Ingress API does not provide a schema to configure rewrites. Rewrites are useful when your upstream/backend URL differs from the path configured in your Ingress rule.

APISIX supports this feature, and you have to use custom annotations to leverage it:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-routes
  annotations:
    k8s.apisix.apache.org/rewrite-target-regex: "/app/(.*)"
    k8s.apisix.apache.org/rewrite-target-regex-template: "/$1"
spec:
  ingressClassName: apisix
  rules:
    - host: local.navendu.me
      http:
        paths:
          - backend:
              service:
                name: bare-minimum-api
                port:
                  number: 8080
            path: /app
            pathType: Prefix

This creates an Ingress resource that configures APISIX to route any requests with the /app prefix to the backend with the prefix removed. For example, a request to /app/version will be forwarded to /version.

Annotations are specific to your choice of an Ingress controller. These "proprietary" extensions limited the scope of portability intended initially with the Ingress API.

Custom CRDs > Ingress API

Being stuck with annotations also sacrifice the usability of the Ingress controllers.

Controllers therefore solved the limitations of the Ingress API by creating their own custom resources. The example below shows configuring Ingress to route traffic between two versions of an application using APISIX's custom resource:

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: api-routes
spec:
  http:
    - name: route-1
      match:
        hosts:
          - local.navendu.me
        paths:
          - /v1
      backends:
        - serviceName: bare-minimum-api-v1
          servicePort: 8080
    - name: route-2
      match:
        hosts:
          - local.navendu.me
        paths:
          - /v2
      backends:
        - serviceName: bare-minimum-api-v2
          servicePort: 8081

These CRDs made it much easier to configure Ingress, but you are tied to the specific Ingress control implementation. Without the Ingress API evolving, you had to choose between usability or portability.

Extending Ingress and Evolution to Gateway API

Ingress API was not broken; it was limited. The Gateway API was designed to overcome these limitations.

(Gateway API) aim to evolve Kubernetes service networking through expressive, extensible, and role-oriented interfaces ...

It takes inspiration from the custom CRDs of different Ingress controllers mentioned earlier.

The Gateway API adds many features "on top" of the Ingress API's capabilities. This includes HTTP header-based matching, weighted traffic splitting, and other features that require custom proprietary annotations with the Ingress API.

Traffic split with APISIX Ingress resource (see ApisixRoute/v2 reference):

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: traffic-split
spec:
  http:
    - name: rule-1
      match:
        hosts:
          - local.navendu.me
        paths:
          - /get*
      backends:
        - serviceName: bare-minimum-api-v1
          servicePort: 8080
          weight: 90
        - serviceName: bare-minimum-api-v2
          servicePort: 8081
          weight: 10

Traffic split with Gateway API (see Canary traffic rollout):

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
  name: traffic-split
spec:
  hostnames:
  - local.navendu.me
  rules:
  - backendRefs:
    - name: bare-minimum-api-v1
      port: 8080
      weight: 90
    - name: bare-minimum-api-v2
      port: 8081
      weight: 10

Another improvement from the Ingress API is how the Gateway API separates concerns. With Ingress, the application developer and the cluster operator work on the same Ingress object, unaware of the other's responsibilities and opening the door for misconfigurations.

The Gateway API separates the configurations into Route and Gateway objects providing autonomy for the application developer and the cluster operator. The diagram below explains this clearly:

Is This the End of Ingress API?

The Gateway API is relatively new, and its implementations are constantly breaking. On the contrary, the Ingress API is in stable release and has stood the test of time.

If your use case only involves simple routing and if you are okay with using custom annotations to get extra features, the Ingress API is still a solid choice.

With the Gateway API being a superset of the Ingress API, it might make sense to consolidate both. Thanks to the SIG Network community, Gateway API is still growing and will soon be production ready.

Most Ingress controllers and service meshes have already implemented the Gateway API along with the Ingress API, and as the project evolves, more implementations will surface.

Personally, at least for now, I would stick with custom CRDs provided by the Ingress controllers like Apache APISIX instead of the Ingress or Gateway API.

Why Should You Contribute to Open Source? - Contributing to Open Source #1

Navendu Pottekkat — Wed, 31 Aug 2022 12:24:41 +0000

In this series of articles, I will share how you can be an open source contributor and make a career out of it.

I have been contributing to, building, and maintaining open source projects for the past three years. Everything I mention in this series is from my personal experience.

We will start the series by talking about why you should contribute to open source.

Understanding the "why" is essential as you will spend your free time and a lot effort to contribute to open source projects. Without clear goals, you are likely to end up wasting your time.

So, here are some reasons why you should consider contributing to open source.

Improve Your Skills

Learning new skills and improving existing ones is important if you are a student or a new developer.

While contributing to an open source project, you can implement your skills and improve them. You can learn from seasoned contributors in the community and have them review your contributions.

Even if you are not contributing code, you can improve other skills like writing, designing, managing, and more. There will always be a place for your talent in an open source project.

Gain Real World Experience

You do not have to go through a whiteboard interview to contribute to an open source project.

If you are starting your career, you can contribute to an open source project and gain actual work experience. Instead of internships and boring jobs, work on top open source projects used by millions of people.

Also, you don't have to limit yourself to your city or country. You can choose to work on any open source project around the world, right from your home.

Gain Recognition

Your contributions to open source is public. This means more people can see your work and recognize you for it.

Even if you don't become an open source superstar, building in public will do you good. What's better than hearing, "Oh, you are a contributor to ABC? I love ABC!".

Build Your Network

Open source projects revolve around open source communities.

Being part of a community and contributing to a project is the best way to meet like-minded people and grow your network.

For your career, these networks can be game changers.

Find New Opportunities

With improved skills, real-world experience, braggable contributions, and a solid network, more opportunities will find you.

You can continue to work in open source or showcase your open source contributions to recruiters. You can use your network to find jobs. And when they ask if you have experience, you can send them a link to your GitHub profile.

When working in open source, you are working with some of the smartest people, learning by looking through vetted lines of code, and contributing to something that a lot of people could use.

As open source projects become sustainable, working on one is turning out to be a stable career option. In the following article, you will learn how you can find a project to contribute to.

Best Practices for Building Reliable APIs

Navendu Pottekkat — Fri, 26 Aug 2022 04:59:00 +0000

As your APIs scale, the need for making them reliable and robust increases.

This article discusses the best practices for building reliable APIs by introducing a special kind of reverse proxies called API gateways.

We will look into:

Problems with traditional API designs
What API gateways are
How API gateways improve APIs and
Patterns and examples using API gateways

But first, what are "reliable" APIs?

What Makes an API Reliable?

As a service provider, you might have service-level agreements (SLAs) with your customers, usually quoted in uptime---the amount of time the service is guaranteed to be online and operational.

Uptime is a myopic view of reliability. To understand what it means to be reliable, you have to look at the factors that affect uptime. Once you understand these factors, you will be in a better position to build reliable services.

Let's look at these factors and the questions they pose:

Latency: How fast does your API respond to requests?
Security: Who can access your API? Is it secure?
Downtime Frequency: How frequently is your API down?
Consistency: Are your API endpoints constant? Do consumers need to change their code often?
Monitoring and Reporting: Can you observe issues and failures in your API? Are you reporting them to your consumers?

As organizations move to cloud native architectures, it becomes difficult for the development teams to account for these factors on each of their services. And as these systems scale, it would be much easier to delegate these responsibilities to a single, separate system. Say hello to API gateways!

API Gateway, the Unified Entrypoint

An API gateway acts as a middleman between your clients and your APIs. It will accept all traffic (API calls) like reverse proxies, forwards the request to the required services in your backend, and returns the needed results.

An API gateway can be the central point that handles all the authentication, security, traffic control, and monitoring concerns, leaving the API developers to focus on business needs and making it easier to improve reliability.

There are a lot of open source and managed API gateway offerings available. In this article, I will be using Apache APISIX.

The following section will describe some of the best practices to make your APIs reliable using API gateways.

Reliability Best Practices with API Gateways

We will focus more on the pattern underneath than the actual implementation, as it can vary based on your API gateway choice.

I will divide these patterns into three categories:

Authentication and security
Monitoring and observability
Version control and zero downtime

We will look into each category in detail below.

Authentication and Security

User Authentication

Authenticated requests with API gateways secure client-API interactions. After a client authenticates, your API gateway can use the obtained client details for fine-grained control.

APISIX handles authentication directly through plugins like key-auth and jwt-auth. APISIX also supports OAuth authentication and role-based access control systems like wolf through plugins like openid-connect and wolf-rbac, respectively.

Rate Limiting

Intentional (DoS attacks) and unintentional (clients making too many requests) traffic spikes to your APIs can bring them down like a house of cards. Setting up rate limiting will improve the reliability of your systems in handling such scenarios.

You can set up rate limiting on your API gateway, and if the number of requests increases above a threshold, the API gateway could either delay or reject the exceeding requests.

With APISIX, you can use any of the three plugins to configure rate limits based on number of requests, number of concurrent requests per client, and count (limit-req, limit-conn, limit-count).

Monitoring and Observability

Your API's reliability and your monitoring setup go hand in hand. And you can monitor your reliability metrics by setting up monitoring on your API gateway.

API logs and traces provide detailed information about an API call. This information will help you know when your API has failed or has an error as soon as possible. Silent fails lead to unfixed errors which can cause problems in the future.

With some configuration, you will also be able to predict and anticipate traffic for the future, helping you scale reliably.

APISIX has plugins that integrate with logging (Apache SkyWalking, RocketMQ), metrics (Prometheus, Datadog), and tracing (OpenTelemetry, Zipkin) platforms/specifications. You can read more on API Observability with APISIX Plugins.

Version Control and Zero Downtime

Canary Release

When switching to new versions of your APIs, you must ensure that you don't drop your traffic. Clients should still be able to make requests to your API and get back the correct response.

With an API gateway, you can setup canary releases. This will ensure that your API remains functional during the transition, and you can also roll back to the older version if there are any issues.

Initially, the API gateway will route all traffic to your API's old version.

When you have a new version, you can configure the API gateway to route some of your traffic to this new version. You can keep increasing the percentage of traffic to your new service and check if everything is working as expected.

Finally, you can route all traffic to your new API.

APISIX uses the traffic-split plugin that lets you control the traffic to your services. You can use it to set up canary releases or your custom release configuration.

Circuit Breaking

When one of your upstream services is unavailable or is experiencing high latency, it needs to be cut off from your system. Otherwise, the client will keep retrying the request, leading to resource exhaustion. This failure can creep into other services in your system and bring them down.

Like how electrical circuit breakers isolate faulty components from a circuit, API gateways have a circuit breaker feature that disconnects faulty services, keeping the system healthy. Traffic to these services are rerouted or delayed until the service becomes healthy.

APISIX comes with an api-breaker plugin that implements this pattern.

Redirects

As you update your APIs, their endpoints might undergo some change. Traditionally, this would mean that the client application should send requests to the /new-api-endpoint instead of the /old-api-endpoint, meaning your consumers must manually change each call to this API endpoint.

If unanticipated, this can break client applications.

With an API gateway, you can provide an abstraction layer and redirect requests to the /new-api-endpoint without having the clients change their requests. With proper redirect status codes and messages, you can gradually depreciate the /old-api-endpoint without your consumers experiencing any downtime.

With APISIX, you can use the redirect plugin to configure redirects.

Conclusion

When reliability becomes a primary concern, it is evident that API gateways are necessary as more organizations split their monoliths into microservices and move to cloud native architectures.

However, this does not mean API gateways are for everyone. Depending on your API's size and usage, an API gateway might be overkill, and you can get away with using a reverse proxy with basic routing and load balancing capabilities.

The use cases mentioned here only scratch the surface of an API gateway's capabilities. You can learn more about API gateways and Apache APISIX at apisix.apache.org.

How and Why I Migrated My Blog From GitHub Pages to Netlify

Navendu Pottekkat — Thu, 25 Aug 2022 12:07:29 +0000

What GitHub Pages Lacked
Enter Netlify
How I Migrated to Netlify
- Create a Netlify Account
- Import Your Website
- Changing the Website URL and Custom Domains
- Configuring Deploy Previews
Features I Might Add in the Future
- Form Handling
- Split Testing
- CDN/git LFS
Too Good to be Free?

My blog is now deployed on Netlify ! I spent a little more than an hour the other day migrating it from GitHub Pages to Netlify.

GitHub Pages is a perfect solution for deploying static websites. But, it made it challenging to implement some of the features I wanted on my blog.

So, on impulse and to procrastinate from finishing a blog post, I migrated the site to Netlify!

What GitHub Pages Lacked

GitHub Pages has been my go-to static website deployment solution for the past three years. And it worked like a charm even if it was free.

I have been looking for ways to set up my Hugo website to show draft blog posts. The problem was that I didn't want these drafts to show up on the homepage listing, but I also wanted a sharable link for people to review.

There are ways I can set this up, but even if I do that, there aren't any ways for people/reviewers to leave feedback on my static website.

I also used a lot of client-side redirects, which is not desirable as opposed to server-side redirects. GitHub Pages did not provide a way for you to configure server-side redirects. So I had to use these hacky, Jekyll redirects on a subdomain. It works, but it could be better.

Enter Netlify

Netlify has deploy previews . So, when you make a pull request to your production branch, Netlify will build the site for you and show a preview of what the change will look like.

Deploy preview feature in Netlify

From github.com/navendu-pottekkat

You know what this is best for? Previewing draft blog posts!

Using Netlify, I can open a pull request with my draft post, and Netlify will generate a preview build of the site without affecting my production site. Reviewers can see the preview and suggest edits as comments on GitHub.

Netlify also brings a better continuous integration experience.

CI checks run by Netlify

From github.com/navendu-pottekkat

Netlify lets you configure the build settings through the Netlify UI or a configuration file (netlify.toml). This gives you a lot more control than the few configuration options GitHub Pages provides.

Netlify dashboard for navendu.me

I added a Plugin and to lint links and it broke the CI because there were a lot of broken links. Task for another day.

With Netlify, configuring redirects is as easy as adding two lines to your configuration file. You also have the option to configure the proper status code. Now that's neat.

Netlify also has a DNS service, supports storing large media (Git LFS), has split testing and rollback features, and more . But, these are only "nice to have" features for me now. I will not be using these anytime soon.

I might use these sometime soon.

How I Migrated to Netlify

The process was pretty straightforward. It only took me a little over an hour to set everything up and test.

In the steps below, I have the following setup:

A Hugo blog published on GitHub Pages.
A custom domain registered on Hostgator (also my DNS service).

You can always refer to the official Netlify docs if you have a different setup somewhere along this guide.

Create a Netlify Account

First, you must create a Netlify account if you don't have one.

Create your Netlify account

Do you also use GitHub for everything?

Import Your Website

You can now add your website to Netlify. You should have the code for your website on a Git provider. If you don't have one, now is the time to git push your code.

Click on "Add new site" and "Import project".

Importing your site
Select the Git provider where you have your website code.

Select your provider
Pick the repository from the Git provider after granting access to Netlify.

Pick the repository to deploy
You can then configure the build settings based on your blog engine. Since I'm using Hugo, I will add the following configurations.

Build settings for my Hugo website

--gc cleans up old resources, --minify reduces the size of the files and the public folder is where Hugo outputs the build

And voila! Netlify will automatically build your first deployment. Now, you will be able to see the production URL for your website.

Production URL of your website

This screenshot is from Netlify's docs

This URL is unique; you can change it to yoursitename.netlify.app or your custom domain.

Changing the Website URL and Custom Domains

You can change your site URL to anything unique. Once changed, this will be your website's address.

From the "Site overview" page, Go to "Domain settings".
Click on the "Options" next to the site name and click "Edit site name".

Edit the generated site name

kung-fu-panda-23 is a cool name
Change the site name and save.

Note: If you use this URL, you might need to change the baseURL in your Hugo configuration file to ensure all the links work.

In my setup, I'm using Hostgator as my DNS service and not the Netlify DNS service.

In "Domain settings", click "Add custom domain".
You can enter the domain you already own or enter something new and purchase the domain. Netlify will set everything up if you are buying a domain.
If you are using a domain you already own with an external DNS provider, you will see a warning. You can ignore that and click on "Add domain".
You can also set up an SSL certificate from the HTTPS section.

Secure your site with an SSL certificate

If you use a different DNS service, you need to configure it to point your domain to your Netlify website. You can check the Netlify docs if you have a different setup than mine.

Login to your domain's control panel and open up your DNS configuration. I'm using Hostgator for my domain.
Create an A Record with your apex domain and point it to Netlify's load balancer IP address 75.2.60.5.
Then, create a CNAME record for the wwwsubdomain and point it to your website address yoursitename.netlify.app.

Now, you must wait for the changes to propagate, and you will have your domain configured.

Configuring Deploy Previews

The main reason to migrate was the deploy previews feature. And it is super easy to set up.

Go to "Site settings".
Select "Build & deploy" from the side menu and then "Continuous Deployment".
Scroll down to "Deploy previews" and set it up as desired. I have enabled it for any pull request against my production branch, and I have also enabled the Netlify Drawer.

That is it. You now have deploy previews!

Note: To ensure that the deploy previews show drafts, I updated my Netlify configuration file (netlify.toml) to change the build command for deploy previews.
[context.deploy-preview]
  command = "hugo --buildFuture --buildDrafts --gc --minify -b $DEPLOY_PRIME_URL"
Here $DEPLOY_PRIME_URL is an environment variable that Netlify sets, used to update the site's baseURL.

That brings an end to my current setup. It is much better than my earlier setup with GitHub Pages and was pretty easy to migrate. I would definitely recommend Netlify for your static websites.

Features I Might Add in the Future

This was my first iteration with Netlify. I have a basic setup that more or less does everything I need. But, I might use these other features if they are fruitful.

Form Handling

I use Mailchimp to handle subscriptions to this blog. If I can find a way to send mass emails, I might set up the free form handling offered by Netlify.

Split Testing

A/B test blog posts? Yes!

CDN/git LFS

My blog contains a lot of images. I do my best to compress these images, but I will reach a point where the images are taking too much space, increasing the repo size. If Netlify's solution is better, I might switch to that.

Too Good to be Free?

Netlify seems too good to be free. I'm on the free tier, and it appears to be generous.

My 24 hour Netlify usage

Petition to add a section that shows the carbon footprint on my blog builds. Take that, people flying on private jets!

But, it will only be some time until I pass these limits and would end up needing to pay for the service. It is not too much money but seeing that the alternative, GitHub Pages, is free, I cannot stop thinking, "maybe I don't need deploy previews".

I hope I don't have to write a post titled "How and Why I Migrated My Blog Back From Netlify to GitHub Pages"!

How I Ask Questions as a Software Engineer

Navendu Pottekkat — Sat, 25 Dec 2021 16:19:17 +0000

Cover Photo by National Cancer Institute on Unsplash

I ask a lot of questions to my peers and to strangers on public forums in the internet. This year, I have been trying to improve this process to ask better questions. Here is how I do it.

But first of all,

What are Good Questions?

Good questions are the ones which are easy to answer.

Our goal for asking a question is to have the other person explain what they know in a way you can understand. A series of good questions is the key to a good answer.

Bad:

J: What happens when we strip the binaries? (Too vague and broad)

N: Stripped binaries don’t have debugging information. So its size is reduced ... (Answers with a lot of irrelevant information)

Good:

J: I see that we are stripping the binaries to reduce its size before publishing. I found that it shouldn’t affect the performance. Is that right? What other implications does this have? (Clear question, easy to answer)

N: Stripping only removes the debugging information. It wouldn’t affect the performance in any way. But it will be difficult to debug if we run into any issues as debug symbols are removed from the traceback.

The Problem with Bad Questions

Bad questions can derail a conversation easily.

For me, asking bad questions have often resulted in:

the person explaining things irrelevant to my question.
the person explaining things I have no clue of.
the person explaining what I already know.
the person not answering the question at all (especially for under researched questions).

All of this boils down to you or both of you walking away frustrated and without a clear answer.

At this point it should be fairly obvious why you should focus on asking questions properly. So, here is my process.

Who are you Asking?

Who you are asking a question should impact how you ask the question. Let me explain.

If you are asking your coworker who works on your project or is familiar of the particular niche, you can fairly assume that the person has some context on what you are asking.

This means that there would be less things for you to explain and you can build your explanation from your shared knowledge. But it is a different game when you are asking questions to the people of the interwebs.

I have had my share of bashing from people in Stack Overflow when I began programming. I get that having a high bar for quality assurance helps Stack Overflow be the go to place to ask questions but some of the moderators are so trigger happy that they will shoot you (your question) down right away.

But anyway, the important thing to remember here is that the person reading your question has very little context about your situation. It is obvious when a person has put very little effort in asking questions and these questions are the first to get the bashing.

When to Ask?

If you have a lot of questions or if you think answering your question will take time, it is better to schedule a time when you are both available.

If your questions are quick, it is better to ask them right away if it ends up saving you a lot of time.

Google First, Ask Later

One of my biggest pet peeve is people who ask technical questions that can be answered by the first result of a Google search. It shows little effort from their part and now I just ask people to Google it and do not bother to answer until they do their homework.

I maintain a project called Meshery and one of the new contributors (who came in to get a GSoC internship) literally asked if I could explain what Meshery is.

We have a website, 100+ pages of documentation, recordings of conference talks and technical documentation, all sent to the user as they join the community.

You know how that conversation went.

It would have been different if they had asked me something along the lines of “I have been going through Meshery’s docs and been trying it out locally. I’m not clear how Meshery adds value if a person is already using a service mesh. Could you point me to any docs where this is explained better?”.

Think for a moment on how you would have answered in these scenarios.

Doing a bit of research can help you build some foundational knowledge to ask a set of better questions.

The “Google first, ask later” motto is only good as a rule of thumb. Nothing has stopped me from asking obvious, googleable (it is a real word) questions when in conversation with someone.

To sum it up, take some effort, do your homework and then ask your questions. Don’t expect to be spoon-fed.

Is that Right?

Let’s go back to the “stripped binary” example.

J: I see that we are stripping the binaries to reduce its size before publishing. I found that it shouldn’t affect the performance. Is that right? What other implications does this have?

See how stating what you already lets you build the rest of conversation?

To ask this question, you have to spend some time and dig through what a stripped binary is and how it is different from a "normal" binary. The time taken to understand and formulate that question is time well spent.

On the receiving end, the person will see that you have spent time in this and is not just asking them to do your work. It will also be easier to answer your question building by on your foundational knowledge.

Vague Precise Questions

J: How do I use a Kind cluster to setup my development environment?

If you ask me this, I would reply with a link to the Kind docs. But this wasn’t what they intended to ask. So they say,

J: I tried this but it is not working.

Well, there are million different reasons for this to not work. I am not Doctor Strange to evaluate all the possibilities in a second! A little bit more context might help.

I will cut to the chase and say how I would ask this question.

N: I was trying to setup Kind for my local development environment. I am on macOS. I have Docker Desktop and Kind running. I have also setup Metallb LoadBalancer and I see the external IP of the service as shown on the logs below. Still, I am not able to reach it from my host machine. Is there something I’m missing?

Then that senior engineer with years of experience can jump right in and say,

S: On macOS, Docker does not expose the docker network to the host. You can try port-forwarding to reach the pods.

See how easy it was to answer?

This goes for all questions. The more precise you are with your questions, more easy it is to answer.

This also prevents the person answering from going off on a tangent explaining irrelevant details which you may either not care about or aren’t relevant to your actual question.

Another way to prevent shooting off on a tangent is to ask questions that can be answered by a simple yes/no.

J: ~~Why are we using this gRPC middleware instead of directly calling the required service?~~

J: Are we using this gRPC middleware to convert between two different configuration formats?

N: Yes.

The person usually goes to explain why yes/no after this but these questions are easy to answer and I almost always get quick responses.

These questions are quite useful when you are in conversation with a person where they are explaining something to you. This segues into my next point.

When in Doubt, Ask More Questions

Imposter syndrome is real.

When I started working with other people, I often stopped myself from saying “I don’t understand” thinking I would look stupid.

I have then come to learn that if you ask a “stupid” question, you are stupid for the day but if you don’t, you are stupid for life (because you will always stop yourself from asking questions, ending up not understanding things completely... umm, you get it right?).

This means when you get an answer and you are not completely satisfied,

say what you don’t understand.
ask more clarifying questions.
stop the speaker and ask more specific questions.

Confronting the imposter syndrome is hard but it has been helpful to me in knowing that everyone else face this too.

When you start thinking “maybe I’m just not smart enough to understand the answer”, remember that people want to help you. You just have to help them help you!

Learning in Public

Ask questions in a public channel instead of DMs.

This may not work in every situation but I try to do this more often now.

This will document the discussions publicly and would also help any others looking in. You can then always point people to this discussion if they ask the same question.

Take Stack Overflow for example. You almost always find answers to problems you are facing from questions asked by someone else.

The imposter syndrome shifts to the next gear here. Face it head-on.

Asking Good Questions is a Skill

And like all skills, it is sharpened with practice.

Asking the right questions will help you extract the answers you want. In most scenarios, it is not that the person answering is incapable, but you are not asking the right questions.

I have gotten better at this over the year and I am still working out the kinks in my process.

This might be a good post to come back in a year to reflect and improve upon.

To summarise this post in a sentence,

Make it easy for people to help you.

Contributing to Documentation in Open-Source

Navendu Pottekkat — Mon, 08 Nov 2021 12:20:58 +0000

Contributing to open-source can be overwhelming.

You are creating a pull request to an open-source project that would open up your code for people to give feedback and criticise.

This turns off a lot of new contributors from making an impact in open-source.

You might also be at a point where you are not confident enough in your programming skills that you hesitate to take that first step in contributing code.

A good way to tackle this and gain confidence to contribute code is to start by contributing to documentation.

Documentation is necessary for every open-source project. Contributing to and maintaining documentation is not an easy task but is highly impactful.

I will drop a bomb and say contributing to documentation is more important than contributing code.

In this post, I will try to share my insights on contributing to documentation from my experience as an open-source contributor and a maintainer.

Being overwhelmed when you first start to contribute to open-source is natural. Start small. But start.

Finding a Project

The first thing to do before you can start contributing is finding a project you can contribute to. And the best project to contribute to is the one you have been using for a while.

That is, if you have been using a framework or a library, a tool or any other open-source project, contribute to it.

With this, you will have a lot of context on what the project is and you will definitely be able to find areas in the documentation you can improve.

If you cannot find such a project to contribute to, look for projects with an active community of contributors.

Community is key in open-source and you will definitely reap the rewards for being part of a thriving community.

All these factors comes secondary to the fact that you should always contribute to projects that YOU are interested in.

Start as a User

As mentioned in the above section, you will be able to make impactful contributions if you are already a user of the project.

So, if you aren't, you should start by exploring the project from the perspective of a user.

As a user, you will likely go through documentation and tutorials as you start out and you are likely to find bugs, out-dated content or things you can improve.

When you find areas to improve, open up issues (or any form of tickets if you are not using GitHub) for these and discuss with a project maintainer to validate it and get it assigned to you.

Once you have been assigned an issue, you can start contributing.

Review the Contributing Guidelines

Most (if not all) open-source projects will have a contributing guideline in their GitHub/GitLab repository.

This document will be geared towards contributors with guidelines on setting up a developer environment and how you can make contributions.

Read these guidelines carefully and make sure that you follow them.

For example, if a project requires you to sign every commit, your pull requests will be rejected very quickly if you do not do it.

What should you Document?

As mentioned in the previous sessions, you are likely to find issues when you start to use the project referring to the documentation.

This could be as simple as an outdated screenshot to outdated or missing documentation.

Open up issues for these as mentioned and discuss it with a maintainer to get it assigned to you.

If you cannot find issues by yourself, you can try to fix already open issues.

There are labels in GitHub issues that can help you filer out only "documentation" or "docs" issues and similar features will be there in any ticketing system being used by the project.

In-Code Documentation

People generally don't think of this when they think about documentation.

In-code documentation refers to the error messages, help texts and other text the user interacts with that doesn't necessarily affect the "logic" of the code.

These are really important as users will definitely interact with this while using the project.

As a new contributor you have the magic eyes to spot issues with these that seasoned contributors are too close to notice.

More Ways to go Beyond the Docs Page

Good documentation is not just limited to the docs page.

It can also involve:

Writing a blog post that takes the user through a new feature
Creating a Twitter thread about the project
Documenting processes that can be used for the community (a contributing guide for example)
Updating the project's website

You can always ask the project maintainers to help you find areas that need contributions.

Before we finish this, I want to point out that non-code open-source contributions are not just limited to writing documentation.

You can check out other ways you can contribute to open-source projects without contributing code.

Article No Longer Available

Originally published on navendu.me

How to Make Non-Code Contributions to Open-Source Projects

Navendu Pottekkat — Sun, 19 Sep 2021 02:48:56 +0000

Do you want to contribute to open-source but you are not sure of your coding skills?

I have good news for you.

Non-code contributions are more valuable than contributing code.

But what are non-code contributions? How can I make non-code contributions? Why are you bashing coders?

I will try to answer these questions in this post.

And with Hacktoberfest right around the corner, I hope this post will encourage more open-source contributions.

Like most of my posts, you can read this in Tweet format like the gods intended here.

What are non-code contributions?

Any contribution that helps an open-source project that does not involve writing code.

These can be anything but I have categorised it broadly into these categories:

✍️ Writing
🎨 Designing
🧑‍🔬 Testing/Using
👥 Mentoring
🧑‍🤝‍🧑 Community Managing/Organising

Let's look at each of these categories in detail.

1. ✍️ Writing

If you are good at technical writing and creating content, then this is for you.

Here is how you can use your writing skills.

1.1 Write Documentation

Contributing to documentation is really impactful.

Even if the project is really amazing, without documentation, nobody would be able to figure out how to use it or at least would not get maximum value from the project.

Here is a tip to help you contribute to docs:

When you first check out an open-source project, go through their existing documentation and see if you can understand or use the project from what you see in the docs.

Most likely, you will find areas that you can improve.

Open issues for these and try to create a pull request adding changes that you think are needed.

It is always helpful for projects to have a new pair of eyes look at things. New perspectives and new ideas often come from new contributors.

1.2 Write Tutorials and Blog Posts

If you have been a user of the project and you are looking to contribute to it, then this is the best way.

You can write tutorials and blog posts to help people who are using the project.

You can also share your own insights and tips while writing these.

If the project has a dedicated blog, you can ask to publish there or there are other platforms like this where you can freely publish content.

I find myself looking for well-written and up-to-date tutorials and blogs instead of going through the documentation.

There is no doubt that these will help the project as well as the users of the project.

1.3 Translate Documentation

People from different parts of the world could use your project.

Language barrier is real and sometimes people may prefer Mandarin over English.

So, if you know Mandarin and English or any other pair of languages, translate the documentation to the other language.

This will open up the project to a whole new user base.

1.4 Take Control of the Social Handles

If your project has social media accounts, take control of it and start creating content.

Even if you directly do not have access to the accounts, get in touch with the social media manager of the project and suggest content and ways to engage with your users and community.

This will drive the project's visibility and will help gain more users and contributors.

2. 🎨 Designing

If you are a creative person and loves designing stuff, this is where you belong.

Here is how you can use your palette to help the project.

2.1 Create Art!

You can put your skills to use to create artworks for the project to share on social media, blog posts and even swags. (who doesn't like swags!)

2.2 Create a Style Guide

Most open-source projects have a large and a geographically distributed set of contributors.

Among other challenges, this makes it difficult to maintain consistency in visual designs.

As a designer and a contributor, you can create a style guide to ensure consistency.

3. 🧑‍🔬 Testing/Using

As a user of the open-source project, there are multiple ways in which you can make valuable contributions.

3.1 Reporting Bugs

If you run into any bugs while using/testing the project, raise them.

Open up an issue and let the code contributors know about the bug.

This will help them identify and fix bugs improving the quality of the project.

3.2 Become and Advocate/Evangelist

Most open-source projects are community-driven and they usually do not have a dedicated marketing team to publicise the project.

This is where you step in as a volunteer advocate and talk about the project in events and social media while encouraging people to use the project.

Here is a tip:

Don't even ask the project owners before advocating for the project.

Everyone likes free marketing.

3.3 Help Improve the User Experience

Most open-source project maintainers/code contributors are usually too close to the project to realise bad and unintuitive UX.

I am and I know a lot of them.

Step in here wearing your user hat and report these issues.

They are always well received.

3.4 Sign Up for Alpha/Beta Testing

Alpha/Beta tests are controlled tests of a new feature or a release to ensure quality and user experience before making it available to the general user base.

As a user, you can sign up for the alpha/beta programs and test out the project before the new features are released and provide feedback.

Feedback from these tests always provide insights that can help in iterating over the features/releases.

4. 👥 Mentoring

Pay it forward by mentoring other contributors.

Here are some of the ways in which you can do it.

4.1 Review Code

The more people there is to review code, the better the quality of the code will be.

Having more eyes on the code would mean less bugs, faster reviews and a better project.

Most projects let anyone review pull requests and comment on it.

If you have opinions, this is the place for you to go.

4.2 Mentor a Contributor

Are you skilled in the project's tech stack?

Can you help contributors make better contributions?

Pay it forward by becoming a mentor!

Share your experiences and knowledge and empower them to become a better open-source contributor.

There are a lot of open-source programs which connect mentors and mentees.

I had curated some of these in a previous article. You can check that out here.

5. 🧑‍🤝‍🧑 Community Managing/Organising

Open-source projects thrive as its community thrives.

But who manages this community? Who manages the contributors? Who manages the project?

This is where a community manager comes in.

5.1 Organise the Project

Stale issues? Follow up!
Issues without proper labels? Add them!
Does this issue still exist? Verify and close them!
Unclear issue descriptions? Ask for clarification!
Unreviewed pull requests? Request for reviews!

Help in every way you can to organise the project and make it whistle as a well oiled machine.

From my experience, project organisers and community managers are well recognised in the community as well!

5.2 Become the Release Manager

A release manager keeps track of what everybody is working on and ensures that a project is ready for a release.

Some of the responsibilities of a release manager are:

Checking with each team
Ensuring different components and features are tested
Organising the alpha/beta programs

This list can go on. Basically you decide when the project is ready to make a new release.

5.3 Organise Events and Meetups

Organise project meetings.
Represent your project in conferences.
Organise community events.

This list is note exhaustive.

Great community managers go above and beyond and are key players in the success of an open-source project.

5.4 Get More Chefs to Cook (I mean, non-code contributors)

Engineers attract other engineers.

But the environment these engineers create are usually not super welcoming to non-code contributors.

So, become a steward of the community and make it welcoming to newcomers!

Host newcomer meetings, share onboarding resources, connect newcomers to mentors.