Creating a web API with Lua using Nginx OpenResty

#lua #nginx #openresty #lapis

I'm a bit of an old fashioned, pragmatic developer and tend to stick with frameworks and patterns that I know are reliable - a 'stick in the mud' if you will.

This dive is deviating from my usual path.

I had never considered Lua to be a viable language to create a backend for a web application. Of course I'm not totally oblivious to the language itself - I do know it's used mostly for embedding in other software and game engines - but that's it really.

I came across the OpenResty project whilst on a learning journey of Nginx web server.

Writing code directly on the web server rather than passing the request to some interpreter is a very interesting concept for me.
Lua runs directly inside the Nginx worker which means a very small barrier between the webserver and the application code.
Additionally, Lua is known for being immensely fast compared to interpreted languages particularly when using the LuaJIT compiler.

Nginx is capable of handling a very high number of concurrent connections at a very low memory footprint.

However, there doesn't appear to be that much excitement about it.
It's been around for some time (at least since 2016) but it does appear to be lacking in developer adoption. Saying that, some big names are using it, including Cloudflare and Tumbler.

Despite that - the performance gain from this sort of backend has peaked my interest.

The easiest way (perhaps the only way) to get started with this is by installing OpenResty.
This provides the Nginx web server with the Lua module.

If you're familiar with Nginx this example will make sense to you. The location directive is being passed a Lua script as a file.
The output from that script is expected to be in JSON in this configuration:

location ~ ^/api(.*)$ {
  default_type 'text/json';
  add_header 'Content-Type' 'application/json';
  content_by_lua_file /etc/openresty/sites/api.lua; # < Points to Lua file
}

You can also pass arbitrary code into content_by_lua and content_by_lua_block.

Now, Lua is a reasonable simple language to pick up if you're used to Ruby or Python. I hadn't used it before but I was still able to quickly write a script that allows creation of API endpoints and parses the body and parameters.

There is an object you can access in Lua called ngx. This provides you with the data passed into Nginx from the request such as the body or the path etc...

Example:

ngx.var.request_method -- POST, GET.. whatever
ngx.req.get_body_data() -- The data passed in from the request
ngx.var.uri -- The request path

The script I wrote allows me to set the Method and Endpoints that are allowed.

--[[ api.lua --]]

-- Helper functions
function strSplit(delim,str)
    local t = {}

    for substr in string.gmatch(str, "[^".. delim.. "]*") do
        if substr ~= nil and string.len(substr) > 0 then
            table.insert(t,substr)
        end
    end

    return t
end

-- Read body being passed
-- Required for ngx.req.get_body_data()
ngx.req.read_body();
-- Parser for sending JSON back to the client
local cjson = require("cjson")
-- Strip the api/ bit from the request path
local reqPath = ngx.var.uri:gsub("api/", "");
-- Get the request method (POST, GET etc..)
local reqMethod = ngx.var.request_method
-- Parse the body data as JSON
local body = ngx.req.get_body_data() ==
        -- This is like a ternary statement for Lua
        -- It is saying if doesn't exist at least
        -- define as empty object
        nil and {} or cjson.decode(ngx.req.get_body_data());

Api = {}
Api.__index = Api
-- Declare API not yet responded
Api.responded = false;
-- Function for checking input from client
function Api.endpoint(method, path, callback)
    -- If API not already responded
    if Api.responded == false then
        -- KeyData = params passed in path
        local keyData = {}
        -- If this endpoint has params
        if string.find(path, "<(.-)>")
        then
            -- Split origin and passed path sections
            local splitPath = strSplit("/", path)
            local splitReqPath = strSplit("/", reqPath)
            -- Iterate over splitPath
            for i, k in pairs(splitPath) do
                -- If chunk contains <something>
                if string.find(k, "<(.-)>")
                then
                    -- Add to keyData
                    keyData[string.match(k, "%<(%a+)%>")] = splitReqPath[i]
                    -- Replace matches with default for validation
                    reqPath = string.gsub(reqPath, splitReqPath[i], k)
                end
            end
        end

        -- return false if path doesn't match anything
        if reqPath ~= path
        then
            return false;
        end
        -- return error if method not allowed
        if reqMethod ~= method
        then
            return ngx.say(
                cjson.encode({
                    error=500,
                    message="Method " .. reqMethod .. " not allowed"
                })
            )
        end

        -- Make sure we don't run this again
        Api.responded = true;

        -- return body if all OK
        body.keyData = keyData
        return callback(body);
    end

    return false;
end

Then I can call Api.endpoint() following to create my endpoints:

--[[ api.lua --]]
Api.endpoint('POST', '/test',
    function(body)
        return ngx.say(
            cjson.encode(
                {
                    method=method,
                    path=path,
                    body=body
                }
            )
        );
    end
)

Api.endpoint('GET', '/test/<id>/<name>',
    function(body)
        return ngx.say(
            cjson.encode(
                {
                    method=method,
                    path=path,
                    body=body,
                }
            )
        );
    end
)

So this has got me started. Obviously the logic inside these endpoints would be much more complex and usually have to connect to databases - all of which OpenResty supports.

One thing I did come across was the Lapis Framework created by Leafo who I know mostly from creating the PHP compilers for Less and SCSS.

Github for code the code here - https://github.com/bambattajb/openresty-api-example,