DEV Community

Cover image for Building a Custom NGINX Module with Ansible: A Dev-Friendly Guide
Athreya aka Maneshwar
Athreya aka Maneshwar

Posted on • Edited on

Building a Custom NGINX Module with Ansible: A Dev-Friendly Guide

#webdev #programming #beginners #devops

Hello, I'm Maneshwar. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product.

Hi there! I’m building a first-of-its-kind tool that helps you automatically index API endpoints across all your repositories. makes it easier to discover, understand, and interact with APIs in large infrastructures.

NGINX is a powerful and versatile web server, and its extensibility through modules is one of its greatest strengths.

Sometimes, however, you need functionality that isn't available out-of-the-box or as a pre-compiled dynamic module.

This is where custom NGINX modules come in.

In this blog post, we'll walk through a real-world scenario: building NGINX with the ngx_http_consul_backend_module and the ngx_devel_kit (NDK) module.

We'll leverage Ansible to automate the entire process, making it repeatable, reliable, and dev-friendly.

Why Ansible for NGINX Builds?

Manually compiling NGINX with custom modules can be a tedious and error-prone process.

Dependencies, compilation flags, and directory structures all need to be precisely managed.

Ansible simplifies this significantly by:

  • Idempotency: Tasks can be run multiple times without causing unintended side effects.
  • Automation: Automate repetitive tasks, saving time and reducing human error.
  • Version Control: Store your build process in version control, enabling easy collaboration and rollbacks.
  • Consistency: Ensure consistent builds across different environments.

Our Goal: NGINX with Consul Backend Module

We'll be building NGINX 1.23.2 and integrating two specific modules:

  1. ngx_devel_kit (NDK): A collection of utilities and APIs that simplify the development of NGINX modules. Many custom modules rely on NDK.
  2. ngx_http_consul_backend_module: A module that allows NGINX to discover backend services from HashiCorp Consul. This is particularly useful in dynamic, microservices-based environments.

Project Structure

Our Ansible project is organized as a role named nginx-with-consul-module.

This structure promotes reusability and maintainability.

ansible
├─ README.md
├─ ansible.cfg
├─ hosts.ini
├─ install_ansible.sh
├─ nginx-build-playbook.yml
└─ roles
   └─ nginx-with-consul-module
      ├─ tasks
      │  ├─ build_consul_backend_module.yml
      │  ├─ build_nginx.yml
      │  ├─ configure_build.yml
      │  ├─ download_sources.yml
      │  ├─ install_dependencies.yml
      │  ├─ main.yml
      │  ├─ purge_deps.yml
      │  └─ systemd.yml
      └─ templates
         └─ nginx.service.j2
Enter fullscreen mode Exit fullscreen mode

The main.yml in the tasks directory orchestrates the entire build process by importing other task files:

---
- import_tasks: purge_deps.yml
- import_tasks: install_dependencies.yml
- import_tasks: download_sources.yml
- import_tasks: build_consul_backend_module.yml
- import_tasks: configure_build.yml
- import_tasks: build_nginx.yml
- import_tasks: systemd.yml
Enter fullscreen mode Exit fullscreen mode

Let's break down each step.

Step 1: Install Dependencies (install_dependencies.yml)

Building NGINX and its modules requires several development tools and libraries.

This task ensures all necessary packages are present on the target system.

---
- name: Install required packages
  apt:
    name:
      - build-essential # For gcc, g++ and make
      - libpcre3
      - libpcre3-dev # For PCRE regular expressions support
      - zlib1g
      - zlib1g-dev # For gzip compression support
      - libssl-dev # For OpenSSL (HTTPS) support
      - git # To clone the Consul backend module
      - wget # To download source archives
      - curl # General utility
    state: present
    update_cache: true
Enter fullscreen mode Exit fullscreen mode

Step 2: Download Sources (download_sources.yml)

Before we can compile anything, we need the source code for NGINX, NDK, and the Consul backend module.

We'll download these to a temporary directory.

---
- name: Create build directory
  file:
    path: /tmp
    state: directory

- name: Download nginx 1.23.2 source
  get_url:
    url: https://nginx.org/download/nginx-1.23.2.tar.gz
    dest: /tmp/nginx.tgz

- name: Extract nginx source
  unarchive:
    src: /tmp/nginx.tgz
    dest: /tmp/
    remote_src: yes

- name: Download NDK module
  get_url:
    url: https://github.com/simpl/ngx_devel_kit/archive/v0.3.0.tar.gz
    dest: /tmp/ngx_devel_kit-0.3.0.tgz

- name: Extract NDK module
  unarchive:
    src: /tmp/ngx_devel_kit-0.3.0.tgz
    dest: /tmp/
    remote_src: yes

- name: Clone Consul backend module
  git:
    repo: https://github.com/hashicorp/ngx_http_consul_backend_module.git
    dest: /go/src/github.com/hashicorp/ngx_http_consul_backend_module
Enter fullscreen mode Exit fullscreen mode

A crucial step here for the Consul backend module is a small patch.

The original code's strlen call with backend might cause a warning or error with newer compilers due to type mismatch.

We're explicitly casting backend to (const char*) to resolve this.

- name: Replace backend string length calculation
  replace:
    path: /go/src/github.com/hashicorp/ngx_http_consul_backend_module/src/ngx_http_consul_backend_module.c
    regexp: 'ngx_str_t ngx_backend = { strlen\(backend\), backend };'
    replace: "ngx_str_t ngx_backend = { strlen((const char*)backend), backend };"
Enter fullscreen mode Exit fullscreen mode

Step 3: Build Consul Backend Module (build_consul_backend_module.yml)

The ngx_http_consul_backend_module is written in Go and needs to be compiled as a C shared library. This involves several steps:

---
- name: Ensure nginx ext directory exists
  file:
    path: /usr/local/nginx/ext/
    state: directory
    owner: "{{ ansible_user | default('root') }}"
    group: "{{ ansible_user | default('root') }}"
    mode: "0755"

- name: Change ownership of Consul backend module directory
  file:
    path: /go/src/github.com/hashicorp/ngx_http_consul_backend_module
    state: directory
    owner: "{{ ansible_user | default('root') }}"
    group: "{{ ansible_user | default('root') }}"
    recurse: yes

- name: Check Go version
  shell: export PATH=/usr/local/go/bin:$PATH && /usr/local/go/bin/go version
  register: go_version_result
  changed_when: false
  failed_when: false

- name: Initialize Go modules
  command: /usr/local/go/bin/go mod init github.com/hashicorp/ngx_http_consul_backend_module
  args:
    chdir: /go/src/github.com/hashicorp/ngx_http_consul_backend_module
  register: go_mod_init_result
  changed_when: go_mod_init_result.rc == 0
  failed_when: go_mod_init_result.rc != 0 and "go.mod already exists" not in go_mod_init_result.stderr
  ignore_errors: true # Ignore if go.mod already exists

- name: Tidy Go modules
  command: /usr/local/go/bin/go mod tidy
  args:
    chdir: /go/src/github.com/hashicorp/ngx_http_consul_backend_module

- name: Print Go version
  debug:
    msg: "{{ go_version_result.stdout }}"

- name: Build Go shared library for Consul backend module
  shell: |
    export PATH=/usr/local/go/bin:$PATH
    CGO_CFLAGS="-I /tmp/ngx_devel_kit-0.3.0/src" \
    /usr/local/go/bin/go build -buildmode=c-shared -o /usr/local/nginx/ext/ngx_http_consul_backend_module.so ./src/ngx_http_consul_backend_module.go
  args:
    chdir: /go/src/github.com/hashicorp/ngx_http_consul_backend_module
Enter fullscreen mode Exit fullscreen mode

Key points:

  • We create a dedicated directory /usr/local/nginx/ext/ to store our compiled dynamic module.
  • We ensure correct ownership for the cloned Consul module directory.
  • We initialize and tidy Go modules to manage dependencies.
  • The go build -buildmode=c-shared command is crucial. It compiles the Go module into a C shared library (.so file) that NGINX can load dynamically.
  • CGO_CFLAGS="-I /tmp/ngx_devel_kit-0.3.0/src" is important because the Consul module depends on headers from the NDK, so we need to tell the Go compiler where to find them.

Step 4: Configure NGINX Build (configure_build.yml)

This step involves running the ./configure script for NGINX. This script generates the Makefile based on the desired modules and features.

# Simplified for brevity, but this is where you'd run ./configure
# with all your desired flags and --add-module directives.
# The original configure command from the user's prompt (commented out) would be used here.

# Example of how it would look if it were a separate task:
- name: Configure NGINX build
  command: >
    ./configure
    --prefix=/etc/nginx
    --sbin-path=/usr/sbin/nginx
    --conf-path=/etc/nginx/nginx.conf
    --pid-path=/var/run/nginx.pid
    --lock-path=/var/run/nginx.lock
    --error-log-path=/var/log/nginx/error.log
    --http-log-path=/var/log/nginx/access.log
    --with-http_ssl_module
    --with-http_stub_status_module
    --with-http_realip_module
    --with-http_auth_request_module
    --with-http_v2_module
    --with-http_dav_module
    --with-http_slice_module
    --with-http_addition_module
    --with-http_gunzip_module
    --with-http_gzip_static_module
    --with-http_sub_module
    --with-mail_ssl_module
    --with-stream_ssl_module
    --with-debug
    --add-module=/tmp/ngx_devel_kit-0.3.0
    --add-module=/go/src/github.com/hashicorp/ngx_http_consul_backend_module
  args:
    chdir: /tmp/nginx-1.23.2
Enter fullscreen mode Exit fullscreen mode

Important configure flags:

  • --prefix=/etc/nginx: Sets the installation prefix for NGINX.
  • --add-module=/path/to/module: This is critical for including our custom modules. We point to the extracted NDK and the cloned Consul backend module source directories.
  • Other --with-* flags enable various built-in NGINX modules like SSL, HTTP/2, etc.

Step 5: Build and Install NGINX (build_nginx.yml)

Once configure has generated the Makefile, we can proceed with compilation and installation.

---
- name: Compile nginx
  command: make
  args:
    chdir: /tmp/nginx-1.23.2

- name: Install nginx
  command: make install
  args:
    chdir: /tmp/nginx-1.23.2

- name: Install apache2-utils # Useful for htpasswd, etc.
  apt:
    name: apache2-utils
    state: present
Enter fullscreen mode Exit fullscreen mode
  • make: Compiles the NGINX source code along with the added modules.
  • make install: Installs NGINX and its components to the paths specified during the configure step.

Step 6: Systemd Integration (systemd.yml)

For a production-ready setup, we need NGINX to run as a system service.

This task typically involves creating a systemd service file.

# Example content for systemd.yml, assuming you have nginx.service.j2
---
- name: Copy nginx systemd service file
  template:
    src: nginx.service.j2
    dest: /etc/systemd/system/nginx.service
    owner: root
    group: root
    mode: '0644'
  notify: Reload systemd

- name: Enable nginx service
  systemd:
    name: nginx
    enabled: true
    daemon_reload: true # Ensures systemd picks up the new service file
    state: started

# Handler for 'Reload systemd'
# handlers/main.yml
# ---
# - name: Reload systemd
#   systemd:
#     daemon_reload: true
Enter fullscreen mode Exit fullscreen mode

The nginx.service.j2 template would define how NGINX starts, stops, and reloads as a systemd service.

Step 7: Cleanup (Optional but Recommended - purge_deps.yml)

After a successful build, you might want to remove temporary build files and even some build dependencies to free up space, especially in a containerized environment.

# Example content for purge_deps.yml
---
- name: Clean up temporary build files
  file:
    path: "{{ item }}"
    state: absent
  loop:
    - /tmp/nginx.tgz
    - /tmp/nginx-1.23.2
    - /tmp/ngx_devel_kit-0.3.0.tgz
    - /tmp/ngx_devel_kit-0.3.0
    - /go/src/github.com/hashicorp/ngx_http_consul_backend_module
Enter fullscreen mode Exit fullscreen mode

Running the Ansible Playbook

To execute this entire process, you would have a playbook like nginx-build-playbook.yml:

---
- name: Build NGINX with Consul module
  hosts: your_nginx_servers
  become: yes # Run tasks with sudo/root privileges
  roles:
    - nginx-with-consul-module
Enter fullscreen mode Exit fullscreen mode

And run it with:

ansible-playbook nginx-build-playbook.yml -i hosts.ini
Enter fullscreen mode Exit fullscreen mode

Conclusion

Building NGINX with custom modules can seem daunting, but by breaking down the process into logical Ansible tasks, we can create a robust, automated, and easily repeatable solution.

This approach is invaluable for consistent deployments across development, testing, and production environments.

You now have a solid foundation for integrating any custom NGINX module into your infrastructure using the power of Ansible!

helps you get all your backend APIs documented in a few minutes.

With , you can generate interactive API docs that allow users to search and execute endpoints directly from the browser.

If you're tired of updating manually or syncing collections, give it a shot.

git-lrc
*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

⭐ Star it on GitHub:

GitHub logo HexmosTech / git-lrc

Free, Unlimited AI Code Reviews That Run on Commit



AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

See It In Action

See git-lrc catch serious security issues such as leaked credentials, expensive cloud operations, and sensitive material in log statements

git-lrc-intro-60s.mp4

Why

  • 🤖 AI agents silently break things. Code removed. Logic changed. Edge cases gone. You won't notice until production.
  • 🔍 Catch it before it ships. AI-powered inline comments show you exactly what changed and what looks wrong.
  • 🔁 Build a…
View on GitHub

Top comments (4)

Collapse
 
dotallio profile image
Dotallio •

Love how you broke the process into clear Ansible roles, really makes custom NGINX builds less of a headache. How do you usually handle upgrades for these modules later on - have you automated that part as well?

Collapse
 
lovestaco profile image
Athreya aka Maneshwar •

Thanks

Collapse
 
parag_nandy_roy profile image
Parag Nandy Roy •

This is gold for DevOps beginners and pros alike ...

Collapse
 
lovestaco profile image
Athreya aka Maneshwar •

Haha so true