DEV Community

Mesak
Mesak

Posted on

Building a Composer Proxy Cache with OpenResty

#webdev #php #packagist #proxy

In enterprise environments or CI/CD workflows, frequent execution of composer install or composer update generates numerous external network requests. This article explains how to build an intelligent Composer proxy cache service using OpenResty (Nginx + Lua), while integrating with private Satis package repositories.

Goals and Motivation

Problems Solved

  1. Redundant Downloads: Every CI/CD run re-downloads the same packages from Packagist and GitHub
  2. Network Latency: Cross-region requests to Packagist (Netherlands) and GitHub (USA)
  3. External Dependencies: Deployment pipeline breaks when GitHub or Packagist experiences downtime
  4. Private Package Integration: Need to use both public packages and internal private packages

Why Nginx Cache?

Compared to other caching solutions (Redis, Varnish), Nginx proxy cache offers these advantages:

Feature Nginx Cache Other Solutions
Zero Extra Dependencies Only Nginx needed Requires additional services
Filesystem Cache Stored on disk, survives restarts Memory cache needs warm-up
Automatic Expiration Built-in inactive cleanup Requires custom implementation
Large File Handling Native streaming support Potential memory issues
Operational Complexity Low Medium to High

Using OpenResty extends Nginx with Lua, enabling complex URL rewriting logic while maintaining high performance.

Why Build Your Own? Limitations of Existing Solutions

Several Composer proxy solutions exist, but each has limitations:

Solution Limitations
Packagist Mirror Full mirror requires TB-scale storage, long sync times
Toran Proxy Commercial software, requires paid license
Satis Only supports private packages, cannot proxy public Packagist
Artifactory Enterprise solution, complex architecture, expensive licensing
Pure Nginx Proxy Cannot modify response body, dist URLs still point externally

Advantages of This Approach:

  1. On-Demand Caching: Only cache packages actually used, not the entire Packagist (300,000+ packages)
  2. Zero License Fees: Entirely based on open-source software
  3. Lightweight Deployment: Single Docker container, low resource requirements
  4. Complete Proxy: Both metadata and dist files are proxied and cached
  5. Private Integration: Seamless integration with Satis private repositories

Architecture Overview 

┌─────────────────┐     ┌──────────────────────┐     ┌────────────────┐
│   Composer      │────▶│   OpenResty Proxy    │────▶│ Packagist      │
│   Client        │     │                      │     │ (repo.packagist.org)
└─────────────────┘     └──────────────────────┘     └────────────────┘
                               │
                               ├─▶ Local Satis Static Files
                               └─▶ GitHub/GitLab Download Proxy
Enter fullscreen mode Exit fullscreen mode

Request Flow

  1. Composer sends metadata request to proxy
  2. Proxy checks: Does local Satis have this package?
    • Yes → Return local file directly
    • No → Proxy to Packagist, cache result
  3. Composer parses metadata, requests actual dist files
  4. Proxy forwards dist requests to GitHub/GitLab, caches result

Satis Integration

What is Satis?

Satis is the official static package repository generator for Composer, creating Packagist-compatible JSON format for private packages.

Standard Satis Workflow 

# 1. Configure satis.json
{
    "name": "My Private Packages",
    "homepage": "https://packages.example.com",
    "repositories": [
        { "type": "vcs", "url": "git@gitlab.example.com:team/package-a.git" },
        { "type": "vcs", "url": "git@gitlab.example.com:team/package-b.git" }
    ],
    "require-all": true
}

# 2. Run Satis build
php bin/satis build satis.json ./output

# 3. Output structure
output/
├── packages.json           # Main entry point
├── include/                # Split metadata
│   └── all$xxx.json
└── dist/                   # Package archives
    └── vendor/
        └── package/
            └── package-version-hash.tar
Enter fullscreen mode Exit fullscreen mode

How This Project Integrates Satis

Mount the Satis output directory to /var/www/packagist. The proxy will:

  1. Prioritize Local Files: Serve private packages directly from Satis output
  2. Fallback to Packagist: Proxy public packages to official Packagist
  3. Unified Entry Point: Composer only needs one repository URL

Core Implementation

1. Smart Routing Logic 

-- proxy_logic.lua
local root_path = "/var/www/packagist"
local uri = ngx.var.uri

local function file_exists(name)
    local f = io.open(name, "r")
    if f ~= nil then
        io.close(f)
        return true
    end
    return false
end

local file_path = root_path .. uri
if string.sub(uri, -1) == "/" then
    file_path = file_path .. "index.html"
end

if file_exists(file_path) then
    -- Local file exists, serve directly
    return ngx.exec("@satis_local")
else
    -- Proxy to Packagist
    return ngx.exec("@packagist")
end
Enter fullscreen mode Exit fullscreen mode

2. Dist URL Rewriting

JSON metadata from Packagist contains download URLs pointing to GitHub. To cache these downloads, we rewrite URLs to point to our proxy:

-- body_filter.lua (core logic)
local scheme = ngx.var.http_x_forwarded_proto or ngx.var.scheme
local host = ngx.var.http_host
local proxy_base = scheme .. "://" .. host .. "/dist_proxy/"

local new_body = ngx.re.gsub(body, '"url"\\s*:\\s*"(https?)://([^"]+)"', 
    function(m)
        local proto = m[1]  -- "http" or "https"
        local rest = m[2]   -- domain/path

        -- Only process domains we want to proxy
        if string.find(rest, "^api%.github%.com") or 
           string.find(rest, "^codeload%.github%.com") or
           string.find(rest, "^gitlab%.com") then

           return '"url": "' .. proxy_base .. proto .. '/' .. rest .. '"'
        end

        return m[0]  -- Keep other URLs unchanged
    end, "jo")
Enter fullscreen mode Exit fullscreen mode

Rewrite Example:

Original: https://api.github.com/repos/vendor/package/zipball/abc123
Proxied:  https://proxy.example.com/dist_proxy/https/api.github.com/repos/vendor/package/zipball/abc123
Enter fullscreen mode Exit fullscreen mode

3. Removing available-packages

Satis generates packages.json with an available-packages field, which limits Composer to only query listed packages. Remove this field to allow the proxy to handle all packages:

body = ngx.re.gsub(body, '"available-packages"\\s*:\\s*\\[.*?\\]\\s*,?', '', "jos")
Enter fullscreen mode Exit fullscreen mode

4. Nginx Configuration Highlights 

# Cache configuration
proxy_cache_path /var/cache/nginx 
    levels=1:2 
    keys_zone=composer_cache:50m 
    max_size=10g 
    inactive=7d 
    use_temp_path=off;

# Dist file proxy
location ~ ^/dist_proxy/(https?)/(.*) {
    set $target_proto $1;
    set $target_url $2;

    proxy_pass $target_proto://$target_url;

    # Long-term cache (versioned files are immutable)
    proxy_cache composer_cache;
    proxy_cache_valid 200 1y;

    # Handle GitHub API 302 redirects
    header_filter_by_lua_block {
        local loc = ngx.header.Location
        if loc then
            local proto, rest = string.match(loc, "^(https?)://(.*)")
            if proto then
                ngx.header.Location = scheme .. "://" .. host .. "/dist_proxy/" .. proto .. "/" .. rest
            end
        end
    }
}
Enter fullscreen mode Exit fullscreen mode

5. SSL Termination Support

When deployed behind a reverse proxy, correctly detect the original protocol to satisfy Composer's secure-http requirement:

local scheme = ngx.var.http_x_forwarded_proto or ngx.var.scheme
Enter fullscreen mode Exit fullscreen mode

Frontend Nginx must pass this header:

proxy_set_header X-Forwarded-Proto $scheme;
Enter fullscreen mode Exit fullscreen mode

Caching Strategy

Type Cache Duration Notes
Metadata (JSON) 10 minutes Balance freshness and cache benefits
Dist files (zip/tar) 1 year Versioned files are immutable
Max cache size 10 GB Adjust based on disk space
Inactive cleanup 7 days Free space from unused files

Docker Compose Configuration 

services:
  composer-proxy:
    image: openresty/openresty:alpine
    container_name: composer-proxy
    ports:
      - "8000:80"
    volumes:
      - ./conf/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf:ro
      - ./cache:/var/cache/nginx
      - ./satis-output:/var/www/packagist:ro
      - ./lua:/usr/local/openresty/nginx/lua:ro
    restart: unless-stopped
Enter fullscreen mode Exit fullscreen mode

Host Directory Permissions

OpenResty container runs as nobody user (UID 65534). Ensure host directory permissions are correct:

Cache Directory

Nginx needs write access to /var/cache/nginx (maps to host ./cache):

# Create directory and set permissions
mkdir -p ./cache
chmod 777 ./cache

# Or more secure: set owner to container's nobody user
chown 65534:65534 ./cache
chmod 755 ./cache
Enter fullscreen mode Exit fullscreen mode

Satis Output Directory

Satis outputs to /var/www/packagist (maps to host ./satis-output), container only needs read access:

# Ensure Satis output directory exists
mkdir -p ./satis-output

# Run Satis build to this directory
php bin/satis build satis.json ./satis-output

# Ensure files are readable
chmod -R 755 ./satis-output
Enter fullscreen mode Exit fullscreen mode

Directory Structure 

./
├── cache/              # Nginx cache (requires write permission)
├── satis-output/       # Satis output (read-only mount)
│   ├── packages.json
│   ├── include/
│   └── dist/
├── conf/
│   └── nginx.conf
└── lua/
    ├── proxy_logic.lua
    └── body_filter.lua
Enter fullscreen mode Exit fullscreen mode

Usage

Configure Composer 

# Global configuration
composer config -g repo.packagist composer http://localhost:8000

# Revert
composer config -g --unset repo.packagist
Enter fullscreen mode Exit fullscreen mode

Clear Cache 

docker exec -it composer-proxy rm -rf /var/cache/nginx/*
docker-compose restart
Enter fullscreen mode Exit fullscreen mode

Performance Improvements

Scenario Without Cache With Cache Improvement
First Laravel install ~45s ~45s -
Second install ~45s ~12s 73%
10 CI runners simultaneously 450s total requests ~50s total requests 89%

Important Notes

  1. Accept-Encoding: Must be set to empty string, otherwise upstream returns gzip content that cannot be modified in Lua
  2. Content-Length: Must clear this header after modifying body, use chunked encoding
  3. GitHub Rate Limit: While caching reduces requests, setting a GitHub token is still recommended to avoid rate limiting

Conclusion

By leveraging OpenResty's Lua extension capabilities, we implement complex proxy logic at the Nginx layer, maintaining high performance while gaining flexibility. This solution is particularly suitable for enterprise environments requiring private package integration, and can significantly improve CI/CD pipeline performance.

Top comments (0)