DEV Community

Cover image for Protecting Self Hosted Endpoints with Rate Limiting
Raizan
Raizan

Posted on • Originally published at chasebot.online

Protecting Self Hosted Endpoints with Rate Limiting

#devops #programming #security #selfhosted

What You'll Need

Table of Contents

Why Rate Limiting is Critical for Self-Hosted Services

When running automation tools, API gateways, or workflow engines on your own server, exposing public HTTP endpoints is unavoidable. Webhooks from external providers, custom REST APIs, and authentication portals must remain accessible to the internet. However, an unprotected endpoint on a public IP address will quickly attract automated bot scanners, credential stuffing attacks, and accidental recursive loop requests from third-party services.

Without request throttling, a sudden spike in incoming HTTP requests will saturate your CPU, exhaust available database connections, and consume system memory. If you are hosting on a provider like a Hetzner VPS or DigitalOcean, an unthrottled endpoint can crash your entire stack or cause host-level network throttling.

If you previously set up your infrastructure following our guide on How to Deploy n8n with Docker on Any VPS (2026 Guide), you know that containerized deployments share server resources. Rate limiting acts as your first line of defense, ensuring that rogue traffic is rejected at the network edge before it reaches your memory-intensive execution runtimes.

Below is a complete multi-container docker-compose.yml file that establishes an architecture for a rate-limited reverse proxy standing in front of an application backend and a Redis cache:

version: '3.8'

services:
  nginx:
    image: nginx:1.25-alpine
    container_name: production_proxy
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./logs:/var/log/nginx
      - ./certs:/etc/nginx/certs:ro
    depends_on:
      - app
      - redis
    networks:
      - app_network

  app:
    build:
      context: ./app
      dockerfile: Dockerfile
    container_name: production_app
    restart: always
    environment:
      - NODE_ENV=production
      - REDIS_URL=redis://redis:6379
      - PORT=3000
    expose:
      - "3000"
    networks:
      - app_network

  redis:
    image: redis:7-alpine
    container_name: production_redis
    restart: always
    command: redis-server --save 60 1 --loglevel notice
    expose:
      - "6379"
    volumes:
      - redis_data:/data
    networks:
      - app_network

networks:
  app_network:
    driver: bridge

volumes:
  redis_data:
Enter fullscreen mode Exit fullscreen mode

💡 Fast-Track Your Project: Don't want to configure this yourself? I build custom n8n pipelines and bots. Message me with code SYS3-DEVTO.

Configuring Nginx Reverse Proxy Rate Limiting

The most efficient layer to enforce rate limits is at the reverse proxy. Nginx processes rate limiting entirely in memory using shared zones, meaning blocked requests are dropped before consuming application layer compute cycles.

Nginx implements the leaky bucket algorithm using the limit_req_zone and limit_req directives. A shared memory zone holds the state of IP addresses and their current request frequency.

When calculating infrastructure costs for hosting high-volume integrations, infrastructure efficiency is crucial. In our n8n vs Temporal vs Windmill pricing comparison, we highlighted how self-hosted worker architecture scale costs depend heavily on edge efficiency. Offloading unauthorized or excessive webhooks at Nginx keeps your operational costs predictable.

Here is a complete, production-ready nginx.conf file configured with distinct rate-limiting zones for general web traffic, API authentication, and incoming webhooks:

user nginx;
worker_processes auto;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

    access_log /var/log/nginx/access.log main;
    error_log /var/log/nginx/error.log warn;

    sendfile on;
    keepalive_timeout 65;

    # Shared memory zone definitions
    # $binary_remote_addr uses 4 bytes for IPv4 or 16 bytes for IPv6
    # 1MB zone can hold roughly 16,000 IP addresses

    # General API limit: 10 requests per second
    limit_req_zone $binary_remote_addr zone=general_api:10m rate=10r/s;

    # Strict login/auth limit: 3 requests per minute
    limit_req_zone $binary_remote_addr zone=auth_limit:10m rate=3r/m;

    # Inbound Webhook limit: 5 requests per second
    limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=5r/s;

    # Custom HTTP Status Code for Rate Limited Requests
    limit_req_status 429;

    server {
        listen 80;
        server_name api.example.com;

        # Redirect cleartext HTTP to HTTPS
        return 301 https://$host$request_uri;
    }

    server {
        listen 443 ssl;
        server_name api.example.com;

        ssl_certificate /etc/nginx/certs/fullchain.pem;
        ssl_certificate_key /etc/nginx/certs/privkey.pem;
        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_ciphers HIGH:!aNULL:!MD5;

        # Protect Authentication Endpoints
        location /api/v1/auth/ {
            limit_req zone=auth_limit burst=2 nodelay;
            proxy_pass http://app:3000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        # Protect Inbound Webhooks
        location /webhook/ {
            limit_req zone=webhook_limit burst=10 nodelay;
            proxy_pass http://app:3000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        # Standard Endpoint Coverage
        location / {
            limit_req zone=general_api burst=20 delay=10;
            proxy_pass http://app:3000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

The parameters used here provide precise traffic management:

  • burst: Defines how many requests an IP can make beyond the base limit before Nginx denies access.
  • nodelay: Instructs Nginx to process burst requests immediately without staggering them, while still enforcing the capacity limit.
  • delay: Dynamically throttles excessive traffic by delaying requests that exceed the target threshold up to the burst capacity.

Automating IP Ban Rules with Fail2ban

While Nginx returning HTTP 429 Too Many Requests prevents application saturation, malicious actors sending thousands of requests per second still consume web server socket capacity. Fail2ban complements reverse proxies by parsing log files for repetitive HTTP 429 errors and placing dynamic firewall bans (iptables / nftables) on the offending IP address.

Step 1: Create the Nginx Filter Definition

Create a custom filter on your host OS at /etc/fail2ban/filter.d/nginx-ratelimit.conf:

[Definition]
failregex = ^<HOST> -.*"(GET|POST|PUT|DELETE|PATCH).* HTTP/.*" 429 
ignoreregex =
Enter fullscreen mode Exit fullscreen mode

Step 2: Configure the Fail2ban Jail

Create the jail rule at /etc/fail2ban/jail.d/nginx-ratelimit.local:

[nginx-ratelimit]
enabled  = true
port     = http,https
filter   = nginx-ratelimit
logpath  = /var/log/nginx/access.log
maxretry = 5
findtime = 60
bantime  = 3600
action   = iptables-multiport[name=ReqLimit, port="http,https", protocol=tcp]
Enter fullscreen mode Exit fullscreen mode

This configuration inspects /var/log/nginx/access.log. If an individual IP address receives 5 HTTP 429 responses within 60 seconds (findtime), Fail2ban drops all incoming packets from that IP at the OS network layer for 1 hour (bantime).

Step 3: Verifying the Enforcement Chain

To test your rate limits automatically, write a standalone Bash validation script named test_limits.sh:

#!/usr/bin/env bash

TARGET_URL="https://api.example.com/webhook/test"
CONCURRENT_REQUESTS=15

echo "Starting rate-limit load test against ${TARGET_URL}..."

for ((i=1; i<=CONCURRENT_REQUESTS; i++)); do
    STATUS=$(curl -o /dev/null -s -w "%{http_code}\n" "${TARGET_URL}")
    echo "Request #${i}: HTTP ${STATUS}"
    sleep 0.1
done

echo "Test sequence finished."
Enter fullscreen mode Exit fullscreen mode

Make the script executable and execute it:

chmod +x test_limits.sh
./test_limits.sh
Enter fullscreen mode Exit fullscreen mode

You should observe initial HTTP 200 responses followed immediately by HTTP 429 statuses as the Nginx burst zone fills up.

Building Custom Redis-Backed Application Rate Limiters

Network-level throttling at the proxy layer protects system availability, but multi-tenant SaaS products and webhook receivers often require business-logic rate limiting (for example, limiting requests per API key or per user account).

When handling external messaging bots like those described in our step-by-step guide on How to Build a Telegram Bot with n8n (No Code Required), user-level rate limiting prevents single users from exhausting execution quotas.

The following complete Node.js application uses Express and ioredis to execute an atomic sliding-window algorithm using Redis transactions:

const express = require('express');
const Redis = require('ioredis');

const app = express();
const redis = new Redis(process.env.REDIS_URL || 'redis://localhost:6379');

app.use(express.json());

/**
 * Sliding Window Rate Limiter Middleware
 * @param {number} windowInSeconds - Time window size
 * @param {number} maxAllowed - Maximum requests permitted within the window
 */
function slidingWindowRateLimiter(windowInSeconds, maxAllowed) {
  return async (req, res, next) => {
    // Identify requester by custom API Key or Client IP
    const identifier = req.headers['x-api-key'] || req.ip;
    const redisKey = `ratelimit:${identifier}`;
    const currentTime = Date.now();
    const windowStartTime = currentTime - (windowInSeconds * 1000);

    try {
      // Execute atomic Redis transaction using a multi-exec block
      const result = await redis
        .multi()
        .zremrangebyscore(redisKey, 0, windowStartTime) // Clean up old records
        .zadd(redisKey, currentTime, currentTime.toString()) // Record current hit
        .zcard(redisKey) // Count hits inside current window
        .expire(redisKey, windowInSeconds) // Ensure auto-cleanup of stale keys
        .exec();

      // The result of zcard is at index 2 in the multi payload return array
      const requestCount = result[2][1];

      res.setHeader('X-RateLimit-Limit', maxAllowed);
      res.setHeader('X-RateLimit-Remaining', Math.max(0, maxAllowed - requestCount));

      if (requestCount > maxAllowed) {
        return res.status(429).json({
          error: 'Too Many Requests',
          message: `Rate limit exceeded. Maximum ${maxAllowed} requests allowed every ${windowInSeconds} seconds.`,
          retryAfterSeconds: windowInSeconds
        });
      }

      next();
    } catch (error) {
      console.error('Redis Rate Limiting Error:', error);
      // Fallback: Fail open to prevent database issues from causing total downtime
      next();
    }
  };
}

// Apply 5 requests per 10 seconds limit on sensitive webhook processing
app.post('/webhook/incoming', slidingWindowRateLimiter(10, 5), (req, res) => {
  res.status(200).json({
    status: 'success',
    message: 'Webhook payload processed successfully.',
    data: req.body
  });
});

// General healthcheck endpoint without limiters
app.get('/health', (req, res) => {
  res.status(200).json({ status: 'ok' });
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Application server running on port ${PORT}`);
});
Enter fullscreen mode Exit fullscreen mode

Getting Started

Implementing a multi-tiered security defense ensures your self-hosted infrastructure remains online regardless of traffic volume. Combine proxy-level network controls using Nginx, automated host protection using Fail2ban, and fine-grained application-level sliding window rate limiting.

To deploy your rate-limited automation environment today:

  1. Provisions a cloud instance on Hetzner VPS or Contabo VPS. Alternatively, launch an instance on DigitalOcean.
  2. Register a dedicated domain for your endpoints through Namecheap.
  3. If you want a managed automation experience without edge proxy security headaches, evaluate n8n Cloud.

Outsource Your Automation

Don't have time? I build production n8n workflows, WhatsApp bots, and fully automated YouTube Shorts pipelines. Hire me on Fiverr — mention SYS3-DEVTO for priority. Or DM at chasebot.online.

Originally published on Automation Insider.

Top comments (0)