Simple BOT blocker with Caddy and Django

Tired of seeing this in your Django logs?

They do no harm because Django handles them properly based on allowed hosts env, but those bot requests pass the reverse-proxy and end up in the Python application - which is not as fast as a reverse proxy built in Go, C, C++, Zig or something else.

Here is how to stop those bot requests using just a django command which generates the Caddyfile template.

The generated Caddyfile will just block by default all routes that are not registered by Django and will block any hosts other that the one specified (ex: bot trying to curl on VPS IP).

The Caddyfile template

We'll use Django Templates to pass existing paths to Caddyfile template.

# Catch any request not matching your domain (like IP scans)
:80, :443 {
    abort
}

{{ app_url }} {

    # Create a named matcher for the host
    @strict_host {
        not host {{ app_url }}
    }
    # Abort if the host isn't exactly yours
    handle @strict_host {
        abort
    }

    # Enable high-performance compression
    encode zstd gzip

    # 1. Static Files (Update with path to staticfiles)
    # Handled first to ensure they are always accessible. 
    handle_path /static/* {
        root * /webapp/staticfiles
        file_server
    }

    # 2. Define Allowed Routes
    # This matcher includes all Django URLs plus your explicit SSE route.
    @allowed {

        path /favicon.ico

        path /robots.txt

        path /sitemap.xml

        path /sse-events*
        {% for route in routes %}
        path {{ route }}
        {% endfor %}
    }

    # 3. Handle Allowed Traffic
    handle @allowed {
        # SSE Sidecar (Specific route handling inside the allowed block)
        reverse_proxy /sse-events* docker-service-name-sse-sidecar:5687

        # Main Django App (Default for other allowed routes)
        reverse_proxy docker-service-name-app:8000
    }

    # 4. Fallback: Block everything else
    handle {
        respond "Access Denied" 403
    }
}

The Django Command

In any app create a new caddyfile_gen.py in your_app/management/commands folder.

Paste this code there in caddyfile_gen.py

from django.core.management.base import BaseCommand
from django.template.loader import render_to_string
from django.urls import get_resolver
from django.urls.resolvers import URLPattern, URLResolver



class Command(BaseCommand):
    help = "Generates a Caddyfile based on registered Django URLs"

    def handle(self, *args, **options):
        # You can set your domain name in env
        app_url = "chatcodfiscal.ro"

        resolver = get_resolver()
        raw_routes = self.get_urls(resolver.url_patterns)

        final_routes = set()
        for r in raw_routes:
            final_routes.add(r)
            # ex: "/chat/" -> adds "/chat"
            if r.endswith("/") and len(r) > 1:
                final_routes.add(r.rstrip("/"))

        # Remove duplicates
        clean_routes = sorted(list(set(final_routes)))

        context = {"app_url": app_url, "routes": clean_routes}

        # Assumes caddyfile.tpl is in your templates/components folder
        caddyfile_content = render_to_string("components/caddyfile.tpl", context)

        with open("Caddyfile", "w") as f:
            f.write(caddyfile_content)

        print("Caddyfile was created!")

    def get_urls(self, patterns, prefix=""):
        urls = []
        for pattern in patterns:
            if isinstance(pattern, URLPattern):
                path_str = str(pattern.pattern)
                full_path = prefix + path_str

                # 1. Remove Regex anchors
                full_path = full_path.replace("^", "").replace("$", "")

                # 2. Ensure it starts with /
                if not full_path.startswith("/"):
                    full_path = "/" + full_path

                if full_path.startswith("/admin"):
                    urls.append("/admin")
                    urls.append("/admin/*")
                    continue

                if "<" in full_path:
                    full_path = full_path.split("<")[0] + "*"

                urls.append(full_path)

            elif isinstance(pattern, URLResolver):
                # Recursively handle included URLconfs
                new_prefix = prefix + str(pattern.pattern)
                urls.extend(self.get_urls(pattern.url_patterns, new_prefix))

        return urls

A Caddyfile file will be generated and it will look similar to this one:

# Catch any request not matching your domain (like IP scans)
:80, :443 {
    abort
}

your-domain.com {

    # Create a named matcher for the host
    @strict_host {
        not host your-domain.com
    }
    # Abort if the host isn't exactly yours
    handle @strict_host {
        abort
    }

    # Enable high-performance compression
    encode zstd gzip

    # 1. Static Files
    # Handled first to ensure they are always accessible.
    handle_path /static/* {
        root * /webapp/staticfiles
        file_server
    }

    # 2. Define Allowed Routes
    # This matcher includes all Django URLs plus your explicit SSE route.
    @allowed {

        path /favicon.ico

        path /robots.txt

        path /sitemap.xml

        path /sse-events*

        path /

        path /admin

        path /admin/*

        path /some-path

        path /some-path/

        path /legal

        path /legal/

        path /login/*

        path /logout

        path /logout/

        path /meniu

        path /meniu/

        path /robots.txt

        path /sitemap.xml

        path /sse-token

        path /sse-token/

    }

    # 3. Handle Allowed Traffic
    handle @allowed {
        # SSE Sidecar (Specific route handling inside the allowed block)
        reverse_proxy /sse-events* docker-service-name-sse-sidecar:5687

        # Main Django App (Default for other allowed routes)
        reverse_proxy docker-service-name-app:8000
    }

    # 4. Fallback: Block everything else
    handle {
        respond "Access Denied" 403
    }
}

The Docker Compose file

In your docker-compose.yml file make sure you don't expose ports to external network.

# DEV
# ports:
#   - "8000:8000"
# PROD - expose port only inside docker network
expose:
    - "8000"

And now all those BOT requests will just stop at Caddy level!