DEV Community: ABDULLAH AFZAL

IP Risk Score Widget: Show Visitors What You See

ABDULLAH AFZAL — Thu, 08 Oct 2026 16:53:18 +0000

An IP risk score decides whether your login form waves someone through or throws a challenge, and the person it describes never gets to see it. Meanwhile your status page says All Systems Operational to a user whose sign-in just failed because their IP sits on a residential proxy list.

Netflix's own help page, for people who get the VPN error with their VPN off, ends with resetting the router and calling the ISP. That's the state of the art. This post builds the alternative: a small "Your connection" widget that tells visitors what your systems see about their IP, in plain language.

It drops into a hosted status page with one script tag. And it leaves out the score on purpose, because a public score checker is exactly what proxy sellers want.

TL;DR

Look up only the caller, on your server: read the client IP yourself and pass it to GET /v3/security. Never accept an IP from the browser.
Request only the fields you'll use with fields=, then send the browser sentences, not data.
Show flags a visitor can act on (VPN, relay, residential proxy, abuse history, corporate gateway). Hide the 0 to 100 score, every confidence value, and every last-seen date.
Cache per IP, limit per IP, cap the daily total, and fail quietly. Each lookup costs 2 credits.
Optional: a browser-side live check behind a button separates "this IP has VPN history" from "this session is on a VPN right now." The widget answers the question your support inbox keeps getting, "why am I blocked?", without turning your status page into a free proxy grader. The server keeps the score and turns flags into plain sentences. The browser only ever sees what you decided to show, and the visitor can act on all of it.

What the widget shows, and what it hides on purpose

The IP reputation API used here returns 27 fields in its security object for one address. Most of them help a person understand their own connection. A handful mainly help someone testing a proxy pool. The rule I use: show a flag when knowing it helps the visitor fix something, and hide anything that measures how sure the detection is or how recently it fired.

Field(s)	What the visitor sees	Why
`is_vpn`, `vpn_provider_names`	"This IP is listed as a VPN exit (SurfShark VPN)"	Naming the provider tells them what to switch off
`is_relay`, `relay_provider_name`	"You're using iCloud Private Relay. It isn't a VPN."	Relays and VPNs deserve different copy and different rules
`is_tor`	"This IP is a Tor exit node"	Honest, and the Tor user already knows
`is_residential_proxy`, `is_proxy`	"Seen in residential proxy networks," plus what usually causes it	The one finding innocent users rarely know about
`is_spam`, `is_known_attacker`	"A recent history of abuse," plus shared-network and malware notes	Common on mobile and shared IPs
`is_corporate_gateway`, `corporate_gateway_provider_name`	"Your employer's security gateway (Zscaler)"	Reassurance; one gateway IP fronts a whole company
`is_cloud_provider`, `cloud_provider_name`	"Comes from a hosting provider (UpCloud)"	The usual false positive for people on cloud desktops
`threat_score`	Hidden. A three-state summary only	The number is the gradient an attacker tunes against
`_confidence_score`, `_last_seen`	Hidden	Freshness tells a proxy operator when to rotate
`is_bot`, `bot_type`, `bot_operator_name`, `is_known_good_bot`	Hidden	Internal classification, meaningless to a person
`proxy_provider_names`	Hidden	Nine proxy brands is noise to a homeowner and a receipt to a proxy user
`is_anonymous`	Not shown separately	It's derived from the flags already shown

The residential proxy row is the one that justifies the whole widget. A home IP lands on those lists when something on the network is selling its bandwidth, often without the owner knowing. When Google disrupted the IPIDEA network in January 2026, its threat intelligence team described devices joining through proxy code embedded in apps, and researchers finding proxy payloads preinstalled on off-brand Android TV boxes. Those people deserve a sentence that says so.

None of this is new for VPN companies. Mullvad's connection check has printed lines like "Your IP is not blacklisted" for years. Your app can do the same for its own rules.

Why the IP risk score never leaves your server

A public IP risk check that prints a number is a tool for whoever owns the proxies. That's not hypothetical. Evomi, a residential proxy seller, lets customers append _fraudscore-0 to the proxy password so they only get exits with a clean Scamalytics score. Show a 0 to 100 number on your status page and you've built the same filter for free, tuned to your own rules.

If you only do one hardening step, drop the score. A yes/no flag tells a proxy operator one exit is burned. A number tells them which exits are almost good enough, and the confidence and last-seen fields tell them when to rotate. IPQS and Scamalytics already publish scores in their public checkers. Your site doesn't need to be another one.

Hiding a field in the UI hides nothing, because the full response sits in the browser's network tab. That rules out the shortcut the API offers. Register your status page as a Request Origin and the browser can call the security endpoint directly, no key required. Tempting for a static page, but every view costs 2 credits, the browser receives whatever you requested, and an Origin header is just a header that anything outside a browser can set. The provider's own auth docs say to prefer a backend proxy in production. That's what we'll build.

Step 1: An endpoint that only looks up the caller

Four small files: the route, the lookup, a cache with limits, and a mapper from flags to sentences. Node 18+ for the built-in fetch, Express 4 or 5, and the cors package.

Read the client IP you can actually trust

The browser can't reliably tell you its own public IP, which is why so many "javascript get client ip" answers call an echo service and post the result back to the server. Don't. An endpoint that accepts an IP from the client will look up any IP anyone sends it, on your credits. Read the address on the server.

Behind exactly one reverse proxy (nginx, a load balancer, a PaaS router), app.set("trust proxy", 1) makes req.ip the address that proxy saw, but only if every request reaches the app through that one-hop path and clients cannot reach the origin directly. If your topology has multiple paths, trust the proxy's actual IPs or CIDRs instead of a numeric hop count. Otherwise a client can supply X-Forwarded-For and turn your caller-only endpoint into an arbitrary-IP lookup.

// server.js: Node 18+, Express 4 or 5
import express from "express";
import cors from "cors";
import { lookupSecurity } from "./ipgeo.js";
import { toFindings } from "./findings.js";
import { getCached, setCached, allowRequest, spendBudget } from "./limits.js";

const app = express();
const UNAVAILABLE = { summary: "unknown", findings: [] };

// Exactly one proxy sits in front of this app. Trusting one hop makes req.ip
// the address that proxy saw, not whatever the client wrote into X-Forwarded-For.
app.set("trust proxy", 1);
app.use(express.static("public")); // serves public/widget.js
app.use("/api", cors({ origin: "https://status.example.com", methods: ["GET"] }));

app.get("/api/connection", async (req, res) => {
  res.set("Cache-Control", "no-store"); // a per-visitor answer must never be shared by a CDN

  // No ?ip= parameter, ever. IPv4 clients can arrive in IPv6-mapped form.
  const ip = req.ip?.replace(/^::ffff:/, "");
  if (!ip) return res.status(400).json(UNAVAILABLE);
  if (!allowRequest(ip)) {
    return res.status(429).json({ ...UNAVAILABLE, message: "Too many checks. Try again in a few minutes." });
  }

  const cached = getCached(ip);
  if (cached) return res.json(cached);
  if (!spendBudget()) return res.status(503).json(UNAVAILABLE);

  try {
    const result = await lookupSecurity(ip);
    const body = result.private
      ? { summary: "unknown", findings: [{ tone: "info", text: "You're on a private network address, so there's nothing to check." }] }
      : { ip, ...toFindings(result.security) };
    setCached(ip, body);
    return res.json(body);
  } catch (err) {
    // Fail open: the widget explains, it never gates. Log it, don't cache it.
    console.error("connection check failed:", err?.message);
    return res.status(503).json(UNAVAILABLE);
  }
});

app.listen(Number(process.env.PORT ?? 3000));

The order matters: rate limit, then cache, then budget. A cached answer never spends credits, and a blocked caller never touches the cache.

Ask the security endpoint for only the fields you'll show

I'm using IPGeolocation's IP Security API because its response keeps relay, corporate gateway, and residential proxy as separate flags, which is exactly what plain-language copy needs. IPQS, IPinfo, and proxycheck.io all do VPN and proxy detection too, so swap freely; findings.js is the only file that knows field names. Security data needs a paid plan, and each lookup costs 2 credits.

One gotcha. If you omit ip, the endpoint looks up the caller. That's handy from a browser and useless from a server, where the caller is you. Server-side, always pass it.

curl -s "https://api.ipgeolocation.io/v3/security?apiKey=$IPGEO_API_KEY&ip=145.223.7.7"

{
  "ip": "145.223.7.7",
  "security": {
    "threat_score": 90,
    "is_tor": false,
    "is_proxy": true,
    "proxy_provider_names": [
      "ProxyScrape", "DataImpulse", "Oxy Labs", "ProxyEmpire", "Proxy-Store",
      "SpyderProxy", "FleetProxy", "Proxy4u", "FlashProxy"
    ],
    "proxy_confidence_score": 99,
    "proxy_last_seen": "2026-10-05",
    "is_residential_proxy": true,
    "is_vpn": true,
    "vpn_provider_names": ["SurfShark VPN", "Ishaan VPN"],
    "vpn_confidence_score": 99,
    "vpn_last_seen": "2026-07-31",
    "is_relay": false,
    "relay_provider_name": "",
    "is_anonymous": true,
    "is_known_attacker": true,
    "is_bot": false,
    "bot_confidence_score": 0,
    "bot_operator_name": "",
    "bot_type": "",
    "is_known_good_bot": false,
    "bot_last_seen": "",
    "is_spam": true,
    "is_cloud_provider": true,
    "cloud_provider_name": "Brander Group Inc.",
    "is_corporate_gateway": false,
    "corporate_gateway_type": "",
    "corporate_gateway_provider_name": ""
  }
}

That's a live response from October 2026: a residential IP that's also a listed SurfShark exit, flagged for attacks and spam, with nine proxy providers attached. Every field is useful to your fraud rules. The lookup below requests 14 of the 27, and the visitor sees sentences built from 13 of those.

// ipgeo.js
const API_KEY = process.env.IPGEO_API_KEY;

// Only what findings.js reads. Confidence scores and last-seen dates never
// reach this server, so a careless log line can't leak them either.
const FIELDS = [
  "security.threat_score",
  "security.is_vpn", "security.vpn_provider_names",
  "security.is_relay", "security.relay_provider_name",
  "security.is_tor", "security.is_proxy", "security.is_residential_proxy",
  "security.is_spam", "security.is_known_attacker",
  "security.is_cloud_provider", "security.cloud_provider_name",
  "security.is_corporate_gateway", "security.corporate_gateway_provider_name",
].join(",");

// Throws on failure; the route's try/catch turns that into a quiet 503.
export async function lookupSecurity(ip) {
  if (!API_KEY) throw new Error("IPGEO_API_KEY is not set");

  const url =
    "https://api.ipgeolocation.io/v3/security" +
    `?apiKey=${encodeURIComponent(API_KEY)}` +
    `&ip=${encodeURIComponent(ip)}` + // omit this server-side and you look up yourself
    `&fields=${FIELDS}`;

  // 1.5s is plenty for a status widget. A slow answer is worse than "try again".
  const res = await fetch(url, { signal: AbortSignal.timeout(1500) });

  // 423 Locked: a private or bogon address, i.e. you're testing from localhost or a LAN.
  if (res.status === 423) return { private: true };
  // 401 usually means a free-plan key; security data needs a paid plan.
  if (!res.ok) throw new Error(`security lookup failed: HTTP ${res.status}`);

  const body = await res.json();
  return { private: false, security: body?.security ?? null };
}

fields here is doing privacy work, not payload trimming. Whatever this server never receives, it can't accidentally return.

Cache per IP, and cap the whole thing

Security data refreshes at least twice a day, so a 30-minute cache costs almost nothing in freshness and turns a visitor mashing the button into one lookup. Then two limits: five checks per IP per ten minutes, and a daily budget that stops spending once it's hit.

// limits.js: in-memory, single instance. Behind several instances, move all of
// this to Redis or your gateway, or each instance quietly gets its own budget.
const TTL_MS = 30 * 60 * 1000;   // security data refreshes at least twice a day
const WINDOW_MS = 10 * 60 * 1000;
const PER_IP_LIMIT = 5;          // checks per IP per window
const DAILY_BUDGET = 5000;       // lookups, i.e. 10,000 credits at 2 each

const results = new Map();       // ip -> { value, expires }
const hits = new Map();          // ip -> { count, resetAt }
let spentToday = 0;
let budgetDay = new Date().toISOString().slice(0, 10);

export function getCached(ip) {
  const entry = results.get(ip);
  if (!entry) return null;
  if (entry.expires < Date.now()) {
    results.delete(ip);
    return null;
  }
  return entry.value;
}

export function setCached(ip, value) {
  if (results.size > 50_000) results.clear(); // crude ceiling; swap in an LRU if this ever fires
  results.set(ip, { value, expires: Date.now() + TTL_MS });
}

export function allowRequest(ip) {
  const now = Date.now();
  if (hits.size > 100_000) hits.clear();      // same ceiling idea for the limiter
  const entry = hits.get(ip);
  if (!entry || entry.resetAt < now) {
    hits.set(ip, { count: 1, resetAt: now + WINDOW_MS });
    return true;
  }
  entry.count += 1;
  return entry.count <= PER_IP_LIMIT;
}

export function spendBudget() {
  const today = new Date().toISOString().slice(0, 10);
  if (today !== budgetDay) {
    budgetDay = today;
    spentToday = 0;
  }
  if (spentToday >= DAILY_BUDGET) return false;
  spentToday += 1;
  return true;
}

At 2 credits a lookup, that budget caps the widget at 10,000 credits a day no matter who's hammering it. A status page with 3,000 unique visitors who all click would use 6,000.

Be honest about what this buys. A proxy user can still check each exit once: the limit is per IP, and they have thousands of IPs. What they can't get is a score to optimize against, and they can't run your bill past the budget.

Turn flags into findings, not data

This is where the transparency actually happens, so spend your editing time on the strings, not the code. Two choices matter.

First, describe the IP, not the person. The data is the address's history, and on a shared or reassigned IP it isn't about them at all.

Second, the summary collapses the score into three states. IPGeolocation's bands run 1 to 19 low observed risk, 20 to 44 use with context, 45 to 79 elevated, and 80 to 100 high risk (what to do at each level). The widget folds the top two into one state on purpose, because "elevated" versus "high" is a gradient again.

// findings.js: flags in, sentences out. Nothing numeric leaves this function.
// The copy describes YOUR policy, so edit it to match what your rules actually do.
export function toFindings(security) {
  if (!security) return { summary: "unknown", findings: [] };
  const s = security;
  const findings = [];
  const add = (tone, text) => findings.push({ tone, text });

  if (s.is_corporate_gateway) {
    const vendor = s.corporate_gateway_provider_name || "a corporate security gateway";
    add("info", `You're browsing through your employer's security gateway (${vendor}). That's normal on a work network.`);
  }
  if (s.is_relay) {
    add("info", `You're using ${s.relay_provider_name || "a privacy relay"}. It hides your address, but it isn't a VPN.`);
  }
  if (s.is_vpn) {
    const name = s.vpn_provider_names?.[0];
    // IP history, not this session: an address can stay listed after it changes hands.
    add("warn", `This IP is listed as a VPN exit${name ? ` (${name})` : ""}. If you're not using a VPN right now, the address may have been reassigned to you.`);
  }
  if (s.is_tor) add("warn", "This IP is a Tor exit node.");
  if (s.is_residential_proxy) {
    add("warn", "This IP has been seen in residential proxy networks. If you're not running a proxy, check for 'bandwidth sharing' apps, free VPN apps, or no-name streaming boxes on your network.");
  } else if (s.is_proxy) {
    add("warn", "This IP has been seen working as a proxy.");
  }
  if (s.is_spam || s.is_known_attacker) {
    add("warn", "This IP has a recent history of abuse. On mobile or shared networks that's usually someone else's traffic. On a home connection, a malware scan is worth ten minutes.");
  }
  // Skip "hosting" when another finding already explains the address, or the copy contradicts itself.
  if (s.is_cloud_provider && !s.is_corporate_gateway && !s.is_residential_proxy) {
    add("info", `This connection comes from a hosting provider${s.cloud_provider_name ? ` (${s.cloud_provider_name})` : ""}, not a home or mobile network.`);
  }

  const rawScore = s.threat_score;
  const score = Number(rawScore);
  const hasScore =
    rawScore !== null &&
    rawScore !== undefined &&
    rawScore !== "" &&
    Number.isFinite(score);

  const summary = !hasScore
    ? "unknown"
    : score >= 45
      ? "attention"
      : score >= 20
        ? "notice"
        : "clear";
    return { summary, findings };
}

For 145.223.7.7, this is everything the browser receives:

{
  "ip": "145.223.7.7",
  "summary": "attention",
  "findings": [
    { "tone": "warn", "text": "This IP is listed as a VPN exit (SurfShark VPN). If you're not using a VPN right now, the address may have been reassigned to you." },
    { "tone": "warn", "text": "This IP has been seen in residential proxy networks. If you're not running a proxy, check for 'bandwidth sharing' apps, free VPN apps, or no-name streaming boxes on your network." },
    { "tone": "warn", "text": "This IP has a recent history of abuse. On mobile or shared networks that's usually someone else's traffic. On a home connection, a malware scan is worth ten minutes." }
  ]
}

No score, no confidence, no dates, no proxy brands. Brander Group's hosting flag got dropped too, because "residential proxy" and "not a home network" in the same card would read as a bug.

Step 2: The widget is one script tag

The widget runs on click, not on page load. Status pages get hit by uptime monitors, link previews, and crawlers, and none of them need a lookup. It renders with textContent only, because provider names are third-party strings and innerHTML turns them into a script injection waiting to happen.

// public/widget.js: loaded by the status page, talks only to your endpoint.
(() => {
  const ENDPOINT = "https://api.example.com/api/connection";
  const root = document.getElementById("connection-check");
  if (!root) return;

  // Edit these to match what your rules actually do to each state.
  const HEADLINES = {
    clear: "Nothing unusual about your connection.",
    notice: "A few things about your connection are worth knowing.",
    attention: "Some actions from this connection will ask for extra verification.",
    unknown: "We couldn't check your connection right now.",
  };

  const button = document.createElement("button");
  button.type = "button";
  button.textContent = "Check my connection";
  const output = document.createElement("div");
  output.setAttribute("aria-live", "polite");
  root.append(button, output);

  button.addEventListener("click", async () => {
    button.disabled = true;
    output.replaceChildren(line("Checking..."));
    try {
      const res = await fetch(ENDPOINT, { credentials: "omit", signal: AbortSignal.timeout(4000) });
      render(await res.json().catch(() => ({ summary: "unknown" })));
    } catch (err) {
      // Network error or timeout: say so plainly, never guess a verdict.
      console.warn("connection check failed:", err?.message);
      render({ summary: "unknown" });
    } finally {
      button.disabled = false;
    }
  });

  function render(data) {
    const items = [line(HEADLINES[data?.summary] ?? HEADLINES.unknown)];
    if (data?.ip) items.push(line(`We see your connection as ${data.ip}.`));
    for (const f of data?.findings ?? []) items.push(line(f?.text, f?.tone));
    if (data?.message) items.push(line(data.message));
    output.replaceChildren(...items);
  }

  // textContent, never innerHTML: provider names come from third-party data.
  function line(text, tone = "info") {
    const p = document.createElement("p");
    p.className = `cc-${tone === "warn" ? "warn" : "info"}`;
    p.textContent = String(text ?? "");
    return p;
  }
})();

On failure it says it couldn't check, and nothing else. The widget is informational, so a broken lookup should never look like a verdict.

Dropping it into Atlassian Statuspage

Statuspage lets you inject custom header and footer HTML, which is all this needs. The catch, per Atlassian's custom HTML docs: custom HTML only renders on a custom domain like status.example.com. Custom HTML also isn't on every plan, so check yours before you promise this to support. Paste this into the custom footer:

<div id="connection-check"></div>
<script src="https://api.example.com/widget.js" defer></script>

Self-hosted status pages (Upptime, Cachet, a /status route in your own app) take the same two lines. The script loads from your API's domain, and only the fetch needs CORS, which the server already scopes to the status page origin.

Optional: a live check for "I'm not on a VPN, I swear"

Everything so far is IP history. The 145.223.7.7 copy admits it: the address is listed as a VPN exit, and that may say nothing about whoever is using it today. For that complaint, the useful question is what this session is doing right now.

IPGeolocation's Real-Time Proxy & VPN Detection is a browser-only script that runs live tests against the connection itself. There's no API key; it's authorized by Request Origin, so register the status page's origin first. A verdict takes 1 to 5 seconds and costs 3 credits, or 5 with the security data attached, which the server already has, so skip it. It's a paid feature. Because it runs in the visitor's browser, your server's limits don't apply to it. It gets its own button and never runs on load, even though the docs suggest starting early.

// public/live-check.js: optional second button inside the same widget.
(() => {
  const SCRIPT = "https://static.ipgeolocation.io/web-assets/static/security/session-analysis.js";
  const root = document.getElementById("connection-check");
  if (!root) return;

  const button = document.createElement("button");
  button.type = "button";
  button.textContent = "Run a live test (a few seconds)";
  const out = document.createElement("p");
  out.setAttribute("aria-live", "polite");
  root.append(button, out);

  const loadScript = () => new Promise((resolve, reject) => {
    if (window.Analysis) return resolve();
    const el = document.createElement("script");
    el.src = SCRIPT;
    el.onload = () => resolve();
    el.onerror = () => reject(new Error("live-check script failed to load"));
    document.head.append(el);
  });

  button.addEventListener("click", async () => {
    button.disabled = true;
    out.textContent = "Testing your connection...";
    try {
      await loadScript();
      // No key here: the request is authorized by the page's registered origin.
      const result = await window.Analysis.startMonitoring().get();
      out.textContent = describe(result?.live_vpn_proxy_detection);
      button.remove(); // one live test per page view; each run costs credits
      // visitor_actual_location is ignored on purpose. Never print the real IP behind a VPN.
    } catch (err) {
      console.warn("live check failed:", err?.message);
      out.textContent = "The live test couldn't run in this browser.";
      button.disabled = false;
    }
  });

  function describe(live) {
    if (!live) return "The live test didn't return a result.";
    // Below 30 the docs call the signal very weak, so claim nothing.
    if ((live.confidence_score ?? 0) < 30) return "The live test was inconclusive.";
    if (!live.is_anonymous) return "Right now, your connection looks direct: no VPN or proxy detected.";
    return (live.proxy_score ?? 0) >= (live.vpn_score ?? 0)
      ? "Right now, your traffic looks like it's going through a proxy."
      : "Right now, your traffic looks like it's going through a VPN.";
  }
})();

Put this next to the server result and you get the sentence support actually needs: listed as a VPN exit, but direct right now. That usually means the address changed hands, and it's the screenshot that closes the ticket.

Pitfall: treat the live verdict as display-only. It runs in the visitor's browser, so anyone can edit what it reports. Never post it back to your server as a trust signal.

A few extra notes

Mobile carriers put many subscribers behind one CGNAT address, so abuse history on a phone connection is rarely about the phone in your hand. That's why the copy says "this IP" everywhere and "you" almost nowhere.

The cache is keyed by exact IP, so a visitor who turns their VPN off gets a fresh lookup the moment their address changes. IPv6 privacy addresses rotate on their own, which can mean more cache misses than you'd expect. Keep security results keyed to the exact address rather than broadening one verdict across a /64; if the budget starts tripping, increase the TTL, tighten rate limits, or raise the budget instead.

Testing on localhost hits the 423 path, because 127.0.0.1 and LAN addresses are bogons. Expose the server through a tunnel, or hardcode a test IP in lookupSecurity and delete it before you commit.

You're showing someone data about their own IP, which is the friendly direction. Don't undo that by logging the findings next to user IDs. The lookup itself is the only record you need.

Ship the server endpoint first, with the score already stripped, and watch the X-Credits-Charged header for a week before adding the live button. If the number surprises you, raise the cache TTL before you touch the budget. And link the widget from your block page, because the people who need it most won't think to open your status page.

Build a Slack Bot That Investigates Any IP You Paste

ABDULLAH AFZAL — Tue, 06 Oct 2026 18:50:48 +0000

A Slack bot that investigates a pasted IP needs three things in one reply: a threat score, the provider behind the address, and the abuse contact to report it to. The first two are one API call. The third is the one everyone skips.

That skip is why a lot of abuse never reaches the network that could actually stop it. Finding an abuse address means a WHOIS query, reading an RIR record, and guessing which of four email addresses is the live one. Nobody does that at 4pm on a Friday.

Most IP lookup bots stop at the score. This one ends at a sendable report, and it handles corporate gateways, known-good bots, and bogons separately so a raw score does not become the whole verdict.

TL;DR

One slash command, two API calls, three credits per investigation.
Returns threat score, VPN and proxy provider names, bot type, and the abuse contact email in a single Block Kit reply.
Uses a slash command rather than a message listener, so the bot only spends credits when someone asks.
Refuses to score three address types that break naive triage: corporate gateways, declared good bots, and bogons.
Full Bolt for Python code, with the ack() timing gotcha that breaks this on Lambda. You'll end up with a /ip command that takes an address, returns a structured verdict in about a second, and tells you exactly who to email if the answer is bad. The triage thresholds are mine, not a vendor's, and I'll show you where to move them.

What the bot does, and what it deliberately doesn't

The output contract is four lines: a score with a band, the flags that explain the score, the provider name where one exists, and an abuse email. Anything else is noise in a channel.

The design decision worth arguing about is the trigger. The obvious build listens for message events and auto-replies whenever an IP appears in text. One of the few existing open-source versions does exactly that. I think that's the wrong call, for three reasons.

It spends credits uninvited. Someone pastes a log excerpt with forty addresses in it and you've just burned 120 credits on data nobody asked for. It's noisy, because half of those addresses are your own load balancers. And it quietly enriches customer IPs that happened to land in a support thread, which is a privacy conversation you don't want to have retroactively.

A slash command is explicit. Someone types /ip 145.223.7.7 because they want to run the lookup. That's intentional invocation, cost control, and a clean audit line in one design choice.

Pitfall: slash-command responses are ephemeral by default, which is exactly what you want for customer IP investigations. Keep them that way unless you deliberately need an in_channel response, and avoid putting the raw enrichment payload into Slack. More on that at the end.

Two calls, three credits

The risk score and the abuse contact live on different endpoints. That sounds like a drawback until you look at what each one costs.

I'll use IPGeolocation for the examples because it returns both from the same key, and the abuse record is the part that's genuinely hard to find elsewhere. IPQualityScore and ipinfo both do risk scoring well; AbuseIPDB is a reporting database rather than a registry lookup; MaxMind GeoIP2 covers anonymity but not abuse contacts. If you already pay for one of those, the Slack and triage code below works unchanged, you'll just need a second source for the abuse address.

Start with the risk call.

curl -s -X GET \
  "https://api.ipgeolocation.io/v3/security?apiKey=$IPGEO_API_KEY&ip=145.223.7.7"

That returns the complete security object. This is the full shape, not a trimmed version, because the fields you ignore today are the ones you'll need the first time a verdict looks wrong:

{
  "ip": "145.223.7.7",
  "security": {
    "threat_score": 90,
    "is_tor": false,
    "is_proxy": true,
    "proxy_provider_names": ["NetNut", "ProxyScrape", "Oxy Labs", "DataImpulse"],
    "proxy_confidence_score": 99,
    "proxy_last_seen": "2026-09-01",
    "is_residential_proxy": true,
    "is_vpn": true,
    "vpn_provider_names": ["SurfShark VPN", "Ishaan VPN"],
    "vpn_confidence_score": 99,
    "vpn_last_seen": "2026-07-31",
    "is_relay": false,
    "relay_provider_name": "",
    "is_anonymous": true,
    "is_known_attacker": true,
    "is_bot": false,
    "bot_confidence_score": 0,
    "bot_operator_name": "",
    "bot_type": "",
    "is_known_good_bot": false,
    "bot_last_seen": "",
    "is_spam": true,
    "is_cloud_provider": true,
    "cloud_provider_name": "Brander Group Inc.",
    "is_corporate_gateway": false,
    "corporate_gateway_type": "",
    "corporate_gateway_provider_name": ""
  }
}

Two things to notice before you write any parsing code. proxy_provider_names and vpn_provider_names are arrays, not strings, and they're frequently empty. And the confidence scores default to 0 when their flag is false, so printing "VPN confidence: 0" next to "VPN: no" makes your bot look broken when it's working correctly.

The abuse call is separate and cheaper.

curl -s -X GET \
  "https://api.ipgeolocation.io/v3/abuse?apiKey=$IPGEO_API_KEY&ip=49.12.0.0"

{
  "ip": "49.12.0.0",
  "abuse": {
    "route": "49.12.0.0/20",
    "country": "DE",
    "name": "Hetzner Online GmbH - Contact Role",
    "organization": "Hetzner Online GmbH",
    "kind": "group",
    "address": "Hetzner Online GmbH, Industriestrasse 25, D-91710 Gunzenhausen, Germany",
    "emails": ["abuse@hetzner.com"],
    "phone_numbers": ["+4998315053", " +4998315050"]
  }
}

Every field in the IP abuse contact record can be empty, including abuse.emails. abuse.route gives you the CIDR range the contact covers, which is what you quote in the report. Plan for "no abuse contact on record" as a normal outcome rather than an exception, because it's the single most common runtime surprise in this build.

You can get both objects from the unified /v3/ipgeo endpoint with include=security,abuse. The docs recommend against it when you only want those two, and they're right: the dedicated endpoints return smaller payloads and charge predictably. Security costs 2 credits, abuse costs 1. Three credits per investigation, 150,000 credits on the $19 Starter plan, so roughly 50,000 investigations a month before you spend another cent.

Heads up: the free plan returns 401 Unauthorized on both of these endpoints. Not a rate limit, not a trimmed response, a flat 401. If you want to follow along end to end you need a paid key. Worth knowing before you copy the code and spend twenty minutes debugging your signing secret.

Setting up the Slack app

Create the app at api.slack.com/apps, then three things matter.

Slash command. Under Features, add a command. Name it /ip, point the Request URL at https://your-host/slack/events, and write a usage hint. Saving the command adds the commands OAuth scope automatically.

Signing secret. Basic Information, App Credentials. This is how you verify that requests actually came from Slack. Bolt checks it for you, including the timestamp freshness check that blocks replay attacks, but only if you pass it in.

Bot token. OAuth & Permissions, after installing to a workspace. Starts with xoxb-.

Then grab an API key for the IP data and put all three in the environment:

export SLACK_BOT_TOKEN="xoxb-..."
export SLACK_SIGNING_SECRET="..."
export IPGEO_API_KEY="..."

pip install slack_bolt requests

The enrichment client

Both calls go through one function. Timeouts are short on purpose: this runs inside a three-second budget and a hung connection is worse than a missing field.

import os
import logging
import requests

log = logging.getLogger(__name__)

API_BASE = "https://api.ipgeolocation.io/v3"
API_KEY = os.environ["IPGEO_API_KEY"]  # fail loudly at boot, not mid-request

# One session for the whole process. Connection reuse saves roughly a
# round trip per call, which matters inside a 3 second budget.
_session = requests.Session()


def _lookup(path: str, ip: str) -> dict | None:
    """Call one IPGeolocation endpoint. Returns None on any failure."""
    try:
        resp = _session.get(
            f"{API_BASE}/{path}",
            params={"apiKey": API_KEY, "ip": ip},
            timeout=(1.0, 1.5),  # connect, read
        )
    except requests.RequestException as exc:
        log.warning("lookup %s failed for %s: %s", path, ip, exc)
        return None

    if resp.status_code == 423:
        return {"_bogon": True}  # private or reserved, costs no credits
    if resp.status_code == 401:
        log.error("401 on %s. Security and abuse need a paid key.", path)
        return None
    if resp.status_code != 200:
        log.warning("lookup %s returned %s for %s", path, resp.status_code, ip)
        return None

    try:
        return resp.json()
    except ValueError:
        log.warning("non-JSON response from %s for %s", path, ip)
        return None

The 423 branch is worth keeping even though we validate before calling. It's the API telling you the address is a bogon or private, and those don't count against your quota, so the failure is free. Catching it explicitly means a /ip 10.0.0.5 from someone pasting an internal address gets a useful answer instead of a shrug.

Fail open, not closed. Both lookups returning None should produce "I couldn't check this one" in the channel, never a blocked user. A chat bot is not an enforcement point.

Parse the IP before you spend a credit

Slash command text is whatever the user typed, and Slack mangles it before you see it.

In several clients, a bare IPv4 address gets auto-linked, so your handler receives <http://145.223.7.7|145.223.7.7> rather than the address. Strip that before anything else touches the string. The first bug report you get will be this one.

import ipaddress
import re

IP_PATTERN = re.compile(r"[0-9a-fA-F:.]{3,45}")


def extract_ip(text: str) -> tuple[str | None, str | None]:
    """Return (ip, error). Strips Slack's auto-linking and rejects non-public IPs."""
    if not text or not text.strip():
        return None, "Usage: `/ip 8.8.8.8`"

    # Slack wraps things that look like links in <...>, sometimes as <http://1.2.3.4|1.2.3.4>
    cleaned = text.strip().strip("<>").split("|")[0]
    cleaned = cleaned.replace("http://", "").replace("https://", "").rstrip("/")

    match = IP_PATTERN.search(cleaned)
    if not match:
        return None, f"`{text[:40]}` doesn't look like an IP address."

    try:
        addr = ipaddress.ip_address(match.group(0))
    except ValueError:
        return None, f"`{match.group(0)}` isn't a valid IPv4 or IPv6 address."

    if not addr.is_global:
        return None, f"`{addr}` is private or reserved. Nothing to look up."

    return str(addr), None

Validating locally rather than letting the API reject the input is partly about speed and partly about the message the user sees. A 400 from an upstream service turns into "lookup failed", which tells nobody anything. \bananadoesn't look like an IP address tells them they typo'd.

addr.is_global covers RFC 1918 space, loopback, link-local, carrier-grade NAT, and the documentation ranges in one check. That last one matters more than it sounds: if a colleague tests your bot with 192.0.2.1 from an RFC, the API would reject it as a bogon anyway, and catching it locally saves the round trip and gives a clearer message.

Ack first, enrich second

Slack gives you three seconds to return HTTP 200 on a slash command. Two API calls plus formatting will usually fit, but "usually" is how you end up with intermittent timeout errors that nobody can reproduce.

from slack_bolt import App

app = App(
    token=os.environ["SLACK_BOT_TOKEN"],
    signing_secret=os.environ["SLACK_SIGNING_SECRET"],
)


@app.command("/ip")
def handle_ip_command(ack, command, respond, logger):
    ack()  # HTTP 200 goes out here, before any network work

    ip, error = extract_ip(command.get("text", ""))
    if error:
        respond(text=error, response_type="ephemeral")
        return

    try:
        security = _lookup("security", ip)
        abuse = _lookup("abuse", ip)
    except Exception:
        logger.exception("enrichment failed for %s", ip)
        respond(text=f"Lookup failed for `{ip}`. Try again shortly.")
        return

    if security is None:
        respond(text=f"No security data for `{ip}` right now.")
        return
    if security.get("_bogon"):
        respond(text=f"`{ip}` is a bogon or private address.")
        return

    respond(
    blocks=build_blocks(ip, security, abuse),
    text=f"IP report for {ip}",
    response_type="ephemeral",
)


if __name__ == "__main__":
    app.start(port=int(os.environ.get("PORT", 3000)))

ack() first is the whole trick. In the default Bolt configuration, calling it flushes the HTTP 200 immediately and the rest of the function runs after Slack has already stopped counting. respond() then posts the real answer through the response_url that came with the payload.

Pitfall: this pattern breaks on AWS Lambda and other FaaS runtimes, where you can't keep working after returning an HTTP response. There you need App(process_before_response=True) plus Bolt's lazy listener mechanism, which splits the handler into an ack function and a lazy list. Same three-second rule, different plumbing. If you deploy this to Lambda with the code above, the ack will be late and users will see a timeout even though the lookup succeeded.

One more constraint on response_url: Slack allows five responses within 30 minutes of the original payload. Plenty for one reply, not enough to build a progress indicator out of.

Reading the response like an analyst

Here's where most tutorials stop, and where the actual work starts. A score on its own doesn't tell anyone what to do.

Score	Check these flags first	Suggested action
0 to 19	`is_corporate_gateway`, `is_known_good_bot`	Allow. Treat gateway and good-bot hits as explicit allow signals, not merely low risk
20 to 44	`is_bot`, `bot_type`	Log it. Rate limit if `bot_type` is `scraper` or `scanner`
45 to 79	`is_vpn`, `is_residential_proxy`, confidence scores	Step up: MFA, email verification, hold the transaction
80 to 100	`is_known_attacker`, `is_spam`, `is_residential_proxy`	Block or queue for manual review, and file the abuse report

Those bands are mine, not the vendor's. The documentation deliberately doesn't publish thresholds, and that's the honest position, because the right cut depends entirely on what a false positive costs you. A blocked signup is cheap. A blocked checkout is not. Start with these numbers, watch a week of real traffic, then move the lines.

What the score can't do is tell you why, which is why the flags sit next to it in the reply. Four real addresses, four correct but completely different outcomes:

IP	Score	What it actually is	Right call
`145.223.7.7`	90	Residential proxy and VPN, 99 confidence on both, known attacker, spam	Block
`223.197.196.92`	40	Bot, `bot_type: brute_force`, confidence 95, known attacker	Rate limit and alert
`4.227.36.0`	15	Bot, operator OpenAI, `bot_type: ai_crawler`, known good bot	Allow, apply crawler policy
`87.58.66.106`	5	Corporate gateway, Zscaler, `secure_web_gateway`	Never block

Notice the middle two. Both are bots. One is trying passwords against your login endpoint, the other is indexing your docs. A bot flag alone would treat them identically.

The corporate gateway trap

87.58.66.106 scores 5 and belongs to Zscaler. It's the egress address of an enterprise secure web gateway, which means employee traffic gets proxied through the vendor's cloud and arrives at your site from their address space.

Block that IP and you lock out an entire company. Not a user, a customer.

The flag exists specifically to stop you doing that, and it carries two more consequences your fraud rules need to know about. Never use a gateway IP for per-user identity or deduplication, because one address legitimately fronts thousands of people. And expect the geolocation to resolve to the vendor's nearest point of presence rather than the user's actual city, so impossible-travel rules will fire on perfectly normal logins.

is_anonymous stays false on these, which is the correct behaviour. The range is published, the organisation is identifiable, and nobody is hiding. It's shared, not anonymous, and those are different problems.

Good bots score low for a reason

4.227.36.0 comes back with is_bot: true, bot_operator_name: "OpenAI", bot_type: "ai_crawler", and is_known_good_bot: true. It's also flagged as cloud provider space, because crawlers run on Microsoft infrastructure like everything else.

The useful pair is bot_type plus is_known_good_bot. The type describes what the automation does, not whether it's welcome, and the same value can appear on both sides. scanner covers both Censys doing published internet-wide research and someone probing your admin paths; is_known_good_bot is what separates them. Four types (exploit, credential_stuffing, brute_force, worm) always arrive with is_known_attacker: true, so those need no interpretation at all.

Treat bot_type as a plain string. New values get added, and a strict enum in your code will throw on the first one that ships.

Formatting the reply

Block Kit sections, not attachments. Attachments are legacy and render inconsistently in newer Slack clients.

BAND = {
    (0, 19): "Low",
    (20, 44): "Watch",
    (45, 79): "Elevated",
    (80, 100): "High",
}


def _band(score: int) -> str:
    for (low, high), label in BAND.items():
        if low <= score <= high:
            return label
    return "Unknown"


def _signals(sec: dict) -> str:
    """Only report flags that are true. A wall of 'false' teaches nobody anything."""
    out = []
    if sec.get("is_residential_proxy"):
        out.append(f"Residential proxy ({sec.get('proxy_confidence_score', 0)}%)")
    elif sec.get("is_proxy"):
        out.append(f"Proxy ({sec.get('proxy_confidence_score', 0)}%)")
    if sec.get("is_vpn"):
        names = ", ".join(sec.get("vpn_provider_names") or []) or "unnamed"
        out.append(f"VPN: {names}")
    if sec.get("is_tor"):
        out.append("Tor exit node")
    if sec.get("is_relay"):
        # Private Relay and WARP are privacy features, not evasion. Flag, don't punish.
        out.append(f"Relay: {sec.get('relay_provider_name') or 'unnamed'}")
    if sec.get("is_spam"):
        out.append("Spam source")
    if sec.get("is_corporate_gateway"):
        vendor = sec.get("corporate_gateway_provider_name") or "unknown vendor"
        out.append(f"Corporate gateway: {vendor}. Do not block")
    if sec.get("is_bot"):
        kind = sec.get("bot_type") or "unclassified"
        good = "declared" if sec.get("is_known_good_bot") else "undeclared"
        out.append(f"Bot: {kind} ({good})")
    if sec.get("is_known_attacker"):
        out.append("Known attacker")
    return "\n".join(f"- {s}" for s in out) or "- No anonymity or abuse signals"

Reporting only the true flags is a small decision that changes how the message reads. Twenty-seven fields rendered as a label-value grid is a data dump. Four lines that each say something is a report.

Two inclusions worth defending. is_spam belongs in a bot whose output feeds an abuse report, because spam is the complaint most networks actually act on. And is_relay is in there separately from is_vpn because iCloud Private Relay and Cloudflare WARP are privacy defaults that ordinary customers have switched on without thinking about it, not evasion tooling. Collapsing relay into VPN is how you end up treating half your iPhone users as suspicious.

def build_blocks(ip: str, sec_payload: dict, abuse_payload: dict | None) -> list:
    sec = sec_payload.get("security", {})
    score = sec.get("threat_score", 0)

    blocks = [
        {
            "type": "section",
            "text": {
                "type": "mrkdwn",
                "text": f"*`{ip}`*\nThreat score *{score}/100* ({_band(score)})",
            },
        },
        {
            "type": "section",
            "text": {"type": "mrkdwn", "text": _signals(sec)},
        },
    ]

    abuse = (abuse_payload or {}).get("abuse", {})
    emails = abuse.get("emails") or []
    if emails:
        org = abuse.get("organization") or abuse.get("name") or "unknown network"
        route = abuse.get("route") or "unknown range"
        blocks.append({
            "type": "section",
            "text": {
                "type": "mrkdwn",
                "text": f"*Report to:* {', '.join(emails)}\n{org} ({route})",
            },
        })
    else:
        blocks.append({
            "type": "context",
            "elements": [{"type": "mrkdwn", "text": "No abuse contact on record."}],
        })

    return blocks

Note emails[0] rather than joining the list. Registry records sometimes carry three addresses, and the first is almost always the one the operator actually monitors. Dumping all of them invites someone to CC a role account that bounces.

From flag to sent report

The abuse email is the point of the whole build, and it's the step most teams never automate because finding the address is tedious.

A word on what you actually get back. For 49.12.0.0 the record resolves to abuse@hetzner.com, a hosting provider that reads its abuse mailbox. For 1.0.0.0 it resolves to helpdesk@apnic.net, which is APNIC's own helpdesk rather than any network operator, because that range is registry research space. Same field, very different usefulness. The data is accurate; what varies is whether anyone on the other end can act.

Response rates vary enormously, and I'd rather say that plainly than imply a lookup guarantees a takedown. Large hosting providers generally act. Some networks have never replied to anything.

What makes a report someone can act on:

Subject: Abuse report: credential stuffing from 145.223.7.7

Source IP:   145.223.7.7
Range:       145.223.7.0/24
Activity:    Automated login attempts against our authentication endpoint
Window:      2026-10-04 02:11:07 UTC to 2026-10-04 02:48:31 UTC
Volume:      ~4,100 requests across 38 accounts
Reply to:    security@yourcompany.example

Sample log lines:
2026-10-04T02:11:07Z POST /api/v1/login 401 145.223.7.7 "python-requests/2.32"
2026-10-04T02:11:09Z POST /api/v1/login 401 145.223.7.7 "python-requests/2.32"

Timestamps in UTC with the timezone written out. A window, not a single moment. Raw log lines, because the operator needs to match them against their own records. And a reply address that a human reads.

If your bot has the data for the top half of that template already, generating it is a formatting exercise. That's the actual return on this build.

Things that will bite you

Caching. Someone pastes the same IP four times while a thread argues about it. That's 12 credits for one answer. A TTL cache fixes it in about fifteen lines:

import time

_cache: dict[str, tuple[float, dict]] = {}
CACHE_TTL = 900  # 15 minutes. Security data refreshes at least twice a day.


def cached_lookup(path: str, ip: str) -> dict | None:
    key = f"{path}:{ip}"
    hit = _cache.get(key)
    if hit and time.monotonic() - hit[0] < CACHE_TTL:
        return hit[1]

    result = _lookup(path, ip)
    if result is not None:  # never cache a failure
        _cache[key] = (time.monotonic(), result)
    return result

Fifteen minutes is deliberately well under the refresh cadence. Use Redis instead of a dict the moment you run more than one process, or your cache hit rate quietly drops to nothing.

Who can run it. Slash commands are available to everyone in the workspace by default. If you're on a paid plan with a real credit budget, check command["user_id"] against an allowlist, or restrict the app to specific channels. Otherwise someone will discover it and loop it over a subnet.

Privacy. A public channel is not where customer IP addresses should live. Use response_type="ephemeral" for anything triggered outside a dedicated security channel, and never log the full enrichment payload where support staff can read it. The address is personal data in several jurisdictions and your retention policy probably didn't anticipate a Slack message.

Bogons are free. Private and reserved addresses return 423 and cost nothing. That makes this API safe to point at raw log data without filtering first, though validating locally is still faster.

The 401 again. Free keys can't call either endpoint. If you're prototyping and everything else is wired correctly, that's the first thing to check.

Where to take it next

Add a button to the Block Kit message that drafts the abuse email into a thread, because the gap between knowing the address and sending the report is where this workflow still leaks. Swap the dict cache for Redis before you run a second worker. And once you've watched a week of real traffic, move those score bands, since the ones above are a starting point rather than a policy.

If your team is pasting more than a few hundred addresses a month, batch the ones that arrive in log excerpts instead of running them one at a time. The bulk endpoint takes up to 50,000 addresses per request at the same credit rate, which turns a noisy channel habit into a scheduled job.

Enrich Nginx Access Logs With Geo and ASN at Write Time

ABDULLAH AFZAL — Fri, 02 Oct 2026 19:18:01 +0000

Your nginx access log format can carry the client's country, city, and ASN. The lookup happens at write time, against a local database file, with no API call in the request path.

Most writing on this treats it as a module install. That is the wrong frame. The two modules worth using read the same file format, so the module is plumbing. The database you load is what decides which fields exist.

TL;DR

Two nginx modules, one file format. Both read MMDB through libmaxminddb.
The database decides which fields you get. GeoLite2 covers country, city, and the ASN number and organization, and it is free.
ASN type, company, hosting provider, and threat flags need a richer database.
Quote every variable in a JSON log format. An unquoted empty value writes "threat":, and hard-fails the parser. I tested this.
Fix the client IP before you log it. One module trusts a client-controlled header by default.
Swap database files with mv, never cp. The module has the file memory mapped. Pick the database for the fields you actually need, point whichever module you already have at it, then quote everything in the log format. The failure modes are all in the last two steps, not the install.

Which database gives you which field

This is the table I wanted when I started and could not find anywhere.

Field	GeoLite2 City	GeoLite2 ASN	IPGeolocation Geo	IPGeolocation ASN	IPGeolocation Security
Country code	yes	no	yes	no	no
City, latitude, longitude	yes	no	yes	no	no
Currency, timezone, calling code	no	no	yes	no	no
ASN number	no	yes	no	yes	no
ASN organization	no	yes	no	yes	no
ASN type (ISP, HOSTING)	no	no	no	yes	no
ASN RIR, allocation date	no	no	no	yes	no
Company name, domain, type	no	no	no	no	separate Company DB
Hosting provider	no	no	no	no	separate Hosting DB
Threat score, VPN, proxy, Tor flags	no	no	no	no	yes

Two things to know before you build a log format around this.

Everything arrives as a string. A threat score of 80 is the text 80, coordinates are text, and booleans come through as 1 or 0. Nginx variables have no other type, so whatever consumes the log does the casting.

ASN number formatting is not consistent across sources. GeoLite2 stores autonomous_system_number as an integer. Ours is documented as a bare integer in the database field reference, while the module docs show the variable rendering as AS1257. That difference decides whether your parser needs to strip a prefix, so check your own file rather than trusting either of us.

Two modules, one file format

Both candidates read MMDB files through libmaxminddb. That is the whole reason the database is the real decision.

	ngx_http_geoip2_module	ngx_http_ipgeolocation_module
Install	Distro package on most systems	Build nginx from source
Field access	You write the path mapping yourself	Ready-made `$ip_*` variables
Multiple databases	One block per file	Repeat the directive, first match wins
Reload on update	`auto_reload`	Reload nginx
Stream module	Yes, with `--with-stream`	HTTP only

On Ubuntu 24.04 the first one is one command:

# Pulls libmaxminddb as a dependency and drops a load_module line
# into /etc/nginx/modules-enabled/ for you.
sudo apt-get install -y libnginx-mod-http-geoip2

That installed cleanly for me at version 1:3.4-5build2 alongside nginx 1.24.0.

Pick by what you already have. If the packaged module is there, keep it and write the mappings. If you are compiling nginx anyway and want ASN type or threat flags without hand-writing a path for every field, the ready-made variables save real config. Rebuilding a working nginx purely to avoid writing ten mapping lines is not a trade I would make.

There is a third module, the original ngx_http_geoip_module in the nginx tree. It reads MaxMind's retired legacy format and is not built by default. Do not start anything new on it.

Recipe A: map the fields yourself

The packaged module takes a file and a set of paths into it. Each path is a sequence of keys, space separated, and the module's directive reference covers the rest.

load_module modules/ngx_http_geoip2_module.so;  # skip if your package enabled it

http {
    geoip2 /etc/nginx/ipgeo/GeoLite2-City.mmdb {
        auto_reload 5m;                        # re-reads the file without a reload
        $geo_country default=ZZ country iso_code;
        $geo_city    city names en;
    }

    geoip2 /etc/nginx/ipgeo/db-ip-asn.mmdb {
        auto_reload 5m;
        $geo_asn      asn as_number;
        $geo_asn_org  asn organization;
        $geo_asn_type asn type;                # not present in GeoLite2-ASN
    }
}

The paths differ per file because the schemas differ, and that is the only thing you change when you swap databases. GeoLite2 nests country data under country and the ASN fields at the top level as autonomous_system_number and autonomous_system_organization. Ours nests everything under asn.

I checked that the packaged module handles an arbitrary nested schema rather than only MaxMind's own, because Recipe A falls apart otherwise. Building a small MMDB with the asn nesting above and pointing the stock Ubuntu module at it returned asn=15169 org=Google LLC type=BUSINESS on a hit. A miss and an RFC 1918 address both came back as empty strings for every variable, with no error and no refusal to start.

Tip: default= only fires when the lookup misses, so default=ZZ gives you a sentinel country you can filter on later instead of a silent empty string. Useful on the country field, noise everywhere else.

Recipe B: use ready-made variables

The other module skips the mapping. You load files, it exposes variables.

http {
    # Declaration order is priority order: the first database
    # containing a requested field answers for it.
    ipgeolocation_db /etc/nginx/ipgeo/db-ip-geolocation.mmdb;
    ipgeolocation_db /etc/nginx/ipgeo/db-ip-asn.mmdb;
    ipgeolocation_db /etc/nginx/ipgeo/db-ip-security.mmdb;

    # Default is `on`, which reads the leftmost X-Forwarded-For value.
    # See the client IP section below before you leave it that way.
    ipgeolocation_trust_forwarded_header off;
}

That gives you $ip_country_code, $ip_city_name, $ip_asn, $ip_asn_organization, $ip_asn_type, $ip_threat_score and about seventy more, with no paths to maintain. Booleans normalise to 1 and 0, and array fields like VPN provider names join into one comma-separated string. Build steps and the full variable list are in the module's build and variable docs.

The cost is the build. It compiles into nginx against libmaxminddb, and a dynamic build has to match your running binary exactly or nginx rejects it as not binary compatible. Reuse the flags from nginx -V and add --with-http_realip_module, which is not built by default and which you will want in a moment.

One genuinely nice property: if a database fails to open, nginx refuses to start. A typo in a path is caught at deploy rather than becoming empty columns you notice three weeks later.

A JSON access log format that survives empty values

An nginx log_format is a string template with no idea what JSON is. That is where these setups break, and it has nothing to do with geolocation.

log_format geo_json escape=json '{'
    '"time":"$time_iso8601","ip":"$remote_addr","status":"$status",'
    '"country":"$geo_country","city":"$geo_city",'
    '"asn":"$geo_asn","asn_org":"$geo_asn_org","asn_type":"$geo_asn_type"'
'}';

access_log /var/log/nginx/access.geo.log geo_json;

Every value is quoted, including the ones that look numeric. That is deliberate. Private addresses, unlisted ranges, and any miss resolve to an empty string, and an unquoted empty value produces this:

{"ip":"127.0.0.1","threat":,"ok":"US"}

That is not valid JSON. I ran it through a parser to be sure, and it fails outright at the empty value. Quoted, the same miss writes "threat":"", which parses and lets you decide what an empty score means downstream.

escape=json handles the other half. City names and organisation names contain quotes, backslashes and non-ASCII characters, and without it one Turkish city name corrupts the line for every parser reading that file.

One subtlety worth knowing, because the nginx log module docs blur it. Their note that a hyphen is logged when the variable value is not found applies to variables that are genuinely unset, like $upstream_addr on a request that never reached an upstream. A geo module on a miss gives you a variable that exists and is empty, which is a different case and logs as empty. I tested both under each escape mode. The practical upshot: under escape=default an unset variable becomes the string -, which sails through a JSON parser and then poisons anything casting that field to a number. Under escape=json you get "" instead.

Get the client IP right before you log it

If nginx sits behind a load balancer, everything above geolocates your load balancer.

The fix differs by module, and one of the defaults is dangerous.

Module	Default source	How to correct it
`ngx_http_geoip2_module`	`$remote_addr`	Set `geoip2_proxy` for trusted ranges, or `source=$variable`
`ngx_http_ipgeolocation_module`	Leftmost `X-Forwarded-For`	Set `ipgeolocation_trust_forwarded_header off` and use realip

The second default is the one to watch. The leftmost X-Forwarded-For value is written by the client. Plenty of proxies append to that header instead of replacing it, so a visitor can send their own and choose which country they appear from. If your log is decorative, that costs you nothing. If anything reads it to make a decision, it is a hole.

http {
    ipgeolocation_trust_forwarded_header off;  # ignore the client-supplied header

    set_real_ip_from 10.0.0.0/8;               # your load balancer CIDR, nothing wider
    real_ip_header   X-Forwarded-For;
    real_ip_recursive on;                      # walk back past trusted hops
}

The realip module rewrites $remote_addr to the real client before the geo lookup runs, using only hops you named. Both modules then geolocate the right address. Get the trusted range wrong and you are back to trusting the client, so keep it tight.

Derive fields at write time

Once the variables exist you can compute things before the line is written, which beats doing it in every query later.

# ASN type casing varies between sources, so match case-insensitively.
map $geo_asn_type $is_hosting {
    default      0;
    "~*hosting"  1;
    "~*business" 0;
}

# Strip an AS prefix if your database renders one, so the field is
# always a bare number for whatever parses the log.
map $geo_asn $asn_num {
    default   $geo_asn;
    "~^AS(?<n>\d+)$"  $n;
}

# Datacenter traffic into its own file. A request is skipped when the
# condition is 0 or empty, so misses stay in the main log.
access_log /var/log/nginx/access.hosting.log geo_json if=$is_hosting;

Splitting hosting traffic out is useful because ASN type tells you whether an address belongs to a hosting network. It does not tell you whether the request is human or automated, so treat it as context rather than identity.

Keeping databases fresh without dropping requests

Both modules memory map the file. Overwriting it in place is how you get workers reading half a file.

#!/bin/sh
set -e
DIR=/etc/nginx/ipgeo
curl -fsSL "$DOWNLOAD_URL" -o "$DIR/db-ip-asn.mmdb.tmp"

# Same filesystem, so mv is atomic. cp and wget -O are not.
mv "$DIR/db-ip-asn.mmdb.tmp" "$DIR/db-ip-asn.mmdb"

# Never reload onto a truncated download.
nginx -t && nginx -s reload

With auto_reload set, the packaged module picks the new file up on its own and you can drop the reload line. Without it, reload. Either way the atomic mv is not optional.

If you are on GeoLite2, the licence terms require you to keep the data current, and geoipupdate is the supported path. Update cadence varies by database: IPGeolocation publishes refreshed databases daily, while MaxMind currently publishes GeoLite ASN every weekday and GeoLite City and Country on Tuesdays and Fridays.

Write time, ingest time, or query time

Doing this in nginx is one of three places, and it is not always the right one.

	Write time (nginx)	Ingest time (pipeline)	Query time (dashboard)
Request latency	none, local lookup	none	none
Database lives on	every web node	one pipeline node	one place
Re-enrich old logs	not possible	reprocess	free
Reflects	the moment of the request	the moment of ingest	today's data

The trade is historical accuracy against flexibility. A write-time country is what that address resolved to when the request happened, which is what you want for an incident review and wrong when an address changes hands and you need to re-attribute.

My rule: enrich at write time when nginx already needs the data to make a decision, because the lookup is happening anyway and the log line is free. Otherwise enrich at ingest, where one node holds the database and you can reprocess. If you are running the Elastic Stack there is a companion piece on enriching at ingest instead, and if you want these fields on a map there is one on shipping them to Loki and plotting them in Grafana.

What to do with the ASN column

A country column gets looked at. An ASN column mostly does not, which is a waste, because it is the field that tells you who actually owns the traffic.

Start by ranking:

jq -r '.asn_org // empty' /var/log/nginx/access.geo.log \
  | sort | uniq -c | sort -rn | head -20

Your top twenty organisations usually cover most of your traffic. That is a small enough set to enrich properly, once, instead of per request:

import os
import sys
import requests

API_KEY = os.environ.get("IPGEO_API_KEY")
ASN_URL = "https://api.ipgeolocation.io/v3/asn"


def describe(asn):
    """Fetch routing context for one ASN. Returns None on any failure."""
    if not API_KEY:
        raise RuntimeError("IPGEO_API_KEY is not set")

    try:
        resp = requests.get(
            ASN_URL,
            params={
                "apiKey": API_KEY,
                "asn": asn,
                # Relationships cost nothing extra; the charge is per ASN.
                "include": "upstreams,peers",
            },
            timeout=(2.0, 10.0),
        )
        resp.raise_for_status()
    except requests.RequestException as exc:
        print(f"lookup failed for {asn}: {exc}", file=sys.stderr)
        return None

    data = (resp.json() or {}).get("asn") or {}
    return {
        "number": data.get("as_number"),
        "org": data.get("organization"),
        "type": data.get("type"),
        "rir": data.get("rir"),
        "ipv4_routes": data.get("num_of_ipv4_routes"),
        "upstreams": [u.get("as_number") for u in data.get("upstreams") or []],
    }

Twenty organisations is twenty lookups, once, cached in whatever you already run. The response carries more than the database does, including the official ASN handle, allocation status, announced route counts, and the upstream and peer relationships:

{
  "asn": {
    "as_number": "AS24940",
    "organization": "Hetzner Online GmbH",
    "country": "DE",
    "type": "HOSTING",
    "domain": "hetzner.com",
    "date_allocated": "2002-06-03",
    "asn_name": "HETZNER-AS",
    "allocation_status": "ASSIGNED",
    "num_of_ipv4_routes": "91",
    "num_of_ipv6_routes": "5",
    "rir": "RIPE"
  }
}

Note the route counts and as_number come back as strings here too. Consistency, at least.

A few extra notes

IPv6. Both modules look up whichever address the client connects with, so the only thing that usually goes wrong is nginx not listening on v6 at all. Check for listen [::]:80;.

Private addresses. Internal health checks and anything behind a NAT will log empty geo fields forever. That is correct behaviour, not a misconfiguration, and it is the main reason the quoting rule above matters.

Retention. An IP address can be personal data, and in access logs where it can reasonably be linked to a visitor it should generally be treated that way. Adding city-level geolocation is additional processing. Enriching at write time means that data lands in a file that may have a longer retention period than your application database, so check the retention policy before turning this on fleet-wide.

Stream traffic. The packaged module works in stream blocks if you compiled with --with-stream, so TCP proxying can carry the same data. The other module is HTTP only.

Decide what an empty value means to you before you write a single if against these variables, because nginx treats both 0 and empty as false and that turns a missing database into a silently permissive rule. And pin your module build to your nginx version in CI. The one that bites is a routine package upgrade moving nginx out from under a dynamic module that was compiled against the old one.

Build a Grafana Geomap of Traffic and Threat Score

ABDULLAH AFZAL — Mon, 28 Sep 2026 16:31:11 +0000

A Grafana Geomap showing traffic by country and coloured by threat score takes about an hour to build. Almost none of that hour is the map.

The map is a built-in panel and it does what you'd expect. The work is upstream: getting a country code and a risk score onto every log line, cheaply enough that you're not paying per request for the privilege. And if you search for this, you'll find a pile of tutorials that solve it with a panel that stopped loading in May 2025.

TL;DR

Use the Geomap panel. The Worldmap plugin is AngularJS, and Angular support was removed outright in Grafana 12.
Enrich the log line at the edge with an Nginx module reading local MMDB files, or batch-enrich unique IPs through an API and cache the result. Both work. They fail at different scales.
Ship with Alloy, not Promtail. Promtail was removed from Loki in 3.7.3.
Colour the map by max threat score, size markers by request count. Don't average them.
The gotchas that will eat your afternoon: latitude and longitude come back as strings, Geomap only auto-detects a field literally named lookup, and X-Forwarded-For is spoofable. You'll end up with one panel that answers two questions at once: where is my traffic from, and how much of it should I be worried about. The second question is the one nobody's dashboard answers.

What you're building

One Geomap panel. Countries shaded by the highest threat score seen from that country in the window, with a circle marker sized by request volume sitting on top. A country that's big and green is fine. A country that's small and red is the one you want to click into.

The pipeline is four pieces: Nginx writes a JSON access log that already contains the country and the risk score, Alloy tails it, Loki stores it, Grafana queries it. Nothing here is exotic. The only decision that matters is how the country and score get into that log line, which is the next section and the bulk of the article.

I'm using Nginx because it's what most people reading this have in front of them. Traefik, Caddy, or an ALB with structured logs all work the same way once the fields exist; only the enrichment step changes.

Why the tutorial you found doesn't work

Worth two minutes before you copy anything from elsewhere.

Worldmap is gone, not deprecated

The Worldmap Panel plugin page says it's deprecated and points you at Geomap, now the only world map panel Grafana ships. The repo was archived on January 23, 2025. More importantly, Worldmap is an AngularJS plugin, and Angular support was removed entirely in Grafana 12. There's no config toggle to bring it back.

So this isn't a "you should migrate eventually" situation. On any supported Grafana, a Worldmap panel doesn't render. Dashboards on grafana.com still ship it, blog posts from December 2025 still teach it, and if you ask ChatGPT this question it'll tell you to install the plugin. I tried. It did.

Promtail was deleted

The Loki 3.7 release notes put it plainly: Promtail was deprecated in Loki 3.0 and has been removed as of 3.7.3. The code lives in Alloy now.

If you've got a promtail-config.yaml you like, alloy convert --source-format=promtail --output=config.alloy promtail-config.yaml will port it. One command, and then you never think about it again.

Getting country and threat score into the log line

Two ways to do this. They're genuinely different architectures, not variations, and the right answer depends entirely on how many unique IPs you see.

For the data itself, the usual suspects are IPGeolocation, MaxMind GeoIP2, ipinfo, ip-api, IP2Location, and IPLocate. They'll all give you a country. Where they diverge is risk: a composite threat score with VPN, proxy, residential proxy, Tor, and known-attacker flags behind it is a narrower field, and it's the reason I'm using ipgeolocation.io for the examples here. Swap in whatever you already pay for; the panel work at the end is identical.

Path A: enrich at the edge

An Nginx module reads MMDB files off local disk and exposes the data as native $ip_* variables. No outbound request, no per-request cost, lookups happen in-process against a memory-mapped file.

http {
    # Declaration order is priority order. First DB with the field wins.
    ipgeolocation_db /etc/nginx/ipgeo/db-ip-geolocation.mmdb;
    ipgeolocation_db /etc/nginx/ipgeo/db-ip-security.mmdb;

    # Off, because the leftmost X-Forwarded-For value is client-controlled.
    # See the note below before you flip this.
    ipgeolocation_trust_forwarded_header off;

    # Let realip establish the true peer from trusted proxies only.
    set_real_ip_from  10.0.0.0/8;
    real_ip_header    X-Forwarded-For;
    real_ip_recursive on;

    # escape=json stops a quote in a city name from breaking every downstream parser.
    log_format geo_json escape=json '{'
        '"time":"$time_iso8601","ip":"$remote_addr","status":"$status",'
        '"country":"$ip_country_code","city":"$ip_city_name",'
        '"asn":"$ip_asn","org":"$ip_asn_organization",'
        '"vpn":"$ip_is_vpn","proxy":"$ip_is_proxy","tor":"$ip_is_tor",'
        '"threat":"$ip_threat_score"}';

    access_log /var/log/nginx/access.geo.log geo_json;
}

Every variable resolves from whichever database contains it, so $ip_country_code comes from the geolocation file and $ip_threat_score from the security file, automatically. Build instructions and the full variable list are in the Nginx module docs; you compile it into Nginx against libmaxminddb, which is the main friction in this path.

The ipgeolocation_trust_forwarded_header off line is the one people get wrong. Default is on, which reads the leftmost X-Forwarded-For address. Plenty of load balancers append to that header rather than replacing it, which means a visitor can send their own X-Forwarded-For and pick which country they show up as on your map. If your map is decorative, fine. If you're going to make decisions from it, turn it off and let realip work out the true client from proxies you actually trust.

Pitfall: never overwrite a live .mmdb in place with cp or wget -O. The module memory-maps the file and workers can read a half-written one. Download to a temp file in the same directory, mv it over, then nginx -t && nginx -s reload.

The trade-off, stated once: this costs nothing per request and adds no latency, but you're rebuilding Nginx from source and licensing a database. That's a real tax if you've got one small VPS. It's nothing if you've got forty.

Path B: batch-enrich unique IPs through an API

No Nginx rebuild. Log the raw IP, then enrich out of band on a schedule and join at query time.

The trick is that you enrich unique IPs, not requests. A million requests a day from twelve thousand distinct addresses is twelve thousand lookups, not a million.

import os
import logging
import requests

API_KEY = os.environ.get("IPGEO_API_KEY")
BULK_URL = "https://api.ipgeolocation.io/v3/ipgeo-bulk"

# The endpoint takes 50,000 IPs per call. Smaller chunks fail cheaper and retry faster.
CHUNK = 1000

log = logging.getLogger(__name__)

Now the enrichment itself. The two things doing real work here are the bogon skip and the X-Credits-Charged header, and neither is obvious until the first time you get a surprise invoice.

def enrich(ips):
    """Return {ip: {"country": str, "threat": int}} for a list of unique IPs."""
    if not API_KEY:
        raise RuntimeError("IPGEO_API_KEY is not set")

    out = {}

    for i in range(0, len(ips), CHUNK):
        batch = ips[i:i + CHUNK]
        try:
            resp = requests.post(
                BULK_URL,
                params={
                    "apiKey": API_KEY,
                    "include": "security",
                    # Trim the payload. Bulk responses get large fast.
                    "fields": "location.country_code2,security.threat_score",
                },
                json={"ips": batch},
                timeout=(3.0, 30.0),
            )
            resp.raise_for_status()
        except requests.RequestException as exc:
            # Fail open: a missing enrichment loses map detail, it doesn't drop logs.
            log.warning("Enrichment batch %d failed: %s", i // CHUNK, exc)
            continue

        # Actual charge, not your estimate of it.
        log.info("Batch %d charged %s credits",
                 i // CHUNK, resp.headers.get("X-Credits-Charged", "?"))

        for rec in resp.json():
            # Private and bogon IPs come back as {"message": "... is a bogon IP address."}
            # and aren't billed. Access logs are full of them.
            if "message" in rec or "ip" not in rec:
                continue

            country = (rec.get("location") or {}).get("country_code2")
            threat = (rec.get("security") or {}).get("threat_score")
            if country:
                out[rec["ip"]] = {"country": country, "threat": threat or 0}

    return out

Write that output to Redis or a Postgres table keyed on IP, run it every fifteen minutes against IPs you haven't seen before, and join it in at query time.

Two things to know before you commit to this path. Bulk and include=security are both paid-only; a free key returns 401 on either. And geolocation plus security is 3 credits per IP, so the $19 Starter plan's 150,000 credits buys you about 50,000 unique IPs a month. That's a comfortable fit for a mid-sized site and completely wrong for a big one.

So which one

If you see under about 50,000 unique IPs a month and you don't control the Nginx build, Path B. It's less setup and the cost is predictable.

Above that, Path A, and it stops being close. At a million unique IPs the API path is a line item and the database path is the same price it was at ten thousand. The Nginx rebuild feels like the complicated option right up until the moment it's the only one that makes sense.

Either way, the log line you end up with is the same shape, and everything below this point doesn't care which one you picked.

Shipping to Loki with Alloy

local.file_match "nginx" {
  path_targets = [{
    "__path__" = "/var/log/nginx/access.geo.log",
    "job"      = "nginx",
  }]
}

loki.source.file "nginx" {
  targets    = local.file_match.nginx.targets
  forward_to = [loki.process.nginx.receiver]
}

loki.process "nginx" {
  stage.json {
    expressions = {
      country = "country",
      status  = "status",
    }
  }

  // Label only what you group by. threat stays in the line on purpose.
  stage.labels {
    values = {
      country = "",
      status  = "",
    }
  }

  forward_to = [loki.write.default.receiver]
}

loki.write "default" {
  endpoint {
    url = sys.env("LOKI_PUSH_URL")
  }
}

Save as config.alloy, run alloy run config.alloy.

That comment about labels is the important line in the file. country and status are bounded sets, so they make fine labels. threat is an integer from 0 to 100, and promoting it to a label would multiply your stream count by up to a hundred for no benefit. Leave it in the log line and let the query parse it. Same goes for ip, which is the single fastest way to melt a Loki instance.

Building the panel

Three settings decide whether a Grafana map renders anything useful: the shape of the query, the location mode, and what you bind colour and size to. In that order.

The queries

Two queries, both set to Instant rather than Range. A map wants one value per country, not a time series per country.

Query A, the colour:

max_over_time(
  {job="nginx"}
  | json
  | unwrap threat
  | __error__=""
  [$__range]
) by (country)

Query B, the size:

sum by (country) (
  count_over_time({job="nginx"}[$__range])
)

A few things worth knowing here. max_over_time needs an unwrap to get a number out of a log line, and count_over_time breaks if you give it one. That asymmetry catches people. The | __error__="" after the unwrap is doing real work: when an IP isn't in the database the field comes back empty, unwrap fails on it, and without the filter those errors poison the result. It has to sit after the unwrap, because only label filters can follow one.

Query B doesn't need | json at all, since country is already a stream label. Parsing you don't need is parsing you pay for.

Then add a Join by field transformation on country to merge the two results into one frame. Without it, Geomap sees two unrelated tables and draws nothing useful.

Location mode, and the trap

This is the step that silently produces an empty map.

Geomap's Auto location mode only detects fields named exactly geohash, latitude / lat, longitude / lng / lon, or lookup. Our field is called country, so Auto finds nothing and you get a basemap with no markers and no error message.

Set it explicitly instead: Lookup mode, with Lookup field country and Gazetteer Countries. The built-in gazetteer takes ISO 3166 alpha-2 and alpha-3 codes, and $ip_country_code / country_code2 are both alpha-2, so they map straight through.

You could also just name the field lookup in your LogQL and let Auto handle it. I'd rather have a readable field name and two extra clicks.

Heads up: if you're tempted to use coordinates instead of country codes, note that the API returns location.latitude and location.longitude as strings, not numbers. Geomap's Coords mode wants numeric fields. You'll need a Convert field type transformation, and for a country-level map the coordinates buy you nothing anyway.

Colour and size

Marker colour from the threat field, marker size from the count field.

For thresholds, the bands I use:

Score	Read it as
0-19	Background noise
20-44	Worth context before acting
45-79	Add friction, don't block
80-100	Manual review territory

And the opinion this whole panel rests on: use max, not mean.

An average threat score per country is worse than no score. A country sending you 40,000 clean requests and 60 requests from a residential proxy network averages out to roughly zero, the map goes green, and the thing you built this to see is the thing it hides. Max surfaces it. Then the size marker tells you whether that max came from a country sending you real traffic or from three requests you can ignore.

If max feels too jumpy once you've watched it for a week, quantile_over_time(0.95, ...) is the sensible next stop. Mean isn't.

Basemap

Default basemap is CARTO, and it's the right default. Set the theme to match your dashboard rather than leaving it on auto, because auto-switching a basemap under a fixed threshold palette makes your reds look different depending on someone's OS setting.

Turn showLabels off. Country names printed under semi-transparent markers is a mess, and the tooltip already tells you which country you're hovering.

Reading the threat map without overreacting

A red country is not an attack. It's a signal that something in that country's traffic tripped a detection, and the detections cover a lot of ground.

is_relay fires on iCloud Private Relay. is_corporate_gateway fires on Zscaler and similar egress. is_anonymous fires on all of it. Those are your customers: someone on an iPhone with Private Relay on, or an employee behind their company's secure web gateway. Blocking on those flags means blocking paying users, and you'll find out slowly through support tickets rather than quickly through an alert.

What's actually interesting is shape. A country with a high max threat score and low volume is usually one scanner and not worth your morning. A country with a high max score, high volume, and a sudden change from yesterday is the one to click into, and that's where the asn, org, and proxy fields you logged start paying off: residential proxy traffic spread across dozens of consumer ISPs in one country looks very different from one datacenter ASN hammering /wp-login.php.

Use the map for triage. Enforce somewhere with more context.

A few extra notes

IPv6 works but the country labels get patchy. Both paths handle v6 addresses fine. Coverage on recently allocated v6 blocks is thinner than v4 across every provider I've used, so expect a few empty country values in a v6-heavy audience. They'll drop out via the __error__ filter rather than breaking anything.

Refresh the databases on a schedule. Path A databases publish daily. A cron entry doing the atomic-swap dance and a reload is enough. Stale security data degrades quietly, which is the worst way for data to degrade.

Empty countries usually mean private IPs. If a chunk of traffic has no country, check whether you're geolocating your load balancer instead of your visitors. That's the realip config, not the database.

Log retention is a GDPR question now. An IP plus a city is personal data in most of the jurisdictions that care. Whatever retention policy you have for access logs already applies; just don't accidentally give this one a longer one because it's in a nicer dashboard.

Where this breaks first

Cardinality, and sooner than you'd think. country and status as labels are fine. The first time someone adds ip or asn to stage.labels because they want to filter on it in the panel, your stream count explodes and query times go with it. Filter on parsed fields at query time instead; it's slower per query and enormously faster overall.

If you're standing this up on a single node, the panel will feel instant for months and then won't. That's the moment to look at what you labelled, not at your Loki resources.

Route Risky Signups Automatically With n8n

ABDULLAH AFZAL — Mon, 28 Sep 2026 08:15:22 +0000

A signup fraud check in n8n often comes down to one IF node: if is_vpn is true, block. It runs, it looks reasonable on the canvas, and it still gets the decision wrong in both directions.
It blocks legitimate VPN users, including employees whose company routes traffic through a corporate VPN. It misses privacy relays such as iCloud Private Relay, and it can wave through a residential proxy that is not also classified as a VPN.

The fix isn't a better boolean. It's a score with four branches under it.

TL;DR

One lookup returns a 0 to 100 threat score plus 28 fields explaining how it got there
Route on the score, override on the flags. Corporate gateway and relay traffic needs handling before the score is read, not after
The published bands are 1-19 allow, 20-44 add context, 45-79 add friction, 80-100 block or send to review
A Switch node with four outputs does the routing. No Code node required
Security lookups cost 2 credits, so a 150,000 credit month covers 75,000 scored signups The workflow is a webhook, one lookup node, one override check, and a Switch with four outputs. Every signup gets a number, every number gets a branch, and the two categories that break naive VPN rules get handled before the score is read. Build time is under an hour.

What one IF node gets wrong

Three failure cases, and they're not edge cases.

Corporate gateways. Zscaler, Netskope and other secure web gateways route an entire workforce through a handful of egress IPs. Take 87.58.66.106: threat score 5, is_corporate_gateway: true, corporate_gateway_provider_name: "Zscaler". And is_anonymous stays false, because a gateway isn't hiding anyone. It's an identifiable company inspecting its own employees' traffic. Block that IP and you haven't blocked a fraudster, you've blocked a company. If you sell B2B, that's the buyer evaluating your product from inside their own network.

Privacy relays. iCloud Private Relay and Cloudflare WARP set is_relay, not is_vpn. A rule written against is_vpn won't catch them, which is the right outcome. Catching them means turning away iOS users who never opted into anything unusual.

Residential proxies. 145.223.7.7 sits on an ASN registered as an ISP. It's also a residential proxy with four provider names attached, a VPN exit, a spam source and a known attacker, scoring 90. A "is this a datacenter range" check clears it. An is_vpn check happens to catch this particular one, but only because it doubles as a VPN. A residential proxy that doesn't would sail straight through.

The pattern in all three: the flag tells you what the IP is. It doesn't tell you what to do. Those are different questions and the second one needs more than one bit.

The signals you actually route on

Here's the complete security object for that residential proxy IP. Not a trimmed version, because the fields you ignore today are the ones you'll want when you start tuning.

{
  "ip": "145.223.7.7",
  "security": {
    "threat_score": 90,
    "is_tor": false,
    "is_proxy": true,
    "proxy_provider_names": ["NetNut", "ProxyScrape", "Oxy Labs", "DataImpulse"],
    "proxy_confidence_score": 99,
    "proxy_last_seen": "2026-09-01",
    "is_residential_proxy": true,
    "is_vpn": true,
    "vpn_provider_names": ["SurfShark VPN", "Ishaan VPN"],
    "vpn_confidence_score": 99,
    "vpn_last_seen": "2026-07-31",
    "is_relay": false,
    "relay_provider_name": "",
    "is_anonymous": true,
    "is_known_attacker": true,
    "is_bot": false,
    "bot_confidence_score": 0,
    "bot_operator_name": "",
    "bot_type": "",
    "is_known_good_bot": false,
    "bot_last_seen": "",
    "is_spam": true,
    "is_cloud_provider": true,
    "cloud_provider_name": "Brander Group Inc.",
    "is_corporate_gateway": false,
    "corporate_gateway_type": "",
    "corporate_gateway_provider_name": ""
  }
}

Twenty-eight fields. Three of them change how you build this.

The confidence scores

proxy_confidence_score and vpn_confidence_score grade each detection 0 to 100 independently of the threat score. A VPN detection at 99 with a vpn_last_seen of last week is not the same signal as one at 40 last seen in March. If you're going to add friction at a threshold, the confidence score is what tells you whether to make that friction an OTP or a hard stop.

The two flags that outrank the score

is_corporate_gateway and is_relay both need special handling, but for different reasons. A corporate gateway is shared enterprise egress and is not treated as anonymous. A relay is an anonymizing privacy route such as iCloud Private Relay or Cloudflare WARP, but relay use alone is normally a low-risk signal. Neither should be blocked simply because of what it is. Check explicit attacker or attack-bot signals first, then use these flags to avoid unnecessary friction from the generic score.

That also means the workflow should not blindly send is_relay: true straight to Allow if is_known_attacker is also true.

bot_type, if you care about automation

is_bot alone treats Googlebot and a credential-stuffing script identically. bot_type separates them by activity: search_engine, ai_crawler, scanner, scraper, brute_force, credential_stuffing, exploit, worm. Pair it with is_known_good_bot and you can allow declared crawlers while blocking the four attack types, which always come back with is_known_attacker: true already set.

For a signup flow this matters less than it does at your edge, since bots rarely complete a signup form. Worth knowing the field exists before you write a rule against a bare is_bot.

The routing matrix

These bands are the published threat score scale, not thresholds I made up. Start here, then tune against your own traffic.

Score	Published guidance	Your branch	What actually happens
1-19	Allow with standard controls	Allow	Create the account. Log the score
20-44	Combine with other signals before acting	Verify	Email verification before first login
45-79	Add friction before allowing access	Review	Create the account, flag it, hold trial credits
80-100	Block, hard challenge, or manual review	Block	Reject, alert, log everything

A worked example per band, using real addresses you can check yourself:

IP	Score	Why	Branch
`87.58.66.106`	5	Zscaler gateway, not anonymous	Allow
2.56.188.35	35	Known attacker history, so the attacker override applies	Block
223.197.196.92	40	bot_type: brute_force, which also sets is_known_attacker	Block
`145.223.7.7`	90	Residential proxy, VPN, spam, attacker	Block

I don't have a verified sample in the 45-79 range to show you. The band exists and the guidance is published, but I'd rather say that than invent a payload that looks authoritative and isn't. If you run this on real traffic you'll find your own inside a week, and those are the ones worth studying anyway.

Pitfall: don't copy those four branch actions as-is. A consumer app and a fintech onboarding flow have different tolerances for a false positive at 40. Run the workflow in log-only mode for a week first, look at where your actual signups cluster, then set the boundaries.

Building it

Send the IP from your app

The first mistake is trying to work out the signup IP from inside n8n. Your webhook sees whatever hop reached it, which behind a proxy or load balancer is the proxy. Capture the client IP in your app, where you already know your own trust-proxy configuration, and send it in the webhook payload.

That makes the IP available as {{ $json.body.ip }} in the node that follows.

Add the lookup node

Search for IPGeolocation in the node panel. It's verified by n8n, so it installs from search in one click with no npm setup. On self-hosted, an instance owner does that once and everyone on the instance gets it.

Pick the Get IP Security action and paste your API key into the credential once. That key then works across every operation from the same node, which matters later if you add an ASN or abuse-contact lookup to the same flow.

One thing to know up front: Get IP Security is a paid operation. The free tier covers geolocation, timezone, time conversion and astronomy, and a free geolocation lookup does still return the ASN and the network owner. That's enough for a crude hosting-versus-ISP split if you want to prototype the shape of the workflow before paying for anything, but it won't give you a threat score.

I'm using IPGeolocation here because the security response carries provider names and confidence scores in the same call as the score, which is what makes a four-branch route possible without a second lookup. Greip has an n8n node with IP scoring, IPQualityScore and AbuseIPDB are both common in n8n fraud templates, and MaxMind minFraud is the enterprise option. They return different shapes. If you swap one in, the routing logic holds and the field names won't.

Override before you score

Add an IF node between the lookup and the Switch. Condition: {{ $json.security.is_corporate_gateway }} is true, or {{ $json.security.is_relay }} is true.

True output goes straight to Allow. These are shared addresses belonging to identifiable organisations, and the useful move is the opposite of blocking: relax your per-IP rate limits, and stop using the IP as a deduplication key. One Zscaler egress will happily produce forty signups from forty real employees, and any "more than three accounts from one IP" rule will read that as an attack.

False output continues to the Switch.

The Switch

Add a Switch node in Rules mode with four outputs, reading {{ $json.security.threat_score }}:

Output	Condition
Allow	less than 20
Verify	20 to 44
Review	45 to 79
Block	80 or above

Set a fallback output and send it to Verify. If the score is missing or arrives as something unexpected, a new signup getting an email verification is a cheap wrong answer. Silently allowing it is not.

Wire whatever you already use to each output. Slack alert on Block, a row in your review sheet on Review, your normal welcome sequence on Allow. The routing is the article; the destinations are yours.

When the lookup fails

Every node that calls an external service will eventually not get an answer. In n8n that's the On Error setting in the node's settings tab, and the choice you make there is a policy decision, not a config detail.

For signups, fail open. Set the lookup node to continue using its error output, wire that output to Verify, and log it. A scoring API being unreachable should never stop someone creating an account, and email verification is a reasonable default for a signup you couldn't score.

For anything where money moves, flip it. A payment or payout flow should fail closed and hold the transaction for review, because the cost of letting one through unscored is higher than the cost of making someone wait.

Say which one you picked in a sticky note on the canvas. Six months from now, when someone asks why unscored signups end up in the verification queue, that note is the answer.

Fraud detection automation without an LLM in the decision path

n8n's template gallery has several fraud workflows that pass the event to a language model and ask it to assess risk. I'd push back on that for this job.

A threat score is a number produced by enumerating proxy infrastructure, running honeypots and watching behaviour over time. Asking a model to look at that number and decide adds latency, cost per signup, and non-determinism to a decision that was already made. It also destroys your audit trail. When a customer emails asking why their signup was blocked, "the model thought it was risky" is not an answer you can defend, and in some jurisdictions it's not an answer you're allowed to give.

Score deterministically. Branch deterministically. Then, if you want, let a model write the Slack message that explains the block in plain English to whoever's on duty. Explanation is a good use of a model. Adjudication isn't.

This is the part of n8n fraud detection people get backwards most often, and it's understandable, because the LLM node is the interesting one on the canvas. The boring Switch is the one doing the work.

What it costs

Billing is in credits rather than requests, and the arithmetic is worth doing before you build.

A security lookup costs 2 credits. The free tier gives you 1,000 credits a day but excludes security entirely, so this workflow needs a paid plan. Starter is $19 a month for 150,000 credits, which is 75,000 scored signups. If you're processing more signups than that you have problems this article can't help with.

If you want location alongside the score, the unified endpoint charges 1 credit for the lookup plus 2 for the security module, so 3 total. Worth it when you're also routing on country. Wasteful when you only need the number.

For backfilling accounts you already have, the bulk security operation takes up to 50,000 IPs in one call at the same 2 credits per valid IP. Bogons and private addresses aren't charged, which makes it safe to throw a messy export at it.

A few extra notes

IPv6 works on every operation, so you don't need a separate path for it. Worth confirming your app is actually capturing the v6 address rather than a mapped form.

If you're storing threat scores against user records and you serve EU users, that's IP data linked to an identifiable person. Set a retention window based on how long the data is actually needed for fraud prevention, document that choice, and make sure your privacy disclosures cover the processing. Scores are cheap to recompute and expensive to hold.

The bogon response is HTTP 423 rather than a 200 with empty fields, which means a private IP leaking into your webhook shows up as an error rather than a silent zero. Route that error output to Verify along with the other failures and you'll spot the leak in your logs instead of scoring 10.0.0.1 as clean forever.

Run this in log-only mode first. Connect all four Switch outputs to the same "create account" path plus a logging step, leave it a week, then look at the distribution before you turn on the branching. If a tenth of your legitimate signups land at 30, your Verify boundary is in the wrong place, and you'd rather learn that from a spreadsheet than from a support queue.

If you'd rather have this as middleware than as a workflow, the same scoring logic in Express and Flask covers the code version, including Redis caching. You'll need an API key with security access either way.

Start with the override IF node even if you skip everything else. Most of the damage a fraud rule does isn't the fraud it misses, it's the customer it turns away at the door.

A Laravel Middleware for Blocking High-Risk Signups

ABDULLAH AFZAL — Mon, 21 Sep 2026 10:43:41 +0000

Every Laravel tutorial about blocking an IP starts the same way: an array of IP addresses, an in_array check, abort(403). That works when you already know which IP is bad. Signup abuse is the case where you never do. The address that just registered four accounts in six minutes is one you have never seen, and by tomorrow it will be a different one.

So the check has to run against a risk score, not a list. The API call is the easy part. The four decisions around it are where this actually gets interesting.

TL;DR

A static blocklist cannot stop signup abuse, because in signup abuse the IP is new every time. Score it instead.
Fix $request->ip() first. Behind a load balancer it returns your own infrastructure unless trusted proxies are configured, and the wildcard * setting is worse than doing nothing.
Do not block on is_vpn. Blocking every VPN exit node blocks your corporate customers. Block on threat_score plus a corroborating signal such as is_known_attacker.
Use four bands, not one threshold: allow, observe, challenge, block. The 45 to 79 band should add friction, not reject.
Cache the verdict, set a 1.5 second timeout, and decide fail-open or fail-closed on purpose rather than by accident.

The code below is a service, a middleware, and a Pest test. It runs on Laravel 11, 12 and 13, with a note for 10 and earlier. Total build time is about twenty minutes, most of it spent choosing thresholds rather than writing code.

Why a blocklist is the wrong shape for this problem

There are good Laravel packages for banning IPs. Banhammer has 87,245 installs and does exactly what it says: bans Eloquent models, IP addresses, and whole countries, with expiry handling and middleware. orkhanahmadov/laravel-ip-middleware gives you ip_whitelist and ip_blacklist middleware in two lines. Both are fine.

They are also ban stores. They record a decision something else made. If you already know 203.0.113.40 belongs to an abuser, Banhammer will keep it out. Nothing in it tells you that the IP registering right now is a residential proxy exit that was last seen running credential stuffing three weeks ago.

That is the gap for signup abuse specifically. A blocklist is reactive by construction: an IP gets on it after it has already done something. At a registration form you get one shot, before the user record exists, and the attacker rotates addresses faster than you can add them.

Scoring flips the order. You ask a question about an address you have never seen and get an answer in about 40 milliseconds. What you do with that answer is your policy, and that policy is the rest of this article.

Get the client IP right before anything else

Every later decision is garbage if this one is wrong, which is why it goes first rather than in a pitfalls section at the bottom.

$request->ip() returns the address that opened the TCP connection. Behind nginx, a load balancer, Cloudflare, or anything else that terminates the connection for you, that is the proxy, not the user. Score it and you will get the same verdict for every signup on your site, which is either "allow everything" or "block everything" depending on your host.

Laravel ships Illuminate\Http\Middleware\TrustProxies in the default global stack. It reads X-Forwarded-For only for requests that arrived from a proxy you named. In Laravel 11 and up you configure it in bootstrap/app.php:

use Illuminate\Foundation\Application;
use Illuminate\Foundation\Configuration\Middleware;
use Illuminate\Http\Request;

return Application::configure(basePath: dirname(__DIR__))
    ->withMiddleware(function (Middleware $middleware): void {
        // Name the actual proxy ranges. REMOTE_ADDR is the one request
        // property a client cannot forge, so it is what the trust check runs against.
        $middleware->trustProxies(
            at: ['10.0.0.0/8', '172.16.0.0/12'],
            headers: Request::HEADER_X_FORWARDED_FOR
                | Request::HEADER_X_FORWARDED_HOST
                | Request::HEADER_X_FORWARDED_PROTO
                | Request::HEADER_X_FORWARDED_PORT,
        );
    })
    ->create();

On Laravel 10 and earlier the same thing lives in app/Http/Middleware/TrustProxies.php as protected $proxies.

Pitfall: trustProxies(at: '*') looks convenient and is the single worst thing you can do to an IP-based control. With the wildcard set, anyone who can reach your app server directly can send X-Forwarded-For: 1.1.1.1 and be whoever they want. Your rate limits, your admin IP allowlist, your audit logs and this middleware all start reading attacker-supplied input. Name the ranges. If you are on a platform whose proxy IPs rotate, use a package that fetches and caches the current list rather than reaching for the asterisk.

Confirm it works before you build on it. Log $request->ip() on a staging route, hit it from your phone on mobile data, and check that what you see is your phone's address and not 10.x.x.x.

What the risk response actually contains

The lookup endpoint is https://api.ipgeolocation.io/v3/security. Here is the full response for 145.223.7.7, which is the example the IP Security API reference publishes:

{
  "ip": "145.223.7.7",
  "security": {
    "threat_score": 90,
    "is_tor": false,
    "is_proxy": true,
    "proxy_provider_names": ["NetNut", "ProxyScrape", "Oxy Labs", "DataImpulse"],
    "proxy_confidence_score": 99,
    "proxy_last_seen": "2026-09-01",
    "is_residential_proxy": true,
    "is_vpn": true,
    "vpn_provider_names": ["SurfShark VPN", "Ishaan VPN"],
    "vpn_confidence_score": 99,
    "vpn_last_seen": "2026-07-31",
    "is_relay": false,
    "relay_provider_name": "",
    "is_anonymous": true,
    "is_known_attacker": true,
    "is_bot": false,
    "bot_confidence_score": 0,
    "bot_operator_name": "",
    "bot_type": "",
    "is_known_good_bot": false,
    "bot_last_seen": "",
    "is_spam": true,
    "is_cloud_provider": true,
    "cloud_provider_name": "Brander Group Inc.",
    "is_corporate_gateway": false,
    "corporate_gateway_type": "",
    "corporate_gateway_provider_name": ""
  }
}

Twenty-seven fields. Four of them decide a signup and the rest are context for your logs.

threat_score is 0 to 100 and is the routing field. It aggregates everything else.

is_known_attacker is the strongest single flag. It is set directly by the attack bot types, so bot_type values of brute_force, credential_stuffing, exploit and worm all arrive with is_known_attacker: true. If you only ever act on one boolean, act on this one.

is_residential_proxy matters more than is_proxy for signups. A datacenter proxy is easy to spot and most abuse tooling has moved past it. Residential proxy networks rent real consumer connections, which is why the address above shows is_residential_proxy: true on an IP that also carries four proxy provider names.

is_corporate_gateway is the one that will save you support tickets. It marks the egress address of an enterprise secure web gateway, Zscaler and Netskope and that category, with corporate_gateway_type and the provider name. One of those IPs can front an entire company's workforce. Note that is_anonymous stays false for them: the range is published and belongs to an identifiable business, so the gateway flag on its own does not raise the threat score.

The rest, including vpn_provider_names, proxy_last_seen and bot_operator_name, are what you write to your audit log so that when someone asks why an account was rejected, you can answer with something better than "the API said no."

Here is the part I want to be blunt about. Do not block on is_vpn. It is the first flag everyone reaches for and it is the wrong one. Somewhere between a lot and most VPN traffic is a privacy-conscious person or an employee whose laptop routes everything through a tunnel by corporate policy. Reject on that flag and you have not built fraud prevention, you have built a support queue. is_vpn belongs in your logs and possibly in a friction decision. It does not belong in a reject branch on its own.

Scoring bands, not a single threshold

One threshold forces a binary you do not actually want. Four bands give you somewhere to put the ambiguous middle, which is where most real traffic lands.

Band	`threat_score`	What the signup gets	Why
Allow	0 to 19	Normal registration	Standard controls are enough
Observe	20 to 44	Registers, flagged in logs	Needs a second signal before it means anything
Challenge	45 to 79	Email verification or OTP before activation	Real risk, but rejecting costs you real customers
Block	80 to 100	Rejected, or queued for manual review	Blocking territory

Two overrides sit on top of the score, and they are the difference between a gate that works and one that generates complaints:

is_known_attacker promotes to Block regardless of score. The flag is specific enough to trust on its own.
is_corporate_gateway caps the band at Observe. Whatever the score says, you are looking at a shared egress for a company. Log it, do not challenge it, and never use that IP for per-user deduplication.

The Challenge band is the one people skip, and skipping it is why blocklists get a bad reputation. A score of 60 means the address has some history, not that this person is a fraudster. An emailed code costs a legitimate user fifteen seconds and costs an attacker a mailbox they have to provision per account. That asymmetry is most of the value in the whole system.

Start conservative. Ship with Block at 80 and Challenge at 45, watch the band distribution against your confirmed-fraud data for two weeks, then tighten. Numbers tuned on someone else's traffic are a guess.

The middleware

Three files: a value object, a service, and the middleware itself.

The verdict

<?php

namespace App\Support\IpRisk;

enum RiskBand: string
{
    case Allow = 'allow';
    case Observe = 'observe';
    case Challenge = 'challenge';
    case Block = 'block';
}

<?php

namespace App\Support\IpRisk;

final readonly class IpRiskVerdict
{
    /** @param list<string> $reasons */
    public function __construct(
        public RiskBand $band,
        public int $score,
        public array $reasons = [],
        // True when we could not reach the provider and fell back to a default.
        // Log this separately; a spike means the gate is open, not that traffic got safer.
        public bool $degraded = false,
    ) {}

    public static function unknown(): self
    {
        return new self(RiskBand::Allow, 0, ['lookup_failed'], degraded: true);
    }
}

The degraded flag exists so that "we allowed this because it was clean" and "we allowed this because the lookup failed" are not the same row in your logs. Without it, a provider outage looks like a quiet day.

The service

<?php

namespace App\Support\IpRisk;

use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;
use Throwable;

final class IpRiskService
{
    private const ENDPOINT = 'https://api.ipgeolocation.io/v3/security';

    public function __construct(
        private readonly ?string $apiKey,
        private readonly float $timeoutSeconds = 1.5,
        private readonly int $cacheTtlHours = 6,
    ) {}

    public function verdict(string $ip): IpRiskVerdict
    {
        // Private, loopback and reserved addresses are rejected by the API with a 423,
        // so short-circuit them here instead of spending a request to be told no.
        if (! filter_var($ip, FILTER_VALIDATE_IP, FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE)) {
            return new IpRiskVerdict(RiskBand::Allow, 0, ['non_public_ip']);
        }

        if ($this->apiKey === null || $this->apiKey === '') {
            Log::warning('IP risk lookup skipped: no API key configured.');

            return IpRiskVerdict::unknown();
        }

        try {
            return Cache::remember(
                "ip-risk:{$ip}",
                now()->addHours($this->cacheTtlHours),
                fn (): IpRiskVerdict => $this->lookup($ip),
            );
        } catch (Throwable $e) {
            // Cache driver problems must not take registration down with them.
            Log::error('IP risk cache failure', ['ip' => $ip, 'error' => $e->getMessage()]);

            return IpRiskVerdict::unknown();
        }
    }

    private function lookup(string $ip): IpRiskVerdict
    {
        try {
            $response = Http::timeout($this->timeoutSeconds)
                ->connectTimeout(1.0)
                ->retry(2, 100, throw: false)
                ->get(self::ENDPOINT, [
                    'apiKey' => $this->apiKey,
                    'ip' => $ip,
                    // Ask for only what the policy reads. Smaller payload, less to parse.
                    'fields' => implode(',', [
                        'security.threat_score',
                        'security.is_known_attacker',
                        'security.is_residential_proxy',
                        'security.is_corporate_gateway',
                        'security.is_tor',
                        'security.is_vpn',
                        'security.is_bot',
                        'security.is_known_good_bot',
                        'security.bot_type',
                    ]),
                ]);
        } catch (ConnectionException $e) {
            Log::warning('IP risk lookup timed out', ['ip' => $ip, 'error' => $e->getMessage()]);

            return IpRiskVerdict::unknown();
        }

        if ($response->failed()) {
            Log::warning('IP risk lookup returned an error', [
                'ip' => $ip,
                'status' => $response->status(),
            ]);

            return IpRiskVerdict::unknown();
        }

        $security = $response->json('security');

        if (! is_array($security)) {
            return IpRiskVerdict::unknown();
        }

        return $this->decide($security);
    }

    /** @param array<string, mixed> $s */
    private function decide(array $s): IpRiskVerdict
    {
        $score = (int) ($s['threat_score'] ?? 0);
        $reasons = [];

        $band = match (true) {
            $score >= 80 => RiskBand::Block,
            $score >= 45 => RiskBand::Challenge,
            $score >= 20 => RiskBand::Observe,
            default => RiskBand::Allow,
        };

        // Attack bot types set this flag directly, so it is specific enough to act on alone.
        if (($s['is_known_attacker'] ?? false) === true) {
            $band = RiskBand::Block;
            $reasons[] = 'known_attacker';
        }

        // Residential proxies are the ones worth adding friction for. Datacenter
        // proxies already show up in the score. Promote from anything below
        // Challenge, because a residential exit with a low score is the exact
        // case the score is worst at.
        if (($s['is_residential_proxy'] ?? false) === true
            && in_array($band, [RiskBand::Allow, RiskBand::Observe], true)) {
            $band = RiskBand::Challenge;
            $reasons[] = 'residential_proxy';
        }

        // One gateway IP can front an entire company. Never reject the workforce.
        if (($s['is_corporate_gateway'] ?? false) === true && $band !== RiskBand::Allow) {
            $band = RiskBand::Observe;
            $reasons[] = 'corporate_gateway';
        }

        // Declared crawlers are not signing up for anything. Do not challenge them.
        if (($s['is_known_good_bot'] ?? false) === true) {
            $band = RiskBand::Allow;
            $reasons[] = 'known_good_bot';
        }

        if (($s['is_vpn'] ?? false) === true) {
            $reasons[] = 'vpn'; // Recorded, deliberately not acted on.
        }

        return new IpRiskVerdict($band, $score, $reasons);
    }
}

Three things in there are load-bearing. The 1.5 second timeout replaces a default that is measured in tens of seconds, because a registration form that hangs is worse than one that lets a bad signup through. The single retry with a 100ms gap covers a blip without doubling your worst case. And every failure path returns IpRiskVerdict::unknown(), which is Allow with degraded: true, so this gate fails open.

That is a choice, not a default. Fail open and an outage means abusers get through for an hour. Fail closed and an outage means nobody can register at all. For a signup form I would take the first every time, and I would alert on degraded so I know it is happening. For a funds-transfer endpoint I would argue the opposite. Make the call explicitly and write it in the code where the next person will find it.

Register it with a bound API key. Note config() rather than env(): once php artisan config:cache runs in production, env() returns null and your gate silently stops working.

// config/services.php
'ipgeolocation' => [
    'key' => env('IPGEO_API_KEY'),
],

// app/Providers/AppServiceProvider.php
use App\Support\IpRisk\IpRiskService;

public function register(): void
{
    $this->app->singleton(
        IpRiskService::class,
        fn () => new IpRiskService(config('services.ipgeolocation.key')),
    );
}

Put IPGEO_API_KEY in .env. You can grab an API key if you do not already have one; any provider returning a risk score works here, the field names just change.

Caching

The Cache::remember wrapper above is not an optimisation, it is the thing that makes this affordable to run. A registration page gets hit repeatedly by the same address during a normal signup flow, and each security lookup costs 2 credits.

Six hours is a defensible TTL. The provider refreshes security data at least twice every 24 hours, and the last_seen fields are day-granularity anyway, so caching for less than that buys you nothing and caching for a week means acting on stale intelligence. Key on the IP, not the session, because the whole point is that you are tracking the address rather than the visitor.

Use Redis if you have it. The file driver will work and will also mean your web nodes each build their own copy of the same verdicts, which is a slow leak rather than a failure.

The middleware class

<?php

namespace App\Http\Middleware;

use App\Support\IpRisk\IpRiskService;
use App\Support\IpRisk\RiskBand;
use Closure;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Log;
use Symfony\Component\HttpFoundation\Response;

final class ScoreSignupIp
{
    public function __construct(private readonly IpRiskService $risk) {}

    public function handle(Request $request, Closure $next): Response
    {
        $ip = $request->ip() ?? '';
        $verdict = $this->risk->verdict($ip);

        Log::info('signup.ip_risk', [
            'ip' => $ip,
            'band' => $verdict->band->value,
            'score' => $verdict->score,
            'reasons' => $verdict->reasons,
            'degraded' => $verdict->degraded,
        ]);

        if ($verdict->band === RiskBand::Block) {
            // A vague message on purpose. Telling an attacker which signal fired
            // tells them what to rotate.
            return back()
                ->withInput($request->except(['password', 'password_confirmation']))
                ->withErrors(['email' => 'We could not complete this registration. Please contact support.']);
        }

        // Hand the band downstream so the controller can require verification
        // without repeating the lookup.
        $request->attributes->set('ip_risk', $verdict);

        return $next($request);
    }
}

The Challenge band is deliberately not handled here. Middleware decides whether the request continues; whether a new account starts in an unverified state is a controller and model concern. Passing the verdict through $request->attributes keeps the lookup to one call per request.

Registering and applying it

On Laravel 11, 12 and 13 you alias it when registering middleware in bootstrap/app.php:

->withMiddleware(function (Middleware $middleware): void {
    $middleware->alias([
        'signup.risk' => \App\Http\Middleware\ScoreSignupIp::class,
    ]);
})

On Laravel 10 that is $middlewareAliases in app/Http/Kernel.php, and on 9 and earlier it is $routeMiddleware. Plenty of tutorials still show the old property names, so check which major version you are actually on before copying anything.

Then attach it to the POST route only:

Route::post('/register', [RegisteredUserController::class, 'store'])
    ->middleware(['guest', 'throttle:6,1', 'signup.risk']);

Not the GET. Scoring the form render doubles your lookups and stops nothing, since nobody registers by loading a page. Keep the throttle in front of it too; rate limiting and risk scoring solve different problems and you want both.

When you want geolocation in the same call

Sometimes the risk decision needs country as well, for example to compare the signup IP's country against the billing country on the card. The official PHP SDK covers /v3/ipgeo, which returns both:

composer require ipgeolocation/ipgeolocation-php-sdk

use Illuminate\Support\Facades\Log;
use Ipgeolocation\Sdk\ApiException;
use Ipgeolocation\Sdk\IpGeolocationClient;
use Ipgeolocation\Sdk\RequestTimeoutException;
use Ipgeolocation\Sdk\TransportException;

// Defaults are 10s connect and 30s read. Those are fine for a batch job and
// far too long for anything sitting in front of a form.
$client = new IpGeolocationClient([
    'api_key' => config('services.ipgeolocation.key'),
    'connect_timeout' => 1.0,
    'read_timeout' => 1.5,
]);

try {
    $response = $client->lookupIpGeolocation([
        'ip' => $ip,
        'include' => ['security'],
        'fields' => ['location.country_code2', 'security.threat_score'],
    ]);

    $country = $response->data->location?->country_code2 ?? null;
    $score = (int) ($response->data->security?->threat_score ?? 0);
} catch (RequestTimeoutException|TransportException|ApiException $e) {
    Log::warning('Combined geo and risk lookup failed', ['error' => $e->getMessage()]);
    $country = null;
    $score = 0;
} finally {
    $client->close();
}

Note the ?-> on both accesses. Optional modules are absent rather than null-filled when they are not returned, so a non-nullable read will throw on you eventually.

For auditing the accounts you have already created rather than gating new ones, POST /v3/security-bulk takes up to 50,000 addresses in one request. That is the right tool for a retroactive sweep and the wrong one for a middleware.

What breaks the first time you run it locally

php artisan serve, submit the form, and the lookup returns HTTP 423 Locked with a message saying the IP is a bogon. That is correct behaviour: $request->ip() is 127.0.0.1 locally, and private, loopback and reserved ranges are rejected rather than scored.

The filter_var guard at the top of verdict() handles this by returning an Allow verdict before the request goes out, which is why local development works and why you are not burning lookups on 10.0.0.0/8 traffic from your own health checks.

To actually exercise the gate locally, fake the response rather than chasing a real risky IP. The test below does exactly that.

Heads up: a very aggressive timeout can produce HTTP 499 responses, which is the server noticing the client hung up first. If you see those in your logs, your timeout is too tight for your network path rather than something being wrong at the other end.

Testing the gate

You cannot tune thresholds you cannot test, and this is the part every tutorial in this space skips.

<?php

use App\Support\IpRisk\IpRiskService;
use App\Support\IpRisk\RiskBand;
use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Http;

beforeEach(function () {
    // Without this the container resolves the service with a null key and every
    // lookup short-circuits to a degraded verdict, so the assertions below would
    // pass for the wrong reason.
    config(['services.ipgeolocation.key' => 'test-key']);
});

function fakeSecurity(array $overrides = []): void
{
    Http::fake([
        'api.ipgeolocation.io/v3/security*' => Http::response([
            'ip' => '145.223.7.7',
            'security' => array_merge([
                'threat_score' => 0,
                'is_known_attacker' => false,
                'is_residential_proxy' => false,
                'is_corporate_gateway' => false,
                'is_known_good_bot' => false,
                'is_vpn' => false,
            ], $overrides),
        ], 200),
    ]);
}

it('blocks a high score', function () {
    fakeSecurity(['threat_score' => 90, 'is_known_attacker' => true]);

    $verdict = app(IpRiskService::class)->verdict('145.223.7.7');

    expect($verdict->band)->toBe(RiskBand::Block)
        ->and($verdict->reasons)->toContain('known_attacker');
});

it('challenges the middle band', function () {
    fakeSecurity(['threat_score' => 60]);

    expect(app(IpRiskService::class)->verdict('145.223.7.7')->band)
        ->toBe(RiskBand::Challenge);
});

it('does not punish a corporate gateway', function () {
    fakeSecurity(['threat_score' => 85, 'is_corporate_gateway' => true]);

    expect(app(IpRiskService::class)->verdict('87.58.66.106')->band)
        ->toBe(RiskBand::Observe);
});

it('fails open when the provider is unreachable', function () {
    Http::fake(fn () => throw new ConnectionException('timeout'));

    $verdict = app(IpRiskService::class)->verdict('145.223.7.7');

    expect($verdict->band)->toBe(RiskBand::Allow)
        ->and($verdict->degraded)->toBeTrue();
});

Disable the cache in your test environment or these will pass for the wrong reason, since the second test will read the first one's stored verdict. CACHE_STORE=array in phpunit.xml is enough.

That fourth test is the one worth keeping. Fail-open behaviour is invisible in production until the day it matters, and an assertion is cheaper than finding out during an outage.

A few extra notes

IP addresses are personal data under GDPR. In a signup system where an IP can reasonably be linked to an account or user, it should generally be treated as personal data. Storing a risk verdict against it is processing, so document an appropriate lawful basis and retention period. Legitimate interests may support fraud-prevention processing, depending on the circumstances, but do not keep verdict logs forever by accident.

Watch the Observe band before you act on it. Scores between 20 and 44 are exactly where a policy tuned on someone else's traffic will be wrong. Let it run for a few weeks, join it against your confirmed-fraud data, and you will have an opinion worth encoding.

IPv6 needs no special handling here. filter_var validates IPv4 and IPv6 alike, the endpoint accepts both, and cache keys are strings. The one thing to check is that your cache key length is not truncating full v6 addresses somewhere in your stack.

Know the ceiling. Residential proxy networks rent real consumer connections, and a determined attacker will rotate through addresses that look like ordinary broadband. IP signals raise the cost of abuse. They do not end it. Device fingerprinting, email reputation and velocity checks on your own data are the next layers, and the risk verdict you are now logging is a useful input to all three.

Start at fail-open, Block at 80, Challenge at 45. Alert on the degraded counter so an outage looks like an outage instead of a quiet afternoon. Then leave it alone for a fortnight and let your own traffic tell you where the thresholds belong.

Why FastAPI geolocation middleware is the wrong tool

ABDULLAH AFZAL — Wed, 16 Sep 2026 07:08:33 +0000

Search for FastAPI geolocation middleware and you'll get the same answer every time: subclass BaseHTTPMiddleware, call an IP lookup API, stash the result on request.state.geo. It works. It's also the wrong tool, and the reason is boring rather than clever.

Middleware runs before routing. It cannot know which endpoint matched, so it fires on everything: /health, /metrics, /docs, /openapi.json, your static mounts, and every CORS preflight OPTIONS. You've just put a network call in front of your liveness probe.

A dependency runs after routing, only where you declare it. That one ordering difference fixes the problem and brings typing and testability along with it.

TL;DR

BaseHTTPMiddleware runs on every request including health checks and docs routes. Depends() runs only on routes that ask for it.
Fix client IP resolution first. request.client.host is your load balancer, and X-Forwarded-For is caller-controlled unless you configure Uvicorn to trust it.
Return a Pydantic model from the dependency so the route signature shows what it gets and your editor can autocomplete it.
Keep geolocation and risk as two separate dependencies. Cheap routes take geo, sensitive routes take both.
Fail open on lookup errors, cache in Redis, and swap the whole thing out in tests with app.dependency_overrides.

You'll end with two composable dependencies, GeoContext and RiskProfile, that any route can pull in by adding one parameter. Roughly 120 lines total, Redis-cached, and testable without mocking HTTP.

Why FastAPI geolocation middleware is the wrong tool

The routing-order problem is the one that actually bites. Starlette runs middleware in the ASGI stack before the router resolves a path, so a geolocation middleware has no way to say "skip this for /health". You end up with a hand-maintained path prefix blocklist inside the middleware, which drifts the moment someone adds a route.

Latency is the part people feel in production. A Kubernetes liveness probe hits /health every few seconds. If your middleware makes an outbound API call on every request, that probe now depends on a third-party network round trip. When the upstream gets slow, probes time out, and your orchestrator restarts a pod that was perfectly healthy. I've watched a team spend most of a day on that one.

Then there's typing. request.state.geo is a bag with no schema. Your editor can't complete it, your reviewer can't see it in the function signature, and a typo in request.state.geo.country_code2 fails at runtime in whichever route nobody tested.

BaseHTTPMiddleware also has its own reputation for edge-case behaviour around exceptions, streaming responses, and background tasks. Plenty of people run it happily. It's still more machinery than this job needs.

What a dependency gives you instead

FastAPI's dependency injection system gives you four things, in the order you'll care about them. Per-route opt-in, so /health stays a pure function. A typed return value that shows up in the signature. Automatic caching within a single request, so declaring the same dependency twice doesn't call the API twice. And dependency_overrides, which makes testing a three-line fixture instead of a mock HTTP layer.

The FastAPI dependency docs cover the mechanics well. What they don't cover is the case for using them where your instinct says middleware, which is most of what follows.

Get the client IP right first

Everything downstream is worthless if you geolocate the wrong address. request.client.host gives you the peer that opened the TCP connection. Behind nginx, an ALB, Cloudflare, or Railway's Envoy layer, that peer is the proxy, and you'll cheerfully look up your own infrastructure for every visitor.

The fix is not to parse X-Forwarded-For in your application code. That header is whatever the caller typed unless something upstream has overwritten it, so trusting the leftmost entry hands any user the ability to claim any country they like. This is the single most common mistake in the geolocation middleware examples floating around.

Let Uvicorn do it. Its proxy header settings rewrite request.client.host from the forwarded headers, but only for peers you explicitly trust:

uvicorn app.main:app \
  --proxy-headers \
  --forwarded-allow-ips="10.0.0.0/8"

The default for --forwarded-allow-ips is 127.0.0.1, which is not what you want the moment your proxy lives on another host. Set it to your load balancer's actual range and nothing else. With that in place, request.client.host is trustworthy and the dependency stays short:

# app/deps/client_ip.py
import ipaddress
from fastapi import Request


def get_client_ip(request: Request) -> str | None:
    """The caller's public IP, or None when we don't have one worth looking up."""
    client = request.client
    if client is None:
        # Some ASGI transports (including parts of the test client) omit this.
        return None

    try:
        parsed = ipaddress.ip_address(client.host)
    except ValueError:
        return None

    # Private, loopback and reserved addresses have no public geolocation.
    # Returning None here saves a pointless round trip on every local request.
    if parsed.is_private or parsed.is_loopback or parsed.is_reserved:
        return None

    return client.host

Returning None rather than raising is deliberate. A missing IP is a normal condition, not an error, and the callers downstream all handle None already.

Running it locally

Every request from your laptop arrives as 127.0.0.1, so get_client_ip returns None and the geo dependency short-circuits before it ever calls out. That's the behaviour you want, and it means local development costs nothing.

When you do want real data in development, hardcode a test IP behind an environment flag rather than pointing the lookup at a private address. IP geolocation APIs reject private and bogon ranges, and you'll spend twenty minutes debugging an error response that was correct all along.

The geolocation dependency

One call, one model. Here's the request shape:

curl -s 'https://api.ipgeolocation.io/v3/ipgeo?apiKey=API_KEY&ip=91.128.103.196'

And the full response, which is worth reading once before you decide what to keep:

{
  "ip": "91.128.103.196",
  "location": {
    "continent_code": "EU",
    "continent_name": "Europe",
    "country_code2": "SE",
    "country_code3": "SWE",
    "country_name": "Sweden",
    "country_name_official": "Kingdom of Sweden",
    "country_capital": "Stockholm",
    "state_prov": "Stockholms län",
    "state_code": "SE-AB",
    "district": "Stockholm",
    "city": "Stockholm",
    "zipcode": "164 40",
    "latitude": "59.40510",
    "longitude": "17.95510",
    "is_eu": true,
    "country_flag": "https://ipgeolocation.io/static/flags/se_64.png",
    "geoname_id": "9972319",
    "country_emoji": "🇸🇪"
  },
  "country_metadata": {
    "calling_code": "+46",
    "tld": ".se",
    "languages": ["sv-SE", "se", "sma", "fi-SE"]
  },
  "currency": { "code": "SEK", "name": "Swedish Krona", "symbol": "kr" },
  "asn": {
    "as_number": "AS1257",
    "organization": "Tele2 Sverige AB",
    "country": "SE"
  },
  "time_zone": {
    "name": "Europe/Stockholm",
    "offset": 1,
    "offset_with_dst": 2,
    "current_time": "2026-09-07 16:55:30.494+0200",
    "current_time_unix": 1788792930.494,
    "current_tz_abbreviation": "CEST",
    "current_tz_full_name": "Central European Summer Time",
    "is_dst": true
  }
}

Two things in there will trip you up. latitude and longitude are strings, not floats, so cast them before any arithmetic. And country_metadata.languages is an array, not the comma-separated string that several other providers return. If you're porting code from another API, that's where it breaks.

I'm using ipgeolocation.io for these examples because the geolocation and threat endpoints share a response envelope, which keeps the two dependencies below nearly identical. IPGeolocation, ipinfo, ip-api, MaxMind GeoIP2, IPLocate and IP2Location all hand back roughly the same country-level payload, so swap in whichever is already in your stack. Only the parsing function changes.

Now the dependency:

# app/deps/geo.py
import logging
import os
from typing import Annotated

import httpx
from fastapi import Depends, Request
from pydantic import BaseModel

from app.deps.client_ip import get_client_ip

logger = logging.getLogger(__name__)


class GeoContext(BaseModel):
    ip: str
    country_code: str | None = None
    country_name: str | None = None
    city: str | None = None
    is_eu: bool = False
    currency_code: str | None = None
    asn_organization: str | None = None
    timezone: str | None = None


async def get_geo(
    request: Request,
    ip: Annotated[str | None, Depends(get_client_ip)],
) -> GeoContext | None:
    if ip is None:
        return None

    api_key = os.environ.get("IPGEO_API_KEY")
    if not api_key:
        logger.warning("IPGEO_API_KEY is not set, skipping geolocation")
        return None

    try:
        response = await request.app.state.http.get(
            "/ipgeo", params={"apiKey": api_key, "ip": ip}
        )
        response.raise_for_status()
        payload = response.json()
    except (httpx.HTTPError, ValueError) as exc:
        # Fail open. A geo lookup should never be why a page returns 500.
        logger.warning("Geolocation lookup failed for %s: %s", ip, exc)
        return None

    location = payload.get("location") or {}
    return GeoContext(
        ip=payload.get("ip", ip),
        country_code=location.get("country_code2"),
        country_name=location.get("country_name"),
        city=location.get("city"),
        is_eu=bool(location.get("is_eu", False)),
        currency_code=(payload.get("currency") or {}).get("code"),
        asn_organization=(payload.get("asn") or {}).get("organization"),
        timezone=(payload.get("time_zone") or {}).get("name"),
    )

The or {} on every nested access is not paranoia. Responses vary by key configuration, and payload["location"]["city"] on an IP with no city resolution is a KeyError in production at 3am.

Fail open or fail closed

The code above fails open: if the lookup breaks, get_geo returns None and the route carries on with whatever default it has. For pricing, currency, or language, that's obviously right. Nobody should see a 500 because a third-party API had a bad minute.

Fail closed is the correct choice in exactly one situation, which is when the lookup is a control rather than a decoration. If you're blocking sanctioned jurisdictions, "we couldn't check" has to mean "denied", because a fail-open geoblock is a geoblock that stops working the moment someone can make your API call time out. Decide which one each route needs and write it down, because the default you pick silently becomes policy everywhere.

Adding risk data as a second dependency

Country tells you where someone claims to be. It says nothing about whether they're behind a VPN, a residential proxy, or a datacenter. For a signup or checkout route you usually want both, and the IP Security API returns the threat side as its own endpoint:

curl -s 'https://api.ipgeolocation.io/v3/security?apiKey=API_KEY&ip=145.223.7.7'

{
  "ip": "145.223.7.7",
  "security": {
    "threat_score": 90,
    "is_tor": false,
    "is_proxy": true,
    "proxy_provider_names": ["NetNut", "ProxyScrape", "Oxy Labs", "DataImpulse"],
    "proxy_confidence_score": 99,
    "proxy_last_seen": "2026-09-01",
    "is_residential_proxy": true,
    "is_vpn": true,
    "vpn_provider_names": ["SurfShark VPN", "Ishaan VPN"],
    "vpn_confidence_score": 99,
    "vpn_last_seen": "2026-07-31",
    "is_relay": false,
    "relay_provider_name": "",
    "is_anonymous": true,
    "is_known_attacker": true,
    "is_bot": false,
    "bot_confidence_score": 0,
    "bot_operator_name": "",
    "bot_type": "",
    "is_known_good_bot": false,
    "bot_last_seen": "",
    "is_spam": true,
    "is_cloud_provider": true,
    "cloud_provider_name": "Brander Group Inc.",
    "is_corporate_gateway": false,
    "corporate_gateway_type": "",
    "corporate_gateway_provider_name": ""
  }
}

Named providers are the part worth keeping. A boolean tells you the IP is a proxy; ["NetNut", "ProxyScrape", "Oxy Labs", "DataImpulse"] tells your fraud team which network it belongs to, which is the difference between an alert and an investigation. is_corporate_gateway matters for the opposite reason: a shared Zscaler or Netskope egress address looks like a proxy by every other measure, and blocking it means blocking an entire enterprise customer.

The dependency mirrors the geo one closely enough that I'll only show what differs:

# app/deps/risk.py
import logging
import os
from typing import Annotated

import httpx
from fastapi import Depends, Request
from pydantic import BaseModel, Field

from app.deps.client_ip import get_client_ip

logger = logging.getLogger(__name__)

BLOCK_THRESHOLD = 80


class RiskProfile(BaseModel):
    ip: str
    threat_score: int = 0
    is_vpn: bool = False
    is_proxy: bool = False
    is_residential_proxy: bool = False
    is_tor: bool = False
    is_anonymous: bool = False
    is_cloud_provider: bool = False
    is_corporate_gateway: bool = False
    vpn_provider_names: list[str] = Field(default_factory=list)
    proxy_provider_names: list[str] = Field(default_factory=list)

    @property
    def should_block(self) -> bool:
        # A corporate gateway scores like a proxy but is a legitimate employer
        # egress. Blocking it locks out everyone behind that company's network.
        if self.is_corporate_gateway:
            return False
        return self.threat_score >= BLOCK_THRESHOLD


async def get_risk(
    request: Request,
    ip: Annotated[str | None, Depends(get_client_ip)],
) -> RiskProfile | None:
    if ip is None:
        return None

    api_key = os.environ.get("IPGEO_API_KEY")
    if not api_key:
        return None

    try:
        response = await request.app.state.http.get(
            "/security", params={"apiKey": api_key, "ip": ip}
        )
        response.raise_for_status()
        security = (response.json() or {}).get("security") or {}
    except (httpx.HTTPError, ValueError) as exc:
        logger.warning("Security lookup failed for %s: %s", ip, exc)
        return None

    return RiskProfile(ip=ip, **{
        key: security[key] for key in RiskProfile.model_fields
        if key in security
    })

That dict comprehension keeps the model and the parser from drifting apart when you add a field. It also means an unexpected key in the response is ignored rather than raising.

Composing them without doubling your calls

Both dependencies declare get_client_ip. FastAPI's dependency injection caches sub-dependency results within a single request by default, so get_client_ip runs once even though two dependencies asked for it. Same applies if a route declares get_risk and a router-level dependency also uses it. You get one call, not two.

If you ever need the opposite, Depends(get_client_ip, use_cache=False) forces a re-run. I've needed that exactly never for this pattern, but it's there.

Wiring it to routes

A type alias per dependency keeps the signatures readable:

# app/routes.py
from typing import Annotated

from fastapi import APIRouter, Depends, HTTPException

from app.deps.geo import GeoContext, get_geo
from app.deps.risk import RiskProfile, get_risk

Geo = Annotated[GeoContext | None, Depends(get_geo)]
Risk = Annotated[RiskProfile | None, Depends(get_risk)]

router = APIRouter()


@router.get("/health")
async def health() -> dict[str, str]:
    # No dependencies. No network calls. This is the whole point.
    return {"status": "ok"}


@router.get("/pricing")
async def pricing(geo: Geo) -> dict[str, str]:
    return {"currency": geo.currency_code if geo else "USD"}


@router.post("/signup")
async def signup(geo: Geo, risk: Risk) -> dict[str, str]:
    if risk and risk.should_block:
        logger.info("Blocked signup from %s, score %s", risk.ip, risk.threat_score)
        raise HTTPException(status_code=403, detail="Signup unavailable")
    return {"country": geo.country_code if geo else "unknown"}

Three routes, three different policies, one set of dependencies. /health does nothing, /pricing pays for geo only, /signup takes both. Middleware cannot express that without a path blocklist.

For a whole section of the API, put the check on the router. The catch worth knowing: router-level dependencies run but their return value is thrown away, so they're for enforcement, not injection.

# app/routes.py, continued. Imports and the Geo/Risk aliases are above.


async def require_acceptable_risk(risk: Risk) -> None:
    """Router-level guard. Return value is discarded, so it raises or passes."""
    if risk and risk.should_block:
        raise HTTPException(status_code=403, detail="Unavailable")


checkout = APIRouter(
    prefix="/checkout",
    dependencies=[Depends(require_acceptable_risk)],
)


@checkout.post("/confirm")
async def confirm(risk: Risk) -> dict[str, int]:
    # get_risk already ran for the guard. The cache means this is free.
    return {"score": risk.threat_score if risk else 0}

Testing, which is where this pays off

app.dependency_overrides swaps any dependency for a fake. No HTTP mocking, no responses library, no network in the test suite:

# tests/test_pricing.py
import pytest
from fastapi.testclient import TestClient

from app.deps.geo import GeoContext, get_geo
from app.main import app


def fake_geo_sweden() -> GeoContext:
    return GeoContext(
        ip="91.128.103.196",
        country_code="SE",
        is_eu=True,
        currency_code="SEK",
    )


@pytest.fixture
def client():
    app.dependency_overrides[get_geo] = fake_geo_sweden
    try:
        yield TestClient(app)
    finally:
        # Clear or the override leaks into every later test in the session.
        app.dependency_overrides.clear()


def test_pricing_uses_local_currency(client):
    assert client.get("/pricing").json()["currency"] == "SEK"

Testing the same logic in middleware means intercepting an outbound HTTP call, which is doable but noisier. The testing-dependencies docs cover the override mechanics if you want the full picture.

One HTTP client for the process

Both dependencies reach for request.app.state.http. Create that once in the lifespan handler, not per request, because a fresh AsyncClient throws away connection pooling and TLS session reuse on every call:

# app/main.py
from contextlib import asynccontextmanager

import httpx
from fastapi import FastAPI

from app.routes import checkout, router


@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.http = httpx.AsyncClient(
        base_url="https://api.ipgeolocation.io/v3",
        # Separate read timeout because a slow upstream is the realistic
        # failure, not a slow handshake. Keep the total well under your
        # own request budget.
        timeout=httpx.Timeout(connect=1.0, read=1.5, write=1.0, pool=1.0),
    )
    try:
        yield
    finally:
        await app.state.http.aclose()


app = FastAPI(lifespan=lifespan)
app.include_router(router)
app.include_router(checkout)

Caching with Redis

The same IP will hit you many times in a session, and paying for a lookup on every request is wasteful whichever provider you use. Redis over an in-process dict, for one specific reason: with four Uvicorn workers, an in-process cache gives you four separate caches and a hit rate a quarter of what you think you're getting. That bug is invisible in staging with one worker.

Two TTLs, because the data ages differently. Geolocation for an IP is stable for days. Threat signals are not, since an address can be clean this morning and a residential proxy exit by lunchtime.

# app/cache.py
import json
import logging

import redis.asyncio as redis

logger = logging.getLogger(__name__)

GEO_TTL = 24 * 60 * 60   # geolocation barely moves
RISK_TTL = 15 * 60       # threat signals go stale fast


async def cached_json(
    client: redis.Redis, key: str, ttl: int, loader
) -> dict | None:
    """Read-through cache. Redis being down degrades to a direct call."""
    try:
        hit = await client.get(key)
        if hit is not None:
            return json.loads(hit)
    except (redis.RedisError, ValueError) as exc:
        logger.warning("Cache read failed for %s: %s", key, exc)

    value = await loader()
    if value is None:
        return None

    try:
        await client.set(key, json.dumps(value), ex=ttl)
    except redis.RedisError as exc:
        logger.warning("Cache write failed for %s: %s", key, exc)

    return value

Wrap the API call in get_geo with cached_json(redis_client, f"geo:{ip}", GEO_TTL, loader) and you're done. Note that Redis failures are caught and logged rather than raised. A cache outage should slow you down, not take you offline.

Tip: Cache geolocation by /24 rather than by exact IP if your traffic is consumer-heavy. Neighbouring addresses in a residential block almost always resolve to the same city, and your hit rate improves noticeably. Don't do this for risk data, where the whole point is per-address precision.

When middleware actually is right

Dependencies aren't a replacement for middleware, and an article claiming otherwise would be selling you something. Middleware is correct whenever the work genuinely applies to every request, or has to happen outside the route's world.

Request ID generation, access logging, CORS, response header mutation, and compression all belong there, and the middleware docs show the shape. So does the thing this article depends on: Uvicorn's proxy header handling is itself middleware, and it works because rewriting the client IP really does apply to every request equally.

The rule of thumb: if the work applies to every request without exception, use middleware. If it applies to some routes, produces a value the route needs, or you want to fake it in tests, use a dependency. Geolocation and risk enrichment are firmly in the second category.

A few extra notes

IPv6 works without changes. ipaddress.ip_address handles both families and the API takes either, so the code above needs nothing extra. Worth an actual test case, though, because IPv6 is where the assumption that an address is four dot-separated numbers usually surfaces.

Rate limits are an ordinary failure. A 429 raises HTTPStatusError, gets caught by the same except block, and fails open like any other error. If you're hitting limits regularly, the Redis TTL is the lever, not a bigger plan.

Bulk endpoints exist for the offline case. If you're enriching historical logs rather than live requests, the bulk lookup takes up to 50,000 addresses per call, and doing that in a loop of single requests is the slow way to learn it exists.

Drop the two dependency files into an existing app and wire them to one route before you touch the rest. The client IP piece is the part that breaks in production and the part you can verify in about a minute: log request.client.host on a deployed route and confirm it isn't your load balancer. If it is, your --forwarded-allow-ips is wrong and everything downstream has been geolocating your own infrastructure.

How to Block Cloud-Hosted Signups Without Blocking Real Users

ABDULLAH AFZAL — Tue, 15 Sep 2026 09:53:14 +0000

A signup from a cloud-hosted IP is almost never a real person. Nobody opens a laptop, routes through an AWS datacenter in us-east-1, and fills in your registration form by hand. Datacenter traffic at a signup endpoint is automation: scrapers, credential-stuffing scripts, mass account creation, the odd misconfigured integration.

So blocking it sounds trivial. Grab a list of cloud IP ranges, drop anything that matches, ship it. Then someone on your own team gets locked out, an entire company behind a corporate proxy can't register, and you're staring at a support ticket wondering what you broke.

The fix isn't a longer blocklist. It's gating on the right signal, and recognizing the handful of cloud IPs that are actually humans. Here's both, with code you can paste into a Node or Python signup route.

TL;DR

A cloud/datacenter IP at signup is a strong automation-risk signal, but it does not prove the request is non-human. If your signup policy blocks cloud-origin traffic, gate on is_cloud_provider, not on "is this user anonymized."

Keep the VPN/proxy/Tor/relay decision separate and softer. Those flags catch real, privacy-minded users, so challenge them, don't hard-block.

Two cloud IPs you must not block: corporate secure-web-gateway egress (is_corporate_gateway, one IP fronts a whole company) and declared crawlers (is_known_good_bot).

Use cloud_provider_name to log an auditable reason for every block. Do not use the provider name alone as an allowlist: shared cloud providers host many unrelated customers.

Static CIDR lists rot and know nothing about gateways or good bots. That's the failure mode behind most "it blocked my own laptop" bugs.

In practice this is one lookup and about fifteen lines of decision logic. The lookup returns a set of boolean flags for an IP; the logic decides which flags mean "a server is pretending to be a user" and which mean "a real user is behind a shared address." Get that split right and you can substantially reduce the false positives of a blunt cloud-IP block.

A datacenter IP is a different signal than a VPN

Most "block anonymous traffic" advice lumps everything together: VPN, proxy, Tor, relay, datacenter, all treated as one risk bucket. That's the mistake. These are two different axes.

A person on a corporate VPN, iCloud Private Relay, or a mobile network that happens to route through a proxy is very often a real user with a real reason to sign up. Block them and you're turning away customers. A request from an EC2 instance or a rented Hetzner box is a different thing entirely: there's no human sitting at that IP. The traffic exists because someone rented compute and pointed it at your form.

So the useful question at a signup endpoint isn't "is this IP anonymized?" It's "is this IP a server?" Those overlap sometimes, but they're not the same, and conflating them is why blunt gates generate support tickets.

The signal you want: is_cloud_provider

Any decent IP intelligence API will tell you whether an IP belongs to a hosting provider. The one I'll use here, ipgeolocation.io, returns it as a boolean plus the provider's name, alongside the rest of the risk signals in a single response. Here's the full shape from the dedicated /v3/security endpoint for a real hosting-provider IP:

{
  "ip": "2.56.188.35",
  "security": {
    "threat_score": 35,
    "is_tor": false,
    "is_proxy": false,
    "proxy_provider_names": [],
    "proxy_confidence_score": 0,
    "proxy_last_seen": "",
    "is_residential_proxy": false,
    "is_vpn": false,
    "vpn_provider_names": [],
    "vpn_confidence_score": 0,
    "vpn_last_seen": "",
    "is_relay": false,
    "relay_provider_name": "",
    "is_anonymous": false,
    "is_known_attacker": true,
    "is_bot": false,
    "bot_confidence_score": 0,
    "bot_operator_name": "",
    "bot_type": "",
    "is_known_good_bot": false,
    "bot_last_seen": "",
    "is_spam": false,
    "is_cloud_provider": true,
    "cloud_provider_name": "Packethub S.A.",
    "is_corporate_gateway": false,
    "corporate_gateway_type": "",
    "corporate_gateway_provider_name": ""
  }
}

The two fields this whole tutorial hangs on are is_cloud_provider and cloud_provider_name. The boolean tells you the IP belongs to cloud-provider address space; the name tells you which provider owns that space. That name is what lets you allowlist providers you trust and log a real reason when you block, instead of shipping a mystery 403. Notice this IP is also flagged is_known_attacker with a low-ish threat_score of 35, which is common for bulletproof hosts: the same address is a datacenter and a repeat offender.

/v3/security is the endpoint to reach for when security is all you need. It costs 2 credits per lookup and returns only the security object, which keeps your credit usage predictable. If you already call /v3/ipgeo for geolocation, you can add include=security and get the same object bundled with location and ASN data in one call instead of two.

Heads up: the security signals are a paid feature. A free-plan key returns 401 on the security endpoint. Plans start at $19/month, which is the honest trade-off to weigh against the free alternatives below.

Before you start

You'll need Node 18+ (for global fetch) or Python 3.10+, and an API key.

Security data on ipgeolocation.io lives on the paid plans; the free plan covers basic geolocation only. Grab an API key and set it as an environment variable so it never lands in your source:

export IPGEO_API_KEY="your_key_here"

If paying for this doesn't fit, there are free options that also flag hosting IPs. ipapi.is has a free datacenter endpoint, IPLocate exposes a free is_hosting flag, and offline libraries like py-cloudip (Python) or cloud-ip-detector (PHP) match an IP against bundled CIDR data with no network call at all. They'll all tell you an IP is a datacenter. What the paid signal here adds is the provider name, the corporate-gateway and good-bot context that prevents the false positives later in this post, and a threat_score in the same response. Pick based on whether you need that context or just a boolean.

A quick smoke test against a known hosting IP before you write any code:

curl -s -X GET \
  'https://api.ipgeolocation.io/v3/security?apiKey=API_KEY&ip=2.56.188.35'

You should get the security object back with is_cloud_provider set to true.

Detect a cloud-hosted signup

curl

The raw call takes an IP as a query parameter. Leave ip off entirely and the API reads the caller's own address, which is handy for a browser-side check:

# Specific IP
curl -s 'https://api.ipgeolocation.io/v3/security?apiKey=API_KEY&ip=2.56.188.35'

# Caller IP (no ip param) — returns the requester's own address
curl -s 'https://api.ipgeolocation.io/v3/security?apiKey=API_KEY'

Node.js

Two functions: one that fetches the security object, one that makes the actual decision. Keeping them separate means the decision logic is easy to unit test without hitting the network.

const IPGEO_KEY = process.env.IPGEO_API_KEY;
const SECURITY_URL = 'https://api.ipgeolocation.io/v3/security';

async function getSecurity(ip) {
  if (!IPGEO_KEY) throw new Error('IPGEO_API_KEY is not set');

  const url = `${SECURITY_URL}?apiKey=${IPGEO_KEY}&ip=${encodeURIComponent(ip)}`;

  // 1.5s ceiling. A signup should never hang on a third-party lookup.
  const res = await fetch(url, { signal: AbortSignal.timeout(1500) });

  if (!res.ok) {
    // 423 = bogon/private IP, 401 = key or plan problem, 429 = quota hit.
    throw new Error(`security lookup failed: HTTP ${res.status}`);
  }

  const data = await res.json();
  return data?.security ?? null; // null if the object is missing
}

Now the decision. This is the part worth reading twice, because it's where the false positives get filtered out:

// Cloud providers you run on or trust. Matched against cloud_provider_name.
const CLOUD_ALLOWLIST = new Set(['Your Own Infra Ltd.']);

function isCloudHostedSignup(security) {
  if (!security) return false;               // no data: treat as not-cloud
  if (!security.is_cloud_provider) return false;

  // Real employees behind an enterprise proxy. One IP fronts a whole company,
  // so blocking it locks out everyone at that customer.
  if (security.is_corporate_gateway) return false;

  // Declared crawlers (Googlebot, ChatGPT, uptime monitors). They won't sign
  // up, but if you reuse this gate on other routes, don't block them.
  if (security.is_known_good_bot) return false;

  // Providers you've explicitly decided are fine.
  if (CLOUD_ALLOWLIST.has(security.cloud_provider_name || '')) return false;

  return true; // a server renting an IP, aimed at your form: gate it
}

Python

Same two-part shape. requests uses a (connect, read) timeout tuple, and .get() with a default keeps a missing field from throwing:

import os
import requests

IPGEO_KEY = os.environ.get("IPGEOLOCATION_API_KEY")
SECURITY_URL = "https://api.ipgeolocation.io/v3/security"

CLOUD_ALLOWLIST = {"Your Own Infra Ltd."}


def get_security(ip: str) -> dict | None:
    if not IPGEO_KEY:
        raise RuntimeError("IPGEOLOCATION_API_KEY is not set")

    try:
        resp = requests.get(
            SECURITY_URL,
            params={"apiKey": IPGEO_KEY, "ip": ip},
            timeout=(1.0, 1.5),  # (connect, read) seconds
        )
        resp.raise_for_status()
    except requests.RequestException as exc:
        # Log and let the caller decide fail-open vs fail-closed.
        print(f"security lookup failed for {ip}: {exc}")
        return None

    return resp.json().get("security")


def is_cloud_hosted_signup(security: dict | None) -> bool:
    if not security or not security.get("is_cloud_provider"):
        return False
    if security.get("is_corporate_gateway"):   # real employees, shared IP
        return False
    if security.get("is_known_good_bot"):      # declared crawler
        return False
    if (security.get("cloud_provider_name") or "") in CLOUD_ALLOWLIST:
        return False
    return True

Gate the signup route

Wiring it into Express takes three things the naive version skips: the correct client IP, a cache, and an explicit failure policy.

const express = require('express');
const app = express();

// Behind a load balancer or CDN? Tell Express how many proxies you actually
// run, then req.ip is the real client. Don't trust more hops than exist, or a
// client can spoof X-Forwarded-For and bypass the gate.
app.set('trust proxy', 1);

// Cache per IP so repeated attempts from one address don't each cost 2 credits.
// A Map is fine to start; swap for Redis once you have more than one instance.
const cache = new Map();
const TTL_MS = 10 * 60 * 1000; // 10 minutes is plenty for signup abuse

async function cachedSecurity(ip) {
  const hit = cache.get(ip);
  if (hit && Date.now() - hit.at < TTL_MS) return hit.security;

  const security = await getSecurity(ip); // may throw
  cache.set(ip, { security, at: Date.now() });
  return security;
}

async function blockCloudSignups(req, res, next) {
  try {
    const security = await cachedSecurity(req.ip);
    if (isCloudHostedSignup(security)) {
      // Log the provider so the block is auditable, not a mystery 403.
      console.warn(`blocked cloud signup: ${req.ip} (${security.cloud_provider_name})`);
      return res.status(403).json({ error: 'Signups from hosting providers are not allowed.' });
    }
  } catch (err) {
    // Fail OPEN: a lookup outage shouldn't stop real users signing up.
    // If this endpoint is genuinely high-risk, flip this to fail closed.
    console.error(`security check skipped for ${req.ip}: ${err.message}`);
  }
  return next();
}

app.post('/signup', blockCloudSignups, (req, res) => {
  // ... your normal signup handler ...
  res.status(201).json({ ok: true });
});

Fail-open is the right default for a signup form. The cost of a lookup outage blocking real users is worse than a few bot signups slipping through during the outage. If you were gating something higher-stakes, like a password reset or a payout request, you'd make the opposite call and fail closed.

Which signal should gate what

The reason to pull the full security object instead of just the cloud boolean is that one response tells you how to treat every kind of traffic, not just servers. Here's how I map the flags to actions at a signup endpoint:

Signal in `security`	What it usually means at signup	Action
`is_cloud_provider` true, nothing else notable	A server renting a datacenter IP	Block
`is_corporate_gateway` true	Real employees behind an enterprise proxy (Zscaler, Netskope)	Allow, and don't rate-limit per IP
`is_known_good_bot` true	A declared crawler (Googlebot, ChatGPT)	Ignore, it won't sign up
`is_known_attacker` true, or `bot_type` is `credential_stuffing` / `brute_force`	An active attack	Block
`is_residential_proxy` true	Often abuse hiding in home IPs	Challenge, or block on high `threat_score`
`is_vpn` / `is_proxy` / `is_relay` / `is_tor` (i.e. `is_anonymous`)	Frequently a real, privacy-minded user	Step-up or challenge, don't hard-block
`threat_score` 80 to 100	Aggregated high risk	Block or send to manual review

The row that matters most for keeping users happy is the anonymized one. is_anonymous is true whenever VPN, proxy, Tor, or relay is detected, and a lot of those are ordinary people. Treat that as a reason to add friction, not to slam the door.

The false positives that lock out real users

This is where naive cloud blocking goes wrong, and it's the reason to use context-aware data instead of a raw CIDR list.

Static lists rot, and they block your own team. The classic bug is a developer who parses a datacenter IP list, drops every match, and then can't sign in from the office because their own egress moved into a range the list now covers. Lists carry no context and go stale between updates. The security data here refreshes at least twice a day and, more importantly, tells you when a "cloud" IP is actually a corporate gateway or a known-good bot.

Corporate gateways are real employees on a shared cloud IP. Enterprise secure web gateways like Zscaler and Netskope route a whole company's browser traffic through the vendor's cloud, so it reaches you from the vendor's address space. That space is registered to the vendor, so is_cloud_provider is true, but blocking it would lock out every employee at that customer. The API marks these separately:

{
  "ip": "87.58.66.106",
  "security": {
    "is_cloud_provider": true,
    "cloud_provider_name": "Zscaler Switzerland GmbH",
    "is_corporate_gateway": true,
    "corporate_gateway_type": "secure_web_gateway",
    "corporate_gateway_provider_name": "Zscaler",
    "is_anonymous": false,
    "threat_score": 5
  }
}

That's why the classifier bails the moment is_corporate_gateway is true. (Full response includes the other fields; trimmed here to the ones that matter.)

Good bots run from cloud space too. Search and AI crawlers mostly operate out of rented infrastructure, so they trip is_cloud_provider. At a signup endpoint this is moot, since crawlers don't POST registration forms, but if you reuse the same gate on content or pricing pages you'd start blocking Googlebot and ChatGPT. The is_known_good_bot flag is the allowlist:

{
  "ip": "4.227.36.0",
  "security": {
    "is_bot": true,
    "bot_type": "ai_crawler",
    "bot_operator_name": "ChatGPT",
    "is_known_good_bot": true,
    "is_cloud_provider": true,
    "cloud_provider_name": "Microsoft Corporation",
    "threat_score": 15
  }
}

Your own automation is a cloud IP. If a partner integration or your own server-to-server flow legitimately creates accounts, it'll come from a datacenter too. Allowlist it, but prefer a shared secret or a scoped API key over an IP allowlist, since IPs change and secrets don't.

Mind the cost. Each lookup is 2 credits, so a burst from one attacker shouldn't cost you 2 credits times a thousand attempts. That's what the per-IP cache is for. Rate-limiting at the edge by ASN, in Cloudflare or your WAF, is an even cheaper first layer before the API call ever happens.

Mobile and CGNAT are not a problem here, residential proxies are. Carrier-grade NAT and mobile ranges don't read as cloud, so the gate won't misfire on phone users. The genuinely hard case is residential proxies, which look residential on purpose. That's exactly why this gate is narrow: it catches servers, and you handle residential-proxy abuse through threat_score and is_residential_proxy, not through the cloud flag.

A few things worth adding

If you want a softer version, drive it off threat_score instead of a hard block: add friction in the 45 to 79 band, block at 80 and up, and let the cloud flag push borderline cases over the line. IPv6 works identically, the flag set is the same. And the cheapest win of all is to log every cloud_provider_name you're about to block for a week before you actually turn blocking on.

Drop the classifier into your signup route, keep the anonymized-traffic decision on its own softer track, and you'll stop the datacenter bots without a queue of real users asking why they can't register. That week of logging first is worth it: you'll find your own CI, a partner's integration, and that one big customer on Zscaler before any of them becomes a support ticket.

Is Anyone Actually Reading Your Abuse Reports?

ABDULLAH AFZAL — Fri, 04 Sep 2026 11:18:01 +0000

Somewhere on a hosting provider's /24, a scanner is working through port 22. It fails on one address, moves to the next in the block, fails again, and keeps walking. Every server it touches that has fail2ban configured with an automated reporting action fires off an abuse report. By evening, the operator's abuse desk holds a few hundred near-identical emails about the same box, each a slightly reworded "SSH brute force from your network."

Nobody reads them. Not because the abuse team is lazy, but because there's nothing left to read. Three hundred reports of one event carry about as much information as one report, and take three hundred times longer to get through.

Automated reporting didn't make that IP more likely to get actioned. It made every report about it less likely to.

That's the whole argument. Reporting abuse is worth doing. The way most of us do it now, on autopilot and at volume, is quietly making it worse.

Reporting got free, and free things get flooded

It used to take effort to report an abusive IP. You noticed the pattern in your logs, found the network's abuse contact, wrote the email, attached the evidence, and sent it. That friction was doing more work than anyone gave it credit for. It meant the reports that went out were the ones a person had decided were worth sending.

Then reporting became a line of config.

# Illustrative: add Fail2Ban's built-in complaint action to the normal ban action.
# This also requires its documented dependencies and a working mail setup.
action = %(known/action)s
         complain[logpath="%(sshd_log)s"]

fail2ban, CrowdSec, honeypots, and mail feedback loops all turned "report this" into a background job that runs thousands of times a day with no human in the loop. High-volume automated reporting became easy. The amount of signal in each report did not necessarily improve.

This is the pattern behind every system that gets cheaper to use faster than it gets smarter. When the marginal cost of sending drops to zero, quantity stops tracking importance. Email spam works this way. So does the notification tray on your phone. Abuse reporting is now in the same bucket, and we got here with good intentions and a cron job.

Volume is not the same as signal

Here's the part that's easy to miss when your dashboard shows a satisfying count of reports sent. The receiving end is either a human with a finite number of hours, or an automated reputation system that's only as good as what you feed it. Both get worse under a firehose.

Think about what a lazy automated report actually contains. An IP. A timestamp in whatever timezone the reporting server happens to run in. A one-line reason like "brute force," and often no evidence at all. Sometimes it contains the reporter's own IP or email, pasted into a comment field by accident. Multiply that by every install reporting the same scanner, and the operator isn't reading intelligence. They're reading log exhaust.

You don't have to take my word for it. Look at what one of the best-known abuse-reporting databases had to build to cope. AbuseIPDB asks you to report a continuing attacker roughly once per day, not once per attempt. The same account cannot report the same IP more than once every 15 minutes, and if the same IP is reported again within 24 hours with the same comment, AbuseIPDB updates the original report's timestamp instead of creating another one. It tells reporters to limit comments to the key information about the abuse and warns against putting personal information into those comments. Its API also enforces rate limits with 429 Too Many Requests.

Those rules aren't decoration. They're scar tissue. Each one exists because undisciplined reporting was degrading the database, and the fix was to make reporters slow down and clean up. When the platform built to absorb abuse reports has to defend itself against abuse reports, the practice has a quality problem, not a volume shortage.

The people doing the reporting suspect it too. Search the sysadmin forums and the same question keeps turning up: is there any point to reporting an abusive IP at all? That doesn't come from people who don't care. It comes from people who've been firing reports into a void and noticed nothing changed.

The strongest case against me

There's a real argument on the other side, and it deserves a fair hearing.

More data beats less data. A reputation system that sees one IP reported by five hundred independent networks has stronger evidence than one that sees it once, even if most of those reports are sloppy. Deduplication and scoring exist precisely to turn noisy inputs into a usable signal, and they mostly work. Reporting, even imperfect reporting, raises the cost of running an attack, because a burned IP is a burned IP. Blocklists built from exactly this kind of crowdsourced volume do block a great deal of bad traffic.

All of that is true, and none of it rescues the specific failures. Dedup collapses the count, but it can't reconstruct evidence a report never carried. Scoring is only as good as its inputs, and a flood of context-free reports drags the average toward noise: a confidence number built on five hundred reports that all say "brute force" with no detail is a number about popularity, not about what the IP did. And the thing volume genuinely can't fix is the human desk. An operator who could have suspended a customer off one clear, actionable report instead closes the ticket, because the one good report is buried under three hundred that say nothing.

So the counter-argument mostly holds for the reputation databases and falls apart for the operators who can actually stop the abuser. That split is the point.

Report less, report better

The fix isn't to stop reporting. It's to treat a report as a message to a stranger who's short on time, and act like it.

Throttle. Report a continuing attacker once per source per day, not once per packet. Whatever your tool fires on, put a window in front of it.

Deduplicate by incident and abuse destination, not blindly by individual IP. If several addresses in the same network are performing the same behavior in the same time window and resolve to the same abuse desk, combine them into one report with the relevant IPs and evidence. Don't automatically collapse an entire routed prefix into one incident, though: the same network can contain different customers and unrelated abusive hosts.

Make it actionable in ten seconds. The offending IP, a UTC timestamp, what it did, the destination or affected system when that helps identify the event, and a few lines of real log. That's the core of the report. Leave out the opinions, threats, and unrelated personal information. If you're posting to a public reputation database, strip your own IP, email, and other PII from comments unless the service explicitly requires it. One report a human can act on immediately beats a hundred they have to decode.

Send it to the operator, not the void. The contact you want is the network operator responsible for the IP, not the registry and not the abuser. ABUSE has been a standardized network-operations mailbox name since RFC 2142 in 1997, although the actual abuse address for an IP may be a different contact registered with the relevant RIR. Resolving it yourself means using registry-aware WHOIS or RDAP data and handling the differences between registries. Or you can skip that plumbing: a single abuse-contact lookup returns the responsible abuse contact and network information directly.

Escalate only when it's earned. If the operator ignores a clean report, an upstream provider, hosting reseller, or relevant blocklist may be a reasonable next move depending on the incident. If the registered abuse contact itself is missing or invalid, raise that contact-data problem with the relevant RIR. Don't treat the RIR as an escalation desk simply because a valid abuse contact chose not to respond. Leading with the blocklist because it's the easiest button is how the noise started.

None of this is hard. It's just less than what your tooling does by default, and that's exactly the point.

What you're actually spending

The internet's abuse handling runs on two things you spend every time you hit send: a volunteer's attention, and the shared reputation data everyone downstream depends on. Both are finite, and a lazy report spends them for nothing. If you're going to automate reporting, and you should, automate the throttling and the formatting and the routing along with it. Otherwise you're not fighting the spam. You're just more of it.

How Do I Route EU Users Data to an EU Region?

ABDULLAH AFZAL — Tue, 01 Sep 2026 12:27:38 +0000

GDPR-aware regional routing is two problems, not one: estimating where a request comes from, and deciding which data region your architecture or residency policy should use. An IP address helps with the first. GDPR does not impose a blanket rule that EU users' data must stay inside the EU, but it does regulate transfers of personal data to third countries.

Most write-ups pick a side. The code-first ones show you a country lookup and stop, with no mention of what the lookup is legally worth. The compliance-first ones explain the regulation and never show you a single line of routing. This is the piece in the middle: working detection and routing, plus an honest map of where IP geolocation stops being an answer.

TL;DR

Detect with an IP geolocation lookup. country_code2 and an is_eu boolean are enough to make the routing decision.
is_eu covers the EU-27, while this residency policy covers the full EEA-30. Add Norway, Iceland, and Liechtenstein explicitly so EEA-detected requests follow the same regional rule.
Route with edge middleware: read the client IP correctly (trust-proxy matters), map country to a data region, and store or process there.
Fail toward the EEA region, not away from it. If your residency policy keeps EEA-detected requests in the EEA, routing an unknown request there is the safer default and avoids an accidental policy violation.
IP is a signal, not a lawful basis. VPNs, proxies, and mobile carriers break it, GDPR scope does not turn on IP, and an IP address is itself personal data. If you want the short version: use the geolocation lookup to route, treat is_eu as a convenience rather than a compliance boundary, widen it to the EEA, and log every routing decision so you can defend it later. The rest of this walks through each step with code and the reasoning behind it.

What data-residency routing actually is

Data residency is a constraint on where data physically sits. Data-residency routing is the mechanism that honors whatever residency policy your product, customer contract, sector, or risk model requires. GDPR itself does not generally require personal data about EU or EEA users to be stored inside the EEA. Keeping relevant storage and processing inside the EEA can, however, simplify compliance by reducing the number of international transfers you need to account for.

Two adjacent ideas get tangled with it, and separating them keeps the design honest.

Data residency is not data sovereignty. Residency is the physical location of the bytes. Sovereignty is about which government's laws and courts have jurisdiction over that data, which can differ from where it is stored (a US-headquartered provider storing data in Frankfurt is a common sovereignty question). Routing solves residency directly. It only mitigates sovereignty.

Routing is not the same as IP minimization. Google Analytics, for instance, drops EU IP addresses before logging rather than routing anything. That is a valid pattern for one field. It is not a substitute for deciding where a user's account, orders, or messages live. Different problem, different tool.

The decision that determines where the data goes

Before any code, define what your routing policy actually means. If your policy is "keep requests detected in the EEA in an EEA region," is_eu alone is too narrow.

is_eu covers the 27 EU member states. The EEA also includes Iceland, Liechtenstein, and Norway, where the GDPR has been incorporated into the EEA framework. Those three are not EU members, so is_eu returns false for them. Add them explicitly when implementing an EEA-based residency policy.

This EEA routing boundary should not be confused with GDPR's territorial scope. Whether GDPR applies to a processing activity depends on Article 3 factors such as establishment, offering goods or services, and monitoring behavior, not simply on the user's detected country.

Set	Members	`is_eu` returns	Route under this EEA-residency policy?
EU-27	Austria, Germany, France, ... 24 more	`true`	Yes
EEA extras	Iceland (IS), Liechtenstein (LI), Norway (NO)	`false`	Yes
Everywhere else	US, IN, BR, ...	`false`	No, unless your residency policy says otherwise

Adequacy is a separate decision from IP-based routing. It concerns where personal data is being transferred, not where the user happens to be located.

If personal data is transferred from the EEA to a country, territory, sector, or organisation covered by a European Commission adequacy decision, that transfer can take place without an additional Chapter V safeguard. The Commission maintains the list and it changes, so treat it as current legal configuration rather than a hardcoded user-country list. Current examples include the United Kingdom, Switzerland, Japan, Republic of Korea, Brazil, New Zealand, Canada for covered commercial organisations, and participating US organisations under the EU-US Data Privacy Framework.

Data path	GDPR transfer position	Practical implication
Relevant processing remains within the EEA	No third-country transfer under Chapter V	Use an EEA region when that matches your residency policy
Transfer from the EEA to an adequate destination	Article 45 adequacy decision	No additional Chapter V transfer safeguard is required for the covered transfer
Transfer from the EEA to another third country	Chapter V mechanism required, unless an applicable exception applies	Assess SCCs or another valid transfer mechanism separately from IP routing

The nuance worth internalizing: keeping relevant storage, processing, and access inside the EEA can reduce international-transfer complexity. But an EEA database region does not by itself eliminate Chapter V questions if the data is remotely accessed, processed, supported, or onward-transferred from outside the EEA. Adequacy decisions and safeguards such as Standard Contractual Clauses address transfers when data does leave the EEA.

Detecting an EEA location by IP

The detection step is one HTTP call. Any IP geolocation provider returns the two fields that matter here: an ISO 3166-1 alpha-2 country code and, if the provider exposes one, an EU membership flag. ip-api, ipinfo, MaxMind GeoIP2, DB-IP, and IPGeolocation all return roughly the same country payload; pick by latency, pricing model (per-call API vs a local database), or what is already in your stack.

I'll use ipgeolocation.io for the examples because its free tier returns country_code2 and an is_eu boolean in a single call, which keeps the routing logic short. You can create a key on the free tier at ipgeolocation.io.

A quick look at the two fields with curl:

# Keep the key in the environment. Never commit it inside a URL.
export IPGEO_API_KEY="your_key_here"

# --max-time caps the whole request; a slow lookup should never block your request path.
curl -s --max-time 2 \
  "https://api.ipgeolocation.io/v3/ipgeo?apiKey=${IPGEO_API_KEY}&ip=8.8.8.8" \
  | jq '{country_code2: .location.country_code2, is_eu: .location.is_eu}'

For 8.8.8.8 (a US address) that returns country_code2: "US" and is_eu: false. The full response carries more than the routing decision needs, and it is worth seeing the shape once so you know what else is there:

{
  "ip": "8.8.8.8",
  "location": {
    "continent_code": "NA",
    "continent_name": "North America",
    "country_code2": "US",
    "country_code3": "USA",
    "country_name": "United States",
    "city": "Mountain View",
    "zipcode": "94043",
    "latitude": "37.42240",
    "longitude": "-122.08421",
    "is_eu": false,
    "country_flag": "https://ipgeolocation.io/static/flags/us_64.png",
    "geoname_id": "5375480"
  },
  "country_metadata": {
    "calling_code": "+1",
    "tld": ".us",
    "languages": ["en-US", "es-US", "fr"]
  },
  "currency": { "code": "USD", "name": "US Dollar", "symbol": "$" }
}

For an EU member the same call returns the member's code ("DE", "FR", and so on) with is_eu: true. For an EEA-but-not-EU country like Norway, country_code2 is "NO" and is_eu is false, which is exactly why the routing code below checks both the flag and a short EEA list.

Heads up: is_eu is a convenience, not a compliance boundary. It answers "is this one of the 27 EU states," not "is this user in GDPR scope." The code widens it deliberately.

Routing the request to an EEA region

Now the mechanism. The pattern that scales is to decide the region once, as early in the request path as possible, and carry that decision downstream to whichever store or queue the data lands in.

Two things break this in practice if you skip them, so they are in the code rather than a footnote. First, reading the client IP correctly behind a proxy or load balancer. Second, deciding what happens when the lookup fails.

Here is Express middleware that resolves an IP to a target region:

// npm i express
const express = require("express");
const app = express();

// If you run behind a proxy/LB/CDN, configure Express to trust only the
// proxy path you actually control. req.ip is then derived from the socket
// address and X-Forwarded-For using that trust configuration.
//
// A numeric hop count is appropriate only when every request reaches the app
// through the same number of trusted hops. In production, trusting explicit
// proxy IPs or CIDRs is safer when your topology allows it.
app.set("trust proxy", 1); // illustrative: exactly one trusted proxy hop

// This residency policy covers the EEA: EU-27 (is_eu) plus these three.
const EEA_EXTRA = new Set(["IS", "LI", "NO"]);

// Country -> data region. Illustrative AWS names; swap for your provider.
// This is a residency-policy decision, separate from any Chapter V
// analysis of transfers to third countries.
const REGION_BY_GROUP = {
  eea: "eu-central-1", // Frankfurt: keep EU/EEA data in the EU
  other: "us-east-1",
};

async function resolveRegion(ip) {
  const key = process.env.IPGEO_API_KEY;
  const url = `https://api.ipgeolocation.io/v3/ipgeo?apiKey=${key}&ip=${ip}`;

  try {
    // 1.5s ceiling: a geolocation lookup must never hold the request open.
    const res = await fetch(url, { signal: AbortSignal.timeout(1500) });
    if (!res.ok) throw new Error(`lookup failed: ${res.status}`);

    const data = await res.json();
    const cc = data?.location?.country_code2 ?? null;
    const isEu = data?.location?.is_eu ?? false;

    // Missing country means the location is unknown, so use the protective default.
    if (!cc) return REGION_BY_GROUP.eea;

    if (isEu || EEA_EXTRA.has(cc)) return REGION_BY_GROUP.eea;
    return REGION_BY_GROUP.other;
  } catch (err) {
    // Fail toward the protective region. Under this residency policy,
    // an unknown location is treated as EEA rather than risking a mis-route.
    console.error(`region lookup error for ${ip}: ${err.message}`);
    return REGION_BY_GROUP.eea;
  }
}

app.use(async (req, _res, next) => {
  req.dataRegion = await resolveRegion(req.ip);
  next();
});

req.dataRegion is now set for every request, and downstream handlers pick the database, bucket, or queue for that region. The two decisions that make it production-safe are the trust proxy setting (so req.ip is the client, not your load balancer) and the catch block that fails toward the EEA region rather than away from it.

The same logic in Python, as a FastAPI dependency:

# pip install fastapi httpx
import os
import httpx
from fastapi import Depends, Request

EEA_EXTRA = {"IS", "LI", "NO"}  # EEA minus the EU-27 that is_eu covers
REGION_BY_GROUP = {"eea": "eu-central-1", "other": "us-east-1"}

async def resolve_region(request: Request) -> str:
    # Behind a proxy, configure your server (e.g. uvicorn --forwarded-allow-ips)
    # so request.client.host is the real client, not the proxy.
    ip = request.client.host if request.client else ""
    key = os.environ.get("IPGEO_API_KEY")
    url = f"https://api.ipgeolocation.io/v3/ipgeo?apiKey={key}&ip={ip}"

    try:
        # (connect, read) timeouts keep a slow lookup off the request path.
        async with httpx.AsyncClient(timeout=httpx.Timeout(1.5, connect=1.0)) as client:
            resp = await client.get(url)
            resp.raise_for_status()
            loc = resp.json().get("location", {})
            cc = loc.get("country_code2")
            is_eu = loc.get("is_eu", False)

            # Missing country means the location is unknown, so use the protective default.
            if not cc:
                return REGION_BY_GROUP["eea"]

            if is_eu or cc in EEA_EXTRA:
                return REGION_BY_GROUP["eea"]
            return REGION_BY_GROUP["other"]
    except Exception as err:
        # Unknown location routes to the EEA region under this protective residency policy.
        print(f"region lookup error for {ip}: {err}")
        return REGION_BY_GROUP["eea"]

# Usage: def handler(region: str = Depends(resolve_region)): ...

Both versions make the same call: check is_eu, widen it with the EEA set, and fall to the EEA region on failure.

Country to region, as a table

The dictionary in the code is the compact version of this decision. Spelled out:

Detected group	Example codes	Route under this residency policy
EU-27 (`is_eu: true`)	DE, FR, ES, IT, PL	EEA region (e.g. `eu-central-1`)
EEA extras	IS, LI, NO	EEA region
Non-EEA	US, GB, JP, BR, IN	Default or configured region
Lookup failed	n/a	EEA region (protective default)

Fail open or fail closed

This is the decision people skip, and for residency it is the one with legal weight. Fail open (route an unknown location to the default region) optimizes for availability and cost. A protective default (route it to the EEA region) is useful when your own residency policy requires EEA-detected traffic to remain there. For that policy, I fail toward the EEA region: occasionally routing a non-EEA request to Frankfurt costs some latency, while routing an EEA request to the wrong region violates the policy the router was built to enforce.

Where IP geolocation stops being enough

Everything above is a routing heuristic. It is genuinely useful and it is not a compliance determination, and conflating the two is how teams get surprised in an audit. Three boundaries matter.

IP location is a signal, not a lawful basis. GDPR's territorial scope does not turn on a user's IP. It applies to processing by an establishment in the EU, and to offering goods or services to, or monitoring, people in the EU, regardless of where the processing happens (GDPR Article 3). A US company with EU customers can be fully in scope even if it never runs an IP lookup. Use geolocation to route data efficiently; do not use it to decide whether the regulation applies to you. The EDPB territorial-scope guidelines are the reference if you need to reason about that line carefully.

VPNs, proxies, and mobile carriers break the signal. A user in Berlin on a corporate VPN that egresses in Virginia looks American. A traveler in the EU on a mobile network can geolocate to the carrier's registered country rather than their cell. Country-level accuracy is high on ordinary residential and business IPs and drops on VPN egress, datacenter ranges, and some mobile carrier blocks. If a mis-route has consequences, corroborate the IP signal with account data (a billing country, a stored preference, an explicit region choice) instead of trusting it alone. Providers that flag VPN, proxy, and hosting IPs let you at least detect when the signal is shaky.

An IP address is personal data. In the context of a service handling identifiable users, you should generally treat client IP addresses as personal data when they can reasonably be linked to a person. In Breyer v Germany (C-582/14), the Court of Justice held that a dynamic IP address can constitute personal data for a website operator where the operator has legal means reasonably available to identify the user with additional information. Practical consequence: the lookup you run to decide residency may itself involve processing personal data. If you send that IP to an external geolocation provider, also account for where that provider processes it and whether the lookup creates a third-country transfer. Keep the lookup purposeful, review the provider's processing arrangement, avoid storing raw IPs longer than you need, and pseudonymize where appropriate.

IP geolocation can establish	IP geolocation cannot establish
A probable country for routing	Legal residence or citizenship
A rough EU/EEA vs non-EU split	Whether GDPR applies to your processing
A fast first-pass region decision	A lawful basis or valid consent
A hint that a connection is anonymized	Where a user actually is, when they use a VPN

Where transfers out of the EEA are genuinely necessary, the mechanisms are adequacy decisions and Chapter V safeguards such as Standard Contractual Clauses, documented on the Commission's pages for adequacy and international transfers. Keeping the relevant data and access paths within the EEA can avoid a third-country transfer. An EEA storage region alone is not sufficient if processing or access still occurs from outside the EEA.

A reference checklist

Keep this next to the implementation:

Route on the EEA-30, not just is_eu (EU-27). Add IS, LI, NO.
Read the client IP through your proxy configuration, not a raw header.
Set a hard timeout on the lookup (1 to 2 seconds) so it never blocks the request path.
Fail toward the EEA region when the lookup is unavailable or ambiguous.
Treat the adequacy list as a lookup, not a hardcoded constant; the Commission updates it.
Log the routing decision (IP-derived country, chosen region, timestamp) so you can defend it.
Corroborate with account data when a mis-route has real consequences.
Remember the lookup itself processes personal data; minimize and pseudonymize. ## A few extra notes

Doing the lookup at the CDN edge and passing a trusted country header to your app beats calling an API on every request. Cloudflare, Fastly, and AWS CloudFront can attach a country code at the edge; your app then reads a header you control instead of paying a network round trip per request. If you are calling a geolocation API synchronously on every request with no cache, that is the first thing I would change. Cache exact-IP results with a short TTL, or preferably move the lookup to a trusted edge layer that already provides the country signal. For policy-sensitive routing, avoid broadening one IP's result across an entire /24.

IPv6 needs the same treatment as IPv4. If your service is dual-stack, make sure the lookup path handles a v6 client address, and test it, because v6 country accuracy on some ranges lags v4.

The UK is worth a specific note. Since Brexit it is a third country, but the European Commission renewed its GDPR adequacy decision in December 2025. That means personal data can currently be transferred from the EEA to the UK under Article 45 without an additional Chapter V safeguard. This is about the transfer destination, not about whether a UK user's data must be stored in any particular region. UK GDPR may separately apply to your processing.

Within the routing condition itself, EEA_EXTRA is the only jurisdiction list you hardcode. Keep IS, LI, and NO in one named constant rather than scattering them across handlers. Treat adequacy decisions, transfer mechanisms, proxy configuration, and your own residency policy as separate configuration that may also change over time.

Where to take it next

Drop the middleware into your request path, point the region map at your actual providers, and add the routing decision to your logs before you ship it. If you only harden one thing afterward, move the lookup to the edge with a cache so you are not paying for it per request. And keep the distinction in view: this routes data correctly, it does not decide whether GDPR applies to you. That question was answered long before the IP lookup ran.

A Self-Correcting Solar System Baseline From Sunrise/Sunset Data

ABDULLAH AFZAL — Thu, 27 Aug 2026 09:36:23 +0000

A fixed-schedule solar baseline drifts out of sync with the sun throughout the year. In Phoenix the sun is up for 13 hours 10 minutes in late August and 10 hours 2 minutes at the December solstice. A flat daily kWh target flags that entire winter as a fault, then stays quiet on the July afternoon when one string dies at 2pm under full sun. The fix is to anchor the baseline to the actual sun instead of the clock, and most of what you need for that does not require an irradiance forecast.

One thing before any code: sun geometry tells you when a system should be producing and when it should peak. It does not tell you how much light actually reached the panels. That is irradiance, and cloud cover swamps it. If you want modeled output in kWh, reach for Forecast.Solar or Solcast, which fold in weather and your array's tilt and azimuth. What follows is the free, dependency-light layer underneath that: the daylight window, the solar-noon peak, and the day-length trend.

TL;DR

Sun geometry (sunrise, sunset, solar noon, day length) catches a specific class of solar underperformance with no irradiance data.

Gate alerts to the real daylight window so your monitor stops crying "underperformance" before sunrise.

Track the daily production peak relative to solar noon. A persistent shift across comparable days can reveal shading, orientation, or system changes that a total-kWh check misses.

Normalize a flat kWh target by day length so winter stops tripping false alarms. First-order fix, not a physics model.

One call to an astronomy endpoint returns all of it. Code below in curl, Python, and Node.

For real production forecasting, use an irradiance API. Sun times are the sanity layer, not the forecaster.

Sun times will not predict your kWh, but they eliminate common timing-based false alarms and can surface useful production-shape anomalies early. Pull sunrise, sunset, solar noon, and day length once a day, gate your alerts to daylight, watch the peak, and scale the target for season.

What sun times can and cannot tell you

Sun geometry gives you three usable things: the window a system can physically produce in, the moment it should peak, and how that window changes through the year. All three come from a plain lat/long lookup and none of them costs you a weather subscription.

What they leave out is the part that actually varies day to day. A clear July noon and an overcast one have the same solar noon and nearly the same day length, but wildly different output. So sun times are a denominator, not a measurement. Treat a gap between expected window and actual production as a question, not a verdict, and reserve the confident "this panel is broken" call for the cases geometry can isolate on its own.

Plenty of tools return sun times: Sunrise-Sunset.org is free with no signup, SunCalc computes positions in the browser without an API at all, and NOAA has a solar calculator. I am using ipgeolocation.io's Astronomy API here because one call returns the window, solar noon, day length, twilight, and the sun's position together, which keeps the code short. Pick whatever you already have in your stack.

The three checks worth building

None of these needs irradiance. Each one closes a real gap in the naive "compare today's kWh to a fixed number" monitor.

Gate alerts to the real daylight window

The cheapest win is to stop evaluating production when the sun is not up. A monitor that compares output to a target every fifteen minutes will happily fire a "low production" alert at 06:00 in December, when sunrise is at 07:25 and the correct output is zero. Gate every underperformance rule behind the sunrise-to-sunset window and that class of alert disappears.

Trim the edges with a margin. Output in the first and last half hour of daylight is near zero because the sun is barely above the horizon, so a low-production alert there is just physics, not a fault. A 30-minute margin inside sunrise and sunset is a reasonable start; widen it for a site with a hard eastern or western horizon. The one edge case to handle is high latitude in midsummer or midwinter, where the API can report the sun as up or down all day; check sun_status before trusting the window there.

Watch the peak drift off solar noon

This is the check that earns its keep. Total daily kWh can look fine while a single string slowly fails, because the healthy strings mask the loss in the daily sum. The shape of the day gives it away sooner.

When the daily peak persistently shifts away from its normal offset, or the curve repeatedly becomes lopsided under comparable conditions, it can point to partial shading, uneven soiling, orientation effects, or inverter behavior. A single cloudy day's peak is weak evidence, so look for the change across multiple days rather than treating one maximum as a fault.

Orientation is the easy thing to get wrong here. A west-facing array peaks after solar noon by design and an east-facing one before it, so the threshold is relative to where your array should peak, not a universal number. Compare today's peak time to solar noon, keep a rolling few weeks of the offset, and alert on the change rather than the absolute value.

Normalize the baseline by day length

A flat kWh target assumes every day has the same amount of daylight, which is why it flags winter. Scaling the target by today's day length against a summer reference removes most of that error in about three lines. A 40 kWh midsummer target becomes roughly 29 kWh at the December solstice (10 hours 2 minutes of daylight against a 14-hour reference); the flat 40 kWh number would have marked a perfectly healthy winter day about 27% low.

Be honest about what this does not fix. Winter days are not just shorter, the sun is also lower, so each daylight hour delivers less energy and real output drops more than the day-length ratio predicts. Day-length scaling gets you from "wildly wrong" to "roughly right," which is enough to stop the seasonal false alarms. Closing the rest of the gap means modeling irradiance and sun elevation, which is exactly the job Forecast.Solar, Solcast, and the pvlib library exist to do.

Pull the sun data

The endpoint is a single GET. The free tier is 1,000 requests a day with no credit card, and one request covers a full day, so a per-site daily pull sits comfortably inside it. Sign up for a key on the Astronomy API and keep it out of source control.

# Free tier: 1000 requests/day, no card. One call returns the whole day's geometry.
# 2s timeout so a slow response can't hang a monitor loop.
curl -sS --max-time 2 \
  "https://api.ipgeolocation.io/v3/astronomy?apiKey=$IPGEO_API_KEY&lat=33.4484&long=-112.0740&date=2026-08-25&time_zone=America/Phoenix"

The response carries far more than the four fields we use. That is the point of one call:

{
  "location": {
    "latitude": "33.44840",
    "longitude": "-112.07400",
    "city": "Phoenix",
    "state_prov": "Arizona",
    "elevation": "331"
  },
  "astronomy": {
    "date": "2026-08-25",
    "current_time": "05:46:58.255",
    "sunrise": "2026-08-25 05:54",
    "sunset": "2026-08-25 19:05",
    "solar_noon": "2026-08-25 12:30",
    "day_length": "13:10",
    "sun_altitude": -3.03,
    "sun_azimuth": 75.14,
    "sun_distance": 151249918.79,
    "sun_status": "-",
    "morning": {
      "civil_twilight_begin": "2026-08-25 05:32",
      "civil_twilight_end": "2026-08-25 05:54",
      "golden_hour_begin": "2026-08-25 05:42",
      "golden_hour_end": "2026-08-25 06:31"
    },
    "evening": {
      "golden_hour_begin": "2026-08-25 18:29",
      "golden_hour_end": "2026-08-25 19:18",
      "civil_twilight_begin": "2026-08-25 19:05",
      "civil_twilight_end": "2026-08-25 19:28"
    },
    "moon_phase": "WAXING_GIBBOUS",
    "moonrise": "2026-08-25 17:48",
    "moonset": "2026-08-25 03:26",
    "moon_illumination_percentage": "93.10"
  }
}

Two things to notice before you write parsers. day_length is a string, "13:10", meaning 13 hours 10 minutes, not a decimal. And because the request set time_zone, the times come back as "YYYY-MM-DD HH:mm" in that zone rather than bare clock times. Both matter for the code. One more, in case it trips you up: sun_altitude and sun_azimuth are the sun's position at current_time, which is when the call ran, not at noon. In this capture that was just before sunrise, hence the negative altitude. The checker below ignores those fields, but they are useful for a live "is the sun even up right now" readout.

Pitfall: timestamps are returned in the time_zone you request. The curl example above asks for Phoenix local time, while the checker below deliberately requests UTC so the timestamps can be parsed unambiguously. Keep now timezone-aware and compare absolute times consistently. The date you request should still be the calendar date at the solar site.

Build the baseline checker

Fetch once per day, then run all three checks against that one response. Calling the API per check wastes credits and adds latency for data that does not change within a day. A small cron job at local midnight that caches the day's response is enough; there is no reason to hit the endpoint again until the date rolls over.

import os
import sys
from datetime import datetime, timedelta, timezone

import requests

IPGEO_API_KEY = os.environ.get("IPGEO_API_KEY")
ASTRONOMY_URL = "https://api.ipgeolocation.io/v3/astronomy"


def fetch_sun_day(lat, lon, date_str):
    """One day of sun geometry. Costs one credit, so call it once per day and
    reuse the result for every check instead of hitting the API per rule."""
    if not IPGEO_API_KEY:
        raise RuntimeError("Set IPGEO_API_KEY in the environment")
    params = {
        "apiKey": IPGEO_API_KEY,
        "lat": lat,
        "long": lon,
        "date": date_str,
        # Request UTC so returned timestamps are unambiguous when parsed.
        "time_zone": "UTC",
    }
    try:
        # (connect, read) timeouts so a slow upstream can't stall the monitor loop
        resp = requests.get(ASTRONOMY_URL, params=params, timeout=(1.0, 1.5))
        resp.raise_for_status()
    except requests.RequestException as err:
        # Fail open: this is a sanity layer, not a safety system. If it's down,
        # skip the geometry checks rather than block the rest of monitoring.
        print(f"sun lookup failed, skipping geometry checks: {err}", file=sys.stderr)
        return None
    return (resp.json() or {}).get("astronomy")


def hhmm_to_minutes(value):
    """day_length arrives as a string like '10:02' (hours:minutes), not decimal."""
    if not value or ":" not in value:
        return None
    hours, minutes = value.split(":")[:2]
    return int(hours) * 60 + int(minutes)


def parse_dt(value):
    """Parse the API's UTC 'YYYY-MM-DD HH:mm' timestamps as aware datetimes."""
    if not value:
        return None
    try:
        return datetime.strptime(
            value, "%Y-%m-%d %H:%M"
        ).replace(tzinfo=timezone.utc)
    except ValueError:
        return None

Those three helpers absorb the two gotchas: the string day length and the dated timestamps. Everything below returns cleanly on missing fields so a partial response never throws.

def in_daylight(now, astro, margin_minutes=30):
    """True only inside the productive window. The margin trims dawn and dusk."""
    status = astro.get("sun_status")

    if status == "Always above the twilight angle":
        return True
    if status == "Always below the twilight angle":
        return False

    sunrise = parse_dt(astro.get("sunrise"))
    sunset = parse_dt(astro.get("sunset"))

    if not sunrise or not sunset:
        return True  # unknown, so don't suppress a potentially real alert

    if now.tzinfo is None:
        raise ValueError("now must be timezone-aware")

    now_utc = now.astimezone(timezone.utc)
    start = sunrise + timedelta(minutes=margin_minutes)
    end = sunset - timedelta(minutes=margin_minutes)

    return start <= now_utc <= end


def peak_drift_minutes(observed_peak, astro):
    """How far the day's production peak sat from solar noon, in minutes.
    observed_peak must be a timezone-aware datetime."""
    solar_noon = parse_dt(astro.get("solar_noon"))
    if not solar_noon or not observed_peak:
        return None

    if observed_peak.tzinfo is None:
        raise ValueError("observed_peak must be timezone-aware")

    observed_peak_utc = observed_peak.astimezone(timezone.utc)
    return abs((observed_peak_utc - solar_noon).total_seconds()) / 60.0


def seasonal_target(base_kwh, astro, reference_day_length="14:00"):
    """Scale a flat kWh target by today's day length against a summer reference.
    Removes the gross seasonal error. It under-corrects, because winter also has a
    lower sun (less energy per daylight hour); closing that gap needs irradiance."""
    today = hhmm_to_minutes(astro.get("day_length"))
    reference = hhmm_to_minutes(reference_day_length)

    if today is None or reference is None or reference == 0:
        return base_kwh

    return round(base_kwh * (today / reference), 1)

Use the daylight gate around real-time production alerts, but run the peak-drift and daily-kWh checks only after the day's production window is complete. A full-day kWh target should never be compared with energy accumulated halfway through the day. The tolerance band below is still only a rough weather buffer, not a substitute for irradiance data.

def evaluate_completed_day(
    observed_peak,
    actual_kwh,
    base_kwh,
    lat,
    lon,
    date_str,
):
    """Run after the day's production window has finished."""
    astro = fetch_sun_day(lat, lon, date_str)
    if astro is None:
        return

    drift = peak_drift_minutes(observed_peak, astro)
    if drift is not None and drift > 90:
        print(
            f"peak offset was {drift:.0f} min from solar noon; "
            "track whether that shift persists across comparable days"
        )

    expected = seasonal_target(base_kwh, astro)

    if actual_kwh < expected * 0.75:
        print(
            f"actual {actual_kwh} kWh below seasonal expectation "
            f"{expected} kWh"
        )

The Node version is the same shape. fetch with a hard timeout, optional chaining on the response, and the identical string-and-timestamp parsing. It assumes Node 18 or newer, where fetch and AbortSignal.timeout are built in; on older runtimes add node-fetch and an abort polyfill.

const ASTRONOMY_URL = "https://api.ipgeolocation.io/v3/astronomy";

async function fetchSunDay(lat, lon, dateStr) {
  const key = process.env.IPGEO_API_KEY;
  if (!key) throw new Error("Set IPGEO_API_KEY in the environment");

  // dateStr should be the calendar date at the solar site.
  // Request event timestamps in UTC so Date parsing is unambiguous.
  const url = `${ASTRONOMY_URL}?apiKey=${key}&lat=${lat}&long=${lon}`
    + `&date=${dateStr}&time_zone=UTC`;
  try {
    // 1.5s timeout so a slow upstream can't stall the monitor loop
    const res = await fetch(url, { signal: AbortSignal.timeout(1500) });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const body = await res.json();
    return body?.astronomy ?? null;
  } catch (err) {
    // Fail open: sanity layer, not a safety system
    console.error(`sun lookup failed, skipping geometry checks: ${err.message}`);
    return null;
  }
}

// day_length is a string like "10:02" (h:mm), not a number
function hhmmToMinutes(value) {
  if (!value || !value.includes(":")) return null;
  const [h, m] = value.split(":");
  return Number(h) * 60 + Number(m);
}

// We request UTC from the API, so the timestamp can be parsed unambiguously.
function parseDt(value) {
  if (!value || value === "-:-") return null;
  const d = new Date(`${value.replace(" ", "T")}:00Z`);
  return Number.isNaN(d.getTime()) ? null : d;
}

function inDaylight(now, astro, marginMin = 30) {
  if (astro?.sun_status === "Always above the twilight angle") return true;
  if (astro?.sun_status === "Always below the twilight angle") return false;

  const sunrise = parseDt(astro?.sunrise);
  const sunset = parseDt(astro?.sunset);

  if (!sunrise || !sunset) return true; // unknown, don't suppress a real alert

  const start = new Date(sunrise.getTime() + marginMin * 60000);
  const end = new Date(sunset.getTime() - marginMin * 60000);

  return now >= start && now <= end;
}

function seasonalTarget(baseKwh, astro, referenceDayLength = "14:00") {
  const today = hhmmToMinutes(astro?.day_length);
  const reference = hhmmToMinutes(referenceDayLength);
  if (today === null || reference === null || reference === 0) return baseKwh;
  return Math.round(baseKwh * (today / reference) * 10) / 10;
}

Peak drift is the same idea on parsed values:

function peakDriftMinutes(observedPeak, astro) {
  const solarNoon = parseDt(astro?.solar_noon);
  if (!solarNoon || !observedPeak) return null;
  return Math.abs(observedPeak.getTime() - solarNoon.getTime()) / 60000;
}

Flag it against your array's orientation, same caveat as before: a west-facing array measures drift from its own expected offset, not from solar noon.

Where this stops and irradiance begins

Run the checker against a week of clear days and it behaves. Run it against a stretch of overcast ones and the seasonal check will nag, because a cloudy day and a broken panel look identical to sun geometry. That is the wall. The 25% tolerance band is a blunt way to ride through normal weather, and it works until a genuinely bad week hides a genuinely bad panel.

The honest boundary is this: use sun times as the reference for checks about timing and geometry. The daylight window is deterministic; peak offset is useful when it persists across multiple comparable days, but clouds can move a single day's observed maximum. Lean on day-length scaling only to keep the seasonal baseline in the right ballpark. The moment you need to tell "cloudy" from "faulty," or you care about temperature derating and soiling trends, you have left what geometry can answer and you want an irradiance model: Forecast.Solar and Solcast expose one over an API (Forecast.Solar drops straight into Home Assistant if that is where your dashboard lives), pvlib gives you one in Python, and any of them will take your panel tilt and azimuth into account.

Drop the daylight gate in first. It is the cheapest change and stops the most false alarms. Add the peak-drift check once you have per-interval data to compare against solar noon. And when geometry-level checks stop being enough, treat that as the cue to pull an irradiance forecast, not to keep widening the thresholds until the alerts go quiet.

Enrich Elasticsearch Logs With GeoIP at Ingest

ABDULLAH AFZAL — Mon, 24 Aug 2026 13:26:25 +0000

An access log full of raw IPs can't answer where a traffic spike came from. Elasticsearch GeoIP fixes that at write time, enriching public-IP events with whatever country, region, city, and coordinate data is available so queries, alerts, and dashboards downstream can use geography directly. This guide sets up that enrichment in an ingest pipeline, fixes the two things that silently break it, and puts your logs on a Kibana map.

TL;DR

The geoip processor ships with Elasticsearch as a module. No plugin install, and it auto-downloads the free MaxMind GeoLite2 databases.

Enrichment happens in an ingest pipeline. Test it with the _simulate API before you point real data at it.

Your Kibana map will be empty unless the location field is mapped as geo_point before you index. Set that in an index template first.

Private and internal IPs (RFC 1918) return no geo fields by design. Handle them, don't ignore them.

GeoLite2 is city-level-optimistic and CC BY-SA. When you need better accuracy or security context, swap the database at ingest without changing the pipeline shape.

The short version: turn on a bundled processor, wire it into a pipeline, map one field correctly, and Kibana renders your traffic on a map. The parts that trip people up are the geo_point mapping and the IPs that have no location, both covered below. The core examples below target modern Elasticsearch 8.x and 9.x clusters with security enabled. Third-party ingest plugins have their own Elasticsearch-version compatibility requirements, so check the plugin's supported version before installing one.

What the Elasticsearch geoip processor gives you

The geoip processor is a module distributed with Elasticsearch, so on a current cluster there's nothing to install. By default it uses the GeoLite2 City, GeoLite2 Country, and GeoLite2 ASN databases from MaxMind, shared under CC BY-SA 4.0, and Elasticsearch downloads and updates them automatically from the Elastic GeoIP endpoint.

Feed it a field holding an IP address and it writes a geoip object onto the document. With the default City database, that object can include continent_name, country_iso_code, country_name, region_iso_code, region_name, city_name, and a location object holding lat and lon. You control which of those land with the properties option; the default set is continent_name, country_iso_code, country_name, region_iso_code, region_name, city_name, and location.

Two option names worth knowing before you write anything: target_field (default geoip) is where the enriched object goes, and ignore_missing (default false) decides whether a document with no IP field quietly passes through or errors. We'll use both.

Step 1: Create the ingest pipeline

An ingest pipeline is a named list of processors that runs on a document before it's indexed. Create one with a single geoip processor pointed at your IP field:

# Create a pipeline named "geoip-pipeline" that enriches the "ip" field.
# --cacert points at the CA cert Elasticsearch generated on first start;
# on 8.x/9.x the HTTP layer is TLS-on by default, so http:// will fail.
curl -sS -X PUT "https://localhost:9200/_ingest/pipeline/geoip-pipeline" \
  --cacert /etc/elasticsearch/certs/http_ca.crt \
  -u "elastic:$ELASTIC_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{
    "description": "Add GeoIP fields from the ip field",
    "processors": [
      {
        "geoip": {
          "field": "ip",
          "target_field": "geoip",
          "ignore_missing": true
        }
      }
    ]
  }'

ignore_missing: true matters more than it looks. Real log streams have events without a client IP (health checks, internal jobs). Without it, every one of those throws and the document is rejected. With it, they index cleanly and simply carry no geoip object.

Before you send a single real document, dry-run the pipeline with the _simulate API. This runs your processors against a sample doc and returns the result without indexing anything:

# Dry-run the pipeline against a real public IP.
# 8.8.8.8 is a convenient public test address. The exact GeoLite2 fields and coordinates can change as the database is updated.
curl -sS -X POST "https://localhost:9200/_ingest/pipeline/geoip-pipeline/_simulate" \
  --cacert /etc/elasticsearch/certs/http_ca.crt \
  -u "elastic:$ELASTIC_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{
    "docs": [
      { "_source": { "ip": "8.8.8.8" } }
    ]
  }'

The response wraps the enriched document under doc._source, and you should see a populated geoip object:

{
  "docs": [
    {
      "doc": {
        "_source": {
          "ip": "8.8.8.8",
          "geoip": {
            "continent_name": "North America",
            "country_iso_code": "US",
            "country_name": "United States",
            "location": { "lon": -97.822, "lat": 37.751 }
          }
        }
      }
    }
  ]
}

If _simulate gives you the fields you expect, the pipeline works. The rest is deciding what data flows through it and making sure Kibana can read the output.

Step 2: Map location as geo_point before you index

Here's the failure that fills the Elastic forums: everything ingests fine, the geoip.location field is populated, and the Kibana map is stubbornly empty. The cause is mapping. Elasticsearch's dynamic mapping sees { "lat": ..., "lon": ... } and guesses two float fields. Kibana Maps only plots a field typed as geo_point, and it won't infer that on its own.

Fix it once with an index template so every matching index is born with the right type:

# Any index matching "logs-geo-*" gets geoip.location typed as geo_point.
# Create this BEFORE the first document lands, or the wrong type sticks.
curl -sS -X PUT "https://localhost:9200/_index_template/logs-geo-template" \
  --cacert /etc/elasticsearch/certs/http_ca.crt \
  -u "elastic:$ELASTIC_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{
    "index_patterns": ["logs-geo-*"],
    "template": {
      "settings": {
        "index.default_pipeline": "geoip-pipeline"
      },
      "mappings": {
        "properties": {
          "geoip": {
            "properties": {
              "location": { "type": "geo_point" }
            }
          }
        }
      }
    }
  }'

Two things happened in that template. The mappings block pins geoip.location to geo_point so Kibana will plot it. And index.default_pipeline sets the pipeline to run automatically on anything written to a logs-geo-* index, so you don't have to name it on every request.

Pitfall: mapping type is fixed at field creation. If documents already landed and geoip.location came out as float, you can't change it in place. You create a new index with the correct template and reindex into it. Get the template in before the first write and you never hit this.

Step 3: Wire the pipeline to your log stream

You have two ways to attach the pipeline. The template above already set index.default_pipeline, which is the clean option: write to the index normally and enrichment is automatic. The other option is naming the pipeline per request with ?pipeline=geoip-pipeline, which is handy for one-off backfills but noisy to repeat on every write.

If you ingest with your own code, the bulk helper is the usual path. Here's a minimal, production-shaped Python ingester using the official client:

import os
from elasticsearch import Elasticsearch, helpers

# Never hardcode credentials. Pull them from the environment.
ES_URL = os.environ.get("ES_URL", "https://localhost:9200")
ES_USER = os.environ.get("ES_USER", "elastic")
ES_PASSWORD = os.environ.get("ES_PASSWORD")
CA_CERT = os.environ.get("ES_CA_CERT", "/etc/elasticsearch/certs/http_ca.crt")

client = Elasticsearch(
    ES_URL,
    basic_auth=(ES_USER, ES_PASSWORD),
    ca_certs=CA_CERT,
    request_timeout=10,  # fail fast instead of hanging a worker
)

def log_events():
    # Replace with your real source (a file tail, a queue, etc.).
    yield {"ip": "8.8.8.8", "path": "/checkout", "status": 200}
    yield {"ip": "2001:4860:4860::8888", "path": "/login", "status": 401}

def index_logs():
    # The index name matches logs-geo-*, so the default_pipeline runs
    # and geo_point mapping applies. No per-doc pipeline needed here.
    actions = ({"_index": "logs-geo-000001", "_source": event}
               for event in log_events())
    try:
        success, errors = helpers.bulk(client, actions, raise_on_error=False)
        if errors:
            # Log rejected docs; don't let one bad IP kill the batch.
            for err in errors:
                print(f"index error: {err}")
        return success
    except Exception as exc:
        # Network or auth failure: surface it, don't swallow it.
        print(f"bulk ingest failed: {exc}")
        return 0

if __name__ == "__main__":
    print(f"indexed {index_logs()} events")

raise_on_error=False plus an error loop is deliberate: one document with a malformed IP shouldn't sink a 10,000-event batch. You log the rejects and keep going.

One convention worth adopting if you use the Elastic Common Schema: put the client address in source.ip and target source.geo instead of a bare geoip field. Elastic's own dashboards and detection rules expect ECS field names like source.geo.country_iso_code, source.geo.city_name, and source.geo.location. Set "field": "source.ip" and "target_field": "source.geo" in the processor and your data drops straight into the built-in tooling.

A word on Logstash, since half the tutorials on this topic start there. The Logstash geoip filter still works and still makes sense if Logstash is already in your pipeline doing Grok parsing and transforms. But if Elasticsearch is your ingest point, the ingest-pipeline approach keeps enrichment inside the cluster with one less moving part to run and monitor. Use the filter when you're already in Logstash; use the processor when you're not.

Step 4: Put it on a map in Kibana

With geo_point mapping in place, Kibana Maps takes a couple of clicks:

Create a data view over your logs-geo-* indices (Stack Management, then Data Views).
Open Maps from the Analytics menu and add a layer from that data view.
Because geoip.location (or source.geo.location) is a geo_point, the documents layer plots each event as a point. Style by count to cluster dense areas. If you followed a tutorial that told you to create a "Tile Map" or "Coordinate Map" visualization, that's why it didn't work. Those classic visualization types were retired; the Maps application replaced them, and it reads geo_point fields directly through a data view.

Private and internal IPs return nothing

The geoip processor looks up public IP geolocation. Private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), loopback, and carrier-grade NAT (100.64.0.0/10) have no geographic location, so those documents come back with no geoip object at all. That's correct behavior, but if a chunk of your traffic is internal, a naive setup leaves you with events that silently have no geo and no explanation.

Tag them instead of dropping them. Add a conditional processor that marks internal traffic before the lookup, so a missing geoip object is intentional rather than a mystery:

# Mark RFC 1918 addresses as internal, then geoip-enrich the rest.
# Explicitly match the RFC 1918 private ranges so internal traffic
# is labeled instead of silently appearing without GeoIP data.
curl -sS -X PUT "https://localhost:9200/_ingest/pipeline/geoip-pipeline" \
  --cacert /etc/elasticsearch/certs/http_ca.crt \
  -u "elastic:$ELASTIC_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{
    "description": "Tag internal IPs, geoip-enrich the rest",
    "processors": [
      {
        "set": {
          "field": "network.scope",
          "value": "internal",
          "if": "def ip = ctx.ip; ip != null && (ip.startsWith(\"10.\") || ip.startsWith(\"192.168.\") || ip.startsWith(\"172.16.\") || ip.startsWith(\"172.17.\") || ip.startsWith(\"172.18.\") || ip.startsWith(\"172.19.\") || ip.startsWith(\"172.20.\") || ip.startsWith(\"172.21.\") || ip.startsWith(\"172.22.\") || ip.startsWith(\"172.23.\") || ip.startsWith(\"172.24.\") || ip.startsWith(\"172.25.\") || ip.startsWith(\"172.26.\") || ip.startsWith(\"172.27.\") || ip.startsWith(\"172.28.\") || ip.startsWith(\"172.29.\") || ip.startsWith(\"172.30.\") || ip.startsWith(\"172.31.\"))"
        }
      },
      {
        "geoip": {
          "field": "ip",
          "target_field": "geoip",
          "ignore_missing": true
        }
      }
    ]
  }'

Internal events now carry network.scope: internal, so one common reason for a missing geoip object is explicit instead of silent. Public IPs with no database match can still have no geoip object. No more staring at half-empty maps wondering where the other half went.

Backfilling indices you already have

Enrichment only touches documents at write time, so turning on the pipeline does nothing for the millions of events already sitting in your indices. To geo-tag existing data, run _update_by_query with the pipeline applied:

# Reprocess existing docs through the pipeline.
# Run this against a copy or a low-traffic window first: it rewrites
# every matched document and costs real I/O.
curl -sS -X POST "https://localhost:9200/logs-geo-000001/_update_by_query?pipeline=geoip-pipeline&wait_for_completion=false" \
  --cacert /etc/elasticsearch/certs/http_ca.crt \
  -u "elastic:$ELASTIC_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{
    "query": {
      "bool": {
        "must_not": [
          { "exists": { "field": "geoip" } }
        ]
      }
    }
  }'

The must_not exists geoip filter avoids reprocessing documents that were successfully enriched. Be aware that private IPs and public IPs with no database match also have no geoip field, so they will be selected again on a later run. If you need a strictly resumable backfill, add a separate processing marker and filter on that instead.
wait_for_completion=false returns a task ID immediately so a long backfill doesn't tie up your connection; poll it with the Tasks API.

One caveat: if the old index has geoip.location mapped as float (because it predates your template), _update_by_query writes the fields but Kibana still won't map them. In that case you reindex into a freshly templated index instead. Backfill in place when the mapping is already correct; reindex when it isn't.

When GeoLite2 is not enough

GeoLite2 is free, bundled, and good enough to see which countries your traffic comes from. It is also the lite tier: MaxMind's own positioning is that GeoLite2 is less accurate than paid GeoIP2, city-level results are approximate, and the CC BY-SA 4.0 license carries attribution and share-alike obligations that some commercial deployments can't accept. When you outgrow it, you swap the database, not the pipeline.

You have a few paths, and they nest cleanly into what you already built:

Point the same processor at a licensed MaxMind database. Elasticsearch can download GeoIP2 directly using your MaxMind license key through the database configuration API, and the geoip processor keeps working unchanged.
Use IPinfo databases via the ip_location processor. Recent Elasticsearch versions ship an ip_location processor that supports IPinfo databases alongside MaxMind, so you can switch providers without leaving the ingest layer.
Load a custom database file. Both processors accept a database_file option, so a compatible MaxMind-format database supported by the processor can back the lookup.
Enrich from a local database that also carries security and timezone context. IPGeolocation publishes an Elasticsearch ingest processor plugin that runs the lookup against a locally installed IPGeolocation database, so alongside country and city you get currency, timezone, and IP-security signals in the same document, with no per-event API call. That last option is worth a concrete look because it combines IPGeolocation's geolocation, timezone, currency, and security data in one local enrichment path. The plugin installs with elasticsearch-plugin install, a setup script downloads the database tier you licensed, and the processor is named ipgeo. Its required options are field (your IP field) and database_version (DB-I through DB-VII, the database tier you licensed); target_field defaults to ipgeo, and include pulls in the extra data classes:

# The ipgeo processor from the IPGeolocation plugin.
# database_version selects the licensed DB tier; include pulls in
# the security and timezone data classes on top of geolocation.
curl -sS -X PUT "https://localhost:9200/_ingest/pipeline/ipgeo-pipeline" \
  --cacert /etc/elasticsearch/certs/http_ca.crt \
  -u "elastic:$ELASTIC_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{
    "description": "Enrich with IPGeolocation databases",
    "processors": [
      {
        "ipgeo": {
          "field": "ip",
          "database_version": "DB-VI",
          "ignore_missing": true,
          "include": "security,timezone"
        }
      }
    ]
  }'

A document enriched by that pipeline carries a richer object than GeoLite2 returns. The relevant part of the response, from the plugin's own docs:

{
  "ipgeo": {
    "country_name": "United States",
    "country_code2": "US",
    "city": "San Francisco",
    "state_prov": "California",
    "zipcode": "94107-2008",
    "location": { "lon": -122.39117, "lat": 37.78229 },
    "currency": { "symbol": "$", "code": "USD", "name": "US Dollar" },
    "time_zone": {
      "name": "America/Los_Angeles",
      "offset": -7.0,
      "is_dst": true,
      "current_time": "2024-05-17 04:52:01.693-0700"
    },
    "security": {
      "is_proxy": false,
      "proxy_type": "",
      "is_tor": false,
      "is_anonymous": false,
      "is_known_attacker": false,
      "is_bot": false,
      "is_spam": false,
      "is_cloud_provider": false,
      "threat_score": 0
    }
  }
}

The trade-off is honest and worth stating plainly: this path costs a plugin install, a licensed database, and an Elasticsearch restart, and the ipgeo.location field still needs the same geo_point mapping as before. What you get back is currency, timezone, and a security block (proxy, Tor, cloud-provider, known-attacker flags, and a threat_score) on every event, computed locally with no lookup latency. If your logs feed fraud rules or security dashboards, that context in the same document is the reason to pay. If you only need a map of where traffic comes from, GeoLite2 is already enough and you can stop at Step 4.

Operational notes

A few things that bite in production and rarely make it into tutorials:

The auto-downloaded databases expire. If Elasticsearch can't reach the Elastic GeoIP endpoint for 30 days, it stops using the stale databases and enrichment quietly goes blank, showing up as _geoip_expired_database tags on your documents. In an air-gapped cluster this is a when, not an if. Plan for a reverse-proxy or custom endpoint, or serve your own databases and turn the downloader off.

To disable the downloader entirely, set ingest.geoip.downloader.enabled: false in the cluster settings. Do this when you run only custom or licensed databases and don't want Elasticsearch reaching out for GeoLite2 updates you aren't using. Leave it enabled if you rely on Elasticsearch's automatically downloaded GeoLite2 databases. Disabling the downloader removes those downloaded databases; the 30-day expiration behavior applies when updates remain enabled but Elasticsearch cannot reach the GeoIP endpoint.

Enrichment runs on ingest nodes and costs CPU per document. On a high-volume cluster, watch ingest node load after you turn on a geo pipeline, and give dedicated ingest nodes room if a lookup runs on every event.

Get the geo_point mapping in before the first write, tag your internal IPs, and decide up front whether the free database is enough or you need licensed accuracy and security context. Those three calls are the difference between a map that just works and a week of debugging why half your events have no location.