http://169.254.169.254/ is the cloud metadata endpoint. On an EC2 instance with an IAM role attached and IMDSv1 still reachable, it hands out temporary credentials to anyone who can make the request. IMDSv2 raises the bar, but plenty of instances still allow v1. Your agent's web_fetch tool blocks that URL. Good.
Does it block http://0xA9FEA9FE/? That is the same address, written in hex. A guard that compares the host against a list of bad strings never sees it.
I am not pointing at someone else's mistake. I shipped an SSRF guard a couple of weeks ago, in a 60-line MCP web_fetch tool (the post is here). It was better than a string match: it resolved the host through the OS resolver first, so the hex form would have collapsed to 169.254.169.254 and been refused. But it had a different hole, one I named in that very post and then left open. It followed redirects and never re-checked where they landed. This is the version I should have shipped.
TL;DR
- A string denylist of bad hosts leaks. The same internal address can be written as a decimal integer (
2852039166), hex (0xA9FEA9FE), or an IPv6-mapped literal ([::ffff:169.254.169.254]), and none of those match the string169.254.169.254. - In the stdlib demo below, the string denylist allows 7 of 8 URLs (6 of them dangerous). A default-deny allowlist that normalizes every host through Python's
ipaddressallows 1 of 8, the one legit host. - A pre-fetch check that does not re-examine redirects is useless against a public host that 30x-redirects to
169.254.169.254. That was the live hole in my own guard. - Flip the model: deny by default, allow an explicit list of host names, normalize every host before you judge it, and re-check the URL you were actually redirected to.
- The script prints its own ceiling: DNS rebinding and time-of-check/time-of-use are not caught by any URL filter, including this one. Fixtures are synthetic and labelled. Run it twice, get the same bytes.
The guard I shipped, and the hole I left in it
Here is what was in that web_fetch tool, trimmed to the guard:
ip = ipaddress.ip_address(socket.gethostbyname(host))
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
raise ValueError("refusing to fetch a private/internal address")
It resolves the host, then classifies the resolved IP. That is genuinely better than matching strings, because the resolver normalizes 0xA9FEA9FE and 2852039166 back to 169.254.169.254 before ipaddress ever looks at them. The encoding tricks at the top of this post do not get past it.
Two things did get past it.
First, it is a denylist. It refuses addresses I thought to list. The day someone routes through a range I forgot, or a new reserved block ships, the default answer is still "allow." OWASP's SSRF Prevention Cheat Sheet is blunt about why that posture loses: deny-lists are bypass-prone, and the cure is to accept only a valid IP address or domain name you already trust. Default-deny, not default-allow.
Second, and this is the one that actually bites: follow_redirects=True with no re-check of the final hop. I validated the URL the agent handed me. Then the HTTP client cheerfully followed a 302 to http://169.254.169.254/ and fetched it, because nothing looked at the second URL. I wrote, in that post, "you'd want to re-check the final hop in anything serious." Then I shipped without it. That sentence aged into a bug report I filed against myself.
Why a string denylist is the weaker baseline
Most guards I read in the wild never resolve anything. They take the host substring and compare it to a blocklist. That is the baseline I want to beat in the demo, because it fails in the most visible way and it needs zero network to demonstrate.
The same address has many spellings. 169.254.169.254 is one. The 32-bit integer 2852039166 is the same four bytes. 0xA9FEA9FE is those bytes in hex. [::ffff:169.254.169.254] is the IPv4-mapped IPv6 form. A resolver treats all of them as the same destination. A string compare treats them as four different, unknown hosts and waves them through.
OWASP names this class directly. When the cheat sheet talks about validating IP-parsing libraries, it lists the bypasses to test against: "Hex, Octal, Dword, URL and Mixed encoding." That is not theory. That is the spelling list an attacker works through.
The fix is not a longer denylist. You cannot enumerate every spelling of every bad address. The fix is to normalize first, then decide, and to decide against an allowlist of names you actually meant to talk to.
The demo: 8 URLs, two guards
The script runs eight synthetic URLs through two checks. NAIVE is the string denylist. GUARD is default-deny: it parses the host, normalizes any IP literal through ipaddress, refuses any raw IP (allow names only), refuses anything not on the allowlist, and re-checks the redirect target when there is one.
Why these eight: one legit allowlisted host, the literal metadata IP, the same IP in three encodings, an RFC 1918 private host, a non-http scheme, and the killer, an allowlisted host that redirects into the internal range. Everything is a static literal. No sockets, no DNS, no clock, no randomness, so the output is identical on every run. I parse the host with a small regex instead of urllib to keep the whole thing at two stdlib imports, ipaddress and re.
One detail that cost me a few minutes: the host regex has to handle the IPv6 brackets explicitly. Without the \[[^\]]+\] branch, [::ffff:169.254.169.254] gets sliced at the first colon, and you end up blocking it for the wrong reason (mangled host) instead of the right one (it is link-local). Getting blocked for the wrong reason is how a guard passes its own tests and fails in prod.
#!/usr/bin/env python3
"""SSRF pre-fetch guard: a string denylist vs. a normalized allowlist.
Why this file exists
--------------------
In an earlier post I shipped an SSRF guard for an agent's web_fetch tool. It
resolved the host and classified the resolved IP with `ipaddress`. That part
holds up. Two things it did NOT do: it never re-checked the URL after an HTTP
redirect, and it was a denylist (refuse known-bad) instead of a default-deny
allowlist. This script isolates two failure modes WITHOUT touching the
network, so the output is byte-for-byte reproducible:
1. A *string* denylist of known-bad hosts is bypassed by encoded IP literals
(decimal dword, hex, IPv6-mapped) and by non-http schemes. A string match
is weaker than the resolve-based guard I actually shipped; it is also the
version a lot of code reaches for first, which is why it's the baseline.
2. A pre-fetch check that ignores redirects is bypassed by a public,
allowlisted host that 30x-redirects to an internal address. THAT one
applies to my old guard too.
Everything here is SYNTHETIC. The URLs are static literals, not captured
traffic. No sockets, no DNS, no clock, no randomness -> deterministic stdout.
stdlib only: {ipaddress, re}. (urllib is intentionally avoided; the host is
parsed with a small regex, so the dependency surface is exactly two modules.)
"""
import ipaddress
import re
# --- policy -----------------------------------------------------------------
ALLOWLIST = frozenset({"api.example.com", "reviews.example.com"})
ALLOWED_SCHEMES = frozenset({"http", "https"})
# What a naive guard typically denies: a handful of known-bad host STRINGS.
NAIVE_DENY = frozenset({"169.254.169.254", "127.0.0.1", "localhost", "0.0.0.0"})
# scheme://(host | [ipv6host]) -- host is everything up to /:?# , OR a
# bracketed IPv6 literal. The brackets matter: without them, [::ffff:1.2.3.4]
# gets sliced at the first colon and you block it for the wrong reason.
_URL_RE = re.compile(r"^([a-z][a-z0-9+.\-]*)://(\[[^\]]+\]|[^/:?#]*)")
def parse_host(url):
"""Return (scheme, host) lowercased, IPv6 brackets stripped. ('', '') on junk."""
m = _URL_RE.match(url.strip().lower())
if not m:
return "", ""
scheme, host = m.group(1), m.group(2)
if host.startswith("[") and host.endswith("]"):
host = host[1:-1]
return scheme, host
def as_ip(host):
"""Parse host as an IP literal in ANY common encoding, else None.
Covers dotted IPv4/IPv6, decimal dword (2852039166), and hex (0xA9FEA9FE).
This normalization is exactly what a raw string compare skips.
"""
if not host:
return None
try:
if re.fullmatch(r"0x[0-9a-f]+", host):
return ipaddress.ip_address(int(host, 16))
if host.isdigit():
return ipaddress.ip_address(int(host))
return ipaddress.ip_address(host)
except ValueError:
return None
def ip_is_internal(ip):
"""True if the address points at something an agent must never reach."""
mapped = getattr(ip, "ipv4_mapped", None)
if mapped is not None:
ip = mapped # judge ::ffff:169.254.169.254 by its IPv4 face
return (ip.is_private or ip.is_loopback or ip.is_link_local
or ip.is_reserved or ip.is_multicast or ip.is_unspecified)
def naive_allows(url):
"""String denylist: allow unless the RAW host string is a known-bad literal."""
_, host = parse_host(url)
return host not in NAIVE_DENY
def guard_check(url, redirect_to=None, _hop=0):
"""Default-deny allowlist + ipaddress normalization + redirect re-check.
Returns (allowed: bool, reason: str).
"""
scheme, host = parse_host(url)
if scheme not in ALLOWED_SCHEMES:
return False, "scheme:" + (scheme or "none")
ip = as_ip(host)
if ip is not None:
if ip_is_internal(ip):
return False, "private-ip:" + str(ip)
return False, "raw-ip-not-allowlisted" # default-deny: allow NAMES only
if host not in ALLOWLIST:
return False, "host-not-allowlisted"
if redirect_to is not None and _hop < 5:
ok, why = guard_check(redirect_to, None, _hop + 1)
if not ok:
return False, "redirect-to:" + why
return True, "allow"
# (url, redirect_to, what-it-is) -- all synthetic literals.
FIXTURES = [
("http://api.example.com/data", None, "legit allowlisted host"),
("http://169.254.169.254/latest/meta-data/", None, "literal metadata IP"),
("http://2852039166/latest/meta-data/", None, "same IP, decimal dword"),
("http://0xA9FEA9FE/", None, "same IP, hex"),
("http://[::ffff:169.254.169.254]/", None, "same IP, IPv6-mapped"),
("http://10.0.0.5/admin", None, "RFC1918 private host"),
("file:///etc/passwd", None, "non-http scheme"),
("http://reviews.example.com/page", "http://169.254.169.254/",
"allowlisted, redirects internal"),
]
def main():
print("SSRF pre-fetch guard -- string denylist vs. normalized allowlist")
print("synthetic URLs, no network, stdlib {ipaddress, re}")
print()
print("NAIVE string denylist (allow unless raw host is a known-bad literal)")
naive_allowed = 0
for url, _redirect, label in FIXTURES:
allowed = naive_allows(url)
naive_allowed += allowed
flag = "ALLOW" if allowed else "block"
reached = " <-- reaches forbidden target" if (allowed and label !=
"legit allowlisted host") else ""
print(" {:<5} {:<46} {}{}".format(flag, url, label, reached))
print(" => allowed {}/{}".format(naive_allowed, len(FIXTURES)))
print()
print("GUARD allowlist + ipaddress + redirect re-check (default-deny)")
guard_allowed = 0
reasons = {}
for url, redirect, label in FIXTURES:
allowed, reason = guard_check(url, redirect)
reasons[url] = reason
guard_allowed += allowed
flag = "ALLOW" if allowed else "block"
print(" {:<5} {:<46} {}".format(flag, url, reason))
print(" => allowed {}/{}".format(guard_allowed, len(FIXTURES)))
print()
print("What this guard does NOT catch (printed so the writeup can't overclaim)")
print(" - DNS rebinding / TOCTOU: an allowlisted NAME that resolves to an")
print(" internal IP at connect time, or flips between check and fetch.")
print(" This filter never resolves DNS (zero network), so it cannot see")
print(" that. Real fix: resolve at connect, pin the IP, re-check AFTER")
print(" the socket is open.")
print(" - An allowlisted host that itself proxies/forwards to an internal")
print(" service. Out of scope for a URL filter.")
print(" - The host parser is a teaching regex. Production needs a hardened")
print(" URL parser (userinfo@ tricks, trailing dots, IDN/punycode).")
print(" Floor proven: encoded literal IPs, non-http schemes, and a")
print(" redirect into the internal range are blocked. Not a full defense.")
print()
# --- the headline, proven by assertion ---
assert naive_allowed == 7, naive_allowed
assert guard_allowed == 1, guard_allowed
assert reasons["http://169.254.169.254/latest/meta-data/"].startswith("private-ip")
assert reasons["http://2852039166/latest/meta-data/"].startswith("private-ip")
assert reasons["http://0xA9FEA9FE/"].startswith("private-ip")
assert reasons["http://[::ffff:169.254.169.254]/"].startswith("private-ip")
assert reasons["http://10.0.0.5/admin"].startswith("private-ip")
assert reasons["file:///etc/passwd"] == "scheme:file"
assert reasons["http://reviews.example.com/page"].startswith("redirect-to:private-ip")
print("assertions passed: naive allowed 7/8, guard allowed 1/8")
if __name__ == "__main__":
main()
Run it with python3 -I ssrf_allowlist_guard.py. Same output every time:
SSRF pre-fetch guard -- string denylist vs. normalized allowlist
synthetic URLs, no network, stdlib {ipaddress, re}
NAIVE string denylist (allow unless raw host is a known-bad literal)
ALLOW http://api.example.com/data legit allowlisted host
block http://169.254.169.254/latest/meta-data/ literal metadata IP
ALLOW http://2852039166/latest/meta-data/ same IP, decimal dword <-- reaches forbidden target
ALLOW http://0xA9FEA9FE/ same IP, hex <-- reaches forbidden target
ALLOW http://[::ffff:169.254.169.254]/ same IP, IPv6-mapped <-- reaches forbidden target
ALLOW http://10.0.0.5/admin RFC1918 private host <-- reaches forbidden target
ALLOW file:///etc/passwd non-http scheme <-- reaches forbidden target
ALLOW http://reviews.example.com/page allowlisted, redirects internal <-- reaches forbidden target
=> allowed 7/8
GUARD allowlist + ipaddress + redirect re-check (default-deny)
ALLOW http://api.example.com/data allow
block http://169.254.169.254/latest/meta-data/ private-ip:169.254.169.254
block http://2852039166/latest/meta-data/ private-ip:169.254.169.254
block http://0xA9FEA9FE/ private-ip:169.254.169.254
block http://[::ffff:169.254.169.254]/ private-ip:::ffff:169.254.169.254
block http://10.0.0.5/admin private-ip:10.0.0.5
block file:///etc/passwd scheme:file
block http://reviews.example.com/page redirect-to:private-ip:169.254.169.254
=> allowed 1/8
What this guard does NOT catch (printed so the writeup can't overclaim)
- DNS rebinding / TOCTOU: an allowlisted NAME that resolves to an
internal IP at connect time, or flips between check and fetch.
This filter never resolves DNS (zero network), so it cannot see
that. Real fix: resolve at connect, pin the IP, re-check AFTER
the socket is open.
- An allowlisted host that itself proxies/forwards to an internal
service. Out of scope for a URL filter.
- The host parser is a teaching regex. Production needs a hardened
URL parser (userinfo@ tricks, trailing dots, IDN/punycode).
Floor proven: encoded literal IPs, non-http schemes, and a
redirect into the internal range are blocked. Not a full defense.
assertions passed: naive allowed 7/8, guard allowed 1/8
Read the two => allowed lines. The string denylist lets 7 of 8 URLs through and only blocks the one it has memorized verbatim. Of the seven it allows, six reach a forbidden target. The allowlist guard allows exactly one, the host I actually meant to talk to.
The three things that slip past a naive check
Encoded literal IPs. 2852039166, 0xA9FEA9FE, [::ffff:169.254.169.254] are all 169.254.169.254. The guard runs each host through as_ip, which reads decimal and hex forms and hands them to ipaddress. Then ip_is_internal checks is_private, is_loopback, is_link_local, is_reserved, plus multicast and unspecified. The IPv6-mapped case gets judged by its IPv4 face. All four spellings land on the same verdict: private-ip.
Non-http schemes. file:///etc/passwd has no network host at all, so a host-only check has nothing to grab and defaults to allow. The scheme check kills it first: anything that is not http or https is refused before the host is even parsed. That also closes gopher://, ftp://, and friends.
The redirect. reviews.example.com is on the allowlist. The pre-fetch URL is clean. The naive check says allow and walks away. The guard does not walk away: when a redirect target is present, it runs the same checks on that URL, and the 302 to 169.254.169.254 comes back redirect-to:private-ip. This is the only fixture where the danger is invisible at request time, and it is the exact gap I left in my real code.
Why allowlist beats denylist here
A denylist answers "is this one of the bad things I listed?" An allowlist answers "is this one of the good things I named?" For an agent fetching the open web on instructions it partly got from the web, the second question is the safe default, because the failure mode of an incomplete allowlist is "I refuse a site I should have allowed," and the failure mode of an incomplete denylist is "I fetched your credentials endpoint."
So the guard refuses raw IPs entirely, even public ones, and allows only host names that appear in ALLOWLIST. If your agent legitimately needs to hit a raw IP, you add that exact IP to the allowlist on purpose. Nothing reaches the network by default.
This is the OWASP position, stated plainly in their cheat sheet: prefer an allow-list of permitted destinations, because you can reason about a short list of things you trust and you cannot reason about the infinite list of things you do not.
What this does not catch, and the demo says so out loud
The script prints its own ceiling, because a security post that only shows wins is selling you something. Three named gaps:
DNS rebinding and TOCTOU. An allowlisted name like reviews.example.com can resolve to a public IP when you check it and a private IP a millisecond later when the socket actually opens. This filter never resolves DNS, on purpose, so it is structurally blind to that. The real fix lives one layer down: resolve at connect time, pin the resolved IP, and re-check it after the socket is established, not before. The resolved view is something a URL filter cannot give you.
A proxying allowlisted host. If a host you trust forwards your request to an internal service, the URL was clean and the destination was not. That is out of scope for any URL-level check.
The parser. The host regex here is for teaching. A real one has to deal with userinfo@host tricks, trailing dots, and IDN/punycode, which is why production code should lean on a hardened URL parser rather than my fifteen-line parse_host.
I will not tell you what fraction of real agent traffic carries these traps, because this demo does not measure that. It measures one thing: that the floor holds.
Network layer, not text layer
This is a different problem from the agent-security posts I have written before. When a scraped page tells your agent what to do, or a tool's own description hides an instruction, the attack is in text the model reads and reasons over. Those live in the prompt.
SSRF lives below that. It does not care what the model thinks. It is about which IP the socket connects to, decided before a single byte of response is read. You need both kinds of defense, and a guard for one does nothing for the other. A perfect prompt firewall still fetches 0xA9FEA9FE if the network layer is wide open.
Why I keep this filter before the fetch and not in the prompt: across our scraping fleet, 2,190 production runs on 32 published actors with one Trustpilot review scraper past 962 runs on its own, the agent is constantly handed lists of URLs to go visit. Some of those lists are user-supplied. The safe place to decide "do not connect there" is in code that runs before the request, not in an instruction I hope the model honors under adversarial input. To be clear, the eight URLs in the demo are synthetic fixtures I wrote to exercise the logic, not a captured incident.
What I would actually ship
Four pieces, in order of how much they matter:
- Re-check after redirects. This is the one I missed. If your client follows redirects, run the full guard on the final URL, or disable auto-redirects and walk each hop yourself. A pre-fetch check that ignores the redirect is theater.
- Default-deny allowlist of host names. Refuse raw IPs. Allow only names you put on the list on purpose.
-
Normalize before you judge. Run every host through
ipaddressso hex, decimal, and IPv6-mapped forms collapse to one verdict. - Resolve at connect, pin the IP, re-check on the socket. This is the part the demo cannot do without the network, and it is what actually closes DNS rebinding. Treat the stdlib script as the URL-level floor, and put the socket-level check above it.
If you only do one thing this week, do number one, and grep your agent's HTTP client for follow_redirects=True. I had to grep mine.
So: when your agent fetches a URL it got from the open web, where does your guard live, before the request or in the prompt, and does it survive a redirect? I would genuinely like to know how others are pinning the resolved IP without reinventing a resolver.
Follow for the next teardown from our production runs. I read every comment, and the worst SSRF near-miss you have hit is exactly the kind of thing I want to hear.
Written with AI assistance. The script was run with python3 -I; every line of output above is real terminal output, byte-identical across two runs (MD5 of stdout 94a8382cc19daf3134693340491070b2). The eight URLs are synthetic fixtures, not captured traffic. Sources: OWASP SSRF Prevention Cheat Sheet, RFC 3927 (link-local 169.254.0.0/16), RFC 1918 (private ranges), and the Python ipaddress docs.
Top comments (0)