DEV Community: Doby Baxter

You Don't Need Perfect Code - You Need a Safety Net That Talks Back

Doby Baxter — Tue, 14 Jul 2026 17:11:14 +0000

TL;DR — My last post was about catching drift: the moment my config tool's idea of "valid" silently diverges from the real Pyxel framework. This post is about the other half of the same system — what happens the instant a human hits one of those checks. A safety net has two jobs: it has to catch you, and it has to talk to you when it does. Perfect code chases the first and forgets the second. I dug into ~60 years of error-message research to work out what a net that talks back should actually say, and mapped each finding onto a concrete rule I now hold my own diagnostics to.

The myth I had to give up first

For a long time I treated "write better code" as the goal, as if enough care up front would let me arrive at software that simply didn't go wrong. That belief survives right up until you maintain a tool whose correctness depends on something you don't own. My browser-based configuration tool exists to help scientists write YAML that ESA's Pyxel detector-simulation framework will accept. Pyxel is the source of truth. My copy of that truth can drift, users will make mistakes I can't anticipate, and no amount of cleverness on my side changes either fact.

So I stopped aiming for code that never fails and started building a net that assumes it will. In the drift post I wrote about the catching layers — freshness checks, bounds tests, API introspection, per-file coverage gates. But a net that catches you silently and then hands you a stack trace is only half a net. The half nobody budgets for is the part that speaks.

Here's the thing I didn't expect: the "speaking" half turns out to be the more researched of the two, and the research is unambiguous.

What the research actually says

I went looking for evidence rather than opinion, because "be nicer to your users" is easy to say and easy to skip. The literature is older and firmer than I expected.

People read error messages — and it's expensive. The comfortable myth is that developers skip the message and jump straight to the code. Barik et al. put 56 developers in front of an eye-tracker and found the opposite: people genuinely read error messages, they allocate around a quarter of their visual attention to them, and the difficulty of reading a message is comparable to the difficulty of reading source code itself. Your error text isn't a footnote to the real work — it is real work you're imposing on someone.

Better messages measurably help. When Becker and colleagues rewrote cryptic compiler errors into enhanced ones, the group receiving them made fewer errors overall, fewer errors per person, and — the metric I care about most — fewer repeated errors, the signature of someone stuck in a loop. The effect has been debated in the literature, which is itself a useful lesson: rewording alone isn't magic. What helps is a message that names the likely cause and the next action, not just prettier phrasing.

It's fundamentally an interface problem. Traver's HCI analysis reframes the compiler as an interface between machine and human, and the error message as the primary channel of that interface. The message has to bridge the gap between the system's internal state and the user's mental model of what they were trying to do. When it fails, it usually fails because it describes the machine's world ("expected identifier") instead of the user's world ("this needs a variable name").

Readability is relative. Later work found that whether a message reads as "clear" depends heavily on the reader's experience and language familiarity — experts and novices don't even agree on what counts as unreadable. There is no universally clear message, only a message clear to a specific audience. Mine is scientists, not compiler engineers, and that changes every wording decision.

And there's a 40-year-old checklist waiting for us. Nielsen's ninth usability heuristic — help users recognize, diagnose, and recover from errors — has been the same since the 1990s: plain language, precise identification of the problem, and a constructive, actionable path out, without blaming the person. His fifth heuristic pairs with it: the best error is the one you prevented from being possible at all.

Mapping the studies to rules I actually hold myself to

Findings are only worth the searches if they change what I do on Monday. Here's the translation, each rule tied to the study behind it and to what it looks like inside my tool.

Rule 1 — Treat the message as a first-class output, budgeted like code

From: Barik (reading a message costs as much as reading code).

If the message is as expensive to read as the code, it deserves as much care as the code. In practice that means the module that turns raw validator noise into human diagnostics is not a "nice to have" I bolt on at the end — it sits behind the same per-file coverage gate as my parsing and bounds logic. The safety net's voice is part of the net, so it gets tested like the rest of it.

Rule 2 — Frontload the fix, because you're spending someone's attention

From: Barik (reading is expensive) plus Nielsen's guidance to frontload the important content so users can scan.

Every message is a withdrawal from someone's attention budget. So the fix goes first, not buried under a paragraph of context. Compare:

Error: instance.detector.optics[2] failed schema validation:
data.cutoff_wavelength must be <= 15

against:

cutoff_wavelength is 18.0 but dark_current_rule07 only accepts 1.7–15.0 µm.
Lower it to within that range, or check you meant a different model.

Same underlying failure. The second one hands back most of those minutes.

Rule 3 — Enhance with cause and action, not just nicer words

From: Becker (enhanced messages reduce repeated errors) plus the debate around it (rewording alone is weak).

A repeated error means someone is stuck in a loop the message failed to break. So a good diagnostic in my tool answers three questions in order: what's wrong, why it's wrong here, and what to do next. The raw AJV output answers only the first, in the machine's dialect. My job is to add the other two — and to resist the temptation to think prettier phrasing is the whole job.

Rule 4 — Speak the user's domain, not the parser's

From: Traver (the message must bridge system state and the user's mental model).

My users think in detectors, wavelengths, and pipeline stages, not in "tokens," "instances," or "discriminated unions." So when a model is placed under the wrong pipeline stage, the message says exactly that in Pyxel's own vocabulary rather than surfacing the JSON-schema machinery underneath. The parser's world leaks out constantly; part of the tool's job is to translate it back into the world the scientist actually inhabits.

Rule 5 — Write for your reader's level, and prove it

From: the readability research (clarity is relative to experience and familiarity).

There's no such thing as an objectively clear message, so "clear to a scientist who is not a career programmer" becomes the target. Practically, that means plain sentences, a low reading grade, and jargon only where it's their jargon (cutoff wavelength, yes; discriminator, no). This is checkable — you can run message text through a readability tool the same way you'd lint code — which turns "be clearer" from a vibe into a measurement.

Rule 6 — Recognize, diagnose, recover — and prevent

From: Nielsen's heuristics 9 and 5.

This is the spine that holds the other rules together, and it maps cleanly onto the layers I already have:

Recognize — surface the problem at the seam where it lives, visibly, so it can't be missed.
Diagnose — say precisely what and where, in the user's terms (Rules 3–5).
Recover — give an actionable next step, ideally with the valid range or the nearest correct option (Rule 2).
Prevent — the best error message is the one that never has to appear. My bounds warnings and schema freshness checks are heuristic 5 in disguise: they stop whole classes of invalid config before they're ever submitted.

Rule 7 — Never blame, and never claim certainty you don't have

From: Nielsen (constructive, non-accusatory) plus my own drift work (a check that can silently discover nothing is worse than no check).

Tone is a design decision. A message that reads as "you did this wrong" produces the person who stops trusting the tool; a message that reads as "here's what happened and how to move" produces the person who fixes it and moves on. And there's a deeper honesty rule that connects straight back to the drift post: when my tool can't verify freshness because Pyxel is unreachable, it says so plainly rather than pretending everything is current. The invariant isn't "the tool is always right." It's "the tool never lies about what it knows." An error message that overstates its own certainty is just a politely-worded version of the silent check that discovers nothing.

The synthesis: a net catches and speaks

Put the two posts side by side and the shape is clear. Perfect code is a single strand — get everything right and you never fall. It's also a fantasy, because in any system whose truth lives somewhere else, you don't control whether you fall. A safety net is the honest alternative, and it's made of two things:

The catching layer — the drift tests, the validation, the coverage gates. This decides whether you fall.
The speaking layer — the error messages. This decides whether the fall teaches you something or just hurts.

I used to pour all my effort into the first strand and call it engineering. The research quietly makes the case that the second strand is where a huge share of your users' time, frustration, and trust actually lives — and that it's been well-understood for decades while most of us kept treating it as an afterthought.

You don't need code that never fails. You need a net that catches you, and then has the decency to tell you — in your own language, without blame, with the fix in hand — exactly what just happened.

Takeaways

Perfect code is the wrong target; a safety net is the right one. When you don't own the source of truth, you can't prevent every fall — so design for the fall.
The message is not a footnote. Reading it costs as much as reading code, so budget and test it like code.
Frontload the fix. You're spending someone's attention; give the answer before the context.
Enhance with cause and next step, not just softer wording — a repeated error is a message that failed to break the loop.
Speak the user's domain and the user's reading level, and measure both instead of guessing.
Recognize, diagnose, recover, prevent — Nielsen's 40-year-old checklist still maps onto every layer of a modern config tool.
Never blame, and never claim certainty you can't back up. An over-confident message is the friendly face of a check that silently discovers nothing.

References

Barik, T. et al. Do Developers Read Compiler Error Messages? ICSE 2017.
Becker, B. A. An Effective Approach to Enhancing Compiler Error Messages. SIGCSE 2016; and An Exploration of the Effects of Enhanced Compiler Error Messages for Computer Programming Novices (2015).
Becker, B. A. et al. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research. ITiCSE-WGR 2019.
Denny, P., Becker, B. A. et al. On Designing Programming Error Messages for Novices: Readability and its Constituent Factors. CHI 2021.
Traver, V. J. On Compiler Error Messages: What They Say and What They Mean. Advances in Human-Computer Interaction, 2010.
Nielsen, J. 10 Usability Heuristics for User Interface Design (Heuristics 5 and 9); Nielsen Norman Group, Error-Message Guidelines.

Building a FastAPI Backend That Serves Live Satellite Imagery and Orbital Position

Doby Baxter — Tue, 14 Jul 2026 03:08:58 +0000

TL;DR — I built a full-stack app that shows one place — the Sheffield Cholera Monument — from two scales at once: a satellite's-eye vegetation view from space, and a centimeter-scale 3D model from the ground. This post is about the backend: a FastAPI service that authenticates against the Copernicus Sentinel Hub, pulls the newest real Sentinel-2 scene over a bounding box, computes NDVI, and then pins a satellite marker to the exact point in orbit where Sentinel-2A passed closest to the monument for that scene. The satellite-data pipeline is the heart of it, so that's where I'll spend most of the words.

What you'll get out of it

How to obtain live satellite imagery from Copernicus: OAuth token, then the Catalog, Process, and Statistics APIs
Why "the newest scene" is a real query, not a timestamp — and how to pin every view to that one scene
Turning a two-line element (TLE) into a satellite position with Skyfield + SGP4, vectorized with NumPy
Designing endpoints that never break the frontend, even when an upstream API is down

The idea: one place, two scales
Getting satellite data, step one: authentication
Step two: finding the newest real scene
Step three: NDVI imagery and statistics
Step four: from a TLE to a position in orbit
Pinning the satellite to the scene
Designing endpoints that can't break the frontend
Keeping it light: caching
Takeaways

The idea: one place, two scales

The project's thesis is simple: observation scale changes what you can understand about a place. From orbit, Sentinel-2 sees vegetation health across a whole neighborhood at ~10 m per pixel but knows nothing of texture. From the ground, a photogrammetric mesh captures sub-centimeter detail of a single monument but sees nothing of its surroundings. Put both on one interactive globe and the contrast becomes the point.

The frontend is a CesiumJS 3D globe; the ground model is a RealityScan .glb. Neither is what makes this a backend story. What makes it a backend story is everything that has to happen server-side to get live, honest, self-consistent satellite data onto that globe — where "self-consistent" means the imagery, the reported date, and the satellite's position all describe the same acquisition.

The stack for that:

Concern	Tool
API server	FastAPI + Uvicorn
Imagery & metadata	Copernicus Sentinel Hub (Catalog / Process / Statistics APIs)
Orbital mechanics	Skyfield (SGP4 propagation)
Live orbital elements	Celestrak TLE feed
Math	NumPy (vectorized)

Four endpoints: latest-scene, ndvi, ndvi-stats, and orbit-at-scene-time. Let's build the data pipeline in the order the data actually flows.

Getting satellite data, step one: authentication

Copernicus Sentinel Hub uses OAuth2 client-credentials. You register an app in the Copernicus Data Space, get a client ID and secret, and exchange them for a short-lived bearer token that every subsequent request carries. The secret never touches the frontend — it lives in backend environment variables (set in the Render dashboard in production), which is the entire reason a backend exists here rather than calling the API from the browser.

TOKEN_URL = "https://identity.dataspace.copernicus.eu/auth/realms/CDSE/protocol/openid-connect/token"

def get_token():
    res = requests.post(
        TOKEN_URL,
        data={
            "grant_type": "client_credentials",
            "client_id": CLIENT_ID,
            "client_secret": CLIENT_SECRET,
        },
        timeout=30,
    )
    res.raise_for_status()
    return res.json()["access_token"]

That's the whole handshake. Every data call below starts by getting a token and putting it in an Authorization: Bearer … header. The backend is, among other things, an auth proxy: it holds the credentials, and the browser only ever talks to the backend.

Step two: finding the newest real scene

Here's the design decision that shapes everything: the dashboard shows the newest available scene, and reports that scene's real acquisition date — not the time you happened to load the page. Satellites don't image every spot continuously; Sentinel-2 revisits a given area every few days, and cloud cover knocks out many passes. So "what's the latest usable image of this exact place?" is a genuine query.

That query goes to the Sentinel Hub Catalog API. I ask for every Sentinel-2 L2A scene intersecting my bounding box in the last 30 days, then sort client-side and pick the newest one under a cloud-cover threshold:

BBOX = [-1.4635, 53.3745, -1.4550, 53.3798]  # around the monument

def get_latest_scene_meta(days=30, max_cloud=30):
    now = datetime.now(timezone.utc)
    start = now - timedelta(days=days)
    body = {
        "collections": ["sentinel-2-l2a"],
        "bbox": BBOX,
        "datetime": f"{start:%Y-%m-%dT%H:%M:%SZ}/{now:%Y-%m-%dT%H:%M:%SZ}",
        "limit": 50,
    }
    token = get_token()
    res = requests.post(CATALOG_URL, headers={"Authorization": f"Bearer {token}"},
                        json=body, timeout=30)
    res.raise_for_status()
    feats = res.json().get("features", [])

    # newest first
    feats.sort(key=lambda f: f["properties"].get("datetime", ""), reverse=True)
    # newest scene under the cloud threshold, else newest of all
    chosen = next(
        (f for f in feats if f["properties"].get("eo:cloud_cover", 100) <= max_cloud),
        None,
    ) or (feats[0] if feats else None)

    if chosen:
        return {
            "datetime": chosen["properties"]["datetime"],
            "cloud_cover": round(float(chosen["properties"].get("eo:cloud_cover", 0)), 1),
        }
    return None

Two small robustness choices worth calling out. Sorting happens client-side rather than via a sortby in the request, because not every catalog deployment supports that parameter reliably — sorting 50 features locally is free and removes a compatibility assumption. And the or feats[0] fallback means that if every recent scene is cloudy, the dashboard still shows the newest one rather than nothing; a cloudy real scene beats an empty panel. The whole function returns None on any failure so callers can degrade gracefully — a pattern that becomes a theme.

The latest-scene endpoint wraps this and returns a clean metadata object. Crucially, that scene's datetime becomes the anchor every other view is tied to.

Step three: NDVI imagery and statistics

NDVI — the Normalized Difference Vegetation Index — is the classic "how green is it" measure: (NIR − Red) / (NIR + Red), using Sentinel-2's B08 (near-infrared) and B04 (red) bands. Healthy vegetation reflects strongly in NIR and absorbs red, so it scores high.

Sentinel Hub's Process API lets you compute this server-side at the source with an evalscript — a small JS function that runs per pixel on the imagery — and hand back a finished PNG. No raw bands cross the network; you receive exactly the visualization you asked for. I bucket the NDVI values into a simple color ramp from bare-earth brown to dense-vegetation green:

evalscript = """
//VERSION=3
function setup() {
  return { input: ["B04", "B08"], output: { bands: 3 } };
}
function evaluatePixel(sample) {
  let ndvi = (sample.B08 - sample.B04) / (sample.B08 + sample.B04);
  if (ndvi < 0.0) return [0.3, 0.2, 0.1];
  if (ndvi < 0.2) return [0.8, 0.7, 0.3];
  if (ndvi < 0.4) return [0.4, 0.8, 0.3];
  if (ndvi < 0.6) return [0.2, 0.6, 0.2];
  return [0.0, 0.4, 0.0];
}
"""

The request sends this evalscript alongside the bounding box, a 30-day time range, a 30% cloud filter, and a 512×512 output size, then streams the PNG straight back to the browser:

res = requests.post(PROCESS_URL, headers={"Authorization": f"Bearer {token}"}, json=payload)
if res.status_code != 200:
    return JSONResponse(content=res.json(), status_code=res.status_code)
return Response(content=res.content, media_type="image/png")

512×512 is a deliberate balance: fine enough to show real spatial structure at Sentinel-2's resolution, small enough to stay snappy for real-time draping over the globe.

The ndvi-stats endpoint does the numerical companion via the Statistics API, aggregating NDVI daily over a 60-day window and pulling out the mean — this time with a dataMask band so cloud/no-data pixels are excluded from the average rather than dragging it down. It returns a single {"mean": 0.342} for the telemetry readout.

Step four: from a TLE to a position in orbit

This is the part I find most fun, and the part most people haven't built. I want a marker on the globe showing where Sentinel-2A actually was when it captured the scene — sitting on its real ground track, not a decorative dot.

The raw material is a TLE (two-line element set): a compact, standardized encoding of a satellite's orbit at a moment in time (its "epoch"). Celestrak publishes fresh TLEs for active satellites. I fetch the active catalog and pull out Sentinel-2A specifically:

def fetch_sentinel2a_tle():
    req = urllib.request.Request(TLE_URL, headers={"User-Agent": "Mozilla/5.0"})
    with urllib.request.urlopen(req, timeout=20) as response:
        lines = [l.strip() for l in response.read().decode().splitlines() if l.strip()]
    for i in range(len(lines) - 2):
        if "SENTINEL-2A" in lines[i].upper() and lines[i + 1].startswith("1 "):
            return lines[i], lines[i + 1], lines[i + 2]
    # …hardcoded fallback TLE if the fetch fails…

A TLE isn't a position — it's the input to a propagator. SGP4 is the standard model that turns "here's the orbit" plus "here's a time" into an actual Earth-centered coordinate. Skyfield wraps SGP4 in a friendly API and does the coordinate conversions. Feeding it the TLE and an array of times gives sub-satellite points (the latitude/longitude directly beneath the satellite):

from skyfield.api import EarthSatellite, load, wgs84

ts = load.timescale()
name, l1, l2 = fetch_sentinel2a_tle()
sat = EarthSatellite(l1, l2, name, ts)

# a whole array of times at once
sp = wgs84.subpoint(sat.at(ts.utc(y, mo, d, h, mi, s + seconds_array)))
lats, lons = sp.latitude.degrees, sp.longitude.degrees

The thing to notice: Skyfield is vectorized. You pass a NumPy array of time offsets and get back arrays of positions in one call, rather than looping second-by-second. That matters a lot for the next step, where I evaluate the orbit at thousands of instants.

Pinning the satellite to the scene

Reporting Sentinel-2A's position at some arbitrary instant would put the marker wherever the satellite happens to be — usually nowhere near Sheffield. What I actually want is its closest approach to the monument around the scene's acquisition time. I find it with a coarse-then-fine search, both fully vectorized:

def _haversine_km(lats, lons):
    dlat = np.radians(lats - MONUMENT_LAT)
    dlon = np.radians(lons - MONUMENT_LON)
    a = (np.sin(dlat/2)**2
         + np.cos(np.radians(MONUMENT_LAT)) * np.cos(np.radians(lats)) * np.sin(dlon/2)**2)
    return 6371 * 2 * np.arctan2(np.sqrt(a), np.sqrt(1 - a))

# Coarse scan: ±1 day around the scene, one sample per 60 s
sec = np.arange(0, 2 * 86400, 60)
sp = wgs84.subpoint(sat.at(ts.utc(y, mo, d, h, mi, s + sec)))
dist = _haversine_km(sp.latitude.degrees, sp.longitude.degrees)
coarse_s = int(sec[int(np.argmin(dist))])

# Refine: ±60 s around that minimum, one sample per second
rsec = np.arange(coarse_s - 60, coarse_s + 61, 1)
rsp = wgs84.subpoint(sat.at(ts.utc(y, mo, d, h, mi, s + rsec)))
rdist = _haversine_km(rsp.latitude.degrees, rsp.longitude.degrees)
closest_s = int(rsec[int(np.argmin(rdist))])

The haversine gives great-circle distance from each sub-satellite point to the monument. The coarse pass (2,880 samples across two days) narrows to the right minute; the refinement (121 one-second samples) pinpoints the exact second of closest approach. Because the newest scene was itself captured as Sentinel-2A flew over Sheffield, that closest approach correctly lands right over the monument — the imagery and the orbit agree because they describe the same overpass.

Finally I build a ±45-minute ground track (181 points at 30-second spacing) centered on that closest second, so the frontend can draw the arc of the orbit with the marker glued to the middle of it:

tsec = closest_s + np.arange(-2700, 2701, 30)   # ±45 min, 30 s steps
tsp = wgs84.subpoint(sat.at(ts.utc(y, mo, d, h, mi, s + tsec)))
track_points = [
    {"lon": float(lo), "lat": float(la), "altitude_m": SAT_ALTITUDE_M}
    for la, lo in zip(tsp.latitude.degrees, tsp.longitude.degrees)
]

One honesty note I put in the README too: this design deliberately only tracks the newest scene, whose date is always recent. That keeps the current Celestrak TLE's epoch close to the scene time, so SGP4 stays accurate without hunting down epoch-matched historical TLEs. Historical-scene accuracy is intentionally out of scope — a scoped limitation is better than a silent inaccuracy.

Designing endpoints that can't break the frontend

The globe refreshes every five minutes and calls all four endpoints. Any of them depends on a third party — Copernicus, Celestrak — that can rate-limit, time out, or briefly vanish. A 500 error here isn't just an empty panel; because of how CORS works, a raised exception can arrive at the browser without the CORS headers, turning a transient upstream hiccup into a confusing cross-origin error in the console.

So the orbit endpoint is built to always return valid JSON, falling back to a safe position over the monument rather than raising:

@app.get("/api/orbit-at-scene-time")
def orbit():
    try:
        meta = get_latest_scene_meta()
        ref_dt = parse_iso(meta["datetime"]) if meta else datetime.now(timezone.utc)
        key = ref_dt.isoformat()
        if _orbit_cache["key"] != key:
            _orbit_cache["result"] = compute_orbit(ref_dt)
            _orbit_cache["key"] = key
        return _orbit_cache["result"]
    except Exception as e:
        print(f"orbit endpoint failed, returning fallback: {e}")
        return _fallback_orbit()

_fallback_orbit() returns a well-formed response with the marker parked over Sheffield and a short straight track. The globe still renders; the user sees the site; the CORS headers are present because FastAPI returned normally. The same philosophy runs through the stack: latest-scene falls back to a null cloud cover, get_latest_scene_meta returns None instead of throwing, and the TLE fetch has a hardcoded backup. Every external dependency has a graceful degradation path, because in a live dashboard "slightly stale but rendered" always beats "broken."

Keeping it light: caching

Two small caches keep the Render instance from re-doing expensive work on every poll. Catalog metadata is cached for 30 minutes — new Sentinel-2 scenes don't appear more often than that, so hammering the Catalog API on every request buys nothing. And the orbit is recomputed only when the newest scene actually changes, keyed on the scene datetime:

if _orbit_cache["key"] != key:
    _orbit_cache["result"] = compute_orbit(ref_dt)
    _orbit_cache["key"] = key

Since the scene changes every few days but the frontend polls every five minutes, this turns thousands of SGP4 computations per day into a handful. The coarse+fine orbit search is cheap in isolation, but not free at 512×512-plus-thousands-of-samples on every poll from every visitor; keying the cache to the scene rather than a clock interval means the work happens exactly when the underlying data does, and never otherwise.

Takeaways

A backend earns its place as a credential boundary. The moment you need an API secret, the browser can't be trusted with it — the backend holds the OAuth handshake and proxies the data. That's the honest reason this isn't a pure frontend app.
"Latest" is a query, not a timestamp. For real observational data, the newest usable record — under a cloud threshold, within a lookback window — is something you look up and then anchor everything else to, so your views stay self-consistent.
Push computation to the data. Sentinel Hub evalscripts compute NDVI at the source and return a finished PNG; you move a picture, not gigabytes of raw bands.
Vectorize the physics. Skyfield + NumPy evaluate an orbit at thousands of times in a couple of calls. A coarse-then-fine search over those arrays finds an exact closest approach without a Python loop in sight.
In a live dashboard, always return valid JSON. Every third-party call gets a graceful fallback, partly for UX and partly because a raised exception can strip your CORS headers and turn a hiccup into a cross-origin error.
Cache to the data's own rhythm. Key expensive work to the thing that actually changes (the scene) rather than a fixed interval, and recompute exactly when — and only when — reality does.

The satellite-data pipeline is the part I'd point anyone to first: authenticate, find the real newest scene, compute vegetation at the source, then turn a live TLE into an exact position in orbit — each step tied to the same acquisition so the whole globe tells one true, consistent story about one small patch of Sheffield.

Testing Against State Drift: Guarding a Config Tool Whose Source of Truth Lives Somewhere Else

Doby Baxter — Tue, 14 Jul 2026 02:54:49 +0000

TL;DR — I maintain a browser-based configuration tool for ESA's Pyxel detector-simulation framework. Its whole job is to help people write configs that a separate, independently-versioned project will accept. That makes drift — my tool's idea of "valid" quietly falling out of sync with Pyxel's — the central failure mode. This post is about the testing and automation I built to catch drift at three different layers, and the one principle that ties them together: test each invariant at the seam where it actually lives.

What you'll get out of it

A concrete way to think about state drift when your correctness depends on an external source of truth
Three drift-defense layers — bundled-schema freshness, real-schema bounds tests, and API-introspection checks — with the actual code
Why I split the suite into automated deterministic logic vs manually-verified rendered output, and how to draw that line
How per-file coverage gates in CI turn "we should keep this tested" into "the pipeline won't let us not"

The problem: correctness you don't own
Three kinds of drift
Layer 1 — Is the bundled schema still fresh?
Layer 2 — Do the real schema's bounds still hold?
Layer 3 — Do the docs still match the actual API?
Testing at the seam
Making the gates non-optional
What's deliberately not automated
Takeaways

The problem: correctness you don't own

Most testing advice quietly assumes you own the definition of "correct." Your code, your tests, your rules. When a test fails, something in your repo changed.

Config tooling breaks that assumption. My tool exists to help scientists produce YAML that Pyxel — a large, actively-developed simulation framework on its own release cadence — will accept and run. Pyxel is the source of truth for what a valid config is. My tool is a helpful intermediary that holds a copy of that truth: a bundled JSON schema, a set of tutorial examples, a mapping of detector types to parameter bounds.

The instant Pyxel ships a new model, renames an argument, or tightens a numeric range, my copy is wrong — and nothing in my repo changed to tell me. The tests still pass. The build still goes green. The tool now confidently blesses configs that Pyxel will reject, or flags valid ones as broken. That's state drift: two systems that are supposed to agree, silently diverging, with no failing assertion at the moment of divergence.

You can't unit-test your way out of that with fixtures alone, because a fixture is also a frozen copy of the truth. If Pyxel drifts, your fixture drifts with it and your green test is lying to you. Guarding against drift means deliberately reaching for the real, external thing at test time — carefully, at the right layer.

Three kinds of drift

Once I framed the tool this way, the drift surface split cleanly into three, each needing a different defense:

Layer	The copy that can drift	The truth it must match	How I detect it
Schema freshness	bundled `pyxel_schema.json`	Pyxel's published schema	live diff of definition names
Argument bounds	my bounds-extraction logic	real values inside the bundled schema	tests run against the real schema file
Tutorial / doc accuracy	70+ tutorial `.md` examples	the installed Pyxel Python API	introspect Pyxel, check every example

The rest of the post is these three, in order.

Layer 1 — Is the bundled schema still fresh?

The tool ships a bundled copy of Pyxel's schema so validation works offline and instantly. The risk is obvious: the day Pyxel adds a model, the bundle is stale and can't validate configs that use it.

Rather than pretend that never happens, the tool checks at runtime and tells the user the truth. It fetches Pyxel's published schema and diffs the set of defined model names against the bundle:

export async function checkSchemaFreshness(bundledSchemaText) {
  try {
    const res = await fetch(PYXEL_SCHEMA_URL, { cache: "no-store" });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const upstream = JSON.parse(await res.text());
    const bundled = JSON.parse(bundledSchemaText);

    const bundledNames = definitionNames(bundled);
    const upstreamNames = definitionNames(upstream);
    const missing = [...upstreamNames].filter((n) => !bundledNames.has(n));

    if (missing.length === 0) return { state: "current" };
    return { state: "stale", missing, messages: [ /* human-readable summary */ ] };
  } catch (err) {
    return { state: "unknown", messages: [ /* "couldn't reach Pyxel…" */ ] };
  }
}

Three states, not two. current and stale are the happy and sad paths — but unknown is the one that matters for trust. If the user is offline, or Pyxel's host is down, or the response is HTML instead of JSON, the tool must not claim freshness it can't verify. It says so plainly: validation reflects the bundled schema only.

That third branch is exactly the kind of thing that rots untested, so it's pinned down hard. The freshness suite mocks fetch and drives every path — superset-is-still-current, singular-vs-plural messaging, five-name truncation with an ellipsis, non-OK HTTP, a rejected promise (offline/CORS), unparseable HTML, and an assertion that the request actually sets cache: "no-store" so a cached response can't produce a false "current":

it("reports 'unknown' when upstream returns unparseable JSON", async () => {
  vi.stubGlobal("fetch", mockFetchResolving("<!doctype html><html>"));
  const result = await checkSchemaFreshness(BUNDLED);
  expect(result.state).toBe("unknown");
});

The invariant being protected isn't "the schema is fresh" — I can't guarantee that. It's "the tool never lies about whether the schema is fresh." That's a property I fully own, so it gets full coverage.

Layer 2 — Do the real schema's bounds still hold?

The tool reads numeric argument bounds (min / max, inclusive / exclusive) out of the schema so it can warn when a value is out of range. That extraction logic has ordinary unit tests against a small synthetic schema — the fast, focused kind you'd expect.

But synthetic fixtures have the drift problem baked in: they prove my parser works, not that it still works against Pyxel's actual data. So a second, smaller set of tests runs the same extractor against the real bundled pyxel_schema.json:

describe("extractArgBounds — against the real bundled pyxel_schema.json", () => {
  const realSchema = JSON.parse(readFileSync(schemaPath, "utf8"));

  it("reads cutoff_wavelength bounds for dark_current_rule07", () => {
    expect(extractArgBounds(realSchema, "dark_current_rule07")).toMatchObject({
      cutoff_wavelength: { min: 1.7, max: 15.0 },
    });
  });

  it("reads an exclusive-minimum arg for usaf_illumination", () => {
    expect(extractArgBounds(realSchema, "usaf_illumination")).toMatchObject({
      multiplier: { min: 0, exclusiveMin: true },
    });
  });
});

These assert on specific values from the shipped schema. If a future schema refresh changes cutoff_wavelength's range, or flips an inclusive bound to exclusive, this test fails loudly at refresh time — which is precisely when I want to find out, not weeks later from a confused user. It turns a silent data change into a red pipeline. The synthetic tests prove the logic; these pin the logic to reality.

Layer 3 — Do the docs still match the actual API?

This is the layer I'm most attached to, because docs are where drift hides best. The tool ships 70+ tutorial files, each with YAML examples showing how to configure a Pyxel model. Prose examples don't get executed, so they rot invisibly: an argument gets renamed upstream, and the tutorial keeps cheerfully showing the old name forever.

So the CI pipeline installs a pinned real version of Pyxel and introspects it — no fixtures, the actual package — to discover the ground truth, then checks every tutorial against it:

def discover_pyxel_models() -> dict[str, dict[str, set[str]]]:
    """Return {group: {model_name: {valid_argument_names}}} from installed Pyxel."""
    catalog = {}
    for group in PIPELINE_GROUPS:
        module = importlib.import_module(f"pyxel.models.{group}")
        for name in dir(module):
            obj = getattr(module, name)
            if not inspect.isfunction(obj):
                continue
            sig = inspect.signature(obj)
            args = { p for p, param in sig.parameters.items()
                     if p != "detector" and param.kind not in (VAR_POSITIONAL, VAR_KEYWORD) }
            catalog.setdefault(group, {})[name] = args
    return catalog

Every YAML block in every tutorial is then parsed and checked against that live catalog. A tutorial fails if it references a model that doesn't exist, puts a real model under the wrong pipeline stage, or passes an argument the actual function signature doesn't accept:

for key in args:
    if key not in valid:
        problems.append(
            f"func '{func}': invalid argument '{key}' "
            f"(valid: {sorted(valid) or 'none'})"
        )

There are two details here I'd happily defend in review. First, there's a guard test asserting introspection discovered more than 40 models — so if an import path changes and discover_pyxel_models() silently returns almost nothing, the suite fails on that rather than falsely reporting every tutorial as clean. A drift check that can quietly discover nothing is worse than no check, because it's green and wrong.

Second, because these tutorials are partly authored with LLM help, the checker also greps for stray assistant chatter — "Certainly!", "Would you like me to…", "Here are the N Mermaid diagrams" — that has no business in a published tutorial. That's a different kind of drift (content contamination rather than API mismatch), but it lives at the same seam, so it's caught in the same pass.

The Pyxel version is pinned in CI (PYXEL_VERSION: "2.17.1"). Bumping it is a deliberate, visible act — and when I bump it, this job tells me immediately which tutorials the new version just invalidated. Drift becomes a code review, not a surprise.

Testing at the seam

Across all three layers, the same principle keeps showing up: test each invariant at the layer where it actually lives, and mock the layer below it.

The freshness checker, the schema loader, and the live telemetry sender all touch the network. None of them hit the network in tests. fetch (and the YAML parser) are replaced with mocks, so the suite exercises the logic around the request — cache hit/miss, HTML-instead-of-YAML detection, missing-models guards, offline fallbacks, the OTLP body shape, the span rate limiter — deterministically, with zero flakiness:

function mockFetchResolving(bodyText, { ok = true, status = 200 } = {}) {
  return vi.fn().mockResolvedValue({ ok, status, text: async () => bodyText });
}

This is the same move as mocking the schema loader when testing pure validation logic, and it's why the JS suite runs in a plain Node environment with no browser and no live calls. The network is a seam — a clean boundary I can substitute at — and testing right at that seam is what makes otherwise-flaky, environment-dependent branches into boring, reliable regression guards that run on every commit.

The pattern generalizes: find the boundary between the logic you own and the world you don't, and put your test double exactly there. Above it, assert everything. Below it, don't pretend to.

Making the gates non-optional

A regression guard nobody enforces is a suggestion. The parts of this codebase that guard against drift — the diagnostics that turn noisy validator errors into friendly messages, the bounds extractor, the shape analysis — are the parts most worth protecting from silent decay, so CI enforces per-file coverage thresholds, not a single blurry project-wide number:

thresholds: {
  "scripts/configShape.js":        { statements: 95, branches: 80, functions: 95, lines: 95 },
  "scripts/modelDiscriminator.js": { statements: 95, branches: 85, functions: 95, lines: 95 },
  "scripts/schemaBounds.js":       { statements: 95, branches: 90, functions: 90, lines: 95 },
  // …
}

Per-file gates matter because a project-wide average lets a well-tested module subsidize a rotting one. Pinning each critical file just under its measured coverage means the pipeline fails the moment someone adds an untested branch to that specific file — the drift-sensitive logic can't quietly lose its safety net behind a healthy-looking global percentage.

One threshold is deliberately low: the module that wires a real AJV instance to the live schema sits at 35%, because most of it is genuinely exercised only in the browser. Rather than fake coverage with brittle DOM tests, the gate is set honestly to what the pure logic covers, and the rest is documented as manual. Which brings me to the last piece.

What's deliberately not automated

Not everything belongs in CI, and pretending it does produces flaky suites that teams learn to ignore. This project draws an explicit line — a method-to-layer split:

Automated: the deterministic logic — parsing, shape derivation, bounds extraction, error de-duplication, freshness diffing, the network seam. Pure functions with clear inputs and outputs. These run on every commit and gate the merge.
Manually verified: the modules that render to or read from the live DOM — schema renderers, validation overlays, the dynamic model loader, the UI components. These are checked by hand in the browser with devtools, where a human eye catches visual and perceptual problems a headless assertion would miss.

The important part is that this line is written down, in the test README, with the reasoning and the next step named: if the rendered layer ever warrants automation, a headless-browser harness like Playwright (whose auto-waiting removes most timing flakiness) is the natural tool. Until then, "manual" is a documented decision, not an accidental gap. A reader — or future me — can see exactly what's covered, what isn't, and why.

That honesty is itself a drift defense. The most dangerous test suite is the one that looks more complete than it is.

Takeaways

When you don't own the source of truth, drift is the primary bug. Frame the whole test strategy around "where can my copy silently disagree with the real thing?" before writing a single assertion.
Fixtures freeze the truth; sometimes you must test against the real thing. Run a subset of tests against the real schema and a pinned real dependency, so an upstream change turns into a red pipeline instead of a confused user.
A check that can silently discover nothing is worse than no check. Guard your introspection with a sanity assertion (">40 models found"), or a broken import will report everything as clean.
Test at the seam. Mock the network (or DOM, or external API) right at its boundary; assert fully on the logic above it. Deterministic branches become reliable guards instead of flaky ones.
Enforce per-file, not project-wide, coverage. An average lets healthy modules hide rotting ones. Pin the drift-sensitive files individually.
Write down what you don't automate. A documented manual-test surface is a decision; an undocumented one is a gap. The suite that overstates its own completeness is the one that eventually bites you.

Drift is quiet by nature — it's the bug that arrives with no failing test and no commit to blame. The only defense is to build the checks that go looking for it on purpose, put each one where it can actually see, and let CI insist on them every single time.

I Instrumented a System the OTel GenAI Conventions Weren't Built For - Here's Where They Broke

Doby Baxter — Tue, 14 Jul 2026 01:55:28 +0000

TL;DR — I instrumented a deterministic, content-blind AI workflow enforcer with the emerging OpenTelemetry GenAI semantic conventions. It technically worked, but pinched in six specific places — each one exposing the same hidden assumption: that a GenAI system is, by definition, a model invoker. This post catalogs the six frictions, shows the design I shipped instead, and follows one friction into a live semantic-conventions discussion.

What you'll get out of it

A concrete map of where the GenAI conventions assume a model sits at the center of every span
A reusable pattern for instrumenting orchestration / governance / policy layers that don't call models
Why "refused" must not be encoded as "errored," and how to fix that in one line
How a friction log — not a formal proposal — can still move a standard

The system under instrumentation
The six frictions (at a glance)
The frictions in detail
What I shipped instead: own the namespace
The pattern underneath
Where one friction ended up
Takeaways

The system under instrumentation

First, a disambiguation, because "LLM router" is an overloaded term.

Most projects with that name are model-selection routers — they pick between a cheap and an expensive model based on query complexity or cost. This system is not that. It's a topology-enforcement layer: given structured metadata about an interaction, it answers exactly one question —

Is this workflow transition permitted?

— independent of model choice or content. It validates workflow topology at load time, enforces invocation limits, and returns one of three terminal decisions: PROCEED, REFUSE, or PAUSE.

The defining property, for instrumentation purposes: it is content-blind by design. It never sees a prompt, never calls a provider, never generates a token. It observes metadata and structure — nothing else.

The experiment wrapped two operations in OTel spans:

Operation	What it does
`validate_workflow_config()`	Static structural validation, before anything runs
`WorkflowEngine.evaluate()`	The per-transition permission decision

The rule I set myself: use the existing GenAI conventions everywhere they fit, and write down every place they don't. That log of mismatches is the heart of this post.

The six frictions (at a glance)

#	Friction	The hidden assumption it exposes
1	No operation name for validation	Observability begins only when inference begins
2	`provider.name` assumes a provider	Every GenAI span calls a model
3	No vocabulary for topology	The unit of interest is a conversation, not a graph position
4	A refusal reads as an error	The only non-success outcome is failure
5	Content observability is the default	Instrumentation means watching prompts and completions
6	No trace-parenting for standalone runs	Workflow spans are children of a model span

Read together, they all say one thing: the conventions model a GenAI system as a model invoker. The detail sections below walk each one; skip to What I shipped instead if you just want the design.

The frictions in detail

Friction 1 — There's no operation name for validation

What broke. The current gen_ai.operation.name values all describe runtime inference: chat, generate_content, invoke_agent, retrieval, execute_tool. None describe static structural validation, so the only accurate value was a non-standard one like validate_workflow_config.

Why it matters. The conventions treat GenAI observability as beginning when model activity begins. But for deterministic orchestration systems, much of the real safety work happens before execution starts. The spec has no vocabulary for that phase at all.

Friction 2 — `gen_ai.provider.name` assumes a provider exists

What broke. The system has no provider. It doesn't call OpenAI, Anthropic, Gemini, or a local inference server, so the attribute is simply omitted — leaving the spans technically incomplete relative to the model.

Why it matters. That's fine when every GenAI span lives inside a model invocation. But it presumes the category of system in scope: model invokers. Orchestration, governance, and policy-enforcement layers that sit adjacent to models have no home here — LangGraph's graph-execution layer, Temporal-style workflow runtimes wrapping LLM steps, or guardrail frameworks like NeMo Guardrails that gate execution without generating anything.

Friction 3 — Workflow topology has no vocabulary

What broke. A meaningful evaluation span needs three facts: the current container, the requested transition target, and the terminal decision. No gen_ai.* attribute represents any of them. The nearest neighbor, gen_ai.conversation.id, describes conversational identity — not structural position in a graph.

Why it matters. Using conversation semantics for topology would produce actively misleading telemetry. This is the clearest point where the model shows its conversational bias: the spec can describe who is talking, but not where in the graph you are.

Friction 4 — A refusal is not an error

What broke. The system emits typed structural refusal reasons — UNKNOWN_CONTAINER, INVALID_TRANSITION, MAX_INVOCATION_EXCEEDED, CONTAINER_REENTRY_BLOCKED. The obvious attribute, error.type, describes failures of execution: timeouts, API errors, exceptions.

Why it matters. A refusal is the opposite of a failure — the system is working correctly when it refuses an invalid transition. Recording enforcement outcomes as errors poisons every downstream error-rate dashboard. "The system failed" and "the system said no, on purpose" are fundamentally different, and the conventions can't tell them apart.

Friction 5 — Content observability is treated as the default

What broke. The conventions center on input/output messages, prompts, completions, tool calls, and retrieval context. This system observes none of those, by design. Its stance is structural, not semantic: it traces topology, transition legality, and enforcement outcomes while staying blind to user content.

Why it matters. That stance is a privacy and an architecture decision. But the spec implicitly treats content observability as the natural shape of GenAI instrumentation, with no notion of a system whose entire telemetry story is structural.

Friction 6 — Standalone evaluations have no trace-parenting model

What broke. Every evaluation produced an isolated trace (parent_id: null), because there was no upstream model-invocation span to inherit context from — and the conventions imply workflow spans exist as children of model or agent spans.

Why it matters. The result is observability fragmentation: config validation and the evaluations that depend on it become disconnected islands instead of one structural lifecycle. There's no guidance for relating these spans when no model runtime sits above them.

What I shipped instead: own the namespace

The conclusion wasn't "force the fit." It was: don't borrow gen_ai.* at all. The shipped instrumentation defines its own documented, versioned convention surface under a wfrouter.* namespace (schema version 1.0.0), on four design rules.

The four design rules

1. Zero hard dependency. The engine never imports the instrumentation; the instrumentation imports the engine. If opentelemetry-api isn't installed, every function degrades to a no-op and behavior is unchanged.

2. The engine stays pure. Nothing is instrumented inside evaluate(). The wrapper traces it from the outside, so the engine stays deterministic, stateless, and trivially unit-testable with no telemetry in the way.

3. Own the namespace, own the versioning. Every attribute lives under wfrouter.*. When the still-evolving gen_ai.* conventions change, nothing moves underneath this schema. If interop ever justifies it, gen_ai.* attributes can be emitted alongside, additively.

4. A refusal sets span status OK. All three terminal states — PROCEED, REFUSE, PAUSE — are valid, successful outcomes. Status ERROR is reserved for genuine failures: a config that fails validation, or an unexpected exception. That's Friction 4, resolved in code.

The convention surface

Three low-cardinality span names — the container name is always an attribute, never part of a span name:

wfrouter.evaluate
wfrouter.validate_config
wfrouter.analyze_topology

Attributes on the wfrouter.evaluate span:

Group	Attributes
Input	`wfrouter.container`, `wfrouter.requested_action`, `wfrouter.previous_state`, `wfrouter.invocation_depth`
Decision	`wfrouter.decision`, `wfrouter.refused`, `wfrouter.reason_code`
Correlation	`wfrouter.correlation_id` — app-level, kept distinct from the OTel trace ID so you can join on either

Two deliberate cardinality decisions worth stealing:

The raw transition history is not emitted — it's unbounded. Only wfrouter.transition_history.length is recorded.
Topology issues become span events (wfrouter.topology_issue), each carrying a severity and a stable, greppable code — not one attribute per issue.

And two metric instruments, both with bounded attribute sets so aggregation stays safe:

Instrument	Type	Attributes
`wfrouter.evaluations`	counter	`decision`, `reason_code`, `container`
`wfrouter.invocation_depth`	histogram	`container`

That first counter gives you things like "REFUSE rate by container" for free.

Using it

pip install "llm-workflow-router[otel]"

from router.engine import WorkflowEngine
from router.observability.otel import ObservableEngine

engine = ObservableEngine(WorkflowEngine(cfg))
result = engine.evaluate(metadata)

ObservableEngine is a drop-in for WorkflowEngine at any call site that wants telemetry. The pure engine underneath never changes.

The pattern underneath

Individually, each friction is small. Together they point at one thing:

The current GenAI conventions model AI systems primarily as systems of generation — prompts in, completions out, tools called, context retrieved.

That's a valid model for a large slice of the ecosystem. But a whole class of systems already exists that participates in AI execution pipelines without generating anything: orchestration engines, workflow runtimes, safety gates, validation layers, execution governors. They need observability too, and their semantics are structural, not conversational. Today, each of them has to do what I did — step outside the spec and own a namespace.

A friction log is valuable precisely because it makes a spec's hidden assumptions visible. This one suggests the conventions may eventually need to separate three things now blurred together:

Inference semantics
Structural orchestration semantics
Governance / enforcement semantics

Where one friction ended up

Here's the part I didn't expect.

I left the friction log as a comment on an open semantic-conventions discussion about agentic authorization — a proposal introducing producer-emitted decision signals like gen_ai.agent.trust_score and gen_ai.agent.drift_score.

A practitioner in that thread, building the producer side in production, surfaced a failure mode you only see once a system is running: a producer's scoring function isn't fixed. Models get re-fit, thresholds move, feature sets change. The emitted score then shifts for a reason that has nothing to do with the agent's behavior — and a consumer correlating scores against outcomes can't tell whether the agent changed or the measurement changed.

That clicked against my frictions, and I framed it the only way I could:

A signal is only worth anything if you can tell when the ruler itself changed length.

The shape the discussion converged on is a minimal provenance pattern — an opaque, producer-scoped method token next to each score:

gen_ai.agent.trust_score.method
gen_ai.agent.drift_score.method
gen_ai.agent.scan_verdict.method

Consumers never parse the token; they compare it for equality only. Same token → scores stay comparable. Changed token → the measurement function was replaced, so reset your baseline instead of chasing a phantom drift event. One disciplined pattern instead of three special cases — and it generalizes to any derived, evolving signal emitted for downstream correlation.

An observation from an odd little friction log, sharpened by someone hitting the same wall from the production side, is now part of a live convention discussion. That's standards work at its best: a practitioner names a blind spot, the principle generalizes, and the fix is smaller than anyone expected.

Takeaways

Instrument the weird system you have. Edge cases are exactly what an in-development spec needs. A mismatch is information — write it down.
Refusals are not errors — encode that in span status. Keep enforcement outcomes out of your error semantics from day one. REFUSE with status OK is one line of code, and it saves every error-rate dashboard downstream.
When a spec doesn't fit, own a namespace instead of forcing the fit. A small, documented, versioned surface is more honest and more stable than misusing someone else's vocabulary — and it leaves room to emit standard attributes alongside later.
Guard your cardinality on purpose. No container names in span names, history length not the raw list, bounded metric attribute sets. Decisions, not defaults.
Friction logs are a real contribution path. You don't need to author a proposal to move a standard. Sometimes you just need to describe, precisely, where the shoe pinched.

The GenAI semantic conventions are still marked experimental — which makes this the cheapest moment to give this kind of feedback. If you're instrumenting anything AI-adjacent that isn't a model invoker, go log your frictions. The spec gets better every time someone maps its edges.

GitLab vs GitHub in 2026: Integrated Platform vs. Composable Ecosystem

Doby Baxter — Mon, 13 Jul 2026 20:33:28 +0000

Pick GitLab or GitHub in 2026 and you're not really choosing between two Git hosts — you're choosing between two philosophies about what a development platform should be. The old comparison ("GitLab has CI/CD built in, GitHub needs external tools") is out of date. GitHub has native CI/CD, native security scanning, and a native container registry now. The real difference is architectural: GitLab bundles the whole DevSecOps lifecycle into one integrated application, while GitHub gives you a strong core and an enormous ecosystem to compose the rest yourself.

This guide walks through where that split actually matters — repository workflow, CI/CD, security, project management, and hosting — and tries to be fair about the trade-offs, because both platforms are genuinely good and the "right" answer depends entirely on your team.

The core distinction

One idea explains most of the differences below, so it's worth stating up front.

GitLab is integration. It's a single application where every stage — plan, code, build, test, secure, deploy, monitor — shares one data model and one interface. A security finding in a pipeline links to the merge request that introduced it, which links to the issue that requested the feature. Nothing is stitched together because it was never separate.

GitHub is composition. It gives you best-in-class code hosting, pull requests, and the largest developer community anywhere, then lets you assemble the rest of your toolchain from a marketplace of 15,000+ Actions and integrations. You wire in the CI/CD, security, and project management pieces you want.

Neither is wrong. Integration reduces context-switching and simplifies governance; composition maximizes flexibility and lets you swap any piece. Almost every concrete difference downstream is a consequence of this one choice.

Repository management

This is the closest to a tie, because both platforms are built on Git and both do the fundamentals extremely well: branching, protected branches, commit history, tags, and a capable web UI.

The one naming difference worth knowing: GitLab calls them merge requests (MRs) and GitHub calls them pull requests (PRs). They're the same concept — a proposed change with review, discussion, and approval before it lands on a protected branch. GitLab's MRs are wired more tightly into its wider DevOps data model (an MR can show pipeline status, security scan results, and linked issues in one view); GitHub's PRs sit at the center of a larger collaboration ecosystem and benefit from that community gravity.

For day-to-day source control, you will be happy on either. The differences show up once you move past the repository itself.

CI/CD

This is where the platforms diverge in feel, even though both are now fully native.

GitLab CI/CD is defined in a .gitlab-ci.yml file and executed by GitLab Runners. It's been native since the beginning, so it's deeply woven into the rest of the platform — pipeline results, environments, and deployments all live in the same interface as your code and issues. Teams with complex delivery workflows often find the unified model reduces configuration drift.

# .gitlab-ci.yml — a simple three-stage pipeline
stages:
  - build
  - test
  - deploy

build-job:
  stage: build
  script:
    - echo "Compiling the project..."
    - ./build.sh

test-job:
  stage: test
  script:
    - echo "Running tests..."
    - ./run-tests.sh

deploy-job:
  stage: deploy
  script:
    - echo "Deploying to production..."
    - ./deploy.sh
  environment: production
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'

GitHub Actions uses YAML workflow files triggered by repository events, executed on GitHub-hosted or self-hosted runners. Its superpower is the marketplace — thousands of prebuilt, reusable actions mean you rarely write automation from scratch. For simple-to-moderate workflows in 2026, Actions has fully closed the historical gap with GitLab; for very complex pipelines, GitLab's directed-acyclic-graph execution model still has an edge.

One 2026 caveat worth flagging for teams that self-host their build infrastructure: GitHub planned to start charging for self-hosted runner minutes in early 2026, and after community pushback the plan was postponed but not cancelled. If your build strategy leans on self-hosted Actions runners, that pricing uncertainty is worth planning around.

The honest summary: GitLab for deeply integrated, complex CI/CD with predictable self-hosted economics; GitHub Actions for speed to first pipeline and an unmatched library of reusable automation.

Security scanning

This is the claim most in need of updating from a few years ago. It is no longer true that GitHub needs external tools for security — both platforms ship meaningful scanning natively.

GitHub includes Dependabot (dependency alerts and automated fix PRs) and secret scanning, with much of it free for public repositories. Its deeper static analysis (CodeQL) and advanced enterprise security features come through a separate, paid security offering.

GitLab bundles a broader set of scan types — SAST, DAST, dependency scanning, container scanning, and more — into its paid tiers without per-scanner licensing, and surfaces all results inside the same interface as your code and merge requests. For teams in regulated industries that have to prove governance, that single unified audit trail — one dashboard tying vulnerabilities to the exact change that introduced them — is often the deciding factor.

So the real distinction isn't "has security vs. doesn't." It's bundled-and-unified (GitLab) vs. strong-core-plus-add-ons (GitHub). Which matters more depends on whether your pain is procurement-and-compliance or flexibility.

Project management

GitLab ships fairly mature agile tooling natively — issues, boards, milestones, epics, and value-stream dashboards — as part of the same application. For teams that want planning to live next to code without a separate tool, it works well as a standalone-ish project manager.

GitHub offers Issues, Discussions, and Projects, which have improved substantially and are genuinely capable for many teams. But organizations running complex agile processes (deep epic hierarchies, roadmaps) still frequently integrate a dedicated tool like Jira. That's consistent with the composition philosophy: GitHub gives you a solid base and expects you to extend it if your process is heavy.

Pages and hosting

Both platforms offer free static site hosting from a repository — GitLab Pages and GitHub Pages — and both are excellent for documentation, portfolios, and project sites. GitLab Pages integrates directly with GitLab CI/CD, so the same pipeline that builds your project can publish the site as a deploy stage. GitHub Pages is famously frictionless for straightforward static sites and benefits from the surrounding ecosystem. Both are hard to beat at their price of zero.

A quick side-by-side

Dimension	GitLab	GitHub
Core philosophy	Integrated single application	Composable ecosystem
CI/CD	Native (`.gitlab-ci.yml`), deeply integrated	Native (Actions), huge marketplace
Security scanning	Broad suite bundled into paid tiers, unified results	Dependabot + secret scanning native; CodeQL/advanced via paid add-on
Project management	Mature native agile (epics, boards, value stream)	Capable native Projects; heavy agile often adds Jira
Community	~30M+ users, enterprise-deep	100M+ users, largest open-source community
Self-hosting	Full DevSecOps suite, single installer	Enterprise Server (paid license + infra)
Static hosting	GitLab Pages (CI/CD-integrated)	GitHub Pages

So which should you choose?

Reduce it to the philosophy and the answer usually falls out on its own.

Choose GitLab if you want one vendor for the entire lifecycle — CI/CD, security, compliance, registries, releases, monitoring — with a unified data model and the option to self-host the whole suite. It's especially compelling for regulated industries and platform-engineering teams who value integrated governance over assembling their own stack.

Choose GitHub if you want the largest community, the deepest third-party ecosystem, best-in-class open-source collaboration, and the flexibility to compose your own toolchain. It's the default for open-source projects and for teams that would rather pick best-of-breed pieces than adopt one integrated platform.

A fair caveat on GitLab, since this guide has been evenhanded elsewhere: packing the entire lifecycle into one product makes the interface dense, and teams newer to DevOps sometimes find it overwhelming to navigate. Breadth has a usability cost. GitHub's narrower core is easier to pick up on day one.

Both are strong platforms. The question isn't which is better in the abstract — it's whether your team would rather have one integrated system or the freedom to assemble the best individual pieces. Answer that, and you've answered the whole comparison.

Making Architecture Diagrams Tell the Truth: Trust Boundaries, Data Flow, and the Things We Leave Out

Doby Baxter — Mon, 13 Jul 2026 20:32:17 +0000

Most architecture diagrams are optimistic. They show the happy path: a client talks to a service, the service talks to a database, arrows point in tidy directions. What they usually leave out is where the risk actually lives — which components hold sensitive data, where trust changes hands, which team is accountable for what, and where data quietly gets copied to a third place nobody remembers to secure.

That omission isn't cosmetic. As a system grows, the parts that get left off the diagram tend to be exactly the parts that cause incidents: the logging pipeline shipping PII to an analytics vendor, the cache holding session tokens, the "internal" service that's actually reachable from the internet. A diagram that hides those things doesn't just fail to help — it actively misleads the people making decisions from it.

This piece lays out a small set of principles for drawing system diagrams that make responsibility, data movement, and trust boundaries explicit, and then walks the same example system through three stages of growth so you can see how the risk surface expands as the architecture does.

What a component label should answer

Every box in a diagram represents a responsibility. If the label is vague, the accountability is vague too. A well-defined component should let a reader answer three questions without asking you:

What does it do? One function, stated plainly. "Auth service," not "backend."
What data does it control? Especially whether that data is sensitive.
Who owns it? Which team, or which external provider, is responsible when it breaks or leaks.

"User Service" is a weak label. "User Service (Auth + PII, owned by Team A)" is a strong one — it tells you the function, the data sensitivity, and the owner in a single line. The extra words are where the accountability lives.

Make data flow directional and typed

Arrows are claims about communication, and the claim should be specific. Two conventions carry most of the weight:

Solid arrows for synchronous calls — the caller waits for a response.
Dashed arrows for asynchronous or background flows — events, log shipping, replication.

Every arrow needs a direction, and ideally a label describing what moves along it. "Read/Write (PII)" tells a reviewer something that a bare arrow doesn't: that this path carries sensitive data and therefore deserves stricter controls. Undocumented data movement is where hidden risk accumulates, because you can't protect a flow you haven't drawn.

Draw the trust boundaries

A trust boundary is any line where the level of trust in the input changes — the edge between the public internet and your internal network, between your services and a third-party API, between an untrusted user-facing layer and a privileged internal one. These boundaries are the most security-relevant lines on the whole diagram, because they mark where authentication, authorization, and validation must happen.

If your diagram doesn't show trust boundaries, it's not showing where enforcement is required. Draw them as explicit enclosures, and make it obvious which components sit inside which zone.

Make persistence and duplication visible

Storage is an architectural decision, not a background detail. Databases, caches, replicated stores, and log sinks all hold data, and every one of them is a place that data can leak from or be retained longer than intended. Label them, mark which ones hold sensitive data, and — critically — show where data is duplicated. A record that exists in the primary database, the cache, the analytics warehouse, and the log aggregator now has four places to secure and four retention policies to reason about. If the diagram only shows one, the other three are invisible risk.

Three stages of the same system

The clearest way to see these principles pay off is to watch a system grow. Here's the same application at three levels of scale. Notice that each step buys capability and spends it on complexity — more communication paths, more data stores, more owners, more audit surface.

Stage 1: the contained monolith

flowchart TD
    Client([Client])
    App[Application Server<br/>Auth + API + Business Logic<br/>Owner: Team A]
    DB[(Primary Database<br/>User Data: PII)]
    Logs[(Internal Logs<br/>Audit Trail)]

    Client -->|HTTPS, token auth| App
    App -->|Read/Write, strict access| DB
    App -.->|Events, PII redacted| Logs

One deployment unit, one primary data store, one owner. Enforcement is concentrated in a single place, data replication is minimal, and auditing is simple because there's only one path in and one team accountable. This is the model with the smallest risk surface — everything sensitive sits behind one trust boundary that one team controls. It scales poorly, but it's honest and easy to reason about.

Stage 2: horizontally scaled with shared services

flowchart TD
    Client([Client])
    LB[Load Balancer<br/>Public Entry]
    App1[App Instance 1<br/>Auth + API]
    App2[App Instance 2<br/>Auth + API]
    Cache[(Cache Layer<br/>Session / Hot Data)]
    DB[(Primary Database<br/>User Data: PII)]
    Logs[(Centralized Logging<br/>Audit + Monitoring)]

    Client -->|HTTPS| LB
    LB --> App1
    LB --> App2
    App1 -->|Read/Write| DB
    App2 -->|Read/Write| DB
    App1 -->|Cached data| Cache
    App2 -->|Cached data| Cache
    App1 -.->|Events| Logs
    App2 -.->|Events| Logs

Now there are multiple runtime instances behind a load balancer, plus dedicated caching and logging. Scalability improves — but notice what else changed. Session data now lives in a cache as well as the database, so PII-adjacent data has two homes. Shared state introduces consistency questions. Responsibility starts to fragment across more moving parts. The trust boundary still holds, but there's more inside it to keep track of.

Stage 3: distributed microservices

flowchart TD
    Client([Client])
    GW[API Gateway<br/>Auth + Routing + Rate Limiting]
    SvcA[Service A<br/>User Management<br/>Owner: Team A]
    SvcB[Service B<br/>Orders<br/>Owner: Team B]
    SvcC[Service C<br/>Payments<br/>Owner: Team C]
    MQ[[Message Queue<br/>Async Events]]
    UserDB[(User DB<br/>PII)]
    OrdersDB[(Orders DB<br/>Transactional)]
    PayDB[(Payments DB<br/>Financial)]
    Ext[External Payment Processor<br/>Third Party]
    Logs[(Centralized Logging)]

    Client -->|HTTPS| GW
    GW --> SvcA
    GW --> SvcB
    SvcA --> UserDB
    SvcB --> OrdersDB
    SvcB -.->|Publish event| MQ
    MQ -.->|Consume event| SvcC
    SvcC --> PayDB
    SvcC -->|External API call| Ext
    SvcA -.-> Logs
    SvcB -.-> Logs
    SvcC -.-> Logs

Service isolation and independent data ownership are real benefits — each team owns its service and its database. But the ethical and security surface has expanded proportionally. Data is now duplicated across services. There's asynchronous movement through a message queue that's easy to under-secure. Payments data crosses a trust boundary to an external processor. Ownership is spread across three teams, which means an audit now requires coordinating three sets of answers. Every one of those is a real property of the system — and every one of them is only manageable if it's actually drawn.

The blind spots to check for

Whatever your architecture, these are the things most commonly missing from the diagram — and disproportionately likely to be where incidents originate:

Logging and observability pipelines (they carry more sensitive data than people expect)
Third-party analytics and their data flows
Data replication and backup copies
Implicit trust relationships — the "internal" call that isn't authenticated because "it's internal"
External APIs that handle sensitive information

If a component is missing from your diagram, it's not being reasoned about. And the components teams most often forget to draw are frequently the highest-risk ones.

Closing

System design isn't only about scalability and performance. It's about responsibility, visibility, and sustainability over time. As a system scales, complexity rises; as complexity rises, accountability diffuses and the data surface area grows. A diagram that makes those dynamics visible — explicit trust boundaries, typed and directional data flow, labeled persistence, named ownership — turns architecture into something you can actually govern, not just something you can deploy. Clarity in the diagram is the first line of defense for everything downstream of it.

Validate Your LLM Workflow Before It Runs (Python + YAML Topology Checks)

Doby Baxter — Mon, 13 Jul 2026 20:30:50 +0000

Your LLM app boots fine. It answers the first few messages fine. Then, ten minutes into a real conversation, it quietly walks from the support step into the billing step that doesn't exist anymore, or loops faq → support → faq → support until something times out. Nothing crashed at startup. The mistake was in the workflow's shape the whole time — you just didn't find out until runtime, in production, with a user watching.

This is the same class of bug as a malformed config file: the error exists the moment you write it, but the tool waits until the worst possible time to tell you. The fix is the same too. Catch it at load time, before a single step executes.

In this tutorial we'll build a small, declarative LLM workflow as YAML, then validate its topology — the graph of which step can hand off to which — before we run anything. We'll catch dead ends, transitions to steps that don't exist, and loops that can never terminate. All the code here runs against a real open-source package, llm-workflow-router, so you can follow along end to end.

Why runtime is the wrong place to find structural errors

Most guardrails for LLM systems limit volume: a max number of tool calls, a timeout, a retry cap. Those are useful, but they treat the symptom. A runaway loop gets cut off after N iterations — it doesn't get prevented, and you learn nothing about why the workflow could loop in the first place.

Structural validation asks a different question: before this workflow ever handles a request, is its graph even valid? A step that transitions to a nonexistent target is invalid no matter what any model says. A step with no way out is invalid. A cycle through a step that isn't allowed to be re-entered is invalid. None of that depends on content, so none of it needs to wait for runtime to be caught.

The mental model: your workflow is a directed graph. Nodes are steps ("containers"). Edges are the transitions each step permits. Validating the workflow means analyzing that graph and refusing to start if it's broken.

The workflow as YAML

Here's a minimal support flow. A user enters at support_entry, which may hand off to faq or refuse; faq may only refuse. REFUSE is a terminal state, not a step.

# good_config.yaml
entrypoints:
  - support_entry

containers:
  support_entry:
    allow_transitions:
      - faq
      - REFUSE
    allow_reentry: false
    max_invocations: 3

  faq:
    allow_transitions:
      - REFUSE
    allow_reentry: true
    max_invocations: 5

Three things are being declared per step, and each one is a rule the validator can check:

allow_transitions — the only steps this one may hand off to. Anything else is an invalid transition, full stop.
allow_reentry — whether the step may be entered more than once. This is what turns "is there a cycle?" into "is there a cycle that's actually a problem?"
max_invocations — a hard ceiling on how many times the step may run. Must be greater than zero, or the step can never execute.

entrypoints names the authoritative roots of the graph. Declaring them explicitly means the validator doesn't have to guess where the workflow starts — and it can flag any step that no path can reach.

Validating from the command line

Install the package:

pip install llm-workflow-router

Then validate the config before you wire it into anything:

python -m router.cli validate good_config.yaml

{
  "ok": true
}

Clean exit, exit code 0. Now let's break it on purpose — the kind of breakage that's easy to introduce and impossible to spot by eye once a config gets large:

# broken_config.yaml
containers:
  support_entry:
    allow_transitions:
      - faq
      - billing        # typo: this step does not exist
    allow_reentry: false
    max_invocations: 3

  faq:
    allow_transitions:
      - support_entry   # loops back into a no-reentry step
    allow_reentry: true
    max_invocations: 5

  orphaned_step:
    allow_transitions:
      - REFUSE
    allow_reentry: false
    max_invocations: 1

There are three separate problems hiding in there. Run validate and it stops you at the first hard error:

python -m router.cli validate broken_config.yaml

{
  "ok": false,
  "error": "CONFIG_VALIDATION_ERROR",
  "message": "Container 'support_entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}."
}

Exit code 2. Notice what the error message actually does: it names the offending step (support_entry), the bad value (billing), and the set of things that would have been valid. That last part is the difference between an error that stops you and an error that helps you. More on that below.

Seeing every problem at once

validate is pass/fail — good for CI, where you want a nonzero exit and the first blocking reason. But when you're actually fixing a workflow, you want the whole picture, not one error at a time. That's what analyze is for: it inspects the topology and reports everything, graded by severity, without raising.

python -m router.cli analyze broken_config.yaml

{
  "container_count": 3,
  "entry_containers": [
    "orphaned_step"
  ],
  "entrypoints_declared": false,
  "cycle_count": 1,
  "cycles": [
    ["faq", "support_entry"]
  ],
  "issues": [
    {
      "severity": "ERROR",
      "code": "CYCLE_WITH_REENTRY_BLOCKED",
      "message": "Cycle ['faq', 'support_entry'] includes containers with allow_reentry=false: ['support_entry']"
    },
    {
      "severity": "ERROR",
      "code": "UNKNOWN_TARGET",
      "message": "Container 'support_entry' transitions to unknown 'billing'."
    },
    {
      "severity": "WARNING",
      "code": "UNREACHABLE_CONTAINERS",
      "message": "Unreachable containers: ['faq', 'support_entry']"
    }
  ]
}

This one report catches all three bugs I planted:

UNKNOWN_TARGET — the billing typo. A transition to a step that isn't in the graph.
CYCLE_WITH_REENTRY_BLOCKED — faq transitions back to support_entry, forming a loop faq → support_entry → faq, but support_entry declared allow_reentry: false. The workflow contradicts itself: the topology requires re-entering a step the config forbids re-entering.
The entrypoint inference falling apart. Because I removed the entrypoints block, the analyzer had to infer the root from graph shape — and the only step nothing points to is orphaned_step, so it wrongly became the "entry." Everything else is now unreachable from that inferred root, hence the UNREACHABLE_CONTAINERS warning. This is a great argument for always declaring entrypoints explicitly: it removes the guessing.

The severity split matters. An ERROR is a contradiction that makes the workflow structurally invalid — it blocks. A WARNING is a smell worth surfacing but not necessarily fatal. Cycles are a nice example of the distinction: a loop through steps that all permit re-entry is a WARNING (it can legitimately terminate via max_invocations), while a loop through a step that forbids re-entry is an ERROR (it can't).

Doing it in Python

The CLI is a thin wrapper. If you're building the workflow programmatically — or want validation inside your own app's startup path — the same checks are one function call.

from router.engine import WorkflowEngine
from router.models.config import ContainerConfig, WorkflowConfig
from router.models.enums import InteractionState
from router.models.metadata import InteractionMetadata
from router.validation.config_validator import validate_workflow_config

cfg = WorkflowConfig(
    containers={
        "support_entry": ContainerConfig(
            allow_transitions=["faq", "REFUSE"],
            allow_reentry=False,
            max_invocations=3,
        ),
        "faq": ContainerConfig(
            allow_transitions=["REFUSE"],
            allow_reentry=True,
            max_invocations=5,
        ),
    },
    entrypoints=["support_entry"],
)

validate_workflow_config(cfg)   # raises ConfigValidationError if the graph is invalid
print("Config valid ✓")

validate_workflow_config raises ConfigValidationError on any hard error, so the natural place to call it is at load time — the workflow simply refuses to start if its shape is wrong. That's the whole idea: an invalid state is made unrepresentable at runtime because it was rejected before runtime began.

Once the config is validated, evaluating a step is deterministic — the same metadata always yields the same decision:

engine = WorkflowEngine(cfg)

metadata = InteractionMetadata(
    container="support_entry",
    previous_state=InteractionState.PROCEED,
    transition_history=[],
    invocation_depth={"support_entry": 1},
    requested_action="faq",
    trace_id="demo-001",
)

print(engine.evaluate(metadata))

EvaluationResult(state=<InteractionState.PROCEED: 'PROCEED'>, container='support_entry', reason=None, trace_id='demo-001')

The engine checks the request against the validated rules: is support_entry a real step? Under its invocation ceiling? Is faq in its allow_transitions? All yes, so the result is PROCEED. Ask for a transition that isn't allowed and you'd get a REFUSE with a machine-readable reason like INVALID_TRANSITION — no exceptions thrown mid-flight, just an explicit, inspectable decision.

The part worth stealing: error messages that point at the fix

Go back and look at that first failure message:

Container 'support_entry' has invalid transition target 'billing'. Must be a container name or one of {'REFUSE', 'PROCEED', 'PAUSE'}.

It would have been easier to write ValueError: invalid transition. It would also have been useless. A good validation error answers three questions without making the reader go digging: where is the problem (support_entry), what is wrong (billing isn't a valid target), and what would be right (a real step name, or a terminal state). The analyze codes do the same job for tooling — UNKNOWN_TARGET, CYCLE_WITH_REENTRY_BLOCKED, ORPHAN_CONTAINER are greppable, stable identifiers you can build dashboards or CI rules around.

If you take one habit away from this, let it be that one. Validation that only says no makes people resent your tool. Validation that says no, here, because of this, and here's what valid looks like makes people trust it. The effort is a few extra f-strings; the payoff is every future debugging session.

Wrapping up

Structural bugs in an LLM workflow — dead ends, phantom transitions, non-terminating loops — are present the instant the config is written. There's no reason to let them surface at runtime. By modeling the workflow as an explicit graph and validating its topology at load time, you convert a whole category of production incidents into a fast, boring failure at startup with a message that tells you exactly what to fix.

Everything here runs against the open-source llm-workflow-router (MIT). Try validate and analyze on your own configs — and if you're building anything where an AI system hands off between steps, put the validation call in your startup path before you need it.

DEV Community: Doby Baxter

You Don't Need Perfect Code - You Need a Safety Net That Talks Back

The myth I had to give up first

What the research actually says

Mapping the studies to rules I actually hold myself to

Rule 1 — Treat the message as a first-class output, budgeted like code

Rule 2 — Frontload the fix, because you're spending someone's attention

Rule 3 — Enhance with cause and action, not just nicer words

Rule 4 — Speak the user's domain, not the parser's

Rule 5 — Write for your reader's level, and prove it

Rule 6 — Recognize, diagnose, recover — and prevent

Rule 7 — Never blame, and never claim certainty you don't have

The synthesis: a net catches and speaks

Takeaways

References

Building a FastAPI Backend That Serves Live Satellite Imagery and Orbital Position

Contents

The idea: one place, two scales

Getting satellite data, step one: authentication

Step two: finding the newest real scene

Step three: NDVI imagery and statistics

Step four: from a TLE to a position in orbit

Pinning the satellite to the scene

Designing endpoints that can't break the frontend

Keeping it light: caching

Takeaways

Testing Against State Drift: Guarding a Config Tool Whose Source of Truth Lives Somewhere Else

Contents

The problem: correctness you don't own

Three kinds of drift

Layer 1 — Is the bundled schema still fresh?

Layer 2 — Do the real schema's bounds still hold?

Layer 3 — Do the docs still match the actual API?

Testing at the seam

Making the gates non-optional

What's deliberately not automated

Takeaways

I Instrumented a System the OTel GenAI Conventions Weren't Built For - Here's Where They Broke

Contents

The system under instrumentation

The six frictions (at a glance)

The frictions in detail

Friction 1 — There's no operation name for validation

Friction 2 — gen_ai.provider.name assumes a provider exists

Friction 3 — Workflow topology has no vocabulary

Friction 4 — A refusal is not an error

Friction 5 — Content observability is treated as the default

Friction 6 — Standalone evaluations have no trace-parenting model

What I shipped instead: own the namespace

The four design rules

The convention surface

Using it

The pattern underneath

Where one friction ended up

Takeaways

GitLab vs GitHub in 2026: Integrated Platform vs. Composable Ecosystem

The core distinction

Repository management

CI/CD

Security scanning

Project management

Pages and hosting

A quick side-by-side

So which should you choose?

Making Architecture Diagrams Tell the Truth: Trust Boundaries, Data Flow, and the Things We Leave Out

What a component label should answer

Make data flow directional and typed

Draw the trust boundaries

Make persistence and duplication visible

Three stages of the same system

Stage 1: the contained monolith

Stage 2: horizontally scaled with shared services

Stage 3: distributed microservices

The blind spots to check for

Closing

Validate Your LLM Workflow Before It Runs (Python + YAML Topology Checks)

Why runtime is the wrong place to find structural errors

The workflow as YAML

Validating from the command line

Seeing every problem at once

Friction 2 — `gen_ai.provider.name` assumes a provider exists