DEV Community: Jay F0x

Provenance in LLMs

Jay F0x — Fri, 29 May 2026 11:11:13 +0000

Just saw another prompt with "Don't make mistakes!"
But what does that actually achieve?

It forces compliance instead of understanding. A model told not to make mistakes will try harder to sound right. It won't optimize for problem-solving but for sycophancy (prioritizing user alignment over truthfulness).

We have access to the most advanced cognitive technology in history, and we're still writing if-statements like it's 1995. Just in Markdown.

Look at most CLAUDE.md files. Same pattern every time: goals, strict rules, edge cases stacked on top of each other. It works, until it doesn't. And when it breaks, it breaks exactly where the constraints failed to anticipate. The fix? More rules. Prompt rot.

Constraints can't cover what genuine context already knows.

The better investment: give the model provenance. Not just what to do — but what this is, who it's for, why it matters. General language models are general for a reason. You have to define the ground they operate on.

Stop coding your prompts. Start anchoring them.
Provenance → direction → constraints.

ps:
Mistakes are how intent gets defined. If your model never does anything wrong, you're not refining anything, you're just accepting defaults. Which kinda defeats the whole purpose of you being in the loop in the first place.

A Refreshing Perspective on AI and Truth

Jay F0x — Thu, 28 May 2026 11:30:28 +0000

Everyone has a favorite movie. Some of us ask why.
A kid might say Spiderverse. A cinephile might insist on Lawrence of Arabia. A film historian might point further back — to a Buster Keaton two-reeler from 1921 that quietly invented half of what cinema still does today.

None of them are wrong. Each is right relative to where they stand: their experience, their era, the conversations they've been part of. Truth, for humans, has an address.
Artificial Intelligence has no address.

During training, a model ingests millions of documents simultaneously — texts from opposing centuries, conflicting political movements, irreconcilable cultures — and flattens them into a single mathematical space. To a film historian, that 1921 Keaton film explains the 2026 blockbuster. To an AI, both exist at the same depth, in the same timeless fog. There is no before. There is no provenance. So when you ask an AI to review your article and it loves a sentence, then in the next session calls that same sentence weak -- that isn't a bug or a bad day. There is no plot, and there is no twist, because there is no story being told from anywhere.

When forced to answer, the model doesn't reason from a position. It calculates a statistical average — blending the kid, the cinephile, and the historian into something that sounds authoritative because it contains all of them and is anchored by none of them.

This is the core paradox: an LLM is never wrong because it is incapable of being right. Not in the way that matters. Being right requires standing somewhere.

Which is why a good prompt is more important than most people think. The prompt is the only provenance the model has. It's the only "when" and "who" and "from where" available to it. A vague prompt doesn't just get a vague answer — it gets an answer from nowhere, averaged from everywhere. A specific, contextual prompt is the closest thing an LLM has to a position in time.

So maybe "truth-seeking AI" isn't entirely a broken idea. It's just that the seeking starts with — and depends on — you (whatever "you" really means).

Mythos Preview - Early news!

Jay F0x — Thu, 09 Apr 2026 16:11:41 +0000

Anthropic just announced Claude Mythos Preview, their most capable model yet. Not released to the public, because it's too good at breaking things.

In testing, Mythos found thousands of zero-day vulnerabilities across every major OS and browser, autonomously. Some bugs had survived decades of human review.

It started as a general purpose AI, not even a specialized security tool, and stumbled into capabilities serious enough to now hold back from the public (or so the tales go).

So instead of a public release, Anthropic launched Project Glasswing (read that name twice): a coalition of 50+ companies (NVIDIA, Apple, the Linux Foundation...) using Mythos to find and patch vulnerabilities across critical infrastructure.

The tools that find vulnerabilities and the tools that exploit them might become the same. And very few are in control of them.
But this capability might not to stay contained. The question becomes how we react when this level of capability becomes widely available.

Rule nr1: Don't panic (42.)

Why the web doesn't need humans anymore

Jay F0x — Fri, 06 Mar 2026 09:44:17 +0000

Why does Google feel different? It isn't just AI. Let's follow the money...

You look for a specific but instead of a solution, you get corporate bs with Reddit threads, Forbes... It feels like the library has been replaced by a shopping mall where every book is written by a marketing committee.

This isn’t a technical failure.

1. The Engineering of "Good Enough"

One of the most chilling discovery of the last two years didn't come from a lab, but from a courtroom. Internal emails from the US v. Google antitrust trial revealed a fundamental civil war inside the company. Ben Gomes, the engineer who helped build Google’s reputation for quality, warned that the company was becoming "too close to the money".

The data suggests they realized they could make the results slightly worse, forcing users to search longer. It’s a "Boiling the Frog" strategy where the goal isn't to find you the best answer, but to keep you in the ecosystem until you settle for "good enough".

2. The Death of the Independent Voice

In May 2024, a massive API leak of 14,000 ranking factors shows part of the reality how Google has changed over the years and why providing quality content is not enough.

Google has effectively hardcoded a preference for big brands like Reddit. This is why a massive, multi-billion dollar site with zero expertise in a topic will now outrank a scientist’s personal blog. The "Human Web"—the era of weird, expert, and independent sites—is being intentionally starved of traffic to make way for a more controllable, corporate-vetted internet.

3. The Browser as training tool

To Google your browser is a sensor and every action you do or don't serves as training data.

This data is used to build "Zero-Click" results. They scrape the best parts of the web, present them in an AI Overview, and ensure you never have to visit the original creator's site. It is a one-way extraction of human knowledge.

4. The Agentic Age

With projects like Jarvis and Mariner, Google is shifting from a tool that helps you find things to an agent that does things for you.

In this new reality, you don't browse the web; you prompt. The agent then navigates a "Ghost Web" (a place where bots talk to bots) and only the information Google deems "authoritative" (or profitable) is allowed through. The internet is being transformed into a vast, silent database for AI training, while the human experience is condensed into a single chat interface.

The Bottom Line:
Google isn't "broken", instead the "Search Engine" is dying so that the "Action Engine" can live.

If we stop visiting websites and start only talking to agents, who decides what information is "true" once the original creators have gone out of business?

Sources & Deep Dives:

Learning basics without misconceptions

Jay F0x — Sun, 01 Feb 2026 13:18:26 +0000

LLM: Predictability VS Determinism - DEV Community

If you search for “how to make an LLM deterministic,” you might find advice like: “Set...

dev.to

LLM: Predictability VS Determinism

Jay F0x — Sun, 01 Feb 2026 13:15:11 +0000

If you search for “how to make an LLM deterministic,” you might find advice like:

“Set temperature to 0, fix the seed, use top-p = 1 or top-k = 1.”

This mixes up two separate ideas: determinism vs predictability.

Predictability: the model tends to give similar outputs because it’s “playing it safe” (low temperature, top-k/p limits).
Determinism / reproducibility: the model gives the exact same output every time, which only happens when the seed is fixed.

Think of the seed like a Minecraft world seed: it doesn’t make the landscape “more likely,” it just makes it repeatable. Same seed + same prompt = same output, every time.

Options like Temperature, Mirostat, top-k, top-p… control style, variety, and “wildness”. They can make outputs more predictable in practice (low temperature = less surprising tokens), but they do not guarantee reproducibility. The seed is the only knob that truly locks the path.

In other words: you can have a wild, creative response that is fully replayable if you fix the seed. That’s why reproducibility in LLMs is really about the seed, not temperature.

Example using Python Ollama:

    ollama.chat(
       ...
       options={"seed": 42 }
    )

TL;DR

Temperature, top-k, top-p, Mirostat → control style and predictability, not determinism.
Seed = true reproducibility. Want the exact same output every time? Lock the seed.

Try it yourself and share your findings!