We now live in a world where an absurd amount of stuff can be generated on demand. Text, images, music, plans, arguments, mockups, and enough “options” to make the concept of a single coherent identity feel like a premium feature you forgot to renew.
The cost of producing a thing has collapsed. That part is no longer controversial. The only people still arguing about it are either selling something, regulating something, or trapped in a meeting where everyone says “paradigm shift” like it’s a prayer.
The obvious conclusion is that creation no longer matters.
That conclusion is wrong, in the way it’s wrong to conclude that because bread is cheap, sandwiches are meaningless. Bread isn’t the problem. The problem is the horrifying number of sandwiches that now exist, many of which are trying to be your personality.
When supply gets close to unbounded, value doesn’t disappear. It migrates. Upstream. Into the part that decides what’s worth keeping.
Generation is cheap. Judgment is still expensive. And somehow we keep trying to pay for judgment with vibes.
I’ve found it useful, for my own sanity, to think about AI in two broad modes. Not because the universe demands categories, but because humans are perfectly capable of confusing themselves to death without help, and I’d like to reduce my reliance on that talent.
There’s the mode where AI is a tool. Instrumental. I have a task, a constraint, an outcome. The model helps me do work. The output goes into a real process that pushes back: compilation, tests, users, production logs, budgets, deadlines, and whatever grim entity governs time estimates.
Reality is the grader.
Then there’s the mode where AI is an oracle. Not necessarily because I want it to be, but because I ask it questions in places where I can’t clearly test whether it’s wrong, or where “wrong” isn’t even well-defined. What should we believe. What does this mean. What will happen. What is best.
This distinction isn’t moral. It’s about control. Tooling keeps agency with me. Oracle use can quietly relocate agency to something that speaks in complete sentences and never looks nervous.
Most of the trouble starts when these two modes get blurred. Which, inconveniently, is the default setting.
My own use leans heavily toward tooling. Not because I’m above temptation, but because I enjoy knowing when I’m being lied to.
Tooling forces contact with consequence. It keeps me inside a feedback loop. If the model helps, great. If it misleads me, I pay for it quickly. That payment is information. It updates my understanding. It’s annoying, but it’s honest.
In practice this looks very unromantic.
I’ll use AI to sketch a refactor path or list tradeoffs, then I validate by doing the boring part: reading the code, running the tests, watching what the system does when it hits reality.
I’ll use it to draft tests or enumerate edge cases, especially where business rules mutate into folklore. Then I check those cases against actual requirements, existing behavior, and whatever production logs are willing to confess.
I’ll use it to explain a module back to me, but I treat it like a rubber duck with a postgraduate vocabulary. Helpful, occasionally brilliant, not legally admissible.
Sometimes I’ll ask it for a few alternative implementations just to see patterns, like shaking a box to hear what rattles. The compiler becomes the lie detector. The runtime becomes the judge. Users become the jury, which is a frightening thought if you dwell on it.
Tooling doesn’t make me smarter. It makes me faster at discovering where I’m wrong.
That matters because speed without feedback is just acceleration, and acceleration is only impressive until you meet a wall.
None of this means oracle use is useless. That would be dishonest. There are places where “correctness” isn’t the goal, because there is no stable ground truth to converge toward. You’re exploring. You’re mapping. You’re trying to see the problem from an angle you don’t naturally inhabit.
Oracles make sense in ideation, in creative exploration, in speculative thinking, in philosophy, in reframing problems you already own responsibility for. They make sense when what you’re buying is perspective, not authority.
The system isn’t there to be right. It’s there to widen the map. To suggest a path you didn’t see. To hand you a weird rock and insist it’s symbolic.
The distinction that matters is authority. Safe oracle use is borrowing perspective, not outsourcing judgment.
Which brings us to the part where people get quietly tricked, usually because the trick is comfortable.
Oracle use is safest when it’s bounded, plural, and owned.
Bounded means I set the frame. I remind myself, explicitly, that this is exploration, not instruction. Plural means I force it to generate multiple perspectives, including the annoying ones. Owned means I treat the output as input to my judgment, not a replacement for it.
If I’m being disciplined, “safe oracle mode” looks like asking for opposing framings, counterarguments, or what would change the answer. It looks like prying assumptions into daylight. It looks like separating facts from interpretations and value claims, because mixing those is how you end up with a toaster as your spiritual advisor.
Oracle use becomes unsafe when confidence replaces accountability.
Especially in politics, culture, ethics, identity, and forecasting complex social systems, which are basically domains where everyone is carrying a sword made of opinions and insisting it’s a measuring instrument. These spaces contain facts, but they aren’t reducible to facts, and the gap is where fluency gets mistaken for truth.
The most dangerous oracle failures aren’t dramatic. They’re quiet. People think they’re using a tool. The system behaves like an oracle. Authority slips in through tone, coherence, and the deeply human habit of mistaking “sounds plausible” for “is reliable.”
A rule of thumb helps here, because humans love rules of thumb even when they fail constantly: if I can’t describe how I’d know the answer was wrong, I’m in oracle mode.
That doesn’t mean don’t do it. It means don’t confuse it with tooling. Don’t install it as your internal government.
This is where “accuracy” starts to mislead people. A lot of AI discourse tries to reduce everything to accuracy because accuracy is measurable, and measurable things make humans feel safe. Like seatbelts. Or quarterly reports.
Accuracy makes sense when there’s a stable external referent. A bridge holds or it doesn’t. A proof is valid or it isn’t. Code compiles or it doesn’t. In those domains, accuracy is a useful metric.
In art and culture, “accuracy” isn’t the metric. In politics and ethics, it’s only part of the story. These are value-laden domains. Outputs aren’t simply correct or incorrect. They’re situated, interpretive, persuasive, contested.
Trying to force these into accuracy problems doesn’t make them safer. It just hides judgment behind math-flavored language.
What matters more here isn’t correctness. It’s epistemic integrity: making assumptions explicit, separating fact from interpretation, and resisting the urge to launder values as facts.
And when “accuracy” becomes slippery, something else becomes dangerous: fluency. Because coherent outputs can feel true even when they’re just well-formed. The model can write like a confident adult in a blazer. Reality does not care about blazers.
Now we get to the part that feels like an upgrade and might secretly be a trap.
GenAI introduces something new into decision-making: a cheap buffer between intent and consequence. We can simulate. Rehearse. Generate alternatives. Explore counterfactuals without immediately acting in the world. Not infinite in any literal sense, but economically unrecognizable compared to what came before. Like going from having one telescope to having a thousand telescopes and no idea which one is pointing at a bird.
This buffer is powerful. In many contexts it’s genuinely beneficial.
But buffers change behavior.
When feedback is delayed, softened, or abstracted, people take bigger risks. Systems scale faster than understanding. Bad ideas survive longer than they should, like a vampire invited in because it looked polite on LinkedIn.
Simulation doesn’t remove cause and effect. It defers it. And deferred consequences tend to arrive concentrated, like interest on a loan you forgot you took out because the app made it feel friendly.
This is the meta-risk that doesn’t require sci-fi. Not “AI decides for us,” but “AI makes it easier for us to avoid feeling the cost of our decisions until the cost becomes unavoidable.”
A buffer can be a safety feature. It can also become a blindfold. Humans are remarkably good at turning safety features into excuses.
Tooling survives this buffer better than oracle use because reality remains the arbiter. Outputs still get graded by the world. Even if I move faster, I’m still constrained by tests, production, users, and failure. Mistakes remain informative. The loop stays intact.
Oracle use erodes that connection. Not because the system lies, but because persuasion replaces validation. Coherence replaces truth-testing. Confidence replaces learning.
The danger isn’t wrong answers. It’s unowned ones.
“The model said so” is a modern method of responsibility diffusion. It’s not malicious. It’s just convenient. And convenience is how civilization usually gets killed, or at least how it ends up in meetings.
When people talk about making AI “trustworthy,” they often mean “make it more accurate.” In many real use cases, trustworthiness is less about accuracy and more about behavior under uncertainty.
A trustworthy system is one that makes clear what it knows versus what it’s guessing, that doesn’t smooth uncertainty into confident tone, that exposes assumptions, that invites verification when verification exists, and that refuses to become an oracle by accident.
This is where grounding, retrieval, and evaluation loops matter. Not because they make the model perfect, but because they tether it to something outside itself when that’s possible. Trust should be earned through traceable process, not vibes.
Vibes are how you end up worshipping a toaster.
There’s an exception worth protecting, because humans need somewhere to be irresponsible without burning down the neighborhood: play.
Using AI for creative inspiration, exploration, or fun is often safe precisely because it makes no claim on truth. Nothing depends on the output. There’s no authority granted. It’s a fountain of variation. Prompts. Seeds. Sparks.
It can hallucinate all it wants in this mode because I’m not confusing it with a decision engine. It’s a toy that sometimes produces interesting artifacts, like a kaleidoscope that occasionally writes a decent paragraph.
Play has boundaries. Hallucination is harmless when it’s not pretending to be instruction.
Where is this heading. The honest answer is: anywhere we let it. But the trend line feels clear.
Generation is trending toward commoditization. The ability to produce outputs will keep getting cheaper and more common. The premium shifts to selection and accountability: systems and practices that preserve agency, surface uncertainty, and keep humans responsible for consequences.
The valuable layer won’t be “more content.” It’ll be better judgment under overload. Tools that help decide what to pay attention to, what to ignore, and when to hesitate.
That’s not glamorous. It’s not oracle-shaped. But it’s load-bearing.
The risk isn’t that we build powerful generators. The risk is that we build them in a way that turns feedback into a suggestion instead of a constraint.
None of this is advice. It’s a constraint I’m choosing to operate under.
I don’t want systems that decide for me. I want systems that help me decide faster while being honest about what they know, what they assume, and what they can’t possibly know.
I want feedback, not reassurance. Friction where it matters. Play where it’s safe.
In a world where almost anything can be generated, preserving responsibility might be the only truly scarce thing left.
Top comments (0)