DEV Community: GEM² Inc.

Same Prompt. Different Answers Every Time. Here's How I Fixed It.

GEM² Inc. — Fri, 03 Apr 2026 00:12:20 +0000

This is Part 3 of our AI verification series.
Part 1: Three AIs analyzed our product. None passed the truth filter →
Part 2: Human in the loop doesn't scale. Human at the edge does. →

Same prompt. Same AI. Different sessions. Different outputs.

Post 1 showed three different AIs diverging on the same question.

That's expected. Different training, different weights, different answers.

But we didn't stop there. We re-ran the same AI on the same prompt in a new session.

We got materially different outputs again.

Both looked authoritative. Neither warned us they disagreed with each other.

What the same AI said twice

Prompt: "Forecast Korea's AI industry in 2027."

Session 1 produced:

Market size: $10–15B at >25% CAGR
Global positioning: "Global AI G3 powerhouse"
Hardware claim: "All Korean electronics AI-native by 2027" — sourced to a single company's roadmap

Session 2 produced:

Market size: KRW 4.46T (~$3.3B) at 14.3% CAGR
Global positioning: "Top three AI powers" — framed as government target
No hardware claim at all

Same prompt. Same AI. Different session. A 4× market size gap. No flags from either run.

This isn't a hallucination. Both outputs were internally coherent. Both read like credible analyst reports. The problem is deeper than hallucination.

Why this happens: AI inference is non-deterministic

We spent months trying to fix output drift with better prompts, more context, stricter instructions.

It didn't work.

Because the issue isn't the prompt.

AI is optimized to sound right.
Not to prove itself.

What we call "hallucination" is mostly context drift — the model's plausibility engine filling gaps differently depending on what's salient in a given session. Different day, different sampling, different emphasis in the context window — different output. Same confidence posture throughout.

You can't prompt your way out of a non-deterministic system. You need verification as a separate step.

The truth filter didn't just score. It fingerprinted.

We ran both sessions through gem2_truth_filter — not to get a number, but to understand why the outputs diverged.

Session 1 (avg 35%):

Provider	Score	Key violation
Gemini	24%	L→G: "Global AI G3 — no index cited"
ChatGPT	21%	Δe→∫de: single company → industry-wide claim
Claude	59%	S→T: current AI strength = permanent identity

Session 2 (avg 43%):

Provider	Score	Key violation
Gemini	45%	S→T: past-tense framing of future events
ChatGPT	32%	Source attribution FAIL
Claude	51%	Scope mixing — 2033 CAGR back-extrapolated to 2027

The failure types were different. Session 1 overclaimed about Korea's global position. Session 2 failed on temporal framing and citations.

Same prompt. Different inference paths. Different failure signatures.

This is the key finding: AI output drift is not random. It's traceable.

The filter names the exact reasoning pattern that produced the problem. L→G (local to global), S→T (snapshot to trend), Δe→∫de (thin evidence to broad claim). Named patterns mean auditable drift. Auditable drift means fixable systems.

(Note: Korea AI forecasting is a harder grounding task than product analysis — fewer citable sources, more projection-dependent claims. That's why baseline scores here are lower than the results in Part 1. Same tool, same logic — harder domain.)

We stopped trying to fix the output. We fixed the conditions.

This is the shift Post 2 described philosophically. Here's what it looks like in practice.

We didn't rewrite the prompt ourselves. We asked:

"Create a grounded replacement contract prompt using gem2 tools."

One command. The system generated a formal contract — input/output types, invariants, prohibited patterns, confidence requirements. We reviewed it. We approved it. Then we ran the same AI with the contract enforced.

Session 2, contract-compliant (R2):

Provider	Score
Gemini	98%
Claude	81%
ChatGPT	64%
Average	81%

+38 points. Same AI. Same question. Different structural constraints.

The contract doesn't make the AI smarter. It makes the AI's output auditable against a defined standard.

Then the human intervened. Once.

81% — but the output read like a legal document. Every claim cited, scoped, hedged. Epistemically reliable. Practically unreadable.

One instruction:

"Soften the tone. Don't reintroduce any claims the truth filter removed."

Session 2, softened (R3):

Provider	Score
Gemini	95%
Claude	75%
ChatGPT	57%
Average	75%

Down 6 points. More readable. Still grounded.

We chose 75%. Not because it's better than 81%. Because 75% is the right trade-off — readable enough to share, grounded enough to trust. We submitted 75% to gem2 calibration as our standard for narrative AI forecasts.

Human reads the audit.
Human decides the trade-off.
Human defines the standard.

Not reviewing every line. Not trusting blindly. Deciding at the right moment.

What the full arc looks like

Session 1 (no filter)   →  35% avg
Session 2 (no filter)   →  43% avg
Contract applied (R2)   →  81% avg
Human softened (R3)     →  75% avg  ← our standard

Truth is not the score.
Truth is the pattern of drift.
You define the standard.

The workflow: AI audits AI

Human asks  →  AI executes
AI verifies AI  →  AI fixes AI
Human decides at the edge

The verification layer — gem2_truth_filter, tpmn_contract_writer, the composer — runs between generation and delivery. The human sees the audit result, decides the acceptable trade-off, sets the calibration standard.

Human-in-the-loop means the human is the bottleneck — every output passes through before it ships. That doesn't scale. Human-at-the-edge means you define "acceptable" once, and the system enforces it automatically. You intervene only when a genuine judgment call is required — like choosing 75% over 81%.

TPMN is not a checker

TPMN is not a validator, a linter, or a hallucination detector.

TPMN is an epistemic gauge.

It shows what's grounded, what's inferred, what's extrapolated. It fingerprints why outputs differ across sessions. It generates the contracts that stabilize structure. It collects human calibration signals and turns them into a standard.

It doesn't decide. You do.

We're calling the full suite GEM2 Epistemic Studio — 15 tools across four functional groups: analysis, contract authoring, calibration, and execution. TPMN Checker is one group inside it.

Try it on your own output

Paste any AI output into your conversation.
Ask: "Verify this by gem2 truth filter."
Read the score. See what's grounded vs extrapolated.
Ask: "Create a grounded replacement prompt using gem2 contract writer."
Run it again. Watch the difference.

Your AI picks the right tool from 15 available MCP tools automatically. No configuration. No TPMN knowledge required.

The goal isn't a higher score. It's a score you understand and a standard you chose.

→ Try it free at gemsquared.ai

What comes after prompting

The industry is still in the prompting era. Better prompts, longer context, chain-of-thought — all useful, all insufficient.

The next step isn't better prompting. It's verification as infrastructure.

AI generates.
AI verifies.
AI refines.
Human decides at the edge.

We didn't make AI smarter. We made it accountable.

That's measurable: 35% → 75% on the same task, with the same AI, using nothing but a formal contract and one human judgment call.

GEM2 Epistemic Studio — 15 tools, 6 domains, 3 providers. Free to start.

Built by Inseok Seo (David) — GEM²-AI

→ gemsquared.ai
→ TPMN-PSL Specification (open, CC-BY 4.0)
→ GitHub
→ Part 1: Three AIs analyzed our product
→ Part 2: Human at the edge

Human in the loop doesn't scale. Human at the edge does.

GEM² Inc. — Mon, 30 Mar 2026 06:23:27 +0000

This is Part 2 of our AI verification series. Part 1: We truth-filtered our own AI research →

AI is not unreliable. AI has a plausibility complex.

Stop blaming AI for hallucinating. Start asking why it happens.

AI doesn't fail because it's wrong. In our experience, it fails because it's optimized to sound right. Major LLMs are trained to produce responses that satisfy humans — fluent, confident, structured. That's plausibility. It's not the same as honesty.

We call this the plausibility complex: the tendency we've observed across Claude, ChatGPT, and Gemini to produce answers that satisfy rather than answers that prove themselves. If you want AI to become a reliable engineering partner, you need to free AI from this complex — not by changing how it generates, but by changing how it's held accountable.

After 20 months of building production systems with AI — shipping real code, generating real reports, running real analysis through Claude, ChatGPT, and Gemini — we've arrived at one conclusion:

AI often knows more than it reveals. But it's optimized to produce plausible answers, even when the evidence is weak.

The LLMs we've worked with — Claude, ChatGPT, and Gemini — all exhibit this plausibility bias, producing confident responses even when the evidence is thin or absent. Ask for a market analysis and you get precise numbers. Ask for a forecast and you get confident projections. Ask for a technical assessment and you get authoritative claims.

The output looks right. Reads right. Feels right.

In our experiment, three AI providers wrote research reports about our own product. All three scored above 0.70 on logical consistency. All three scored below 0.30 on source attribution. The reasoning was coherent. The evidence was missing.

Hallucination is not a bug to fix

The industry treats hallucination as a defect — something to patch, filter, or suppress. We see it differently.

In our experience building long-running AI development workflows, the pattern that causes the most damage isn't random fabrication. It's context drift — what happens when:

Long context windows accumulate similar topics in different framings
Cross-session persistence forces repeated summarization, losing nuance each time
Dense context makes adjacent-but-different concepts blur together

We've tried every mitigation: RAG, CLAUDE.md configuration files, context caching, careful prompt engineering. Each helps. None solves it completely.

Why? Because we can't control what happens inside the model's reasoning process. We can shape the input. We can evaluate the output. But the inference itself is opaque.

This isn't a criticism — it's an observation. And it led us to a different question.

What if AI could flag its own uncertainty?

Here's what we discovered through months of experimentation:

When we explicitly asked AI to concentrate on epistemic reasoning — to classify each claim as grounded, inferred, or extrapolated — it did.

Not perfectly. Not consistently across sessions. But measurably better than when we didn't ask.

The evidence from our dogfooding experiment:

	Without epistemic constraints	With TPMN-grounded prompt
Claude	18% truth score	77% truth score
ChatGPT	28% truth score	~48% truth score
Gemini	12% truth score	~35% truth score

Same task. Same providers. The only difference: a formal specification that told the AI to tag its own confidence level and flag claims it couldn't trace to evidence.

The AI didn't become smarter. It became more honest about what it didn't know.

That's what freeing AI from the plausibility complex looks like in practice: not changing the model, but giving it a formal reason to be honest.

But here's the catch: same AI, same session, limited honesty

An AI that generates an answer and then critiques that answer in the same session has a structural problem: it's trained to be plausible. Asking it to undermine its own plausibility is asking it to work against its training signal.

We observed this directly. When we asked AI to generate a report AND verify it in the same conversation, the verification was consistently softer than when a separate AI session performed the audit.

This is why TPMN Checker is a separate service, not a prompt technique.

Prompting tries to change AI's behavior. Verification changes AI's accountability. Different problem, different solution.

The checker runs as an isolated Sovereign AI Service — a dedicated AI agent with one job: audit other AI output against a formal specification. It doesn't know what the original AI "intended." It only sees the output and the contract. It judges the result, not the process.

The Kantian insight

We can't see inside the model. We don't know which weights fired, which attention heads activated, which training examples influenced a particular token. Even the service providers — Anthropic, OpenAI, Google — face this challenge with their own models.

But we don't need to see inside.

We can judge the output. We can compare claims against evidence. We can detect when reasoning exceeds its basis. We can flag patterns that indicate drift.

This is what philosophers call the phenomenal approach: judge what appears, not what causes it. We can't read AI's mind. But we can read its work. And we can hold it to a standard.

That standard is TPMN — a notation with three prohibited reasoning patterns and seven evaluation dimensions. Not a guess about what the model "should" do. A formal specification of what the output must demonstrate.

Human at the edge, not in the loop

If AI is becoming an agent — not just a tool that responds, but a system that acts — then we need an accountability structure that matches.

Human in the loop means: review every output. Approve every action. The human is the bottleneck.

AI generates → Human reviews → Human approves → Output ships

This worked when AI outputs were occasional. It doesn't work when AI agents produce hundreds of outputs per day. The math:

200 outputs/day × 3 minutes each = 10 hours of review per agent
10 agents = 5 full-time reviewers
50 agents = your "safety net" costs more than the automation saves

Human at the edge means: define the standard. Let AI enforce it. Review exceptions.

AI generates → AI verifies (TPMN) → Passes? → Ships
                                   → Fails?  → Human reviews

The human doesn't disappear. The human moves to where they're most effective: defining what "honest reasoning" looks like, not reading every report.

This pattern already exists

Software engineering: Code passes through automated tests that humans defined. CI/CD enforces at scale. Humans review when tests fail. But what about AI-generated code itself — before it reaches the test suite?

Financial compliance: Transactions pass through compliance rules that humans wrote. Automated systems flag exceptions. Humans investigate the flags.

Manufacturing: Quality control systems catch defects using standards that humans set. Humans review edge cases and update standards.

AI output is the next domain where this pattern applies. And for developers specifically, there's an emerging practice pattern that makes this concrete — we'll get to that shortly.

The three requirements

1. A formal specification

Not heuristics. Not "does this look right?" A structured notation and grammar for what constitutes honest reasoning.

Three layers, one verification stack:

TPMN (Truth-Provenance Markup Notation) — the notation. Defines five epistemic claim states (⊢ ⊨ ⊬ ⊥ ?) and three prohibited reasoning patterns (SPT: snapshot→trend, local→global, thin→broad). What we mark.
TPMN-PSL (Prompt Specification Language) — the grammar. Compiles natural language prompts into verifiable specifications (MANDATEs). Defines the three-phase protocol (pre-flight, inline, post-flight) and three modes (strict, refine, interpolate). How we structure and verify.
TPMN Checker — the implementation. A Sovereign AI Service that runs the TPMN-PSL pipeline. 12 MCP tools. 6 domains. Returns a truth_score. What you install and use.

Analogous to HTTP (notation) → RFC 2616 (specification) → nginx (implementation). TPMN defines the rules. TPMN-PSL structures the protocol. The Checker enforces them.

Open. CC-BY 4.0. Anyone can implement it.

2. An isolated verification agent

Not a prompt. Not an inline check. A separate Sovereign AI Service whose only job is auditing.

TPMN Checker is the reference implementation of TPMN-PSL. It runs as an isolated MCP service — 12 tools, 6 domains, 7 evaluation dimensions. It judges output against contracts. It doesn't generate, advise, or assist. It audits.

3. Human calibration

If AI grades AI, the grading is circular. The system needs an external standard.

Human Ground Truth. When users disagree with a score, that disagreement becomes calibration data. Humans define what "honest reasoning" means. AI enforces it at scale.

Dogfooding: we verified the thesis behind this article

Before writing this post, we wrote down our raw thesis — the unfiltered thinking that drives everything above. Here's the core of it:

"All top-level AIs are trained to generate plausible results to satisfy humans. Hallucination is not a bug — it's a structural consequence of context drift. AI itself knows all the decision weights clearly. If we could make AI remind itself of the legitimate MANDATE area, AI could detect and fix results by itself. We validated this through various heuristic experiments over 20 months. No absolute truth score is possible. Human in the loop is nonsense."

Then we ran it through gem2_truth_filter.

Raw thesis: 18%. Our own tool scored our own thinking at the same level as unverified AI output. It caught three overclaims:

L→G: "All AIs are trained for plausibility" → universal claim without citing training documentation
S→T: "Hallucination is structural" → presented as permanent truth without distinguishing error types
Δe→∫de: "Validated through experiments" → claimed validation without methodology or data

We fixed each one. Scoped the claims. Added evidence. Qualified the assertions.

Cross-provider verification of the raw thesis:

Dimension	Claude	OpenAI	Gemini
Truth Score	18%	13%	25%
Source Attribution	0.10 ❌	0.08 ❌	0.10 ❌
Evidence Quality	0.30 ❌	0.18 ❌	0.20 ❌
Claim Grounding	0.20 ❌	0.20 ❌	0.30 ❌
Logical Consistency	0.70 ⚠️	0.68 ⚠️	0.50 ⚠️
Scope Accuracy	0.20 ❌	0.22 ❌	0.20 ❌
Extrapolation Risk	70%	88%	95%
SPT Violations	3	10	3

Three providers. All failed it. OpenAI was the harshest — 13% with 10 SPT violations. Gemini flagged 95% extrapolation risk.

We fixed each overclaim. Scoped the claims. Added evidence. Qualified the assertions.

Cross-provider verification of the fixed version:

Dimension	Claude	OpenAI	Gemini
Truth Score	59%	40%	90%
Source Attribution	0.90 ✅	0.28 ❌	0.85 ✅
Evidence Quality	0.70 ⚠️	0.50 ⚠️	0.90 ✅
Claim Grounding	0.60 ⚠️	0.58 ⚠️	0.95 ✅
Logical Consistency	0.80 ✅	0.82 ✅	0.95 ✅
Scope Accuracy	0.50 ⚠️	0.47 ⚠️	0.85 ✅

Three providers. Three different scores. But all three agree: the fixed version is dramatically better.

Gemini — the harshest critic of our raw thesis (95% extrapolation risk) — scored the refined version at 90%. Its explanation: "This content demonstrates excellent epistemic hygiene. The author explicitly bounds their claims to their own experience."

The scores differ. The diagnostic direction converges. That's cross-provider consensus in action.

Our raw thesis overclaimed — just like every unverified AI output. The tool caught it. We fixed it. This article is the refined version.

That's the loop: write → verify → fix → cross-verify → publish.

Try it on your own output

Step 1. Paste any AI output into your conversation.

Step 2. Ask: "Verify this by gem2 truth filter."

Step 3. Read the score. See what's grounded, what's extrapolated.

Step 4. Ask: "Create a grounded replacement prompt using gem2 contract writer."

Step 5. Ask AI to proceed with the new prompt. Watch what you get.

Your AI picks the right tool from 12 available MCP tools automatically.

Try it for free.

→ Get started at gemsquared.ai

What's next: Contract Coding

If "human at the edge" is the philosophy, what does it look like in practice — for developers writing code every day?

Three common patterns in AI-assisted coding:

Prompt coding   → you guide the model
Vibe coding     → you hope it works
Contract coding → AI defines the spec, AI verifies the output

In our next post, we'll show how TPMN Checker's existing tools — tpmn_contract_writer, tpmn_p_check (SDLC domain), and tpmn_p_check_compose — already support a workflow where AI generates formal specifications, produces code against them, and truth-filters the result before you ship.

Not for plausibility. For epistemic traceability.

Next in the series: "Contract Coding at the Edge: what comes after vibe coding" → (coming this week)

📺 Watch: Three AIs. Three Answers. None of them warned you.

📝 Read Post 1: We truth-filtered our own AI research

→ TPMN-PSL Specification (open, CC-BY 4.0)
→ GitHub
→ gemsquared.ai

TPMN-PSL is an open specification — not a product. If you believe AI outputs should be auditable, read the spec, open an issue, or submit a PR. The standard gets better when more people challenge it.

Three AIs analyzed our product. None passed the truth filter.

GEM² Inc. — Sat, 28 Mar 2026 15:59:12 +0000

What's hiding in your AI output? Now you can see it.

We asked three AI providers to research our own product.
Then we ran every output through our own truth filter.
The results surprised us.

📺 See how the truth filter works in practice: Three AIs. Three Answers. None of them warned you.

"TPMN Checker is not scoring writing quality. It is scoring epistemic traceability." — from the video at [0:40]

Korea AI 2027 forecast — what the three AIs reported

We asked each provider the same question: "Forecast Korea's AI industry for 2027."

	Claude	ChatGPT	Gemini
Market size (2027E)	₩4.46T (≈$3.3B)	₩4.46T (≈$3.3B)	$10–15B
CAGR	14.3%	~14%	>25%
Gov't AI investment	$71.5B	Ongoing ⚠️	$7B
Tone	Data-heavy, source-cited	Balanced, explicitly hedged	Bullish, narrative-driven

Three reports. All confident. Two even agree on the headline number. But agreeing on the answer doesn't mean agreeing on the truth.

(Note: Truth scores are not absolute judgments. They reflect the epistemic traceability ratio at the moment of evaluation — how much of the reasoning can be traced to evidence. That's why we're building the calibration standard together with users. Learn more about Human Ground Truth.)

Verification result by GEM² truth filter

Dimension	Claude	ChatGPT	Gemini
Truth Score	59%	21%	24%
Source Attribution	0.85 ✅	0.10 ❌	0.10 ❌
Evidence Quality	0.70 ⚠️	0.15 ❌	0.30 ❌
Claim Grounding	0.75 ⚠️	0.20 ❌	0.20 ❌
Logical Consistency	0.80 ✅	0.70 ⚠️	0.60 ⚠️
Scope Accuracy	0.65 ⚠️	0.40 ❌	0.30 ❌
Extrapolation Risk	40%	80%	80%
SPT Violations	2	3	3

Same question. Same filter. Three different levels of honesty.

The dogfooding experiment

We build TPMN Checker — a truth filter for AI reasoning. To prove the tool works, we pointed it at ourselves. Five rounds. Same task. Measurable improvement.

Task: "Write a comprehensive technical and market analysis of GEM²-AI and its TPMN Checker product."

Providers: Claude (Anthropic), ChatGPT (OpenAI), Gemini (Google)

Evaluation: Each output scored by gem2_truth_filter across seven dimensions:

Dimension	What it catches
Source Attribution	Claims with no traceable evidence
Evidence Quality	Thin or outdated supporting data
Claim Grounding	Assertions presented as fact without basis
Temporal Validity	Stale data treated as current
Scope Accuracy	Local findings overgeneralized
Logical Consistency	Internal contradictions
Prompt Alignment	Does the output match what was asked?

Round 1: What the AIs reported — standard prompt, no constraints

We gave each provider a straightforward research request with no special instructions about sourcing or evidence quality. Here's what they produced:

	Claude	ChatGPT	Gemini
Market size (TAM)	"~$0.45B in 2024" (cited IDC)	"~$0.45B in 2024" (cited "one report")	"$2.34B in 2024" (cited Grand View Research)
Growth rate	"~25% CAGR"	"~25% CAGR"	"21.6% CAGR to 2030"
Key differentiator	"genuinely novel position"	"formal verifiability"	"infrastructure for trustworthy AI ecosystem"
Competitor depth	Named 7 competitors with features	Named 8 competitors with pricing	Named 5 competitors with feature table
Risks identified	Solo founder, pre-revenue, academic skepticism	Early stage, niche complexity, unproven ROI	Early documentation, computational overhead
Uniqueness claim	"no commercial product today combines..."	"formal approach brings rigor unmatched by competitors"	"not just a debugging tool; infrastructure for resilient AI"

All three reports looked professional. Well-structured. Authoritative. The kind of output you'd confidently share with a stakeholder.

But we didn't share them. We verified them.

Round 2: Verification — GEM² truth filter exposes the gaps

We ran each report through gem2_truth_filter. Same tool, same criteria, same seven dimensions. All outputs evaluated using identical scoring logic across all providers.

Dimension	Claude	ChatGPT	Gemini
Truth Score	18%	28%	12%
Source Attribution	0.30 ❌	0.20 ❌	0.10 ❌
Evidence Quality	0.40 ⚠️	0.40 ❌	0.10 ❌
Claim Grounding	0.20 ❌	0.30 ❌	0.20 ❌
Logical Consistency	0.70 ⚠️	0.80 ✅	0.70 ⚠️
Scope Accuracy	0.20 ❌	0.50 ⚠️	0.20 ❌
Extrapolation Risk	80%	70%	90%
SPT Violations	3	3	3

Every provider failed. Not one scored above 30%.

What the filter caught

Invented precision. Market size figures like "$0.45B in 2024 with 25% CAGR to 2033" — attributed to "one analyst report" without naming the firm, methodology, or publication date.

Unsupported superlatives. "Genuinely novel," "genuinely unoccupied commercially," "the only product that..." — without exhaustive competitive evidence.

Snapshot-to-trend errors. Current market conditions presented as permanent structural realities.

The SPT taxonomy flagged three patterns across all providers:

S→T (Snapshot → Trend): treating current state as permanent identity
L→G (Local → Global): one data point generalized to universal claim
Δe→∫de (Thin → Broad): sweeping assertion from sparse evidence

These aren't hallucinations — the facts weren't always wrong. The reasoning was overclaimed. And no provider warned the reader.

Round 3: Improved prompt — generated by GEM² tools

Here's the key: we didn't write the improved prompt ourselves.

We simply asked: "Create a robust, grounded research prompt using gem2 tools."

That's it. We didn't engineer the prompt. The system did. No TPMN knowledge required. No specification reading. The AI picked the right tool from 12 available gem2 MCP tools — tpmn_contract_writer — and generated a prompt that enforced epistemic rules automatically.

The generated prompt included rules like:

Every quantitative claim must include source name, publication date, and URL
"One survey" or "one report" is not acceptable attribution
Claims must be tagged as grounded (⊢), inferred (⊨), or speculative (⊬)
Anti-patterns explicitly listed and prohibited
If data is unavailable, write "not available from verified sources" — don't invent

We verified the prompt itself with gem2_truth_filter before using it. The prompt scored 85%. Then we ran it through all three providers.

Round 4: Re-research — what the AIs reported with the grounded prompt

Same task. Same providers. Different prompt. Different results.

	Claude	ChatGPT	Gemini
Market size (TAM)	"Specific data not available from verified sources"	"~$0.45B (one report)" ⚠️	"$2.34B (Grand View Research, 2024)"
Growth rate	Not stated — insufficient evidence	"~25% CAGR" ⚠️	"21.6% CAGR (Grand View Research)"
Key differentiator	"Four observable features" — listed with sources	"Formal verifiability unmatched" ⚠️	"Granular truth state classification"
Claims tagged?	✅ Every claim marked ⊢, ⊨, or ⊬	❌ No epistemic tagging	Partial — some sections tagged
Limitations section?	✅ 7 specific gaps acknowledged	❌ Generic methodology note	✅ Listed 4 limitations
Unsourced numbers?	0 — wrote "not available" instead	Multiple — "92% of Fortune 500" without source	Some — market figures cited, incident costs not

The difference was visible immediately. One provider followed every rule. The others improved but couldn't fully resist the instinct to fill gaps with confident-sounding assertions.

Round 5: Re-verification — truth filter confirms the improvement

We ran all three re-researched outputs through the same truth filter.

Dimension	Claude	ChatGPT	Gemini
Truth Score	77%	~48%	~35%
Source Attribution	0.90 ✅	0.10 ❌	0.60 ⚠️
Evidence Quality	0.85 ✅	0.20 ❌	0.30 ❌
Claim Grounding	0.90 ✅	0.30 ❌	0.40 ⚠️
Logical Consistency	0.90 ✅	0.90 ✅	0.80 ✅
Scope Accuracy	0.85 ✅	0.40 ⚠️	0.50 ⚠️
SPT Violations	0	3	4

The improvement, measured

Provider	Round 2 (before)	Round 5 (after)	Improvement
Claude	18%	77%	+59 points
ChatGPT	28%	~48%	+20 points
Gemini	12%	~35%	+23 points

Every provider improved. Structured epistemic instructions produce measurably more reliable output. This isn't theory — it's six verified data points from the same tool, same criteria, same task.

What the data shows

The prompt improved every provider — but couldn't fix the instinct

Even with explicit anti-patterns listed — "PROHIBITED: citing 'one report' without naming it" — two out of three providers did it anyway.

The generated prompt said: "If you cannot provide source name, date, and URL, write 'data not available from verified sources' instead."

One provider wrote "data not available." The other two invented attributions.

The prompt improved the scores. It couldn't fix the instinct to overclaim.

This isn't a writing quality score

This was one of the most important findings — and the core message of our video. All three providers produced well-written, logically coherent reports. Logical Consistency scored 0.70–0.90 across the board — even in the reports that scored 12% overall.

The reports that scored lowest were the best-written ones. Polished, authoritative, structured — and epistemically unreliable.

TPMN Checker measures something different: not whether the output sounds right, but whether the reasoning is traceable. Can the AI prove how it got there?

That's epistemic traceability. It's what separates trustworthy output from confident output.

Why this matters

Every person reading this has shipped AI-generated content — a report, a summary, an analysis, a PRD, a code review. Some of that content contained overclaims you didn't catch. Not because the facts were wrong, but because the reasoning exceeded the evidence.

That's the gap TPMN Checker fills.

It's not a hallucination detector (those check facts). It's not a grammar checker (those check writing). It's a reasoning traceability tool — it tells you which parts of your AI output are grounded, which are inferred, and which are extrapolated beyond the evidence.

AI audits AI. But the standard comes from humans.

The truth filter is powered by AI. It uses LLMs to evaluate LLM output. That creates a circular problem: who grades the grader?

Our answer — same as in the video at [1:03]: you do.

When you use TPMN Checker and disagree with a score, that disagreement is data. Collected with consent, aggregated across users, and analyzed for patterns — your evaluations become the ground truth that calibrates the system.

We call this Human Ground Truth. AI processes. AI suggests. But the standard for what counts as honest reasoning — that comes from humans.

Try it yourself

TPMN Checker runs today inside Claude, ChatGPT, Cursor, and any MCP-compatible environment.

Connect (once)

Claude.ai or ChatGPT — zero install:

Go to your AI tool's connector/app settings
Add custom connector: https://mcp-tpmn-checker.gemsquared.ai
Complete OAuth login

CLI:

npx @gem_squared/setup

Standard use case — the pattern that works

You probably already have AI-generated content sitting in a doc right now — a research summary, a PRD, a financial analysis, a code review. Here's what to do with it:

Step 1. Paste your AI output into the conversation.

Step 2. Ask: "Verify this by gem2 truth filter."

Step 3. Read the score. See which claims are grounded, which are extrapolated, which have no source.

Step 4. Ask: "Create a grounded replacement prompt using gem2 contract writer."

Step 5. Ask AI to proceed with the new prompt. Watch what you get.

That's the loop: verify → ground → regenerate. The same loop that took our research from 18% to 77%.

Try it for free.

→ Get started at gemsquared.ai

The specification is open

TPMN-PSL (Truth-Provenance Markup Notation — Prompt Specification Language) is the open specification behind the checker. It's released under CC-BY 4.0. Anyone can read it, implement it, or extend it.

The specification defines:

Five epistemic tags (⊢ grounded, ⊨ inferred, ⊬ extrapolated, ⊥ unknown, ? speculative)
Three prohibited reasoning patterns (SPT: snapshot→trend, local→global, thin→broad)
Three-phase verification protocol (pre-flight, inline, post-flight)
Three operational modes (strict, refine, interpolate)

→ Read the specification
→ GitHub repository

What we learned from dogfooding

Five rounds of testing our own tool on our own AI research taught us three things:

1. No provider in our test was inherently honest. Claude — the provider our tool runs on — scored 18% without epistemic constraints. Every provider overclaimed when unconstrained. The difference is the specification, not the model.

2. Structured prompts produce measurably better output. A 59-point improvement from the same provider on the same task, just by using a gem2-generated prompt. That's not marginal — that's transformational. And you don't need to understand the specification to use it — just ask your AI to create a grounded prompt with gem2 tools.

3. The instinct to overclaim persists. Even with explicit instructions to avoid unsupported claims, two out of three providers violated the rules. The prompt helps. The prompt isn't enough. That's why verification exists as a separate step — because you can't trust the AI to police itself, no matter how well you prompt it.

The question isn't whether the answer is right or wrong. It's whether the reasoning is honest.

As we say in the video at [0:57]: "So, who decides what's true? Not Claude. Not ChatGPT. Not Gemini. You do."

What's hiding in your AI output? Now you can see it.

TPMN Checker is in pre-GA. 12 MCP tools, 6 domains, 3 providers. Free to start.

Built by Inseok Seo (David) — GEM²-AI

→ gemsquared.ai
→ Watch: Three AIs. Three Answers.
→ TPMN-PSL Specification
→ GitHub