DEV Community: Jon Scott

I Built a Math AI That Verifies Its Answers. Then I Made It Free.

Jon Scott — Mon, 26 Oct 2026 02:30:53 +0000

What if you built a badass math AI, gave it a full simulation lab and graphing tools, and then made it all free?

Well, I did.

Meet Pythos.

I started building Pythos because I wanted to tackle a problem with AI and mathematics: language models can explain concepts beautifully, but they can also make algebra mistakes while sounding completely confident.

For students trying to learn, that matters. A convincing explanation isn't worth much if the answer is wrong.

So I started building something different.

More than another math chatbot

Pythos combines an AI math tutor with deterministic verification engines designed to independently check mathematical claims.

The idea is straightforward: let the AI handle explanations and tutoring, while dedicated mathematical tools verify the claims they can formally prove.

Depending on the problem, that can involve symbolic computation, algebraic checks, domain restrictions, and other deterministic methods.

When a claim can't be verified, the system needs to distinguish that from a confirmed result. That's a core engineering challenge I'm continuing to work on.

The learning tools matter, too

I didn't want to build a box that spits out answers and calls itself an educational platform.

Pythos also brings together graphing tools and a simulation lab, giving students ways to explore mathematical ideas and experiment with problems instead of simply copying solutions.

The bigger goal is to make it useful as a learning environment, not just an answer generator.

Why make it free?

Because students already have enough barriers to education.

I want people to be able to use these tools without another subscription standing between them and learning something new.

Pythos is part of my work under LANZAR. It's an ongoing project, and I'm continuing to improve the verification pipeline, tutoring behavior, and overall experience.

I'm not claiming it's finished or infallible. Building a system that can reliably distinguish a correct answer from a plausible-looking mistake is an ongoing engineering problem—and one I think is worth tackling.

Try it yourself

Pythos: https://pythos.lanzar.me

I'd genuinely like feedback from developers, math enthusiasts, students, and educators.

Try a difficult problem. Test the explanations. Look for edge cases. If something breaks, I want to know about it.

I'm especially interested in feedback on deterministic verification, how AI tutoring should work, and what features would make this more useful in a real classroom.

I built it because I thought math education tools could be better.

Now I want to see what people do with it.

Pythos Oracle Is Live: Building AI Math and Physics Tutoring Around Deterministic Verification

Jon Scott — Sat, 10 Oct 2026 16:56:45 +0000

Pythos Oracle Is Live: Building AI Math and Physics Tutoring Around Deterministic Verification

I've been building something I want to share with the developer community.

Pythos Oracle is live: pythos.lanzar.me

The idea behind Pythos comes from a problem I've been thinking about for a while: large language models can produce convincing mathematical explanations, but a convincing explanation doesn't necessarily mean the answer is correct.

What happens when we stop treating an AI-generated answer as the final authority and start building systems that independently verify the mathematics?

That's the engineering problem I'm exploring with Pythos.

The Core Idea

Rather than relying exclusively on a language model to generate and validate its own mathematical output, Pythos uses a verification-oriented approach that separates language-model capabilities from deterministic mathematical checks.

The language model provides the conversational interface and helps explain concepts. Deterministic tools can independently evaluate supported calculations and mathematical claims.

The distinction matters:

Generating an answer and establishing that it's correct are two different operations.

I've written more about the thinking behind this approach in my earlier article:

Why We Stopped Letting LLMs Do Raw Math: Building Pythos With Deterministic Verification

Why Build This?

I believe we're in the middle of a technological revolution, but public discussion about AI is often stuck between two extremes: treating it as magic or treating it as an existential threat.

Neither approach helps us build better systems.

AI has real capabilities, real limitations, and real infrastructure costs. We should understand those limitations and engineer around them wherever possible.

For mathematics and physics education, reliability matters. Students need more than a confident answer; they need useful explanations and ways to check the underlying work.

That's the direction I'm pursuing with Pythos Oracle: making STEM learning more accessible while exploring how deterministic verification can improve the reliability of AI-assisted problem solving.

This Is an Ongoing Engineering Project

I'm not claiming to have eliminated AI hallucinations or solved mathematical verification for every possible problem.

Verification has a defined scope, and the system needs continued testing against difficult problems, edge cases, and unsupported inputs.

That's part of the work.

I'd rather identify the limits, test them, and improve the implementation than make claims the evidence can't support.

I'd Like Your Feedback

If you're interested in AI engineering, mathematics, physics, or educational technology, take a look:

Live project: Pythos Oracle

I'm particularly interested in hearing from developers who have experimented with deterministic verification, symbolic mathematics, or hybrid AI architectures.

What kinds of mathematical problems would you use to stress-test a system like this?
Where do you think deterministic verification provides the most value?
Where does it become difficult to apply?
What would you want to see in an AI-powered STEM learning tool?

I'm building Pythos to be useful, not just impressive in a demo.

Honest technical feedback is welcome.

From Dev.to Comment to Production in 24h: Building an Inspectable Math Verification Contract in Pythos (and Fixing the Pearson Trap)

Jon Scott — Wed, 07 Oct 2026 03:24:55 +0000

Earlier this week, I published an article here on DEV: "Why We Stopped Letting LLMs Do Raw Math: Building Pythos With Deterministic Verification".

The response was humbling, but one particular comment from an enterprise AI engineer at IT Path Solutions stood out. They validated our core thesis—treat the LLM as a conversational interface, never the mathematical source of truth—and left a game-changing suggestion:

"Make the verification result part of the data contract... each step could carry the expression evaluated, the verification method, the assumptions used, and the exact state that was checked."

They were 100% right.

In Pythos, our dual-engine verifier (in-process Math.js for exact arithmetic and a sandboxed SymPy CAS daemon for symbolic algebra/calculus) was already validating derivations step-by-step behind the scenes. But once the delivery decision gate passed, we were condensing all that rich telemetry into simple binary flags before shipping the text to the client.

We decided to build what they suggested.

Here is how we turned mathematical verification into an explicit, wire-level data contract, built a client-side derivation inspector, and—in a hilarious twist of real-world battle testing—fixed a production bug where Pearson homework boilerplate tricked our deterministic router into computing the area of a right triangle.

1. The Architecture: The `VerificationStepAudit` Data Contract

Prior to v1.8.35, Pythos knew internally whether a claim was true or false, but the frontend only received markdown text with basic metadata.

We formalized a new data contract: VerificationStepAudit. Every evaluated claim emitted over our streaming NDJSON and JSON endpoints now carries structured audit telemetry:

interface VerificationStepAudit {
  stepIndex: number;
  expressionEvaluated: string;
  verificationMethod: 'SYMPY_CAS' | 'MATHJS_EXACT' | 'DETERMINISTIC_FASTPATH';
  engine: 'sympy' | 'mathjs' | 'native';
  assumptions: {
    domain?: 'real' | 'complex' | 'integer';
    tolerance?: number;
    quadrant?: string;
  };
  checkedState: {
    proposed: string;
    expected: string;
    discrepancy?: string;
  };
  certified: boolean;
  rejectionRationale?: string;
}

Why This Matters for AI Tutoring

Zero Black-Box Math: The student (and educator) isn't just told "Trust me, the answer is $x = \frac{1}{2}$." They can inspect the actual symbolic reduction that certified the step.
Targeted Socratic Revision: When an LLM generates a multi-step derivation and makes a sign error on Step 4, we don't ask it to "try again" blindly. The verifier catches the exact step failure and feeds the structured VerificationStepAudit back into the Socratic revision prompt: > "Step 4 failed: evaluated $\sin^2\theta + \cos^2\theta = 2$, expected $1$. Discrepancy: Pythagorean identity violation." The model corrects itself with mathematical precision.

2. The UX: The Derivation Inspector

Exposing verification metadata without cluttering the learning experience is tricky. If you dump raw JSON or giant debug banners on students, it overwhelms them.

Instead, we added a subtle, accessible badge on the bottom-right action bar of assistant messages, sitting cleanly alongside the Report and Copy buttons:

[ 🛡️ Verified (3 steps) ]  [ 📋 Copy ]  [ 🐞 Report ]

Clicking Verified expands a slide-out Derivation Inspector panel directly underneath the response:

Highlights each individual derivation step.
Displays the exact mathematical expression tested.
Shows the certified CAS engine badge (SymPy CAS or Math.js).
Confirms the active assumptions (e.g., $x > 0$, real domain).

Now, students aren't just reading AI generated text; they can audit the mathematical proof behind it in real time.

3. The Multimodal Polish: Magic Bytes & Vision History

Alongside the audit contract, we upgraded our multimodal screenshot ingestion pipeline. Students frequently screenshot math worksheets and paste them directly into Pythos.

We ran into two edge cases with upstream vision models (Llama 3.2 Vision on Groq):

MIME-Type Rejections (400 Bad Request): Some browsers generate clipboard image blobs with generic or missing MIME headers. We implemented client-side magic-byte sniffing (inspecting PNG \x89PNG, JPEG \xFF\xD8\xFF, and WebP headers) to guarantee strict Content-Type tagging.
Vision TPM Throttling (429 Rate Limit): In a multi-turn conversation, re-transmitting multi-megabyte base64 images on every turn quickly incinerates Token-Per-Minute (TPM) limits on free-tier inference gateways. We bounded conversational image history so previous turns retain verified OCR transcripts while releasing redundant base64 payloads once grounded.

4. The "Pearson Trap": When Homework Boilerplate Met Deterministic Routing

No deployment story is complete without a hilarious production edge case.

Shortly after releasing v1.8.35, I was using Pythos myself to work through college trigonometry homework from Pearson MyLab. I pasted a problem:

Use a right triangle to write the following expression as an algebraic expression. 
Assume that x is positive and that the given inverse trigonometric function is defined for the expression in x.
tangent left parenthesis cosine Superscript negative 1 Baseline 4 x right parenthesis
Question content area bottom
Part 1
tangent left parenthesis cosine Superscript negative 1 Baseline 4 x right parenthesisequals

I expected Pythos to derive:
$$\tan(\cos^{-1}(4x)) = \frac{\sqrt{1 - 16x^2}}{4x}$$

Instead, Pythos instantly and cheerfully responded:

📐 Right-Triangle Area

Here is the right triangle with base 3 and height 4:

$\text{Area} = \frac{1}{2} \times \text{base} \times \text{height} = \frac{1}{2}(3)(4) = 6$

Wait... what? Where did a 3-4-5 triangle and an area of 6 come from?!

The Root Cause

To keep latency near zero and save GPU compute costs, Pythos features an in-process Deterministic Router that short-circuits simple calculations (arithmetic, unit conversions, geometry requests) before invoking the LLM.

Two bugs in the router had collided:

Screen-Reader Web Boilerplate: Pearson MyLab outputs accessibility tags including the phrase: > "Question content **area* bottom"*
Overly Eager Regex: The router's geometry parser checked for right triangle queries, followed by:

   const isArea = /\barea\b/i.test(clean);

It matched the word "area" in "Question content area bottom"!

The Dummy Fallback: Because the prompt asked to "Use a right triangle..." to solve an algebraic expression, no numerical leg lengths were given. The legacy geometry routine fell back to a default 3, 4, 5 triangle and happily computed $\frac{1}{2} \times 3 \times 4 = 6$.

The Hardened Fix

We patched server/deterministicRouter.js immediately:

Added an isAlgebraicOrInverseTrig guard that completely blocks prompts containing algebraic keywords (algebraic expression, simplify, expand) or inverse trig operators (cos^-1, arcsin, StartFraction, Superscript negative 1) from ever being hijacked as static geometry drawings.
Sanitized web platform boilerplate (question content area, content area bottom) before testing for geometric intents.
Required geometric area calculations to have explicit numerical side lengths and dedicated question phrasing (e.g., "find the area").

5. Adding It to the Master Regression Suite

A bug once discovered must never happen again.

We created Group G in our fast-path hardening suite (test-deterministic-fastpath-hardening.js) with tests specifically checking Pearson screen-reader transcripts, algebraic inverse trig phrases, and fractional boilerplate:

runTest('G1: Pearson screen-reader inverse trig prompt must not trigger dummy triangle area', () => {
  const prompt = 'Use a right triangle to write... tangent left parenthesis cosine Superscript negative 1 Baseline 4 x right parenthesis Question content area bottom...';
  const intent = deterministicRouter.analyzeDeterministicIntent(prompt, []);
  assert.strictEqual(intent, null); // Must pass through to LLM + CAS verification!
});

We then promoted this into Suite 25 of our master regression test harness (test-regression-harness.js).

All 25 regression suites—covering SymPy CAS, Math.js adversarial arithmetic, student privacy redaction, streaming verification, and fast-path hardening—are passing 100% green.

Summary

In less than 24 hours:

A great comment from the DEV community gave us the exact conceptual blueprint we needed.
We turned verification into a first-class, inspectable VerificationStepAudit data contract.
We added an accessible client-side UI inspector to prove mathematical derivations to students.
We caught and eliminated a wild edge-case where Pearson homework boilerplate masqueraded as a geometry problem.

Huge thanks to the DEV community and our commenter from IT Path Solutions for pushing us to make Pythos better.

You can try out Pythos for free anytime at pythos.lanzar.me or explore the code and verification engine on GitHub.

What are your thoughts on exposing deterministic verifier contracts directly in AI chat interfaces? Would you find step-level mathematical receipts helpful when studying or building math tools? Let's discuss in the comments!

Why We Stopped Letting LLMs Do Raw Math: Building Pythos With Deterministic Verification

Jon Scott — Tue, 06 Oct 2026 03:26:17 +0000

Large Language Models are incredible at conceptual analogies, Socratic dialogue, and breaking down complex ideas.

But when it comes to raw mathematics and physics derivations, they are notoriously unreliable calculators.

They drop negative signs, fabricate intermediate arithmetic steps, and deliver confidently incorrect answers with total poise. In creative writing, a hallucination is a quirk; in mathematics and physics education, it completely derails a student's confidence and understanding.

To solve this, I built Pythos—a free, open-access AI math and physics tutor engineered with a fundamentally different philosophy: never let the language model deliver unchecked math.

The Problem: The Confidence Trap in STEM Chatbots

Most AI tutoring tools take a student’s math problem, pass it to an LLM, and stream the generated response directly to the screen.

When the model makes an algebraic error in Step 3 of a 6-step calculus integration, the final result is wrong. If the student questions it, the model often apologizes, scrambles its numbers, and hallucinates a second, equally flawed derivation.

Instead of maximizing model "confidence," we asked a different question:

What if an AI tutor was designed with a strict deterministic verification gate that checks derivations before outputting them—and withholds answers if it cannot prove them?

The Architecture: The Deterministic Verification Gate

In Pythos, the AI does not operate in a vacuum. Instead, we split the reasoning process into pedagogical explanation and deterministic execution:

Student Input ➔ Sent to LLM for pedagogical reasoning
LLM Formulates Steps ➔ Generates explanation & step derivations
Deterministic CAS Gate ➔ Math engine validates each algebraic step
Automated Audit ➔ Checks signs, values, and consistency
Verification Gate ➔ If verified, delivers to student; if unverified, withholds and reroutes!

Socratic Pedagogical Layer: The language model breaks the problem into guiding steps and conceptual intuition rather than just spitting out a single answer.
Deterministic Calculation Safeguard: Mathematical claims, factorizations, limits, and algebraic operations are verified through deterministic calculation kernels.
The "Refusal" Safeguard: If a mathematical step fails verification or cannot be guaranteed with high confidence, Pythos is programmed to withhold the response rather than guessing.

Proving It: 110,000+ Problem Validation Record

To measure reliability, we ran multi-tier benchmark campaigns testing tens of thousands of real exam and homework problems across algebra, calculus, and classical mechanics.

You can inspect our public mathematical validation record directly here:

👉 Pythos Validation Benchmark

By pairing language model reasoning with strict algorithmic verification, the pipeline eliminates mathematical hallucinations on verified paths, achieving a 0% incorrect delivery rate across our validation sets by safely withholding unverified steps.

Interactive Physics & Math Instruments

Math and physics shouldn't just be static text. We also built real-time, interactive visual instruments right into the tutoring canvas:

2D Projectile Motion Simulator: Interactive PhET-style ballistics canvas allowing students to adjust launch angle, velocity, and gravity sliders with live trajectory arcs and calculated range readouts.
Right Triangle & Trig Inspector: Live geometric recalculations showing exact trigonometric ratios and step-by-step Pythagorean derivations.
Function Grapher: Real-time 2D plotting engine constrained for fast, touch-accessible mobile exploration.

100% Free for Students and Educators

Pythos is non-profit, ad-free, and requires no paywalls or subscriptions. It was built by a student for students.

🌐 Try Pythos Live: pythos.lanzar.me
📚 Study Guides: Algebra | Calculus | Physics
💻 Open Source Repository: github.com/JonScott79/Pythos

I would love to hear feedback from other developers and educators: How are you handling hallucination boundaries and deterministic validation in your AI applications?

DEV Community: Jon Scott

I Built a Math AI That Verifies Its Answers. Then I Made It Free.

What if you built a badass math AI, gave it a full simulation lab and graphing tools, and then made it all free?

More than another math chatbot

The learning tools matter, too

Why make it free?

Try it yourself

Pythos Oracle Is Live: Building AI Math and Physics Tutoring Around Deterministic Verification

Pythos Oracle Is Live: Building AI Math and Physics Tutoring Around Deterministic Verification

The Core Idea

Why Build This?

This Is an Ongoing Engineering Project

I'd Like Your Feedback

From Dev.to Comment to Production in 24h: Building an Inspectable Math Verification Contract in Pythos (and Fixing the Pearson Trap)

1. The Architecture: The VerificationStepAudit Data Contract

Why This Matters for AI Tutoring

2. The UX: The Derivation Inspector

3. The Multimodal Polish: Magic Bytes & Vision History

4. The "Pearson Trap": When Homework Boilerplate Met Deterministic Routing

The Root Cause

The Hardened Fix

5. Adding It to the Master Regression Suite

Summary

Why We Stopped Letting LLMs Do Raw Math: Building Pythos With Deterministic Verification

The Problem: The Confidence Trap in STEM Chatbots

The Architecture: The Deterministic Verification Gate

Proving It: 110,000+ Problem Validation Record

Interactive Physics & Math Instruments

100% Free for Students and Educators

1. The Architecture: The `VerificationStepAudit` Data Contract