DEV Community

Cover image for AI Didn’t Solve a Physics Problem. It Remembered One.
Simon Paxton
Simon Paxton

Posted on • Originally published at novaknown.com

AI Didn’t Solve a Physics Problem. It Remembered One.

Yuji Tachikawa's X thread: he showed Claude Fable a six-month-stalled physics problem; it flagged a calculation error, expanded the approach and solved it, verified with SymPy, and he reflects that understanding may be an illusion.

Yuji Tachikawa (@yujitach) describing the episode on X, July 12, 2026 — translated from Japanese. This thread is the primary source for the article.

Claude Fable likely recalled and recombined known methods in the Yuji Tachikawa episode, not verifiably produced a new physics result. The strongest public evidence as of July 13, 2026 is still a viral social-media retelling and secondary reporting, not a paper, preprint, or independent replication.

Yuji Tachikawa is a professor at the Kavli Institute for the Physics and Mathematics of the Universe at the University of Tokyo, working in string theory and quantum field theory. That matters because this was not a random user praising a chatbot; it was a world-class specialist saying a frontier model helped on a problem his group had been stuck on.

Tachikawa’s reported account was that he showed Claude Fable research notes, the model pointed out a derivation mistake, suggested a different route, and wrote SymPy code to check the expression. Useful does not equal verified original reasoning. A model trained on a broad corpus of scientific text and code is built to do exactly that sort of recall, recombination, and tool use.

What Tachikawa actually claimed

The public claim, as it spread, was that Claude had “essentially solved” a physics problem after researchers had been stuck for months, based on Tachikawa’s own social-media description as reproduced in viral posts and screenshots. What is missing is the part that would make this a scientific result: there is still no linked paper, no preprint, no full derivation, and no independent verification in the public record tied to the viral story.

That gap is the whole story. A theoretical physicist saying a model was helpful is evidence that the tool may be valuable. It is not evidence, by itself, that the model generated a novel result outside its training distribution.

This is the same basic caution that applies in smaller failures too. In ArXiv hallucinated papers, the problem was fabricated references rather than an impressive derivation, but the lesson is similar: an LLM output is not self-validating just because it looks fluent and domain-specific.

Anthropic itself describes Claude Fable 5 as a system built for advanced reasoning, coding, and tool use, and frontier labs broadly use that language. The language is real in the sense that it reflects product positioning and some measurable performance gains, but it is also incentive-laden marketing language, not a settled scientific verdict about what is happening internally.

Anthropic presents Claude Fable 5 as its most capable model for advanced reasoning and coding, with strong tool use, in its own launch materials.

Why this looks like recall and recombination, not verified novel reasoning

A language model can help on a hard derivation without “understanding physics” in the human sense because the task can be broken into patterns the model has seen many times before.

First, spotting a derivation mistake does not require a human-style internal grasp of field theory. In a huge corpus of papers, lecture notes, textbooks, Stack Exchange posts, and code notebooks, wrong symbolic steps are often followed by the kinds of corrections experts make. The model can learn those local correction patterns and rank likely fixes. That is more sophisticated than lookup, but it is still consistent with statistical pattern use rather than original physical insight.

Second, suggesting a different approach can also be recall plus recombination. A specialist can get trapped in one framing; a model with broad exposure to neighboring subfields can surface a standard alternative technique that the human team was not currently reaching for. Research on science-focused evaluation suggests LLMs can be good at retrieving inspirations and making new associations between known ideas, even when contamination is actively controlled. That is impressive. It still is not the same thing as proving the system created a genuinely new physics concept.

Third, writing SymPy verification code is one of the least mysterious parts of the episode. SymPy is a widely used symbolic-math library with abundant examples online and in public repositories. Turning a symbolic derivation into executable checks is a common code-generation task for frontier models, especially those explicitly optimized for coding and tool use, as Anthropic claims for Claude Fable 5.

The cleanest way to think about this is:

  • Error spotting: likely pattern-matching over familiar derivation structures.
  • Alternative method: likely retrieval of a known technique from adjacent literature.
  • SymPy check: likely code synthesis over a well-represented library and workflow.
  • Scientific novelty: not established by any of the above alone.

How this might work, concretelyan illustration, not Tachikawa’s actual (unpublished) problem.

1. Spotting the error. Suppose the notes evaluate a standard Gaussian integral and drop a factor:

∫ e−ax² dx = √(π / 2a)  ←  wrong

The correct identity, √(π / a), appears in thousands of textbooks, papers, and homework sets. A model that has read them doesn’t need to understand the integral — the corrected line is simply the overwhelmingly likely continuation of the wrong one. It flags the missing factor the way autocomplete finishes a familiar sentence.

2. Suggesting another route. Say the team has been grinding a sum term by term and stalling. A model with broad exposure can surface a standard move from an adjacent corner of the literature — “that sum is a known generating function, close it in one step,” or “impose the symmetry and most terms cancel.” That is not invention. It is retrieving a technique the humans, fixated on one path, hadn’t reached for.

3. Checking with code. Then it writes a few lines of SymPy to test its own claim symbolically:

from sympy import symbols, integrate, exp, sqrt, pi, simplify, oo
x, a = symbols('x a', positive=True)
lhs = integrate(exp(-a*x**2), (x, -oo, oo))
print(simplify(lhs - sqrt(pi/a)))   # -> 0, the identity holds

SymPy is everywhere in public code, so generating this is a well-worn task. Notice what each step needed: pattern-completion, retrieval, and code synthesis over material the model had already seen. At no point did it have to grasp the physics. That is exactly why a machine can be genuinely useful here without doing anything that clears the bar for original reasoning.

A world expert can still be out-recalled here. Tachikawa has deep mastery in a narrow region of theory; a frontier model has shallow exposure to a vast amount of mathematical physics writing and symbolic code. Breadth beats depth on recall tasks. That is not embarrassing for the scientist. It is the expected outcome when one side is a person and the other is a machine trained on enormous text and code corpora.

There is also a long-running benchmark problem here. Papers on contamination have shown that apparent reasoning gains can be inflated when evaluation material overlaps with training data or with near-duplicates of it. A 2024 NAACL paper on benchmark contamination documented how contamination complicates capability claims, and a 2024 survey summarized the broader problem. A 2026 EACL Findings paper went further, arguing that in some settings 25% to 50% of evaluation datasets appeared in training corpora, which blurs memorization and reasoning even more.

That does not prove Claude memorized this answer. The stronger and more defensible claim is recombination of techniques and patterns drawn from prior scientific literature and code examples. On the other side, it is also true that some newer work finds real generalization improvements on code-reasoning tasks; one 2025 study argues newer models outperform simple pattern-matching expectations on parts of code reasoning. So the honest position is not “all reasoning is fake.” It is that this specific episode does not supply the evidence needed to call the result verified original physics reasoning.

That is also why headlines about AI cracking famous open problems should be read carefully. In AI and the Erdős problem, the interesting question was not whether the output sounded clever, but whether it survived expert scrutiny as something genuinely new.

What would count as genuine novelty

A real claim of original physics reasoning would need evidence that is missing here.

Requirement What would satisfy it
Public artifact A paper, preprint, notebook, or full derivation showing the model’s actual contribution
Independent check Verification by other physicists who were not involved
Novelty test Evidence the key move was not a standard method already present in the literature
Distribution test A case that the result was outside what the model plausibly absorbed from training text and code
Robustness Reproducing similar results across multiple fresh problems, not one anecdote

Human researchers also remain more robust than LLMs on many out-of-distribution reasoning tasks. A 2022 benchmark study found humans were still more flexible when problems changed in ways that broke familiar patterns. That is exactly the kind of bar an “AI discovered new physics” claim should have to clear.

(adsbygoogle = window.adsbygoogle || []).push({});

There is a more modest claim that already seems plausible: tool-using LLMs can be genuinely useful scientific assistants. Anthropic’s own Economic Index research argues that users employ Claude on harder knowledge work, not just boilerplate writing. A system that recalls relevant methods, translates derivations into executable SymPy, and catches likely algebraic slips can save experts real time even if it does not possess human-like understanding.

That is enough to matter. It is just not the same as autonomous scientific discovery.

The next thing worth watching is simple: whether Tachikawa or collaborators publish the problem, the model’s exact input and output, and an independent verification of what Claude contributed. Until then, the Tachikawa-Claude episode is best read as evidence of powerful recall-and-recombination with tools, not a verified case of AI doing new physics.

Key Takeaways

  • The best-supported reading is that Claude recalled and recombined known techniques, rather than verifiably producing a novel physics result.
  • As of July 13, 2026, the story still rests on social-media posts and screenshots, not a paper, preprint, or independent replication.
  • Spotting a derivation mistake, suggesting an alternative method, and writing SymPy checks are all tasks a literature-trained coding model can plausibly do without human-like physical understanding.
  • A world expert can be out-recalled by a model with broad exposure to scientific literature and code, especially when the expert is stuck in one framing.
  • Calling an LLM’s contribution genuine new physics would require a public derivation, independent verification, and evidence that the key move was outside familiar methods in the training distribution.

Further Reading

Frequently Asked Questions

Did Claude solve a new physics problem?

There is no verified public evidence that it did. The public story is still a social-media account saying Claude helped by catching a mistake, suggesting an approach, and writing a symbolic check, but there is no paper or independent verification attached to the viral claim.

Why can an LLM help a physicist without understanding physics?

Because much of the visible work can be framed as pattern use over a vast literature and code base. If a model has absorbed common derivation structures, standard alternative methods, and many examples of SymPy workflows, it can often produce a useful next step without possessing the kind of conceptual understanding a human physicist has.

Could this still involve real reasoning?

Possibly, in a limited sense, but this episode does not prove it. Some recent studies report better generalization on code-reasoning tasks in newer models, so it would be too strong to say all apparent reasoning is fake. The narrower claim is that this anecdote does not establish verified original physics reasoning.

Why would a top expert miss something a model finds?

Because expertise and recall are different strengths. A specialist has deep understanding but finite memory and can get locked into one line of attack; a frontier model has broad, shallow coverage over enormous amounts of adjacent material and can surface a familiar method quickly.

References

Last reviewed: 2026-07


Originally published on novaknown.com

Top comments (0)