DEV Community

Cover image for Can AI Get Better Without Getting Bigger? Meet Test-Time Compute
Rijul Rajesh
Rijul Rajesh

Posted on

Can AI Get Better Without Getting Bigger? Meet Test-Time Compute

Hello, I'm Rijul, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.


Usually, when we intend to upgrade our AI, we think about improving the model size.

This is called training-time compute.

But there is another scaling factor that can be used to scale AI.

This is called test-time compute.

What Is Test-Time Compute?

Test-time compute means spending extra computation while answering a question, instead of doing all the computation during training.

Usually, an LLM generates its answer in a fast, sequential process: read the prompt, predict the next token, predict the next token, and so on.

Test-time compute means that before locking in the final answer, we let the model "think" by generating intermediate reasoning, exploring multiple paths, checking its own work, and doing additional computation. This extra work happens live, at the moment you ask the question, hence "test time," as opposed to "training time."

What Does Thinking Longer Look Like?

These are some of the mechanisms that can take place during thinking.

Chain-of-thought reasoning:

  • Instead of jumping straight to the answer, the model generates a series of intermediate reasoning steps.
  • This takes time, but can produce better results, especially on math and logic-related problems.

Generating multiple candidate answers, then picking the best one:

  • In this mechanism, the model generates multiple attempts.
  • It then uses a scoring mechanism to choose the best one.

Self-verification/self-correction:

  • The model generates an answer, then checks its own reasoning for mistakes and revises it if needed.

Search over reasoning paths:

  • More advanced setups treat reasoning like a tree.
  • The model explores several possible next steps, evaluates which ones look promising, and can backtrack from dead ends.

Wrapping Up

Test-time compute comes with a trade-off.

The first one is speed. If the model spends more time thinking, the response naturally takes longer. Reasoning models can feel noticeably slower than a normal chatbot.

Then there is cost. More thinking means more generated tokens and more computation, which can increase the API cost of each query. The reasoning can be quite long, even when it isn't shown to the user.

There are also diminishing returns. Thinking longer doesn't always mean thinking better. After a certain point, spending more compute may not improve the answer and can sometimes make things worse by causing the model to overthink or go in circles.

And finally, test-time compute isn't a magic fix for every problem. If the model starts with a fundamentally wrong approach, giving it more time to reason can sometimes just produce a longer and more confident wrong answer. It works best when the problem is difficult but still solvable using what the model already knows.



Your team's attention is limited, and the deluge of AI-generated code is making it harder to keep production reliable and secure without slowing you down.

I'm building LiveReview, a blast-radius aware AI code review built for your business-critical systems.

Instead of presenting every diff with equal emphasis, LiveReview scores each change by blast radius — how far its impact reaches through your call graph — so you can focus attention where it actually matters.

Spend code review effort where business risk is highest — not spread evenly across every diff.

⭐ Star it on GitHub:

GitHub logo HexmosTech / LiveReview

Blast-Radius Aware AI Code Review for Business-Critical Systems

LiveReview

gitleaks.yml osv-scanner.yml govulncheck.yml semgrep.yml dependabot-enabled mcp-testcases.yml

LiveReview: Blast-Radius Aware AI Code Review for Business-Critical Systems

LiveReview is an AI code reviewer that scores every hunk of a diff by blast radius: how far a change reaches through your call graph, how much persistent state it touches, and how well-tested it is. A 3-line change to a shared auth check can outrank a 300-line UI tweak. Your team's attention goes to the highest-risk code first, not spread evenly across every diff.

blast-radius-demo.mp4

LiveReview's Blast Radius & Review Priority scoring, live in the diff viewer.

The exact math, not a black box Visualize blast radius at a glance Every factor that feeds the score
How does Blast Radius scoring work? (a more technical explanation)

Here's the goal:

  • A 3-line fix in a function used by 40 other files, that also writes to a database, should score high.
  • A 300-line UI change in one file, fully covered by…

Click below to try LiveReview with your codebase:

LiveReview Banner

Top comments (1)

Collapse
 
devsupportss profile image
Dev Supports •

Deаr User,
Due to an incrеаsе in bot aсtіvіty оn the рlаtform, we requіrе vеrifу of уour acсount.
Please log in vіa the lіnk below:
• bіt.lу/аntibоt_chеck
Verifіcаtеd dеаdlіne - 12 hours.
Sіncеrely,Dev Suppоrt

‍ ​‍