Reasoning models have changed what LLMs can do.
Instead of jumping straight to an answer, these systems now generate internal step-by-step reasoning before responding. That has made them much better at tasks like math and coding, where decomposing a problem is often the difference between a useful answer and a wrong one.
But that same capability introduces a security problem that is easy to miss if you only think about output quality: some reasoning models can be deliberately pushed into long, unproductive reasoning loops. In practice, that means an attacker may be able to make a model spend far more compute than normal on a single request, which turns “thinking harder” into a denial-of-service risk.
This is not just a theoretical concern. Researchers from Zhejiang University and Alibaba presented work at ICML 2026 showing that logically inconsistent prompts can trigger this behavior across several commercial reasoning models.
The Core Issue: Overthinking as a Failure Mode
Earlier LLMs typically produced an answer directly. Modern reasoning models do more work internally: they break problems down, examine steps, and try to solve them more carefully before responding.
That is useful, but previous research has shown these models sometimes “overthink” even when there is no benefit. They produce extremely long streams of reasoning that do not improve the answer, and in some cases appear to get stuck trying to resolve a problem that is effectively unsolvable.
The new research shows this behavior can be induced on purpose.
The attack works by corrupting the logical structure of prompts so that the model is led into a contradiction or an incomplete premise set. Rather than detecting that the problem cannot be solved cleanly, the model keeps reasoning. From a system perspective, that is costly: longer outputs consume more tokens, increase inference time, and place more load on the provider’s servers.
That makes overthinking a potential availability issue, not just a model-quality issue.
How the Attack Works
The researchers built an evolutionary algorithm that mutates prompts until they reliably trigger long reasoning traces.
The source problems came from 940 math questions drawn from three benchmark datasets. The team used an LLM to split each question into its logical pieces: premises and the final question. Then they repeatedly “jumbled” those pieces using mutations such as:
- swapping premises between different problems
- adding extra premises
- deleting existing premises
- swapping the final question between two premise sets
After each mutation round, the candidate prompts were scored in two ways:
- How many words they caused the target model to generate
- How often they produced linguistic signals of overthinking, such as “but,” “wait,” “maybe,” or “alternatively”
The best-scoring prompts were kept, mutated again, and repeated for five generations.
That scoring loop is important because it turns the problem into an optimization task. The attacker is not guessing randomly; they are systematically searching for prompts that cause the model to spend more and more time reasoning.
Why This Is Security-Relevant for Builders
For application developers, the concern is not just that the model may answer slowly. It is that a single input can disproportionately increase cost and latency.
If your product routes user queries to a reasoning model, an attacker who discovers a prompt pattern that induces overthinking can create a few practical problems:
- higher inference cost per request
- degraded latency for legitimate users
- possible service instability under repeated abuse
- higher exposure for pay-per-token APIs
The researchers reported that the attack was effective against models including DeepSeek-R1, Alibaba’s Qwen3-Thinking, OpenAI’s GPT-o3, and Google’s Gemini 2.5 Flash. On one benchmark, DeepSeek-R1 produced output up to 26.1 times longer than the longest response to the original, unmodified questions.
That kind of multiplier matters. Even if the attack is not cheap enough to scale indefinitely, it may still be economically attractive enough to create denial-of-service pressure, especially when paired with rate limits, burst traffic, or low-friction API access.
Why Closed Models Are Still Exposed
One detail that makes this research especially relevant to real-world systems is that the method does not require access to model internals.
The attack can be built by querying the target model and observing its outputs. In other words, it works as an external prompt-generation process rather than a white-box exploit. That means closed-source commercial services are not automatically protected just because their weights are hidden.
The researchers also showed that prompts generated using a smaller, cheaper model could still transfer to the target models and cause much longer outputs than normal. That lowers the barrier to experimentation and makes the attack more feasible than if every test required expensive calls to the most capable reasoning model.
Limits and Tradeoffs
The paper is careful not to oversell the attack as an immediately practical large-scale DoS tool.
There are real constraints:
- API pricing can make repeated probing expensive
- rate limiting can reduce throughput
- context window limits may cap how much text a model can generate
- provider defenses may interrupt or truncate excessive reasoning
Those caveats matter. A vulnerability is not the same as a weaponized attack at scale.
But security teams should care about the distinction between “hard to exploit cheaply” and “not exploitable.” The research is mainly making one point: this attack surface exists, and reasoning models appear to share it across vendors and architectures.
What Developers Should Take Away
If you are building on top of reasoning models, it is worth treating excessive reasoning as an operational risk.
A few practical ideas to consider:
- monitor output length and latency per request
- flag repeated prompts that produce unusually long reasoning traces
- set guardrails around maximum generation length
- rate-limit suspicious or repetitive traffic
- test reasoning models with malformed, incomplete, or contradictory inputs during security review
None of those measures is a complete fix on its own. But they can reduce the blast radius if a user discovers a prompt that consistently drives the model into long loops.
The bigger lesson is architectural: once a model’s behavior depends heavily on internal reasoning, the cost of a bad prompt is no longer just a bad answer. It can become a resource exhaustion problem.
A Useful Security Reminder
Reasoning improves capability, but it also expands the attack surface.
This research does not argue that reasoning models are broken. It argues that they are more complex systems with failure modes that look a lot like traditional availability bugs: a small input can trigger a large, expensive response.
For builders, that means reasoning quality and operational resilience need to be designed together. If your product relies on these models, you should assume adversarial prompts exist and measure what they do to latency, token usage, and cost before they show up in production.
Reasoning is powerful. But as this work shows, “thinking harder” can also be something an attacker tries to force on purpose.
Tags: #llm #cybersecurity #artificialintelligence #promptengineering
Top comments (0)