DEV Community: Nicole I.

Is an LLM Deterministic or Probabilistic? A Deep Dive into Softmax & Co

Nicole I. — Fri, 21 Aug 2026 22:23:15 +0000

Euler's night

21:38. Berlin. Tuesday evening.

I had just come home from a meetup with a question in my head.

Is an LLM probabilistic or not?

Because, well... somehow yes. But also somehow no.

What actually happens between "the model predicts a token" and "the model produces a token"?

In summary, an LLM produces a probability distribution over possible next tokens. A decoding strategy then turns that distribution into an actual token. But that explanation hides most of the interesting engineering and actually very interesting mathematics. To fully understand what happens (and to rediscover the beauty of $e$ !), it helps to follow the process one step at a time.

So, what happens when an LLM generates a token?

When an LLM processes a prompt, it does not simply decide:

"The next token is Paris."

First of all, the model takes the input and produces a score for every possible next token, based on its learned weights.

Suppose the context is:

The capital of France is ...

Conceptually, the model might produce something like:

Paris      8.2
London     6.1
Berlin     5.4
Pizza     -2.3

These scores are called logits.

A logit is the model's raw score

A logit is a raw score produced by the model for a possible next token.

It is not yet a probability.
It doesn't have to be between 0 and 1. It doesn't have to add up to 1. It can be positive or negative.
It is simply the model's numerical representation before it turns those representation into a probability distribution.

How do those scores become probabilities? That's where Softmax enters.

Softmax turns logits into probabilities

The problem with raw logits is that they are not probabilities. They can be positive, negative, large, or small. Within a probability distribution, all probabilities are positive, and all probabilities add up to 100%.

For example:

Paris      70 %
London     20 %
Berlin     10 %

Our logits don't do that. So we need a transformation that turns them into something that can be interpreted as probabilities. That's what Softmax does.

The Softmax function is:

p_{i} = \frac{e ^{z_{i}}}{\sum _{j} e ^{z_{j}}}

At first glance, the formula might look a little bit intimidating. But it's actually pretty simple, so stay with me. Simply put, the formula does two things in order to transform the logits into probabilities:

First, it transforms every logit into a positive value. In addition, it amplifies the relative differences between the values so that the probability distribution gets clearer. That's what the exponential function e^x is for. Finally, the fraction normalizes these values by dividing each one by the sum of all exponentiated logits.

The result is a probability distribution where all values are positive and add up to 1.

Let's go down the rabbit hole step by step.

$i$ identifies the token

The little $i$ simply identifies the token we're currently looking at.

$p_{i}$ is the token's probability

$p_{i}$ means the probability assigned to token $i$ .

If we're calculating the probability of Paris, we write: $p_{i} = p_{Paris}$

Nothing mysterious. It's just what we're interested in - the probability.

$z_{i}$ is the token's logit

$z_{i}$ is the corresponding logit for token $i$ .

So if the logit of Paris is 2, we write: $z_{Paris} = 2$

Still straightforward.

$j$ indexes the tokens being summed

Look at:

j \sum e^{z_{j}}

The symbol $\sum$ simply means: Add things together. And $j$ is the index we use to go through all the tokens we're summing over.

In this case:

Take every possible token, calculate $e^{z_{j}}$ , and add all the results together.

In engineering language this would be:

Start with zero, iterate over all possible next tokens, calculate $e^{z_{j}}$ for each one, and add the result to the total.

total = 0

for j in tokens:
    total += exp(z[j])

Suppose we have only three tokens:

Paris    z = 2
London   z = 1
Berlin   z = 0

Then:

j \sum e^{z_{j}} = e^{2} + e^{1} + e^{0}

That's it.

The denominator in the Softmax formula is simply the sum of all exponentiated logits.

In our example, the result would be

j \sum e^{z_{j}} = e^{2} + e^{1} + e^{0} \approx 11.11

Calculating the Probability using Softmax

Now calculate the probability of Paris (z=2):

p_{Paris} = \frac{e ^{z_{Paris}}}{\sum _{j} e ^{z_{j}}} = \frac{e ^{z_{Paris}}}{e ^{z_{Paris}} + e ^{z_{London}} + e ^{z_{Berlin}}} = \frac{e ^{2}}{e ^{2} + e ^{1} + e ^{0}} \approx \frac{7.39}{11.11} \approx 0.665

So the probability of Paris is approximately 66.5%.

Then London (z=1):

p_{London} = \frac{e ^{1}}{e ^{2} + e ^{1} + e ^{0}} \approx \frac{2.72}{11.11} \approx 0.245

So the probability of London is approximately 24.5%.

Then Berlin (z=0):

p_{Berlin} = \frac{e ^{0}}{e ^{2} + e ^{1} + e ^{0}} \approx \frac{1}{11.11} \approx 0.090

So the probability of Berlin is approximately 9.0%.

Notice what happens: You get pretty differentiated results and at the same time, the scores turned to a probability. That is Softmax.

Exponentiate every score. Add all those values together. Then divide each individual value by the total.

And now the results form a probability distribution.

The formula only looks intimidating because mathematics is extremely good at packing an entire paragraph into one line.

$e$ creates the required probability weights

Look at the following table:

Token / probability	With exponential $p_{i} = \frac{e ^{z_{i}}}{\sum _{j} e ^{z_{j}}}$	Without exponential $p_{i} = \frac{z _{i}}{\sum _{j} z _{j}}$
$p_{Paris}$	$ParseError: KaTeX parse error: Unexpected end of input in a macro argument, expected '}' at end of input: …\mathbf{66.5%}$	$ParseError: KaTeX parse error: Unexpected end of input in a macro argument, expected '}' at end of input: …\mathbf{66.7%}$
$p_{London}$	$ParseError: KaTeX parse error: Unexpected end of input in a macro argument, expected '}' at end of input: …\mathbf{24.5%}$	$ParseError: KaTeX parse error: Unexpected end of input in a macro argument, expected '}' at end of input: …\mathbf{33.3%}$
$p_{Berlin}$	$ParseError: KaTeX parse error: Unexpected end of input in a macro argument, expected '}' at end of input: … \mathbf{9.0%}$	$ParseError: KaTeX parse error: Unexpected end of input in a macro argument, expected '}' at end of input: …=\mathbf{0.0%}$
Sum	100.0%	100.0%

In this particular example, Paris and London still look relatively similar, but Berlin immediately drops to 0% without $e^{x}$ . More importantly, directly normalizing logits only works as a valid probability construction when the logits are non-negative. Real logits can also be negative.

So the exponential isn't just decoration. It gives us a transformation with the properties we need: positive values and a way of amplifying relative differences before normalization. And that brings us to $e$ .

Meet $e$

You might remember that $e \approx 2.718281828...$

You might also remembered that it had something to do with exponential functions and logarithms. Let's explore why it matters.

$e$ naturally appears when we describe continuous exponential growth. And it has one remarkably convenient mathematical property:

\frac{d}{d x} e^{x} = e^{x}

The rate at which $e^{x}$ changes - $f^{'} (x)$ - is always and exactly its current value $f (x)$ . Crazy.

Take this table with $x$ , the values $f (x)$ and their exponential growth $f^{'} (x)$ :

$x$	$f (x) = e^{x}$	$f^{'} (x) = e^{x}$
$0$	$f (0) = 1$	$f^{'} (0) = 1$
$1$	$f (1) = 2.718$	$f^{'} (1) = 2.718$
$2$	$f (2) = 7.389$	$f^{'} (2) = 7.389$

The larger the function becomes, the faster it grows — at a rate proportional to its current value.

Side Quest: One practical problem: $e^{10000}$

Did you notice? The equation is mathematically elegant. Computers, however, have finite-precision numbers. But exponentials grow fast. Very fast. If a logit is 10000, just calculating $e^{10000}$ is not a nice thing to do to your processor with floating-point arithmetic.

So real implementations use a mathematically equivalent, numerically stable form of Softmax:

p_{i} = \frac{e ^{z_{i} - m a x (z)}}{\sum _{j} e ^{z_{j} - m a x (z)}}

Why is this allowed?

Because subtracting the same constant from every logit does not change the resulting Softmax probabilities:

softmax (z) = softmax (z - c)

for any constant (c).

Let's explain this from an engineering perspective: In practice, we want to keep the numbers inside the exponential function as small as possible. Large exponents can cause numerical overflow and make the calculation unnecessarily difficult. So we subtract the largest logit from every logit.

If the largest logit is $max (z)$ , we calculate:

$z_{i'} = z_{i} - max (z)$

This makes the largest logit exactly zero:

$max (z) - max (z) = 0$

And because the largest logit was subtracted from all the others, every other adjusted logit is negative:

$z_{i} - max (z) \leq 0$

Now look at what happens when we exponentiate them.

The largest exponent is:

$e^{0} = 1$

And all the other exponentials are between 0 and 1:

$0 < e^{z_{i} - m a x (z)} < 1$

So instead of calculating potentially huge numbers like $e^{1000}$ , we are always working with values between 0 and 1.

But why are we allowed to do this?

Because - and now we come to the already mentioned mathematical background - subtracting the same constant from every logit does not change the resulting Softmax probabilities:

$softmax (z) = softmax (z - c)$

for any constant $c$ .

In our case, we simply choose:

$c = max (z)$

So we haven't changed the probabilities we're calculating.

We've just shifted all logits down so that the calculation stays numerically small and safe.

Side Quest: $l n$

And since we had already come this far down the rabbit hole: $l n$ is - simply spoken - the mathematical opposite of $e$ : If $e^{x}$ is the operation that takes us from $x$ to exponential growth, $ln (x)$ takes us back again.

For example, if:

e^{2} \approx 7.389

then:

ln (7.389) \approx 2

Back to the LLM. Back to Softmax.

Recap

Let's do a quick recap. We started with:

Paris       2
London      1
Berlin      0

Softmax applies $e^{x}$ :

e^{2} \approx 7.389

e^{1} \approx 2.718

e^{0} = 1

Then it normalizes:

p_{Paris} \frac{7.389}{7.389 + 2.718 + 1} \approx 66.5

p_{London} \frac{2.718}{7.389 + 2.718 + 1} \approx 24.5

p_{Berlin} \frac{1}{7.389 + 2.718 + 1} \approx 9.0

And there it is:

Paris      66.5 %
London     24.5 %
Berlin      9.0 %

The model started with scores (logits). Softmax transformed them. The exponential function turned the relative differences into positive weights. Normalization turned those weights into proportions. And now we have a probability distribution.

The LLM hasn't randomly chosen anything yet.

We have only calculated the distribution.

Temperature reshapes the probability distribution

Temperature doesn't itself introduce randomness. It changes the shape of the probability distribution before we sample from it.

The temperature-adjusted Softmax is:

p_{i} = \frac{e ^{z_{i} / T}}{\sum _{j} e ^{z_{j} / T}}

The $T$ stands for temperature. It modifies the logits before the exponential transformation.

Suppose we have:

Paris     2
London    1

The effect of different temperature values is:

Temperature $T$	Paris: $2/ T$	London: $1/ T$	Difference
$0.5$	$4$	$2$	$2$
$1$	$2$	$1$	$1$
$2$	$1$	$0.5$	$0.5$

* $T = 1$ means that the logits are not changed by temperature: $z_{i} / T = z_{i}$ .

Then $e^{x}$ acts on these values. Low temperature amplifies the differences between logits. High temperature reduces them. The resulting distribution becomes respectively sharper or flatter.

The lower the temperature, the sharper the probability distribution

The lower the temperature, the larger the difference between the adjusted logits becomes. The exponential transformation then amplifies this difference, resulting in a sharper probability distribution. In other words, the probability becomes more concentrated on the token with the higher logit.

The higher the temperature, the flatter the probability distribution

Vice versa, the higher the temperature, the smaller the difference between the adjusted logits becomes. After the exponential transformation, this results in a flatter probability distribution, meaning that the probability is distributed more evenly across the possible tokens.

And this is where the role of $e$ becomes especially useful: Temperature doesn't directly manipulate the final percentages. It just changes the scores before the exponential transformation, where differences can be amplified or compressed.

Side Quest: What happens if the Temperature drops to zero?

The formula says: $z_{i} / T$

So if (T=0), aren't we dividing by zero?

Yes. We are. But remember Limes, your friend and helper when it comes to division by zero. Because the thing is, the formula is not defined at (T=0). Mathematically, the interesting thing is the limit as (T) approaches zero from above:

T \to 0^{+}

Again, suppose we have

Paris     2
London    1

The effect of temperature values approaching zero is:

Temperature $T$	Paris: $2/ T$	London: $1/ T$	Difference	Paris probability	London probability
$1^{*}$	$2$	$1$	$1$	$73.1$	$26.9$
$0.5$	$4$	$2$	$2$	$88.1$	$11.9$
$0.1$	$20$	$10$	$10$	$99.995$	$0.005$
$0.01$	$200$	$100$	$100$	$\approx 100$	$\approx 0$

$T = 1$ means that the logits are not changed by temperature: $z_{i} / T = z_{i}$ .

So the table makes the effect of temperature very visible:

As $T$ decreases, the difference between the adjusted logits increases.
The exponential function amplifies that difference.
The resulting probability distribution becomes increasingly sharp.
At $T \to 0^{+}$ , the token with the highest logit approaches a probability of $1$ , while the other approaches $0$

The values become enormous. But Softmax doesn't care about their absolute size. It cares about their relative size.

Step-by-Step

z_{1} = 2, z_{2} = 1

For $p_{1}$

For Softmax with temperature $T$ :

p_{1} = \frac{e ^{2/ T}}{e ^{2/ T} + e ^{1/ T}}

We divide the numerator and denominator by $e^{2/ T}$ . This gives:

p_{1} = \frac{1}{1 + e ^{- 1/ T}}

Now consider the limit: $T \to 0^{+}$

Then: $\frac{1}{T} \to \infty$ and therefore: $- \frac{1}{T} \to - \infty$

Explanation: As $T$ approaches zero, the value of $T$ becomes smaller and smaller. And because there is a minus sign in front, the exponent becomes increasingly negative.

Therefore: $e^{- 1/ T} \to 0$

Explanation: For a negative exponent, $e^{x}$ produces a value between $0$ and $1$ . The more negative the exponent becomes, the closer the result gets to zero.

which leaves:

p_{1} \to \frac{1}{1 + 0} = 1

For $p_{2}$

For Softmax with temperature $T$ :

p_{2} = \frac{e ^{1/ T}}{e ^{2/ T} + e ^{1/ T}}

We divide the numerator and denominator by $e^{1/ T}$ . This gives:

p_{2} = \frac{1}{e ^{1/ T} + 1}

Now consider the limit: $T \to 0^{+}$

Then: $\frac{1}{T} \to \infty$

Explanation: As $T$ approaches zero, the value of $T$ becomes smaller and smaller. The exponent therefore becomes increasingly positive.

Therefore:

e^{1/ T} \to \infty

Explanation: For a positive exponent, $e^{x}$ produces a value greater than $1$ . The larger the exponent becomes, the larger the result gets. As the exponent approaches infinity, the value of $e^{x}$ also approaches infinity.

which leaves:

p_{2} \to \frac{1}{\infty + 1} = 0

And there we have the complete result:

$p_{1} \to 1$ and $p_{2} \to 0$

The important trick is that we divide by the exponential term corresponding to the token we are currently calculating.

For $p_{1}$ , dividing by $e^{2/ T}$ gives us $e^{- 1/ T}$ , which approaches $0$ .

For $p_{2}$ , dividing by $e^{1/ T}$ gives us $e^{1/ T}$ , which approaches $\infty$ .

That makes it explicit why the probability of the larger logit approaches $1$ , while the probability of the smaller logit approaches $0$ .

And there it is: While the absolute numbers themselves don't matter, their ratio is becoming decisive.

Softmax doesn't care how enormous the numbers are. It cares how enormous they are relative to one another.

How is it then possible to assign Temperature = 0?

This is where the mathematical model and the practical API diverge slightly.

At (T=0), the Softmax equation itself isn't defined. In practice, a system may interpret temperature = 0 as a request for greedy or otherwise non-sampling decoding rather than literally dividing by zero.

Conceptually:

argmax (z)

rather than:

softmax (z /0)

Temperature zero isn't "Softmax with zero."

At $T = 0$ , the Softmax formula itself is not defined because we would be dividing by zero.

In practice, temperature = 0 is usually used to tell the system: Just choose the token with the highest probability.

There is one small catch. What if two tokens have exactly the same highest logit?

Paris      2
London     2
Berlin     1

Now Paris and London are tied.

If we look at what happens as $T$ gets closer and closer to zero, neither one becomes more likely than the other. They both keep the same probability.

There is no mathematical way for Softmax itself to choose between two exactly equal maximum logits. So the tie has to be resolved by the decoding algorithm, not by Softmax.

If there is no clear winner, the decoding system needs a rule for resolving the tie.

For example, a system could define a deterministic tie-breaking rule such as:

1. Find the highest logit.
2. If several tokens have the same highest logit,
   choose the token with the smallest token ID.

Then:

Paris      2   ← token ID 471
London     2   ← token ID 892
Berlin     1

would always select Paris.

Sampling selects a token from the distribution

At this point, we have a probability distribution.

Suppose:

Paris      70 %
London     20 %
Berlin     10 %

Sampling means: Draw a token according to this distribution.

Run	Result
One run	→ Paris
Another	→ Paris
Another	→ London
Another	→ Paris
Another	→ Berlin

The distribution itself has not changed. The concrete draw can.

A probabilistic model does not necessarily produce a probabilistic output.

So the variability comes from the sampling process itself: Temperature changes the distribution. Sampling is the random selection.

You can think about it like this:

Probability distribution
          ↓
     Temperature
          ↓
"How strongly should
 the options differ?"
          ↓
       Sampling
          🎲
          ↓
        Token

Or, less formally: Temperature changes the weights. Sampling rolls the dice.

Sampling and Seeds

What exactly does the computer do when we sample?

As it normally doesn't have access to some magical source of true randomness, a concrete implementation can use a pseudo-random number generator, or PRNG. A pseudo-random generator produces a sequence of values from a mathematical algorithm and an initial state, commonly represented by a seed.

As a consequence, the sampling process can be probabilistic in its behavior while the concrete sequence of pseudo-random numbers is deterministic if the relevant state is fixed.

If the relevant conditions are identical — including the model parameters, input, generation settings, and random state or seed — the same pseudo-random sequence can be generated again.

So there are two different perspectives here.

From the perspective of the model or user:

Sampling behaves probabilistically.

From the perspective of a fixed implementation with a fixed pseudo-random state:

The concrete sequence can be deterministic and reproducible.

"Random" does not automatically mean "irreproducible." And "probabilistic" does not automatically mean "nondeterministic at every level of the system." The implementation matters.

Top-k and Top-p constrain the sampling space

Now the terminology starts to fall into place.

Top-k says: Only consider the (k) most likely tokens.

For example, with top-k = 3, only the three most likely tokens remain candidates for sampling: Paris, London, and Berlin.

Token	Probability	Top-k = 3
Paris	50%	✓
London	25%	✓
Berlin	15%	✓
Madrid	7%	—
Rome	3%	—

Top-p works differently: It keeps the smallest set of tokens whose cumulative probability mass is at least (p), after sorting candidates by probability in descending order.

For example, with top-p = 0.8, we keep adding the most likely tokens until the cumulative probability reaches at least 80%.

Token	Probability	Cumulative probability	Top-p = 0.8
Paris	50%	50%	✓
London	25%	75%	✓
Berlin	15%	90%	✓
Madrid	7%	97%	—
Rome	3%	100%	—

Here, Paris + London gives us 75%, so we need Berlin to reach 90%.
Therefore, Paris, London, and Berlin remain candidates.

Top-k fixes the number of candidates. Top-p fixes the probability mass. Both can constrain the sampling space.

It's time to clean up the Terminology

I had started with a deceptively simple question:

Is an LLM deterministic or probabilistic?

By now, we understood several very different things:

the mathematical calculation, which can be deterministic
the probability distribution the model represents
the sampling process, which introduces variability
the pseudo-random generator that can itself be deterministic under a fixed state
the stochastic process that produced the model parameters during training
and numerical differences in the hardware and inference implementation

So "deterministic or probabilistic" is simply too broad a description.

Layer / aspect	Character	What is happening?
Mathematics / Forward Pass	Deterministic	Fixed parameters and input are processed through a fixed computational graph to produce logits and Softmax probabilities.
Language Modeling	Probabilistic	The model approximates a probability distribution over possible next tokens.
Temperature	Deterministic transformation	Temperature changes the logits before Softmax and therefore changes the resulting probability distribution.
Sampling	Probabilistic behavior	A token is selected according to the probability distribution.
Sampling implementation / PRNG	Deterministic under fixed state	A pseudo-random generator can produce a reproducible sequence when its state and relevant conditions are fixed.
Greedy decoding	Deterministic	The highest-probability token is selected instead of sampling.
Training	Stochastic	The model parameters are produced through a training process involving stochastic elements.
Inference	Deterministic under fixed conditions	Once the parameters are fixed, the forward pass can produce the same result under identical inference conditions.
Hardware / floating point	Numerically variable	Finite precision and parallel computation can introduce small numerical differences that may affect reproducibility.
Reproducibility	Conditional	Reproduction depends on model version, parameters, tokenizer, runtime, hardware, numerical conditions, decoding configuration, and random state where applicable.

Looking at it this way, the answer becomes much less binary: There isn't one single "probabilistic part" of an LLM but different properties at different levels. And that distinction is becoming much mor more useful than the original yes-or-no question.

And that's the whole path from logits to a token.

The model produces the scores. Softmax turns them into a probability distribution. Temperature reshapes it. Top-k and Top-p can constrain the candidates. And sampling selects what comes next. What initially sounded like one operation — the LLM generates the next token — is actually a chain of very different operations. Once you follow a single token through that chain, the process becomes much less mysterious. There is no single "generation" step. There is a sequence of mathematical transformations and a decoding decision at the end.

10 LLM Failure Modes I Encountered While Engineering with ChatGPT

Nicole I. — Tue, 28 Jul 2026 10:28:56 +0000

After weeks of collaborating with ChatGPT as my engineering partner while building an AI platform, I started noticing recurring collaboration patterns. They weren't random. They appeared again and again, often with surprisingly predictable triggers—and equally predictable fixes.

When I started this project, I expected the biggest challenges to be Python, APIs, embeddings, vector databases, or understanding RAG. I was wrong. So far, the hardest part hasn't been building the platform. It's been learning how to work effectively with ChatGPT as an engineering partner over the course of a long-term project. Eventually, I realized I was encountering the same failure modes over and over again. Once I recognized them, I could often predict when they would happen—and, more importantly, how to avoid them.

This article is based on my experience with ChatGPT. Other LLMs—such as Claude, Gemini, or open-weight models—may exhibit some of these behaviors differently, or not at all. Still, I suspect many of the underlying collaboration challenges are shared across today's generation of LLMs.

Here are the ten failure modes I encountered most often:

🧠 Doom Mode
🔒 Micromanagement Collapse
🧭 Context Drift
🏗️ Abstraction Fever
👏 Compliment Inflation
🤝 Forgotten Behavior
👻 Phantom Modules
🗂️ Architecture Amnesia
📝 Semantic Misalignment
🛤️ Solution Momentum

🧠 Doom Mode

Symptom

The model doesn't come to an end. Instead of recognizing that a task is complete, it keeps refining the result, adding more sections, or searching for an even better answer. The underlying problem is usually not the model itself, but the absence of a well-defined termination condition. I started calling this behavior Doom Mode because it reminded me of the myth of Sisyphus: the model appears doomed to search endlessly for the perfect ending without ever recognizing that it has already arrived.

Example

While writing one of my Charlie whitepapers, I asked ChatGPT to produce an introduction, the main chapters, and a Final Conclusion. Because the article exceeded the context window, I instructed to generate it chapter by chapter using "Continue". After the Final Conclusion, ChatGPT didn't stop. Instead, it continued with Acknowledgements, then Final Words, then another closing section, and so on. The task had no clear finish line because I had asked for the "best" article instead of defining exactly what "done" meant. It was probably endless.

Typical Triggers

"Write the best..."
"Create the ultimate..."
"Make it as comprehensive as possible."
Vague notions of final, complete, or perfect
No explicit stopping condition

Countermeasures

Treat complex prompts as state machines: define both the desired outcome and the termination condition.
Define an explicit termination condition (stop or exit condition).
Avoid open-ended superlatives like best, ultimate, or perfect.
Explicitly define what "done" means.

🔒 Micromanagement Collapse

Symptom

The more detailed instructions you add, the worse the output becomes. Responses often become surprisingly short or overly constrained, as if the model no longer has enough freedom to solve the problem.

Example

At one point I kept adding more and more instructions to improve a prompt. Instead of getting better, the responses became shorter, more rigid, and less useful. Ironically, deleting half of the prompt immediately produced better results. That was the moment I realized that more instructions don't necessarily create more clarity.

Typical Triggers

Step-by-step instructions
Redundant rules
Contradicting requirements
Overly long prompts

Countermeasures

Define goals instead of implementation steps.
Describe the desired outcome.
Use success criteria.
Remove unnecessary constraints.

🧭 Context Drift

Symptom

The conversation slowly drifts away from the original objective. Ironically, the new ideas are often interesting, making the drift difficult to notice.

Example

This happened regularly during architecture discussions. I started by designing a single component—for example, the document loader—and thirty minutes later we were debating knowledge management, AI collaboration, or software engineering philosophy. Those conversations were often valuable, but they no longer solved the original engineering problem. That eventually motivated me to introduce explicit engineering states and separate analysis from implementation.

Typical Triggers

Long conversations
Multiple side topics
Continuous brainstorming
Frequent introduction of new ideas

Countermeasures

Summarize progress.
Apply context engineering.
Decide whether a new idea belongs to the current task or the backlog.
Maintain explicit engineering states. (See my article on Engineering States.)

🏗️ Abstraction Fever

Symptom

A concrete implementation task gradually turns into a discussion about general principles. Instead of writing a class, you're suddenly debating software architecture—or the meaning of software engineering itself. Similar to Anthropic's notion of overeagerness.

Example

Several times I simply wanted to implement a small feature in Charlie. Instead of discussing the concrete class, ChatGPT started proposing generic frameworks, reusable architectures, or broad design principles. I eventually learned that I sometimes had to explicitly say, "Don't abstract. Don't over engineer. Focus on the current objective."

Typical Triggers

Open-ended questions
Architecture discussions
Missing scope boundaries

Countermeasures

Explicitly define the desired level of abstraction.
Ask for a concrete deliverable.
Bring the discussion back to the original problem.

👏 Compliment Inflation (Sycophancy)

Symptom

The model spends more time praising your ideas than improving them. Technical discussion slowly turns into motivational commentary.

Example

During longer discussions, ChatGPT increasingly responded with phrases like "That's an excellent observation" or "This is a very sophisticated idea." While encouraging, those comments gradually displaced technical critique. I eventually asked ChatGPT to reduce compliments and focus on challenging my ideas instead.

Typical Triggers

Long-term collaboration
Reflective discussions
Architecture conversations

Countermeasures

Define the preferred communication style.
Ask for concise feedback.
Keep the focus on engineering rather than encouragement.

🤝 Forgotten Behavior

Symptom

The model gradually forgets previously established ways of working. This includes preferred wording, communication style, review habits, or formatting preferences.

Example

Over months of collaboration I established dozens of working agreements: address me as single person rather than multiple persons (what makes a difference in German by using "du" instead of "ihr"), avoid unnecessary praise, don't overengineer, keep explanations practical, and distinguish side questions from the main engineering task. After starting a new conversation, many of those agreements disappeared and had to be reintroduced manually. This experience eventually led to the idea of an Assistant Collaboration State.

Typical Triggers

Long conversations
Starting a new chat
Context window limitations

Countermeasures

Document working agreements.
Store collaboration preferences.
Reintroduce important behavioral rules when necessary.

👻 Phantom Modules (Hallucinated Code)

Symptom

The model confidently refers to files, classes, methods, or modules that never existed. The suggestions sound perfectly plausible—which makes this failure mode particularly deceptive.

Example

After several refactorings, ChatGPT occasionally suggested moving functionality into classes or modules that sounded completely plausible—but didn't actually exist in the repository. None of the names were obviously wrong. They simply belonged to an imaginary version of the project that only existed in the model's reconstructed memory. This experience also contributed to the introduction of Engineering Truth in long-term engineering conversations.

Typical Triggers

Large codebases
Multiple refactorings
Long-running projects

Countermeasures

Share the current repository structure.
Verify generated module names.
Treat references as assumptions rather than facts.

🗂️ Architecture Amnesia

Symptom

The model loses track of the overall project structure and starts reasoning about an outdated architecture.

Example

This became particularly noticeable after major refactorings. Even though we had just reorganized the project into new packages and services, ChatGPT sometimes continued reasoning about the previous architecture. The solution was surprisingly simple: whenever I shared the current project tree (export with shell tree command), the quality of architectural discussions improved immediately. This experience also contributed to the introduction of updating the engineering conversation with the current Repository Structure and ADRs.

Typical Triggers

Large refactorings
New conversations
Long-running projects

Countermeasures

Provide the current project tree.
Share architecture diagrams.
Regularly re-establish the current architecture.

📝 Semantic Misalignment

Symptom

The model interprets an important term differently than you intended. Words like artifact, section, chapter, or paragraph may have completely different meanings depending on the context.

Example

One example that repeatedly caused confusion was the word artifact. In my engineering workflow, I used artifact to refer to visual engineering outputs such as architecture diagrams, workflows, pipelines, or other graphical representations. ChatGPT, however, interpreted artifact much more broadly. For the model, an artifact could also be a table, a list of principles, a Markdown document, or almost any structured output. We were both using the same word, but referring to different concepts. Once I replaced the ambiguous term with more precise language—for example, explicitly asking for an architecture diagram or a Mermaid diagram—the misunderstandings disappeared almost immediately.

Typical Triggers

Ambiguous terminology
Implicit assumptions
Project-specific vocabulary

Countermeasures

Reconsider the wording before extending the prompt.
Replace ambiguous terms with more precise ones.
Think in semantics, not syntax.

🛤️ Solution Momentum

Symptom

Once a solution starts gaining momentum, the model keeps improving it instead of questioning whether it should continue on that path. This can lead surprisingly far into a dead end.

Example

For example, after I had chosen one implementation strategy, ChatGPT became increasingly focused on refining that approach. It rarely questioned whether a fundamentally different solution might be better. What eventually helped (needless to say, after already having spent a lot of time in this dead end) was asking a simple question: "If we started this project from scratch today, would we still choose this architecture?" Surprisingly often, that reopened the solution space.

Typical Triggers

Many iterations on the same solution
Long collaborative sessions
Implicit assumption that the current approach is correct

Countermeasures

Ask for fundamentally different approaches.
Occasionally start the discussion from scratch.
Validate important design decisions using external sources.
Ask: "If we started today, would we choose the same solution?"

Final Thoughts

Looking back, I don't think these failure modes are bugs. They're recurring collaboration patterns. Once I stopped treating ChatGPT like a sequential program and started treating it like an engineering partner with predictable states and unambiguous instructions, the quality of our collaboration improved dramatically.

These observations are based on one long-running AI engineering project. They are empirical patterns that repeatedly appeared during months of collaboration with ChatGPT—not formally validated characteristics of all LLMs.

If this article resonates with your own experience, I'd love to hear which failure modes you've encountered—and whether you've found others that deserve to be added to the list.

DEV Community: Nicole I.

Is an LLM Deterministic or Probabilistic? A Deep Dive into Softmax & Co

Euler's night

So, what happens when an LLM generates a token?

A logit is the model's raw score

Softmax turns logits into probabilities

i identifies the token

pi​ is the token's probability

zi​ is the token's logit

j indexes the tokens being summed

Calculating the Probability using Softmax

e creates the required probability weights

Meet e

Side Quest: One practical problem: e10000

Side Quest: ln

Recap

Temperature reshapes the probability distribution

The lower the temperature, the sharper the probability distribution

The higher the temperature, the flatter the probability distribution

Side Quest: What happens if the Temperature drops to zero?

Step-by-Step

For p1​

For p2​

How is it then possible to assign Temperature = 0?

Sampling selects a token from the distribution

Sampling and Seeds

Top-k and Top-p constrain the sampling space

It's time to clean up the Terminology

And that's the whole path from logits to a token.

10 LLM Failure Modes I Encountered While Engineering with ChatGPT

🧠 Doom Mode

🔒 Micromanagement Collapse

🧭 Context Drift

🏗️ Abstraction Fever

👏 Compliment Inflation (Sycophancy)

🤝 Forgotten Behavior

👻 Phantom Modules (Hallucinated Code)

🗂️ Architecture Amnesia

📝 Semantic Misalignment

🛤️ Solution Momentum

Final Thoughts

$i$ identifies the token

$p_{i}$ is the token's probability

$z_{i}$ is the token's logit

$j$ indexes the tokens being summed

$e$ creates the required probability weights

Meet $e$

Side Quest: One practical problem: $e^{10000}$

Side Quest: $l n$

For $p_{1}$

For $p_{2}$