The hardest part of solving a difficult problem is often not finding the solution, It is figuring out what problem you are actually solving.
You probably have some experiences similar to these:
- A team spends two weeks optimizing a database query, only to discover that the real bottleneck was somewhere else.
- An engineer proposes a sophisticated caching layer for a performance problem that could have been solved by removing an unnecessary API call.
A team debates whether to use PostgreSQL, Redis, Kafka, or some new distributed system—before anyone has clearly defined what constraint the system actually needs to satisfy.
Or perhaps the most familiar one: "We've tried everything. Nothing works". Then someone makes a small change in the way the problem is presented, and suddenly the solution becomes straigh-forward.
This isn't just an engineering phenomenon. It's a consequence of how human mind tries to solve problems.
The first point is, we don't interact with a problems directly. We construct a mental representation of it, search through possible solutions using different strategies, rely on memories and heuristics, and sometimes get trapped by the very strategies that previously made us successful.
Understanding how this works is useful for anyone who solves difficult problems (like software engineer which their profession is built around problem solving).
This article series look at the first three layers of that process from psychology's perpective:
- What a problem actually is
- How the mind represents a problem
- How the mind searches for solutions
And then discuss about how we can use it to make better decisions, as engineers and engineering managers.
1. What Is a Problem?
Let's start with this question: What exactly is a problem?
In cognitive psychology, a problem can be thought of as a situation where we have:
- a current state
- a desired state
- a set of possible actions
- and no immediately obvious sequence of actions that gets us from current state to desired state.
Let's start by an example: suppose your API takes 3 seconds to respond. That's not yet a complete problem. You might define the problem as: "The API is too slow."
But that's only an observation. A more useful representation might be:
Current:
API latency = 3s
Desired:
API latency < 500ms
Constraints:
- same infrastructure
- same database
- no breaking API changes
- 2 days available for the fix
Now we have something closer to a problem. But, suppose one engineer starts with: "How do we make this query faster?".
This brings us to one of the most powerful problem-solving mechanisms: reframing.
You might search for:
- better indexes
- query optimization
- caching
- database tuning
But suppose another engineer reframes it as: "How do we avoid executing this query?". Now completely different solutions appear:
- caching
- denormalization
- precomputation
- changing the API
- changing the workflow
- removing the requirement entirely
The problem didn't change. The representation changed. And therefore the search space changed.
Solution Search Strategies
Starting with a metaphorical search in mathematical space is a useful way to visualize what we're actually doing when we "search for a solution."
Imagine that every possible system configuration is a point in some enormous space. Most of these points are irrelevant or impossible. We have constraints that define a feasible region. The problem-solving process is, in a very abstract sense, a search through this space.
We start from the current state and try to find a destination point that satisfies our requirements and a set of actions to reach us to this point. Our search path can reach us to some local optimum or gobal optimum point (A working option or a best practice). We might even have no global optimum point, and have local optimums (There is no best practices. Only some good practices at best).
Different problem-solving strategies correspond to different ways of exploring that space.
2. How We Search for Solutions
After discussing representation of the problem, we can start searching for solutions. Our mind have several different strategies for finding valuable solutions, which is dicussed below.
2.1 Algorithms
The most explicit strategy is an algorithm: Follow a KNOWN sequence of steps that leads toward a solution. For example, debugging a production incident might follow:
1. Check symptoms
2. Check metrics
3. Identify failing component
4. Inspect logs
5. Form hypothesis
6. Test hypothesis
Algorithms are powerful when the problem is well understood and has a clear boundary. But real engineering problems are often too open-ended for a complete algorithm. That's where heuristics come in.
2.2 Heuristics
A heuristic is a rule of thumb that reduces the amount of search we need to perform. For example:
"If a database query suddenly becomes slow after a deployment, check the query plan."
Or:
"If latency increases while throughput stays constant, investigate downstream dependencies."
A heuristic doesn't guarantee a solution. It simply tells us to: "Try looking this part of seach space first."
This is really valuable in searching for solution because the theoretical solution space is enormous. Without any heuristics, we'd have to explore wider solution space with far too many possibilities.
2.3 Intuition and Pattern Recognition
Sometimes the search becomes even faster. An experienced engineer sees a problem and immediately says: "This smells like a connection pool issue."
That can look like magic. Usually it isn't. It is pattern recognition. Experience has built a large collection of associations.
The conscious reasoning might only see the final result: "Check the connection pool". But underneath it may be years of accumulated examples.
This is why experienced engineers can sometimes solve problems without being able to immediately explain every step of their reasoning. Their brain may have performed much of the search implicitly. However, intuition has an important limitation: Intuition is only as good as the patterns it was trained on. It tends to work better when:
- the environment is familiar
- patterns repeat
- feedback is reliable
- the consequences of previous decisions are observable
It becomes much less trustworthy when the problem is novel, the system is changing, or feedback is poor. In such cases, relying on senior's intuition would become a trap!
2.4 Trial and Error
Another strategy is simply: "Try something -> observe what happens -> update the model."
Software engineers do this constantly:
- prototype
- benchmark
- run experiments
- add instrumentation
- change one variable
- deploy behind a feature flag
- reproduce bugs
- test hypotheses
This isn't necessarily "random guessing". A good experimentation is structured search. Instead of trying to reason about the entire solution space (What could possibly work?), we ask:
"What small experiment would eliminate the largest number of possibilities?"
That is a very different way of thinking.
2.5 Means–Ends Analysis
Another common strategy is to compare the current state with desired state.
Current state ----Some Actions------> Desired state
Then ask:
"What action would reduce this difference?"
For example:
Current:
API latency = 3s
Goal:
API latency < 500ms
Difference:
2.5s
Possible actions:
- remove query
- optimize query
- cache result
- parallelize requests
- reduce payload
- eliminate downstream call
This is a very natural engineering strategy.
But it has a weakness: This form of search for solution paths are not the best solution for most situations. The biggest improvement isn't always the one that reduces the most visible difference (Think of Greedy Search algorithms and limitations these algos have).
You can optimize the wrong layer of system architecture extremely efficiently hoping it answers to all kinds of latency issues.
2.6 Working Backward
Sometimes it's easier to start from the goal and instead of asking:
"What can I do from here?"
ask:
"What must be true immediately before I reach the goal?"
For example if the goal is:
- Payment successfully completed
What must be true?
Payment authorized
What must be true?
Valid payment requestWhat must be true?
Order contains valid payment information
Working backward is particularly useful for complex workflows, algorithms, planning, and debugging. Reducing steps between current and desired state, will reduce amount of possible solutions drastically.
2.7 Analogical Reasoning
Another powerful strategy is:
"Have we solved something structurally similar before?"
For example:
"This isn't really a new caching problem. It's structurally similar to the invalidation problem we solved in the recommendation system."
The important word is structurally.
Two problems can look completely different on the surface while sharing the same underlying structure. This is one of the ways expertise becomes reusable.
It shares a lot with intuition, but they are different in how solution patterns are memorized or retrieved from memory: Intuition has an implicit way of remembering them (often the expert has no idea how he has reached this solution), but analogical reasoning is about explicitly finding the similar cases, where the expert knows what these two cases have in common and what are the differences.
2.8 A short wrap-up
Let's have a short summary of different search strategies, which I like to symbolize with solution search in multi dimentional space:
Then concepts become intuitive:
| Concept | Explanation |
|---|---|
| Local search | climb the nearby hill |
| Global search | move to another region |
| Mental model | determines what the landscape looks like |
| Heuristic | tells you where to look |
| Intuition | recognizes promising regions |
| Bias | keeps pulling you toward a familiar region |
| Exploration | discover new regions |
| Experiment | get information about the landscape |
| Feedback | update your map |
| Expertise | better map + better search heuristics |
| Overengineering | optimizing a hill nobody needs to climb |
3. Sequential and Parallel Processing Search
So far, we've mostly described problem solving as if a single thinker were walking through a search tree from A to B to C. But human cognition is more complicated than that.
Some mental processes require focused and sequential attention. Others can happen relatively automatically and in parallel. Think about what happens when you're debugging a difficult problem. You might consciously reason:
The request reaches the API, then the service calls the database, so the next thing I should inspect is...
Your conscious attention is following a fairly narrow sequence. At the same time, other processes are operating outside that focused stream:
- Memories are being activated.
- Associations are being triggered.
- Patterns are being compared.
- Pieces of previously encountered problems may become relevant.
You might suddenly think:
"Wait. This looks like that connection-pool problem from six months ago."
You didn't necessarily consciously enumerate that possibility. It surfaced.
This distinction is useful, but it needs a caveat: "conscious" and "subconscious" are not two perfectly separate processors inside the brain. Human cognition involves many interacting processes, and the popular picture of a fully conscious serial processor running alongside a fully unconscious parallel processor is not explaining how our minds work correctly.
Still, as a working model, the distinction helps explain something familiar.
Focused reasoning is expensive
When you're actively reasoning about a difficult problem, your attention is limited. You can hold only a relatively small amount of information in active working memory.
So you might spend 30 minutes trying to reason through:
A -> B -> C -> D -> ?
And feel completely stuck. Then you walk away. You make coffee. You take a shower. You work on something else. And suddenly:
"Oh. What if C isn't necessary?"
The solution seems to have appeared from nowhere. It probably didn't.
4. Why Leaving a Problem Alone Can Help
This phenomenon is often described as incubation.
The basic idea is that temporarily stopping conscious work on a problem can sometimes improve the chance of finding a solution later. Stepping away removes the problem from the center of your attention. And by moving your attention, the approach you are trying to make it work becomes less dominant in you mind. That reduces the cognitive load associated with continuously maintaining the same unsuccessful line of reasoning.
There is a practical pattern worth taking seriously:
Continuous effort is not always the same thing as continuous progress.
Sometimes the best next step in a difficult search is to stop searching temporarily.
5. Why Good Engineers Get Stuck
At this point, we can explain one of the strangest things about problem solving:
Intelligence and experience don't make you immune to getting stuck.
In some situations, experience can actually make you more resistant to a new solution.
The reason is that successful problem solving creates habits. If a particular strategy has worked many times before, the mind becomes increasingly likely to use it again.
This is efficient but also needs to be careful about another point we discuss next.
5.1 Mental Set
A mental set is a tendency to approach a problem using a strategy that has worked previously. Imagine an engineer who has solved several performance problems by adding caching. Eventually, a new performance problem appears.The immediate thought becomes: "We need a cache."
Maybe that's correct, but perhaps the actual problem is: "We're doing work that shouldn't happen at all."
The previous solution has become the starting point for the new search. Experience has reduced search cost—but it has also narrowed the search.
This is the central trade-off:
Expertise makes familiar problems easier by shrinking the search space. But for novel problems, that same compression can hide alternative solutions.
5.2 Functional Fixedness
Functional fixedness is a related tendency: seeing something primarily in terms of its familiar function.
In engineering, this happens in many situations when a queue becomes "the thing that handles background jobs." or a database becomes: "where we store persistent data."
Those definitions are useful. But they can also constrain our thinking. What if the queue could become a coordination mechanism? What if the database could eliminate an entire service?
Sometimes the existing architecture doesn't just implement our mental model, it reinforces it.
5.3 Anchoring
Anchoring happens when an initial piece of information disproportionately influences subsequent reasoning.
For example, A new incidemt has happened and the incident started after the database migration.
This fact may be useful. But once it becomes the anchor, the investigation can quietly turn into: How did the migration break the system?, instead of: What changed in the system, and what explains the observed symptoms?
The first question searches around the anchor. The second searches around the evidence.
5.4 Premature Closure
Another common failure is finding an explanation that is good enough to stop searching.
For example:
Symptom -> Plausible explanation -> "That's probably it." -> Stop
The explanation may even be correct. But sometimes it explains only part of the evidence. A useful discipline is to ask: "What evidence would we expect to see if this explanation were true?" and "What evidence would contradict it?"
That turns an explanation into a hypothesis that can actually be tested.
5.5 Confirmation Bias
Once we have a hypothesis, we naturally tend to notice information that supports it. Suppose:
"The database is overloaded."
You might immediately look for:
- high CPU
- slow queries
- connection saturation
But you might pay less attention to:
- a downstream API timing out
- a network problem
- increased application-side serialization
- a deployment that changed request behavior
The problem isn't that engineers are irrational. It's that search is guided by the current hypothesis. Once you've decided what you're looking for, you don't explore the entire problem space equally.
5.6 Sunk Cost
Another trap is continuing with a solution because you've already invested heavily in it. But the time already spent is not a reason for the solution to be correct.
This is particularly difficult in engineering because abandoning a technical direction can feel like admitting failure.
Sometimes, however, abandoning a bad search path is exactly what good problem solving looks like.
5.7 Overengineering
There is one more failure mode worth mentioning because it is particularly common among experienced engineers.
We don't always solve the problem we have, sometimes we solve the problem we expect to have later.
A requirement says "We need to process 10,000 events per day", and we design for "What if we eventually process 10 billion events per second?"
The resulting system may be technically impressive, but the additional complexity is itself a cost. This is another search-space problem.
We've expanded the set of constraints until we're optimizing for a hypothetical future rather than the actual problem.
Conclusion
I hope this post has sparked your curiosity! :)
The goal of this first part was to clarify a few concepts and help us better understand how our minds work. One valuable outcome would be for readers to start recognizing these patterns in their own thinking and when approaching different problems.
In the next sections, I'll share my thoughts on what we can do to become better problem solvers.
Top comments (1)
Really thoughtful post! Documenting real-world engineering hurdles and actionable solutions like this brings immense value to the community.