Part 2 of the "Pragmatic AI Adoption" series
In the first article, I explored a question I've been thinking about more often:
How much AI do we actually need?
That naturally leads to another question:
If AI is the right direction, what kind of solution should we actually build?
Interestingly, I don't think this is where the decision starts.
Too often, solution discussions quickly become:
- Should we build a chatbot?
- Should we use RAG?
- Should we build an AI agent?
Those are important questions.
But I believe they're implementation decisions, not architecture decisions.
The architecture decision comes first.
Start with the Problem, Not the Technology
When a new requirement arrives, our instinct is often to think about technology.
Instead, I think we should first understand the characteristics of the problem.
For example:
- Is the outcome deterministic?
- Does it depend on business rules?
- Does it require searching large amounts of information?
- Does it involve reasoning or interpretation?
- Does it require taking actions across multiple systems?
The answers to those questions often narrow the solution considerably.
Different Problems Need Different Solutions
Here's a simple mental model I've found useful.
| Problem Characteristics | Solution Direction |
|---|---|
| Well-defined business rules | Traditional application or workflow |
| Structured data and reporting | Analytics / BI |
| Finding information | Search or RAG |
| Content generation or summarization | LLM |
| Multi-step reasoning and action | AI Agent |
The point isn't that one technology is better than another.
It's that they solve different kinds of problems.
AI Is One Capability in the Architecture
Sometimes discussions make AI sound like the application itself.
I see it differently.
AI is another architectural capability.
Just like:
- APIs
- Databases
- Search
- Workflow engines
- Rules engines
- Analytics platforms
The challenge isn't choosing AI over traditional software.
It's choosing the right combination of capabilities.
The Cost of Choosing the Wrong Pattern
Every architectural choice introduces trade-offs.
For example:
A traditional application offers:
- Predictability
- Explainability
- Lower operational complexity
An AI-enabled solution introduces:
- Greater flexibility
- Better handling of ambiguity
- New governance requirements
- Evaluation and monitoring needs
- Higher operational complexity
Neither approach is universally better.
They simply optimize for different outcomes.
A Question Worth Asking Early
Instead of asking:
"Which AI technology should we use?"
I've started asking:
"What characteristic of this problem makes AI necessary?"
Sometimes the answer is obvious.
Sometimes the answer is:
It doesn't.
And that's perfectly acceptable.
Architecture Before AI
Technology decisions are often easier once the problem is well understood.
Choosing between:
- Traditional software
- Search
- RAG
- LLMs
- AI agents
should be a consequence of understanding the problem—not the starting point.
Perhaps the most important architecture decision isn't selecting the most advanced technology.
It's selecting the simplest solution that satisfies the business need.
What's Next
In the next article, I'll explore another question that I think is becoming increasingly important:
When NOT to use AI.
Sometimes the best architecture decision isn't selecting a different AI technology.
It's deciding that AI isn't the right solution at all.
Final Thoughts
AI has expanded what's possible in software.
But it hasn't changed one of the fundamental principles of architecture:
Technology should follow the problem—not the other way around.
As AI continues to evolve, I believe the organizations that gain the most value won't necessarily be the ones using the most AI.
They'll be the ones making thoughtful decisions about where AI genuinely belongs.
Top comments (0)