Over the last few months, I’ve been building an AI application with three priorities:
Quality. Robustness. Accessibility.
But there is another constraint that I’ve started thinking about much more seriously:
How do we make AI intelligent without making every request expensive?
It is very easy to build an AI application today.
Take an LLM API.
Add a system prompt.
Add some context.
Connect a few tools.
Build a UI around it.
And you have an AI application.
There is absolutely nothing wrong with this approach. In fact, this is how many useful AI products are built.
But while working on my own system, I started thinking about a different question:
What happens when we stop thinking of the LLM as the entire intelligence of the application and instead make it one component of a larger learning system?
That question led me toward reinforcement learning, decision-making policies, context selection, adaptive routing, and, most importantly, learning how to use an LLM intelligently.
The AI Wrapper Problem
Let's start with something that is often misunderstood.
An AI wrapper is not inherently bad.
A wrapper can provide enormous value by combining an existing model with:
- Better prompts
- Retrieval
- APIs
- Tools
- Memory
- Structured outputs
- Application-specific workflows
- A good user experience
The problem isn't the wrapper itself.
The problem is assuming that adding more layers around an LLM automatically means that the application itself is learning.
Consider a simple architecture:
User
↓
Prompt
↓
Context
↓
LLM
↓
Response
We can make this architecture considerably more sophisticated:
User
↓
Prompt
↓
RAG
↓
Vector Database
↓
Tools / APIs
↓
LLM
↓
Response
Now we have a powerful application.
But ask yourself:
What happens tomorrow if the same situation occurs again?
If the system doesn't remember the outcome or change its decision-making strategy based on feedback, it is fundamentally executing a predefined workflow again.
The system may have memory.
It may have retrieval.
It may have tools.
But that doesn't necessarily mean it has learned.
Prompting ≠ Learning
This distinction is probably the most important one in my thinking.
With prompting, we essentially tell the model what we want it to do.
For example:
If the user asks about X,
retrieve information from Y,
then answer using Z.
We're defining the behaviour.
A learning system is different.
Instead of explicitly defining every decision, we provide a mechanism through which the system can learn which decisions produce better outcomes.
Conceptually:
Observe
↓
Choose Action
↓
Execute
↓
Observe Outcome
↓
Receive Feedback
↓
Update Policy
↓
Choose Better Action Next Time
The difference is subtle but important.
One approach primarily says:
"Follow these instructions."
The other asks:
"Given what happened previously, which action should I prefer now?"
That is a very different engineering problem.
Fine-Tuning ≠ Reinforcement Learning
Another distinction worth making is between fine-tuning and reinforcement learning.
Fine-tuning generally involves adapting a model using additional training data so that its behaviour becomes better aligned with a particular task, domain, or style.
For example:
General LLM
+
Domain-specific examples
↓
Fine-tuned model
Reinforcement learning approaches the problem from another direction.
Instead of simply saying:
"Here are examples of what I want."
we can define an objective and provide feedback about the quality of actions.
Conceptually:
State
↓
Policy
↓
Action
↓
Environment
↓
Reward / Feedback
↓
Policy improvement
The important word here is policy.
A policy is essentially a strategy for choosing actions given a particular state or situation.
And this is where I find reinforcement learning particularly interesting for LLM-based systems.
Do We Actually Need to Train the LLM?
This is where my thinking changed.
When people hear "reinforcement learning," they often immediately imagine:
"We're going to train the entire LLM."
But that isn't necessarily what we need.
Instead, we can think about the LLM as one component inside a larger decision-making system.
For example:
┌───────────────┐
│ User │
└───────┬───────┘
↓
┌─────────────────┐
│ Context / State │
└────────┬────────┘
↓
┌─────────────────┐
│ Decision Policy │
└────────┬────────┘
↓
┌──────────────┼──────────────┐
↓ ↓ ↓
Retrieve Use Tool Call LLM
│ │ │
└──────────────┼──────────────┘
↓
Outcome
↓
Feedback
↓
Policy Update
The LLM doesn't have to change every time.
The decision-making layer around the LLM can learn.
This is a much more interesting and potentially more practical architecture.
What Could the System Learn?
This is where things get interesting.
Imagine an AI application receiving a request.
It has several possible actions:
Action 1 → Answer directly
Action 2 → Retrieve documents
Action 3 → Call an external tool
Action 4 → Ask for clarification
Action 5 → Use a stronger model
Action 6 → Perform deeper reasoning
A traditional application might contain hard-coded rules:
if question_is_complex:
use_big_model()
if question_is_about_documents:
use_rag()
if question_is_calculation:
use_calculator()
These rules can work.
But they eventually become complicated.
What if the system encounters a situation we didn't anticipate?
Instead, we could potentially allow a policy to learn which actions tend to produce better outcomes.
For example:
Context A
→ Direct answer
→ Good result
→ Positive reward
Context B
→ Direct answer
→ Poor result
→ Negative reward
Context B
→ Retrieval + LLM
→ Good result
→ Positive reward
Over time, the policy can learn that certain states are better handled using retrieval rather than direct generation.
Token Efficiency
This is one of the main reasons I'm interested in this approach.
There is a common assumption in AI development:
More context = better answer.
That isn't always true.
Suppose an application has 100 pieces of potentially relevant information.
Do we really need to send all 100 to the model?
Maybe only 10 matter.
The other 90 increase:
- Input tokens
- Latency
- Cost
- Processing requirements
- Potential distraction
- Context complexity
So instead of:
Retrieve everything
↓
Send everything to LLM
↓
Hope the model figures it out
I'm interested in:
Retrieve candidates
↓
Evaluate relevance
↓
Select useful context
↓
Send only what matters
↓
Generate response
And eventually, the selection mechanism itself could improve from feedback.
More Tokens ≠ More Intelligence
This is probably one of the biggest misconceptions I have encountered while building AI applications.
If we give a model a massive amount of context, we haven't necessarily made the system smarter.
We may simply have made it more expensive.
Consider these two approaches.
Approach A
100 documents
↓
50,000 tokens
↓
LLM
↓
Answer
Approach B
100 documents
↓
Relevance selection
↓
8 useful documents
↓
4,000 tokens
↓
LLM
↓
Answer
If Approach B produces the same or better result, then the additional 46,000 tokens weren't intelligence.
They were overhead.
This is why I believe token efficiency should be treated as an engineering objective, not merely a billing concern.
The LLM as a Tool
This also changes how I think about LLMs.
Instead of:
"The LLM is the application."
I prefer:
"The LLM is one of the tools available to the application."
The system can decide:
Do I need an LLM here?
If yes:
Which model?
How much context?
Should I retrieve something?
Should I call a tool?
Should I reason more deeply?
Should I use a cheaper model?
Should I escalate to a stronger model?
That creates an additional layer of intelligence.
The model generates.
The system decides when and how generation should happen.
A Possible Architecture
A system I'm interested in exploring looks something like this:
USER
│
▼
┌─────────────────┐
│ State / Context │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Decision Policy │
└────────┬────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
Retrieval Tools Models
│ │ │
│ ┌────┴────┐ ┌───┴────┐
│ │ APIs │ │ Small │
│ │ Search │ │ Large │
│ │ DB │ │ Expert │
│ └─────────┘ └────────┘
│ │ │
└──────────────┼──────────────┘
▼
Outcome
│
▼
┌─────────────────┐
│ Feedback / Eval │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Policy Learning │
└────────┬────────┘
│
└──────────► Better decisions
The interesting part isn't necessarily the individual components.
We already have LLMs.
We already have vector databases.
We already have APIs.
We already have agents and tool calling.
The interesting part is the feedback loop connecting them.
Where Reinforcement Learning Fits
Reinforcement learning can potentially be applied to the decision layer.
For example, suppose the state is:
User query
+
Previous conversation
+
Available context
+
System state
+
Historical outcomes
The policy chooses:
Action = retrieve_context
The system performs retrieval and generates an answer.
Then we evaluate the outcome.
Perhaps the user accepts the answer.
Perhaps they ask the same question again.
Perhaps they correct the answer.
Perhaps an evaluator scores it highly.
Perhaps the task fails.
These signals can contribute to the reward.
Conceptually:
State
↓
Policy
↓
Action
↓
Result
↓
Reward
↓
Policy improvement
The goal isn't to predict the future perfectly.
It is to improve the probability of choosing better actions in future situations.
Reinforcement Learning Doesn't "Know the Future"
This distinction is important.
Sometimes discussions around AI learning make reinforcement learning sound almost magical.
It isn't.
An RL system doesn't suddenly understand the future.
It learns from experience.
If an action repeatedly produces good outcomes in particular situations, the system can learn to prefer that action.
If another action consistently performs poorly, the policy can learn to avoid it.
So the objective is closer to:
Past experience
↓
Learn patterns
↓
Estimate useful actions
↓
Make better future decisions
rather than:
Train once
↓
Know the future
The quality of the learning system therefore depends heavily on the quality of the feedback and reward design.
The Hardest Part: Reward Design
This is where the idea becomes significantly harder than simply integrating an API.
If I tell a system:
"Maximize reward."
I've still left a huge question unanswered:
What exactly is a good outcome?
Suppose the system generates an answer.
Should we reward:
- Accuracy?
- User satisfaction?
- Speed?
- Low token usage?
- Low cost?
- Completeness?
- Safety?
- Conciseness?
Usually, the answer is:
Some combination of all of them.
For example, we might conceptually define:
Reward =
Accuracy
+ User Satisfaction
+ Task Success
- Token Cost
- Latency
Obviously, a real system would require much more careful normalization, weighting, and evaluation.
But this illustrates the fundamental challenge.
If you optimize only for token usage, the system may become too aggressive about avoiding computation.
If you optimize only for answer quality, it might use expensive models and enormous contexts for everything.
The real challenge is finding the right balance.
Efficiency Shouldn't Destroy Quality
This is an important constraint in my own thinking.
I don't want a system that is cheap but produces mediocre results.
The goal is not:
"Use fewer tokens at any cost."
The goal is:
"Spend computation where it actually improves the outcome."
For an easy request:
Small model
+
Minimal context
+
Direct answer
For a complex request:
Retrieval
+
More context
+
Tool usage
+
Stronger model
+
Deeper reasoning
The system should ideally learn the difference.
That's much more interesting than forcing every request through the same pipeline.
Adaptive Model Routing
This idea can also extend to model selection.
Imagine having:
Small / inexpensive model
Medium model
Large / expensive model
Specialized model
A traditional implementation might always use the largest model.
That works, but it is expensive.
Another implementation might always use the cheapest model.
That saves money but can hurt quality.
An adaptive system could potentially learn:
Simple task
→ Small model
Moderate task
→ Medium model
Complex task
→ Large model
Specialized task
→ Specialized model
The decision isn't necessarily based on a manually written rule.
The policy can learn from outcomes.
Again, the goal isn't to eliminate expensive models.
It's to use them when they are actually worth the cost.
This Changes the Definition of an "AI Application"
This is probably the biggest conceptual shift for me.
I used to think about AI applications primarily as:
Application
+
LLM
+
Prompt
+
Tools
Now I increasingly think about them as:
Application
+
Models
+
Tools
+
Memory
+
State
+
Decision Policy
+
Feedback
+
Evaluation
+
Learning
The model is still extremely important.
But it becomes one part of the overall intelligence.
The system itself becomes responsible for deciding how that intelligence should be used.
What Makes This Difficult?
There are several problems that make this significantly harder than building a simple LLM application.
1. Reward Design
Bad rewards produce bad behaviour.
If the system is rewarded for minimizing tokens, it might sacrifice quality.
If it is rewarded only for user satisfaction, it may overuse expensive computation.
The objective needs to reflect the actual goal.
2. Feedback Quality
Not every user interaction provides useful feedback.
A user leaving the application doesn't necessarily mean the answer was bad.
A user saying "thanks" doesn't necessarily mean the answer was perfect.
Feedback needs to be interpreted carefully.
3. Exploration vs Exploitation
A learning system needs to balance:
Exploitation: use strategies that already work.
Exploration: try new strategies that might work better.
Too much exploitation can make the system stuck in a mediocre strategy.
Too much exploration can make it unpredictable.
4. Evaluation
You cannot simply say:
"The model seems better."
You need measurable evaluation.
For example:
Task Success Rate
Answer Quality
Token Usage
Latency
Cost per Successful Task
Tool Accuracy
Retrieval Precision
Failure Rate
The system needs to improve according to metrics that actually matter.
5. Safety and Constraints
Learning systems should not be allowed to optimize without boundaries.
There should be constraints around:
- Safety
- Privacy
- Tool access
- Cost
- Data usage
- Allowed actions
- Reliability
Learning doesn't eliminate engineering rules.
It changes where some of the decision-making happens.
I Don't Think Bigger Models Are the Only Answer
The AI industry has understandably focused heavily on model scale.
Larger models.
More parameters.
Larger context windows.
More compute.
Better reasoning.
And these improvements are incredibly important.
But there is another dimension to intelligence:
How effectively do we use the intelligence we already have?
Imagine two systems.
System A:
Huge model
+
Huge context
+
Every tool
+
Maximum computation
System B:
Appropriate model
+
Relevant context
+
Relevant tools
+
Adaptive computation
+
Feedback
System B may not have more raw intelligence.
But it may be a better system.
That's the direction I find interesting.
The Future: Bigger Models or Smarter Systems?
I don't think the answer has to be one or the other.
We'll probably continue building increasingly capable foundation models.
But I believe the systems built around those models will become just as important.
The next generation of AI applications may not simply ask:
"Which LLM should I use?"
They may ask:
"What is the best way to solve this particular problem?"
And the answer could involve:
Retrieve?
Reason?
Use a tool?
Ask the user?
Use a small model?
Use a large model?
Use more context?
Use less context?
Try again?
Stop?
That decision itself can become a learning problem.
My Current Perspective
I'm not trying to build another application that simply puts an LLM behind a nice UI.
I'm interested in building a system where the LLM is one component of a larger architecture.
A system that can:
Observe → Decide → Act → Evaluate → Learn → Improve
And importantly, the objective isn't simply to make AI more powerful.
It's to make it:
More adaptive.
More efficient.
More robust.
More accessible.
If an AI system can learn that it doesn't need 20,000 tokens to solve a problem that requires only 2,000, that's not just a cost optimization.
That's better engineering.
If it can learn when a tool is actually useful instead of calling it every time, that's better decision-making.
If it can learn when a stronger model is worth the additional cost, that's adaptive intelligence.
And if it can improve these decisions based on real outcomes rather than requiring us to manually encode every possible scenario, then we're moving beyond simply wrapping an AI model.
We're building a system that learns how to use intelligence.
Final Thought
Maybe the future of AI isn't only about building bigger models.
Maybe it is also about building systems that know when, where, and how to use those models.
The model provides capability.
The architecture provides control.
The feedback provides learning.
And the policy determines how the system evolves.
Not just AI that generates.
AI that learns how to make better decisions.
Top comments (0)