When making changes to your codebase, it's easy to accidentally break existing functionality while implementing new features or fixing bugs. The Oracle in Amp excels at analyzing code changes and identifying potential regressions that might not be immediately obvious from a simple diff review.
This guide walks through a practical scenario where we'll make changes to a simple application, then use the Oracle to detect and fix any regressions in core functionality.
Our example application
For this walkthrough, we'll use a simple task management API. Here's the core task completion logic we want to protect:
// server.js \- Key functionality we're monitoring
app.patch('/tasks/:id/complete', (req, res) \=\> {
const taskId \= parseInt(req.params.id);
const task \= tasks.find(t \=\> t.id \=== taskId);
if (\!task) {
return res.status(404).json({ error: 'Task not found' });
}
task.completed \= true;
task.completedAt \= new Date();
res.json(task);
});
With the code above, let’s assume we have committed this working version as our baseline in git
## **Making changes that might introduce regressions**
As per usual development processes, we will take our baseline PoC code and begin to enhance it. Let’s say we are looking to add task editing capabilities and improve validation. Here are the key changes we might make to our completion logic:
// Enhanced validation function (NEW)
function validateTask(title, priority) {
const validPriorities \= \['low', 'medium', 'high', 'urgent'\];
if (\!title || title.trim().length \=== 0\) return 'Title required';
if (priority && \!validPriorities.includes(priority)) return 'Invalid priority';
return null;
}
// Updated task completion with new validation
app.patch('/tasks/:id/complete', (req, res) \=\> {
const taskId \= parseInt(req.params.id);
const task \= tasks.find(t \=\> t.id \=== taskId);
if (\!task) {
return res.status(404).json({ error: 'Task not found' });
}
// NEW: Prevent double-completion
if (task.completed) {
return res.status(409).json({ error: 'Task already completed' });
}
task.completed \= true;
task.completedAt \= new Date().toISOString(); // Changed format
res.json(task);
});
With these changes in place, let's commit the changes in git either manually, or via Amp:
Using Oracle to detect regressions
Now we want to ensure our changes haven't broken the core task completion functionality. Let's ask the Oracle to analyze our changes*.* Here's our prompt to Amp, which will invoke the Oracle to begin its analysis:
Use the oracle to review the last commit's changes. I want to make sure that the actual logic for task completion is unchanged and still works as it did before.
Then, in Amp, we will see the Oracle take over and begin its work.
Oracle response: No regression detected
In the case of the code we added, we know that the logic for task completion remains intact and doesn’t suffer from any regression issues. The Oracle’s output confirms this:
Oracle response: Regression detected
But what if our changes had introduced a regression? Let's instead imagine that we’ve unintentionally added this problematic change to our code base:
// Problematic version that breaks task completion
app.patch('/tasks/:id/complete', (req, res) \=\> {
const taskId \= req.params.id; // BUG: Missing parseInt()
const task \= tasks.find(t \=\> t.id \=== taskId);
if (\!task) {
return res.status(404).json({ error: 'Task not found' });
}
task.completed \= true;
task.completedAt \= new Date().toISOString();
res.json(task);
});
This seemingly minor change creates a critical bug that's easy to miss in code review. When a request comes in to /tasks/1/complete, req.params.id returns the string "1", not the integer 1. Our task IDs are stored as integers in the array.
The line tasks.find(t => t.id === taskId) now compares integer 1 with string "1". Due to JavaScript's strict equality (===), this comparison always returns false, making the function unable to find any task, regardless of whether it exists. Let’s use Amp to push this code up to git.
This is precisely the type of subtle regression the Oracle excels at detecting: a change that looks innocuous but completely breaks functionality through type system interactions. Without running the code and testing it, reviewers may not notice this immediately. As an extra line of regression defense, let’s once again run the Oracle to prove that it can find this type of regression:
Now, we can see that the Oracle has detected the issue and also given a suggested fix. Since the Oracle is read-only, we will need to get the Amp agent to make the fix for us.
Using Amp to fix the detected regression
When the Oracle detects a regression, we can immediately request that Amp fix it. In our particular case above, we could use the following prompt:
The oracle found a regression in task completion due to the missing parseInt(). Please fix this issue while preserving all the other improvements we made in the last commit.
From here, Amp gets to work on fixing the regression:
Amp then restores the correct type conversion, tests the fixed endpoint, and then commits the fix to git.
Then, Amp puts out a final message to us letting us know that the code is committed and fixed. This demonstrates how we can utilize the Oracle to identify potential regressions and how to utilize Amp to resolve them, ensuring everything functions as intended.
Best practices for regression detection
Although the Oracle is quite smart, there are still certain best practices when it comes to regression detection that you’ll want to keep in mind. The Oracle is most effective when you:
Be specific about what functionality to check: Instead of "check for regressions," specify "check that user authentication logic is unchanged."
Focus on critical paths: Prioritize checking core business logic, security-sensitive code, and frequently-used features
Use after significant refactoring: The Oracle excels at spotting subtle changes in complex refactoring operations
Combine with testing: Use Oracle analysis alongside your existing test suite for comprehensive coverage
Ask follow-up questions: If the Oracle finds issues, ask for specific fix recommendations or impact analysis
When to use Oracle for regression analysis
Ideal scenarios:
- After large refactoring operations
- When modifying shared utilities or core business logic
- Before deploying critical fixes to production
- When multiple developers have worked on related code
Less suitable for:
- Simple, isolated changes
- Purely additive features that don't touch existing code
- Changes with comprehensive test coverage already
The Oracle transforms regression detection from a time-consuming manual process into a systematic analysis that catches issues before they reach production.
Top comments (0)