Introduction
Production incidents are an unavoidable part of software engineering. When an application starts returning HTTP 500 errors, database timeouts, authentication failures, or unexpected latency spikes, engineers need to quickly understand what happened, identify the root cause, and restore the service. The challenge is that many incidents are not completely new. Similar problems may have happened weeks or months earlier, but the knowledge about how those incidents were diagnosed and resolved is often difficult to retrieve.
This led us to build ResolveIQ, an AI-powered incident response agent that combines AI reasoning with long-term memory. The goal is not simply to ask an LLM to analyze an error, but to give the agent access to the organization's previous incident experience so that it can use what was learned from earlier failures.
The project was built around the idea of AI agents that learn using Hindsight.
The Problem
Traditional incident investigation often involves multiple steps. An engineer receives an alert, checks application logs, searches monitoring dashboards, investigates the database or infrastructure, looks through previous tickets, and then decides how to resolve the problem.
The difficult part is that previous solutions may already contain valuable information.
For example, suppose a Payment API starts returning HTTP 500 errors because the database connection pool is exhausted. If the same problem occurred previously and the engineering team solved it by increasing the connection pool from 50 to 100, that historical experience could be extremely useful.
However, a normal LLM does not automatically know this organization's previous incident history.
ResolveIQ addresses this gap by giving the AI agent long-term incident memory.
What Is ResolveIQ?
ResolveIQ is an AI-powered incident response system that helps engineers investigate production incidents using both current incident information and historical experience.
The system has five major stages:
Incident
↓
Hindsight Recall
↓
Groq AI Reasoning
↓
Engineer Action
↓
Hindsight Retain
↓
Future Incident
The important idea is that the system does not stop after generating a recommendation.
After the incident is resolved, the resolution becomes part of the agent's memory.
Therefore:
Every resolved incident becomes experience for future investigations.
ResolveIQ Dashboard
The first component is the ResolveIQ Dashboard.
The dashboard provides engineers with a centralized view of active production incidents. Each incident contains information such as the incident ID, service name, severity, error rate, affected users, current status, and incident signals.
For our demonstration, one of the main incidents is:
INC-245 — Payment Service HTTP 500 Spike
The incident contains information such as a 35% error rate, approximately 12,430 affected users, and signals indicating database connection acquisition timeouts and connection pool saturation.
Instead of manually searching through different systems, the engineer can select the incident directly from the ResolveIQ dashboard.
AI Investigation
After selecting an incident, the engineer can start an AI Investigation.
The React frontend sends the incident information to the FastAPI backend. The backend coordinates the investigation by retrieving structured incident information from PostgreSQL and requesting relevant historical experience from Hindsight.
The current incident is then combined with the historical information before being sent to the Groq-powered reasoning layer.
This gives the AI more context than simply looking at the current error.
PostgreSQL — Structured Incident Data
PostgreSQL is responsible for storing structured operational information.
For example:
Incident ID
Service
Severity
Error Rate
Affected Users
Logs
Status
Resolution
For INC-245, PostgreSQL contains the structured details of the Payment Service incident.
PostgreSQL answers the question:
“What is happening in the current incident?”
Hindsight answers a different question:
“Have we experienced something like this before?”
Hindsight — Long-Term Memory
Hindsight is the memory layer of ResolveIQ.
Hindsight is designed as an agent memory system focused on helping agents learn rather than simply remember. Its documented core operations include Retain, Recall, and Reflect.
In ResolveIQ, we use Hindsight to store production incidents, root causes, resolutions, and outcomes.
For example, a previous incident might be stored as:
INC-127 — Payment API Failure
Problem:
Database connection pool exhaustion.
Root Cause:
Connection pool reached its maximum capacity.
Resolution:
Increase connection pool from 50 to 100.
Outcome:
Successful.
This information becomes part of the ResolveIQ incident memory.
Hindsight Recall
When INC-245 occurs, ResolveIQ doesn't only analyze the current error.
It asks Hindsight to recall relevant previous experiences.
The query can be conceptually expressed as:
Find previous incidents where:
Payment API
+
HTTP 500
+
Database connection timeout
+
Connection pool exhaustion
Hindsight retrieves relevant memories using its memory and retrieval mechanisms. Its architecture combines different retrieval signals including semantic, keyword, graph, and temporal information.
For our example, Hindsight can retrieve INC-127 because it contains a similar Payment API failure caused by database connection pool exhaustion.
This is where the memory layer becomes valuable.
Groq AI Reasoning
The next stage is Groq AI reasoning.
Groq receives two important pieces of information:
Current Incident
+
Historical Experience
The current incident tells the model what is happening now.
Hindsight tells the model what happened previously.
Groq then reasons over both pieces of information to generate an investigation result.
For example:
Current:
Payment Service → HTTP 500
ConnectionPoolTimeoutException
Connection pool saturated
Previous:
Payment API → HTTP 500
Connection pool exhausted
Pool increased 50 → 100
Successful
The agent can then identify the likely relationship between the two incidents.
Root Cause Analysis
For our demonstration, the probable root cause is database connection pool exhaustion.
The sequence is:
High production load
↓
Database connections become saturated
↓
New requests cannot obtain connections
↓
Connection acquisition timeout
↓
Payment request fails
↓
HTTP 500
ResolveIQ presents this reasoning to the engineer instead of requiring them to manually reconstruct the entire chain.
Recommended Resolution
Based on the current incident and the historical incident retrieved from Hindsight, the agent can recommend:
Increase database connection pool
from 50 → 100
Restart affected service
Monitor HTTP 500 rate
and database connections
The recommendation is presented to the engineer as decision support.
The system does not need to automatically execute potentially destructive production operations.
The engineer remains responsible for reviewing and applying the appropriate action.
Learning From the Resolution
This is the most important part of ResolveIQ.
After the engineer resolves INC-245, the system records the outcome.
For example:
INC-245
Root Cause:
Database connection pool exhaustion
Resolution:
Pool increased from 50 → 100
Outcome:
Successful
ResolveIQ then retains this information in Hindsight.
This means INC-245 becomes another piece of historical experience.
The next time a similar incident occurs, Hindsight can potentially retrieve both INC-127 and INC-245.
The Continuous Learning Loop
This creates the core learning loop:
New Incident
↓
Hindsight Recall
↓
Historical Experience
↓
Groq AI Reasoning
↓
Root Cause + Recommendation
↓
Engineer Applies Fix
↓
Hindsight Retain
↓
New Learned Experience
↓
Future Incident
This is the central concept behind ResolveIQ.
The system becomes more useful as more resolved incidents are retained in its memory.
Hindsight Memory Graph
One of the interesting aspects of Hindsight is its ability to organize memories and relationships.
In our Hindsight interface, memories can be visualized as a graph containing nodes and connections. Hindsight's documented architecture separates different types of information and relationships, allowing agents to work with structured long-term memory rather than only flat text.
For ResolveIQ, the memory graph can conceptually connect:
Payment Service
|
↓
HTTP 500
|
↓
Connection Timeout
|
↓
Database Connection Pool
|
↓
Pool Exhaustion
|
↓
Resolution
|
↓
50 → 100
|
↓
Successful Outcome
This gives us a visual representation of how different pieces of incident knowledge are connected.
System Architecture
The overall ResolveIQ architecture is:
React Frontend
|
↓
FastAPI Backend
|
┌─────────┴─────────┐
↓ ↓
PostgreSQL Hindsight
Structured Data Long-Term Memory
| |
└─────────┬─────────┘
↓
Groq LLM
|
↓
AI Investigation
|
↓
Root Cause + Recommendation
|
↓
Engineer Action
|
↓
Hindsight Retain
Each component has a specific responsibility.
React provides the engineer interface.
FastAPI coordinates the backend workflow.
PostgreSQL stores structured incident information.
Hindsight provides long-term memory.
Groq performs AI reasoning.
Together, these components form the ResolveIQ incident investigation and learning system.
Why This Approach?
The main idea is that an AI agent should not have to start from zero every time an incident occurs.
A conventional approach might look like:
New Incident
↓
LLM analyzes current information
↓
Recommendation
ResolveIQ adds organizational experience:
New Incident
↓
Retrieve Previous Experience
↓
LLM analyzes Current + Historical Information
↓
Recommendation
↓
Resolution
↓
Store New Experience
This makes long-term memory a central part of the incident-response workflow.
Technology Stack
ResolveIQ was built using:
React.js — Frontend dashboard
FastAPI — Backend API
Python — Agent and backend services
PostgreSQL — Structured incident storage
Hindsight — Long-term agent memory
Groq — LLM reasoning
Hindsight is open source under the MIT license, and its official repository documents its memory concepts and integration options.
Future Improvements
The current project demonstrates the core memory-driven incident response workflow. A production version could connect ResolveIQ directly to observability and monitoring systems.
For example:
Application
↓
Logs / Metrics / Alerts
↓
Incident Detection
↓
ResolveIQ
↓
Hindsight + Groq
↓
Engineer
Future versions could integrate with systems such as application logging, metrics, alerting, tracing, and incident-management platforms.
Another potential improvement would be adding more sophisticated incident correlation, evaluation of recommended resolutions, and automated post-incident summaries.
Conclusion
ResolveIQ explores how long-term memory can change the way AI agents assist software engineers.
Instead of treating every production incident as an isolated event, ResolveIQ connects the current incident with previous engineering experience. Hindsight provides the memory layer, Groq provides reasoning, PostgreSQL stores structured incident data, FastAPI coordinates the backend workflow, and React provides the engineer-facing interface.
The central idea is simple:
Production incidents should not just be resolved. They should become knowledge for the next incident.
That is the idea behind ResolveIQ — an AI incident response agent that learns from every production failure.
Top comments (1)