I Built an AI Security Assistant That Remembers Previous Investigations

adithya9666 — Mon, 28 Sep 2026 19:37:36 +0000

Security alerts are rarely completely new.

A security analyst might see dozens of failed-login alerts, suspicious IP addresses, unusual data transfers, or large file movements. Many of these incidents resemble cases the team has already investigated.

The problem is that a typical LLM starts each analysis from scratch.

It can understand the alert in front of it, but it doesn't automatically know how the security team handled a similar incident last week.

I built ThreatMemory to explore a different approach: an AI security alert triage assistant with persistent memory.

The key idea is simple:

Instead of only asking an AI what it thinks about an alert, let it remember what the security team learned from previous alerts.

The problem with stateless alert analysis

Consider a security alert like this:

36 failed authentication attempts against payroll@company.com from IP 185.20.4.21 between 01:50 AM and 02:05 AM.

A normal AI assistant can analyze the information contained in that alert.

It might identify several possibilities:

A legitimate employee repeatedly entering the wrong password
A VPN-related authentication problem
A brute-force attempt
Credential stuffing
A compromised device

Without additional context, the AI has no way to know how this organization's security team handled similar events previously.

That means every alert becomes a new investigation.

ThreatMemory adds a memory layer to this process.

The architecture

The application has three main components:

Security analyst → ThreatMemory → Hindsight + LLM

The workflow is:

The analyst provides a security alert.
ThreatMemory sends the alert to Hindsight for relevant historical cases.
Hindsight recalls previous investigations and analyst decisions.
The retrieved cases are provided to the LLM as context.
The LLM produces a recommendation and investigation steps.
The analyst makes the final decision.
The analyst's decision and reasoning are retained in Hindsight.
Future alerts can retrieve that experience.

This creates a continuous loop:

Alert → Recall → Analyze → Analyst Decision → Retain → Future Recall

Hindsight is therefore not just another component in the application. It is the part that allows previous investigations to become usable context for future ones.

The before-and-after difference

The most important part of ThreatMemory is the difference between analyzing an alert with memory and without memory.

Memory OFF

When Hindsight memory is disabled, ThreatMemory explicitly tells the LLM:

Memory is OFF.

Do not use any historical cases.
Analyze this alert only from the information contained in the alert itself.

For the example alert, the AI identified the unusually high number of failed attempts and recommended further investigation.

That is reasonable.

But it is working with only the current alert.

Memory ON

Now the same alert is analyzed with Hindsight enabled.

ThreatMemory retrieves relevant historical investigations.

For example, previous cases may show that similar failed-login alerts originated from the organization's corporate VPN infrastructure and were ultimately classified as false alarms.

The LLM can now consider that history alongside the current alert.

Instead of starting from zero, it has organizational context.

The important distinction is that ThreatMemory does not blindly copy an old decision.

The prompt explicitly tells the model:

When historical cases are provided, use them as context.
Memory should influence the recommendation naturally.
Do not blindly copy an old decision.
The analyst always makes the final decision.

This matters because a similar-looking alert can still represent a completely different incident.

How Hindsight is used

The core memory operation is retrieval.

Conceptually, ThreatMemory takes the current alert and asks Hindsight:

“What previous investigations are relevant to this alert?”

The application then passes the retrieved cases to the LLM as historical context.

A simplified part of the implementation looks like this:

if memory_enabled and memories:
memory_text = "\n\n".join(
f"PAST CASE {i + 1}:\n{m['text']}"
for i, m in enumerate(memories)
)

context = f"""

Historical security cases retrieved from Hindsight:

{memory_text}
"""

The important part is that the model isn't receiving an arbitrary collection of documents.

The historical cases are retrieved specifically in response to the current alert.

That allows semantically similar incidents to become relevant even when the wording or exact details differ.

Memory doesn't replace the analyst

One design decision was important from the beginning:

ThreatMemory is decision support, not an autonomous security system.

The AI provides:

A recommendation
Confidence
Reasoning
Recommended investigation steps
Relevant historical cases

But the analyst makes the final decision.

The interface contains a dedicated Analyst Decision section with three options:

False Alarm
Real Threat
Needs Investigation

The analyst can also provide a reason for the decision.

That decision is then stored back into Hindsight.

For example:

The source IP was confirmed as corporate VPN infrastructure and the employee verified the login attempts as legitimate.

This transforms an analyst's investigation from a one-time action into future context.

The learning loop

This is where the system becomes more interesting than a simple RAG application.

Suppose an analyst investigates an alert and discovers that it was a legitimate VPN event.

ThreatMemory stores that outcome.

Later, another alert appears with similar characteristics.

Hindsight can retrieve the previous investigation, allowing the AI to consider the team's previous experience.

The system therefore has two directions of memory:

Recall:
“What have we learned about cases like this?”

Retain:
“What did the analyst learn from this case?”

Over time, the memory store can contain different types of experience:

Repeated false alarms
Confirmed attacks
Legitimate backup activity
Business-travel login events
Analyst overrides
Previously unknown patterns that were later resolved

This makes the memory useful beyond simply remembering conversations.

What I learned building it

The biggest lesson was that adding memory isn't automatically useful.

The memory has to affect the actual workflow.

A system that retrieves five old records but doesn't change how the current task is handled isn't meaningfully using agent memory.

For ThreatMemory, the memory layer is directly connected to the decision process:

Current alert → Relevant experience → AI reasoning → Analyst decision → New experience

Another important lesson was to keep the scope narrow.

Instead of trying to build a complete security operations platform, ThreatMemory focuses on one workflow: security alert triage.

That makes the role of memory easy to understand and easy to demonstrate.

A limitation

ThreatMemory is currently a prototype using realistic synthetic security cases.

That means its historical memory is only as useful as the information stored in it.

A production system would need much stronger safeguards around data quality, access control, privacy, retention policies, auditability, and the accuracy of analyst decisions.

It would also need integration with real security systems such as authentication logs, SIEM platforms, endpoint telemetry, and incident-management systems.

The current application deliberately stops before autonomous response.

It recommends.

The analyst decides.

Project Links

🔗 Live Demo: https://threatmemory-f2kdfnwvpefa9isiquicgxv.streamlit.app

💻 GitHub Repository: https://github.com/adithya9666/threatmemory

What's next

The next step would be to connect ThreatMemory to real organizational security data and allow the memory to grow naturally from real investigations.

The broader idea goes beyond cybersecurity.

Many professional workflows have the same problem: people repeatedly make decisions using information that their organization has already learned, but that knowledge is scattered across previous cases.

Persistent agent memory creates a way for AI systems to carry that experience forward.

For ThreatMemory, the goal is straightforward:

Don't make the analyst investigate every alert as if it has never happened before.

Give the AI access to what the team has already learned — while keeping the human analyst in control.

DEV Community: adithya9666

I Built an AI Security Assistant That Remembers Previous Investigations

Project Links