DEV Community: Priyanshu Kumar

ForgeMind: Building a Memory-Backed Factory Troubleshooting System

Priyanshu Kumar — Tue, 29 Sep 2026 18:22:34 +0000

By Bhupathi Mahesh Varun Kumar · Hindsight / AI / memory

When we started building ForgeMind, we wanted to solve a specific problem in factory troubleshooting: an AI system can generate a convincing explanation, but it is difficult to know whether that explanation came from the current incident, a previous factory incident, or the model's own reasoning.

We designed ForgeMind around a simple principle:

Historical experience should remain identifiable when it reaches the model.

That led us to combine Hindsight memory, a backend API layer, an investigation UI, analysis logic, integration testing, and a separate research and debugging workflow.

The result is a prototype where an incident can be analyzed against historical experience, recommendations can carry evidence identifiers, and confirmed outcomes can become future memory.

The architecture we built

The system has several layers.

The frontend collects factory incident information such as the machine, issue, error code, severity, and symptoms. The backend receives that information through FastAPI and forwards analysis requests to the M1 analysis service.

M1 is responsible for interacting with Hindsight. It can recall relevant historical incidents and reflect on them before the analysis stage combines historical evidence with the current incident.

The overall flow is:


text
Factory Incident
      ↓
Frontend
      ↓
FastAPI Backend
      ↓
M1 Analysis
      ↓
Hindsight Recall
      ↓
Historical Evidence
      ↓
AI Analysis
      ↓
Recommendations
      ↓
Operator Resolution
      ↓
Outcome Retained
      ↓
Future Hindsight Recall

I Made Factory Outcomes the Memory, Not Reports

Priyanshu Kumar — Tue, 29 Sep 2026 18:14:47 +0000

I Made Factory Outcomes the Memory, Not Reports

By Shaikh Nargish · Research / debugging

From a research and debugging perspective, I start with a provenance question: which values describe the original report, and which record captures what happened after an operator acted? A report describes a problem; it does not necessarily tell us what fixed it. If every incoming incident is retained as if it were a confirmed lesson, a memory system can preserve guesses beside verified repairs. I see that distinction as the central design problem in ForgeMind's outcome path: make the operator's resolution a separate input, turn it into a Hindsight record, and leave the initial analysis free to be wrong without automatically promoting it to history.

Analysis is not retention

ForgeMind has separate M1 operations for analyzing an incident and recording its outcome. The outcome request travels from the frontend to FastAPI and then to the M1 service. The M1 service validates that an outcome object includes an incidentId; its retention function then formats a text document and calls Hindsight. An analysis response alone does not run that retention function.

The implementation in m1-runtime/m1/services/incidentMemoryService.js makes the conversion visible:

const content = [
  "ForgeMind factory incident outcome",
  `Incident ID: ${outcome.incidentId}`,
  `Action taken: ${outcome.actionTaken || "unknown"}`,
  `Result: ${outcome.result || "unknown"}`,
  `Resolution status: ${outcome.resolutionStatus || "unknown"}`,
  `Notes: ${outcome.notes || ""}`
].join("\n");

return hindsight.retain(bankId, content, {
  context: "ForgeMind factory incident outcome",
  timestamp: new Date().toISOString(),
  document_id: `incident-${outcome.incidentId}-outcome`,
  tags: [
    `incident:${outcome.incidentId}`,
    "domain:factory",
    "type:outcome"
  ]
});

This record has a stable document identifier derived from the incident ID, plus tags for the incident, domain, and record type. The content is readable text rather than an opaque object dump, and it explicitly distinguishes the action, result, and resolution status. Hindsight is responsible for the memory operation; ForgeMind is responsible for deciding which fields to send and how to label them.

Before and after

Consider the local INC-007 example for COOL-01: reduced coolant flow, higher machining temperature, a blocked filter, and a resolution that cleaned the filter and verified flow. Before a confirmed outcome is sent through the retention path, a later similar report may retrieve no Hindsight record. The local example in data/incidents.json is not automatically inserted into Hindsight by incident analysis.

After an operator records the cause and outcome through the application flow, M1 can retain a separate outcome document. A later recall query may then return that memory if Hindsight considers it relevant. That is a conditional code-path description, not evidence from a live retrieval run; the repository contains no measured recall result for this example.

The history view helps explain the user-facing workflow, but it reads local incident data. I would not use its visible rows as proof that a Hindsight bank contains those same memories. The retention request and the browseable local history are different paths, even if they describe overlapping incidents.

Why explicit outcomes are useful

Separating the write operation creates a useful checkpoint. An operator can report that a proposed cause was wrong, record an unsuccessful action, or confirm that a repair restored operation. That gives future retrieval more than a model's initial guess. The distinct operation also gives the application a clean place to attach an outcome status and notes, rather than treating every analysis as confirmed learning.

The Hindsight operations fit a straightforward loop: retain an outcome record, recall records while analyzing a later incident, and reflect on relevant history. The important engineering choice is not just calling all three operations; it is deciding what data crosses each boundary and what its source means. Hindsight's documentation describes its memory capabilities, while its GitHub repository is the implementation home. Vectorize's agent-memory overview provides broader context for memory beyond a chat transcript.

An honest limitation

The record is only as complete as the outcome payload. Missing action, result, or status values become the literal string unknown; notes become an empty string. That avoids an exception for those fields, but it can retain a low-information document. The M1 route validates the presence of incidentId, not the quality or completeness of the resolution. The code also does not demonstrate a review workflow that checks whether the recorded cause is confirmed before retention.

A second limitation is that the retained outcome text includes incident ID, action, result, status, and notes, but not all original report fields such as machine ID, title, or symptoms. Tags identify incident and domain, but they do not restore omitted symptoms to the retained content. Later matching therefore depends on the searchable text and Hindsight's retrieval behavior; the code does not promise that a particular outcome will be recalled.

My lesson is to treat retention as a data-quality boundary. A deliberate operator outcome is a better memory candidate than an unverified analysis, but a dedicated record should still make confirmation explicit and preserve enough context for a future comparison. Retain, recall, and reflect are capabilities; reliable learning depends on the application data that flows through them.

Why I Separated Hindsight Evidence From Model Reasoning

Priyanshu Kumar — Tue, 29 Sep 2026 18:11:48 +0000

Why I Separated Hindsight Evidence From Model Reasoning

By Bhupathi Mahesh Varun Kumar · Hindsight / AI / memory

When I inspect an AI troubleshooting answer, I want to know which part came from the machine report, which part came from historical experience, and which part is a model's inference. A paragraph that simply says “this happened before” is not enough. It needs a source that the next stage can carry forward. ForgeMind's Hindsight integration makes that problem concrete: retrieve candidate memories, shape them into a bounded evidence set, and pass their identifiers alongside the current incident.

The problem: retrieval results are not yet usable evidence

A memory system can return several records that overlap, vary in relevance, or describe different machines. Passing every result directly into a model increases prompt size and makes it harder to inspect why a record was selected. Passing only a summary loses identity: later output cannot point back to the record that supported a claim.

ForgeMind builds the recall query from the current machine, title, description, and symptoms. In m1-runtime/m1/hindsight/recall.js, the request is explicit:

const query = [
  `Machine: ${incident.machineId || "unknown"}`,
  `Title: ${incident.title || ""}`,
  `Description: ${incident.description || ""}`,
  `Symptoms: ${(incident.symptoms || []).join(", ")}`,
  "Find historically relevant incidents, root causes, actions, and outcomes."
].join("\n");

return hindsight.recall(bankId, query);

That function delegates retrieval to Hindsight; it does not decide whether a result is a diagnosis. The distinction matters. A similar phrase in a memory can be useful context without proving that the same cause applies to the current machine.

Normalize, preserve identity, then bound

The orchestration layer in m1-runtime/m1/services/incidentMemoryService.js maps Hindsight's results into a shape the analysis step can inspect. It keeps the returned ID, final score, summary text, tags, and a source label. It also sorts by relevance, groups duplicate summaries, keeps the highest-scoring item within each duplicate group, then returns at most three records.

normalized.push({
  memoryId: item.id,
  relevance:
    typeof item.scores?.final === "number"
      ? item.scores.final
      : 0,
  type: inferEvidenceType(summary),
  summary,
  source: "hindsight",
  tags: Array.isArray(item.tags) ? item.tags : []
});

The final slice(0, MAX_HISTORICAL_EVIDENCE) is a small but meaningful boundary. It gives the later prompt a bounded set and keeps each candidate's identity attached to its content. ForgeMind then places evidence in a named prompt section and tells the model to use only supplied memory IDs for historical claims. The returned recommendation schema includes evidenceMemoryIds, making provenance visible in the response shape.

I think of this as an evidence ledger, not a confidence oracle. The ID says which record was supplied; it does not certify that the memory is correct or that the model used it correctly.

Before and after

Before this mapping, imagine passing raw recall text into analysis. The model might describe a repair as precedent, while the application has no reliable way to show which record it meant. Similar summaries can also appear more than once and consume prompt space.

After mapping, the analysis input contains a short list of structured records. For a hypothetical later report of reduced coolant flow, a recalled outcome could travel with its Hindsight memory ID, score, tags, and summary. The model can cite that ID in a cause or recommendation. If recall returns no records, ForgeMind passes an empty evidence list and the prompt says that no historical Hindsight evidence was retrieved. This is a description of the code path, not a claim that a live query returned a particular record or improved diagnosis accuracy.

The screen above is deliberately not presented as a Hindsight recall result. Factory Memory currently searches local JSON incidents and shows sample totals; Hindsight recall occurs in M1's analysis path. That mismatch is worth making visible because a UI label can otherwise suggest the wrong source of truth.

What I would keep and what I would change

The useful design choice is carrying identifiers all the way into model output. The code also keeps reflection separate from the raw evidence list, so a consumer can inspect both rather than treating one generated paragraph as the whole history. Hindsight supplies retain, recall, and reflect operations through its client; ForgeMind chooses the query, record shape, cap, and response contract.

There are limits. The evidence type is assigned by keyword checks such as “caused,” “replaced,” and “lesson.” That label is display metadata, not verified causality. The code requests only the top three after deduplication, so a relevant fourth record is excluded. And while the prompt instructs the model to reference supplied memory IDs, the application does not validate generated IDs against the actual evidence set. A schema can require an array of strings without proving those strings are valid references.

My engineering takeaway is to treat retrieval as the beginning of provenance, not the end. Preserve IDs and source metadata, make the selected set small enough to inspect, and validate references after generation if downstream users rely on them. Hindsight makes memory operations available; the application still owns the evidence contract.

DEV Community: Priyanshu Kumar

ForgeMind: Building a Memory-Backed Factory Troubleshooting System

The architecture we built

I Made Factory Outcomes the Memory, Not Reports

I Made Factory Outcomes the Memory, Not Reports

Analysis is not retention

Before and after

Why explicit outcomes are useful

An honest limitation

Links

Why I Separated Hindsight Evidence From Model Reasoning

Why I Separated Hindsight Evidence From Model Reasoning

The problem: retrieval results are not yet usable evidence

Normalize, preserve identity, then bound

Before and after

What I would keep and what I would change

Links