Building an Accounts Payable Agent That Remembers Why It Made a Decision

The first invoice is easy to process. The difficult part starts when the same vendor sends the tenth invoice and the system still behaves as if it has never seen them before.
That was the problem I focused on while building our Accounts Payable Agent.
An accounts payable workflow contains many repetitive tasks: reading invoices, extracting important fields, checking information, validating the invoice, and deciding whether it can continue through the payment workflow.
Traditional automation can handle many of these steps with rules.
But an agent has an additional opportunity: it can use relevant experience from previous interactions.
That is where I integrated Hindsight agent memory into the workflow.
What the Accounts Payable Agent does
The basic workflow starts with an invoice.
The agent needs to understand information such as:
Vendor name
Invoice number
Invoice amount
Purchase order
Payment terms
Invoice status
Validation results
The workflow can be represented as:
Invoice
↓
Extract information
↓
Validate invoice
↓
Retrieve relevant memory
↓
Make a decision
↓
Approve / Reject / Escalate
↓
Store useful outcome
The important part is the memory step.
Without it, every invoice is mostly an isolated task.
With it, the agent can use relevant context from previous interactions.
Why memory matters in accounts payable
Consider a vendor that regularly sends invoices.
The first invoice may contain an unusual situation.
For example:
Vendor: ABC Supplies
Invoice: INV-1042
PO: PO-2051
Amount: ₹48,500
Issue: Amount does not match the expected purchase order
The agent may send the invoice for human review.
A human checks the situation and confirms that the invoice is valid because an additional delivery was included.
If the next invoice from the same vendor has a similar situation, a stateless system starts again.
A memory-enabled agent can retrieve the previous interaction.
The previous decision does not automatically determine the new decision.
Instead, it provides additional context.
That distinction is important.
The role of Hindsight
I used Hindsight as the memory layer for the agent.
The idea is simple:
Current invoice
↓
AP Agent
↙ ↘
Current data Hindsight
↘ ↙
Decision
↓
Outcome
↓
Hindsight
The agent can retrieve relevant information before making a decision and store useful information after the workflow finishes.
Conceptually, the integration looks like this:
context = retrieve_relevant_memory(invoice)

decision = agent.evaluate(
invoice=invoice,
context=context
)

store_outcome(
invoice=invoice,
decision=decision
)
The exact implementation should match the project code, but the important architectural idea is that memory becomes part of the agent's reasoning loop.
Memory should contain useful information
One mistake when building memory systems is assuming that storing more information automatically makes an agent better.
It doesn't.
For an AP Agent, useful memories are things that can help with future invoices.
For example:
Vendor: ABC Supplies

Previous issue:
Purchase order amount mismatch.

Resolution:
Human reviewed the invoice and confirmed the additional
delivery was valid.

Final decision:
Invoice approved.
This is more useful than simply storing:
Invoice INV-1042 processed.
The first memory contains context.
The second is mostly a log entry.
Before and after memory
The difference can be represented simply.
Before
Invoice arrives
↓
Extract information
↓
Apply rules
↓
Make decision
↓
Finish
After
Invoice arrives
↓
Extract information
↓
Retrieve relevant previous context
↓
Apply current validation
↓
Make decision
↓
Store useful outcome
The second workflow gives the agent continuity.
Memory does not replace validation
A previous decision should never become an automatic answer for a new invoice.
Suppose a vendor's previous invoice was approved.
That doesn't mean the next invoice should automatically be approved.
The correct approach is:
Previous interaction
↓
Relevant context
↓
Current invoice
↓
Current validation
↓
Current decision
The current invoice remains the source of truth.
Memory provides additional context.
This separation makes the system easier to reason about.
Handling uncertainty
An AP workflow also needs to know when it should not make an automatic decision.
For example:
Invoice information is complete
↓
Validation succeeds
↓
Continue processing
But:
Invoice information conflicts
↓
Historical context is unclear
↓
Human review
This is especially important because accounts payable involves financial transactions.
An agent should not be forced to make a decision when the available information is insufficient.
The feedback loop
The most interesting part of the architecture is what happens after a decision.
Suppose an invoice is escalated.
A human reviews it and provides the final outcome.
That outcome can become useful memory:
Invoice
↓
Agent
↓
Escalation
↓
Human decision
↓
Outcome stored
↓
Future invoice
This creates a feedback loop.
The system can accumulate relevant experience instead of treating every interaction as completely independent.
What I learned
The biggest lesson from building this workflow is that agent memory is not the same thing as storing application logs.
A log tells me what happened.
Useful memory tells the agent something that may help it understand a future situation.
That means memory needs to be designed around future decisions.
I would ask:
What will the agent need to know when a similar invoice arrives tomorrow?
That question is more useful than:
What information can I store?

Three lessons I would reuse

1. Store outcomes, not just events The fact that an invoice arrived is less useful than knowing how an unusual invoice was resolved.
2. Retrieve memory before the decision Historical context is most valuable when the agent is actually evaluating the current invoice.
3. Keep human escalation Memory should improve context, but it should not remove the ability to send uncertain cases to a human. Final thought The interesting part of an Accounts Payable Agent isn't simply reading an invoice. The more interesting problem is what happens when the same situation appears again. A stateless system sees another invoice. A memory-enabled agent can see another invoice plus relevant experience from previous interactions. That is the reason I treated Hindsight as an important part of the AP Agent architecture rather than just another storage component.