Every AI photo tool today is an amnesiac critic. Lightroom edits, culling apps, and ChatGPT/Gemini/Qwen will happily critique a single photo — then forget you exist. You re-explain yourself every session, get the same beginner advice forever, and nothing knows whether you're actually improving.
Memory is the difference between a critic and a coach: someone who knows where you started, notices what you've mastered, and moves the goalposts as you grow.
So for the Global AI Hackathon with Qwen Cloud (Track 1: MemoryAgent), I built Engram — an AI photography coach whose most important organ isn't the critique. It's a forgetting-aware memory engine.
The loop
Engram runs one loop, forever:
-
Critique — upload a photo;
qwen-vl-maxscores five dimensions with glass-box reasoning grounded in real photography principles. - Remember — every critique writes memory: salience-scored facts, skill evidence, a genre identity. The app then tells you what it learned from this photo.
- Focus — skills you're working on are watched; three consecutive strong sessions and a skill graduates — celebrated on your timeline, then retired from coaching.
- Adapt — the next critique and every chat reply are built from what's still true about you.
Home stops being a dashboard and becomes a memory: a mentor's read of you, genre "memory threads" you step through like a photo-app Memories reel, and a coaching plan that changes as you do.
The hard part wasn't remembering — it was forgetting
Anyone can append facts to a database. The interesting problem Track 1 named is timely forgetting: what happens when a fact stops being true?
You switch from a Canon body to a Sony mirrorless. A memory that remembers everything forever keeps coaching you on a camera you sold last month. So Engram does two things most "memory" layers don't:
-
Supersession. A contradicted fact gets a
superseded_bylink — excluded from recall, but kept in an audit trail. Forgetting you can't inspect is just data loss. - Graduation. A mastered skill is retired from active coaching — forgetting rendered as promotion, not deletion.
Proving it: the FAMA benchmark
Claims are cheap. I froze 26 scripted photographer histories — gear switches, twice-replaced habits, multi-hop questions, and controls where nothing changes — and scored the engine against two baselines with a metric I called FAMA (Forgetting-Aware Memory Accuracy: rewards recalling every still-true fact, penalizes surfacing outdated ones).
| config | mean FAMA | recall of still-true facts | context tokens |
|---|---|---|---|
| Engram (forgetting on) | 1.00 | 100% | 1.72× fewer |
| recency-only (keep the 5 newest facts) | 0.64 | 100% | baseline |
| never-forgets (full history) | 0.64 | 100% | baseline |
The result worth sitting with: the two baselines tie. Keeping only the newest facts scores exactly the same as keeping everything — because recency can't tell that a newer fact invalidates an older one. The win isn't about trimming context; it's about knowing what's stale. And recall stays at 100% across all three, so the engine isn't trading recall for forgetting — it gets both, at 1.72× lower token cost.
One command reproduces it: python -m eval.run --compare.
Building on Qwen Cloud
Every model call goes through Alibaba's DashScope (OpenAI-compatible) endpoint, across three tiers behind one client:
-
qwen-vl-max— vision critique + genre -
qwen3.7-max— reasoning / shape repair -
qwen3.6-flash— mentor chat (SSE streaming), summaries, cheap repairs
Two things I learned building on it:
Qwen trusts your prompt where other stacks enforce a schema. My first live critique came back with the critique as prose, an invented overall score, and dropped arrays. The fix was a defense-in-depth chain: an explicit JSON skeleton in the prompt, then a local deterministic salvage layer that repairs known deviations (near-miss enums, out-of-range scores) in ~0.1ms — replacing a model-based repair loop that once burned 93 seconds and 502'd in front of me. Reliability became a feature.
Sometimes the model is right and your schema is wrong. One early response returned genre: "still_life" — refusing my enum because the photo genuinely was a still life and my taxonomy lacked it. It's in the enum now.
The whole thing — SPA, API, memory engine, and a custom engram-mcp server (so any Qwen agent can mount the same memory over MCP: recall / forget / get_memory_stats) — ships as one Docker image on an Alibaba Cloud ECS instance in Singapore, behind Caddy TLS, with photo storage one env flip from Alibaba OSS.
The takeaway
Users don't care that facts persist. They care that the coaching changes because of them. Committing to a forgetting metric didn't just measure the product — it shaped it: supersession semantics, audit trails, receipts on every reply. Memory is a behavior, not a store.
Try it (no login): https://engram.prasadtilloo.com/?judge=1
Code (Apache-2.0): https://github.com/prasadt1/engram
Built with Qwen Cloud (DashScope) + Alibaba Cloud ECS for the Global AI Hackathon, Track 1: MemoryAgent.
Top comments (10)
How does FAMA hold up once the histories stop being scripted? The 26 frozen cases are perfect for proving the mechanism, but the hard part of real forgetting is detecting the contradiction in the first place, and clean scripted swaps hand you that for free. A real photographer says "moving to Sony" in one session and "back on the Canon for this shoot" three sessions later, and now supersession has to figure out whether that's a switch, a temporary thing, or noise. Curious if you've thrown any fuzzy or ambiguous contradictions at it yet. The supersession-with-audit-trail idea is genuinely the right shape though, forgetting you can't inspect really is just data loss.
This is the most accurate criticism of the benchmark and I'd rather concede it
than argue it.
The traces declare supersession as a field — every fact carries an
invalidated_by_session— so what FAMA measures is whether the engineexcludes correctly given a known invalidation, not whether it detects
the contradiction. Detection is handed over for free, exactly as you say. And
no, I haven't run fuzzy or ambiguous contradictions through it yet.
Your Canon example would break the current design outright. Both statements
land as gear facts, the newer one wins, and "back on the Canon for this shoot"
would incorrectly retire the Sony fact — a frame-scoped statement getting
promoted to a durable identity claim.
What I think actually helps isn't better NLP on the sentence, it's a different
substrate. EXIF is already extracted on every upload but currently only stored
as portfolio metadata; wiring it into supersession is the top roadmap item.
Camera make/model from the file is a fact about that frame, which is much
cleaner input than a sentence — one Canon frame in a Sony month is evidence of
a lens choice, not a switch. It doesn't solve ambiguity in prose, but it moves
the durable gear claim onto ground where "temporary vs permanent" becomes a
frequency question instead of an interpretation one.
Detection is the unsolved half. The benchmark proves the exclusion half works,
and I tried to be explicit in
eval/README.mdthat the traces are authored —but you're right that it's the easier half.
I found the concept of a "forgetting-aware memory engine" in Engram to be particularly intriguing, as it highlights the importance of timely forgetting in AI systems. The idea of using
superseded_bylinks to exclude contradicted facts from recall while keeping them in an audit trail is a great approach to balancing memory and forgetting. The FAMA benchmark results are also impressive, showing that Engram's forgetting mechanism can achieve 100% recall of still-true facts while reducing context tokens by 1.72×. I'm curious to know more about how theqwen-vl-maxmodel is used in the critique phase and how its output is integrated with the memory engine to inform the coaching plan.Thanks — and the thing that took longest to get right is that memory is an input to the critique, not just an output of it.
Order of operations on an upload:
The supersession pattern is the part that deserves more attention. Most memory layers treat forgetting as deletion — but deletion without audit trail is data loss, and data loss makes the system untrustworthy. Keeping a superseded_by chain is the right call because it makes forgetting inspectable.
The FAMA result is clean: recency-only ties with never-forgets at 0.64 because recency cannot detect invalidation. That is exactly the failure mode — the newer fact does not just add to the context, it contradicts the older one. Knowing which facts are stale is a different capability than knowing which facts are recent.
One question on the graduation mechanic: when a skill graduates after three consecutive strong sessions, what happens if the photographer regresses? Is there a re-activation path, or does graduation assume monotonic improvement? The non-monotonic case is where memory systems tend to break down — the user improves, regresses, then the coach is still celebrating an old milestone.
You've found the real gap, and the honest answer is no — there's no
re-activation path today.
SkillStatushas exactly two states, WATCHING andCLEARED, and the transition only fires one way.
The nuance is that the data for the reverse transition is already being
collected. A cleared skill still records evidence on every new session, and its
consecutive-above-bar counter still resets when a frame comes in under the bar.
The engine can already see the regression. Nothing acts on it.
I left it out deliberately rather than shipping half of it. Demotion needs its
own hysteresis — one weak frame is noise, not regression, and the whole design
principle elsewhere is "one photo doesn't rewrite your plan." It also needs
narration that doesn't read as punitive when it fires. Get the threshold wrong
in either direction and it's worse than absent: too sensitive and the skill
yo-yos, too lax and you get exactly the stale-milestone problem you're
describing.
So the guarantee today is only "a cleared skill stops consuming coaching
attention," which is honest but incomplete. It's the top engine-level item on
the roadmap for precisely the reason you gave — non-monotonic is where this
design has to prove itself.
Forgetting is underrated in agent design. Everyone focuses on memory, but stale or over-broad memory can make a system worse. The useful version is scoped recall: keep durable facts, expire task noise, and make it clear which memory influenced the current answer.
"Make it clear which memory influenced the current answer" is the one I'd argue
hardest for. It started as an explainability feature and turned into my main
debugging tool — most of the recall bugs I found were visible in the receipt
before they were visible in the output.
Scoping is the other half and it's underrated the same way. Chat scoped to a
single photo and chat across a whole portfolio shouldn't draw from the same
pool, because "what's wrong with this frame" and "how am I doing this year" are
different questions. Durable facts vs task noise is exactly the split — the
failure mode I kept hitting was durable-sounding facts that were really scoped
to one session.
The forgetting logic is just half the battle — the platform can silently sabotage it. On Ollama, when I requested num_ctx=65536, the GGUF quietly clamped it to its native 40960: no error, no warning. My session compression kept failing without a trace, and the context would reinflate with every turn until I noticed. Whatever your forgetting strategy, first verify that the context window you think you have is the one you actually got.
Silent clamping is a nasty class of bug because every layer above it keeps
reporting success.
I hit a cousin of it on DashScope: the stream's trailing usage-only chunk
(empty
choices) crashed my generator, and it only ever showed up against livetraffic. Every mocked test passed, because the mock emitted the API I imagined
rather than the one that exists.
Your point generalises to something I ended up building for a different reason.
The recall receipt reports
budget_usedagainstbudget_totaland lists whatgot dropped for budget — so the packing step reports what actually fit, not
what it intended to fit. That started as an explainability feature, but it
doubles as the check you're describing: if the real window were smaller than I
thought, the receipt would show it instead of the context quietly reinflating.
Verifying the window you actually got, rather than the one you asked for, is
good advice regardless of the forgetting strategy sitting on top of it.