How I built an open source Python memory layer with persistent drafts, explicit human approval, tamper evident finalization, revision history, and provenance aware AI exports.
Personal AI systems are becoming increasingly capable of remembering.
They can summarize conversations, identify recurring people, infer locations from photos, detect emotional patterns, and connect events across years.
But that creates a deeper question:
When does an AI suggestion become part of someone’s personal history?
A model might suggest that a photograph was taken in Seoul. It might identify who was present, infer how someone felt, or create a title for the moment.
The suggestion might be helpful.
It might also be wrong.
Either way, it should not silently become fact.
That is the principle behind version 0.4.0 of the open source DiaryVault Memory Layer:
AI may suggest. People confirm.
This release introduces an end to end review workflow for personal memory records, from initial capture through explicit approval, cryptographic finalization, and provenance aware export.
What is DiaryVault Memory Layer?
DiaryVault Memory Layer is an open source Python SDK for portable, tamper evident personal memory records.
Before this release, the SDK already supported:
Local memory storage
SHA 256 content hashing
AES 256 GCM encryption
HMAC SHA 256 integrity signatures
Tamper detection
Selective context sharing
JSONL exports
RAG ready chunks
Conversation history exports
Personal knowledge graphs
Portable .dvmem records
Version 0.4.0 adds the missing layer between capture and permanent memory:
human review.
Instead of allowing AI generated information to flow directly into a permanent record, applications can now create a draft, attach suggestions, record explicit decisions, approve the result, and finalize it as a verified memory.
The review workflow
The lifecycle is intentionally explicit:
Capture
↓
ReviewDraft
↓
AI suggestions
↓
Accept, edit, or reject
↓
Explicit approval
↓
Finalized Memory
↓
Provenance aware exports
Each stage has a distinct meaning.
A suggestion is not a confirmed value.
An open draft is not an approved memory.
An approved draft is not permanent until it has been finalized.
Downstream AI systems can determine whether a memory was reviewed instead of treating every stored value as equally authoritative.
Creating and persisting a draft
A review workflow begins inside MemoryVault:
from diaryvault_memory import MemoryVault
vault = MemoryVault(
encryption_key="replace-with-a-private-secret",
storage_dir="./memory-data",
)
draft = vault.create_draft(
content="She laughed when the dog sneezed.",
tags=["family"],
)
The draft is stored separately from finalized memories.
That separation matters. Drafts can change during review, while finalized memories represent completed records.
Applications can save, retrieve, list, and delete drafts:
vault.save_draft(draft)
restored = vault.get_draft(draft.draft_id)
open_drafts = vault.list_drafts(state="open")
vault.delete_draft(draft.draft_id)
Draft records live in their own storage directory, so they do not appear in normal memory searches or exports before finalization.
AI suggestions remain unconfirmed
An AI system can attach a suggestion without changing the user’s confirmed record:
draft = draft.add_suggestion(
field_name="location",
value="Seoul",
source="echo",
model="example-model",
process_version="memory-card-v1",
confidence=0.88,
suggestion_id="suggestion-location",
)
draft = draft.add_suggestion(
field_name="title",
value="The first laugh",
source="echo",
suggestion_id="suggestion-title",
)
vault.save_draft(draft)
Each suggestion may include:
The proposed field
The proposed value
Its source
The model that created it
A process or prompt version
Confidence when available
A creation timestamp
None of those suggested values appears in resolved_fields() until a person accepts it.
Accepting, editing, and rejecting suggestions
The reviewer may accept the proposed value:
draft = draft.accept(
"suggestion-location",
reviewer="parent",
)
They may accept it with a correction:
draft = draft.accept(
"suggestion-location",
reviewer="parent",
value="Seoul Forest",
)
Or reject it entirely:
draft = draft.reject_suggestion(
"suggestion-title",
reviewer="parent",
)
The corrected value becomes the confirmed value.
The original AI suggestion remains preserved as provenance.
This distinction is important. The system does not rewrite the suggestion to make it appear as though the model was correct.
Approval is a real state transition
A draft cannot be approved while suggestions remain undecided:
draft = draft.approve(
reviewer="parent",
)
Approval records who completed the review and when.
The review objects also validate their own invariants. Invalid states are rejected, including:
Duplicate suggestion identifiers
Decisions referencing unknown suggestions
Multiple decisions for one suggestion
Multiple accepted values for the same field
Approved drafts with pending suggestions
Open drafts carrying completion metadata
Rejected decisions carrying accepted values
Those checks also run during JSON deserialization.
Loading a malformed record cannot bypass the same rules used during normal construction.
Deep immutability
The review domain uses frozen value objects, but freezing only the outer dataclass is not enough.
A supposedly frozen suggestion could still contain a mutable dictionary or list.
Version 0.4.0 recursively freezes JSON style values so callers cannot mutate nested data after creation.
Conceptually, this is no longer possible:
draft.suggestions[0].value["people"].append("Someone else")
Serialization converts the frozen structures back into normal JSON compatible values.
This gives the domain model immutability during execution without sacrificing portable storage.
Finalizing an approved memory
Once approved, the draft can be finalized:
memory = vault.finalize_draft(draft)
Finalization reuses the same processing pipeline as ordinary memory creation:
Content
↓
SHA 256 hash
↓
AES 256 GCM encryption
↓
HMAC SHA 256 signature
↓
Local storage
The resulting memory contains the complete review record under:
memory.metadata.custom["review"]
That record includes:
Original content
Suggestions
Model provenance
User decisions
Accepted corrections
Rejections
Reviewer identity
Approval time
Other explicitly accepted fields remain available under confirmed metadata.
For example, an accepted location can become:
memory.metadata.location
A finalized draft can be finalized only once.
It can no longer be replaced or deleted because it has become provenance for the permanent memory.
That rule also survives a process restart.
memory_id = vault.finalized_memory_id(draft.draft_id)
Revision history without duplicate state
Review activity is available through a derived revision history:
for revision in draft.revision_history():
print(
revision.occurred_at,
revision.action,
revision.actor,
)
Typical actions include:
draft_created
suggestion_added
suggestion_accepted
suggestion_rejected
draft_approved
The revision history is not stored as a second mutable event log.
It is derived from the timestamps already present on the draft, suggestions, decisions, and terminal approval.
This avoids two sources of truth.
The history cannot drift away from the record it describes.
The missing piece: provenance aware exports
Preserving review information inside the vault is not enough.
The distinction between an AI suggestion and a human confirmed value must survive when the data enters another system.
Version 0.4.0 therefore carries review provenance into the export layer.
RAG chunks
from diaryvault_memory import VaultExporter
exporter = VaultExporter(vault)
chunks = exporter.to_rag_chunks()
review = chunks[0].metadata["review"]
A reviewed memory may expose a compact summary such as:
{
"reviewed": True,
"approved_by": "parent",
"approved_at": "2026-07-19T10:30:00+00:00",
"suggestion_count": 2,
"accepted_count": 1,
"confirmed_fields": ["location"],
}
An ordinary memory without a review record is marked clearly:
{
"reviewed": False,
}
A retrieval system can now rank, filter, or label memories based on whether a person explicitly reviewed them.
Generic JSONL
Review summaries are included in the generic JSONL format:
examples = exporter.to_jsonl(
format="generic",
)
Each record can carry both the memory content and its confirmation status.
Conversation history
Conversation exports include the review summary in their metadata:
history = exporter.to_conversation_history()
review = history[0]["metadata"]["review"]
This allows an assistant to distinguish reviewed personal history from unreviewed information when reconstructing context.
Knowledge graphs
Memory nodes in the personal knowledge graph also contain review metadata:
graph = exporter.to_knowledge_graph()
memory_nodes = [
node
for node in graph.nodes
if node.node_type == "memory"
]
The provenance therefore stays attached as memories become nodes and relationships.
Schema clean fine tuning exports
The OpenAI and Anthropic fine tuning formats intentionally remain schema clean.
The SDK does not inject custom review fields into formats that expect a specific message structure.
Applications that need the provenance can use the generic JSONL, RAG, conversation, or graph exports.
Why this matters for personal AI
Imagine a retrieval system containing these two values:
Location: Seoul
Location: Seoul Forest
Without provenance, a downstream model cannot know:
Which value came from the AI
Which value the user corrected
Whether either value was reviewed
When the decision happened
Who approved it
With the review record, the system can understand that:
AI suggested: Seoul
User confirmed: Seoul Forest
Reviewer: parent
Status: approved
That is not just more metadata.
It changes the meaning of the record.
A complete example
from tempfile import TemporaryDirectory
from diaryvault_memory import MemoryVault, VaultExporter
with TemporaryDirectory() as storage_dir:
vault = MemoryVault(
encryption_key="synthetic-example-key",
storage_dir=storage_dir,
)
draft = vault.create_draft(
content="She laughed when the dog sneezed.",
tags=["family"],
)
draft = draft.add_suggestion(
field_name="location",
value="Seoul",
source="echo",
model="example-model",
confidence=0.88,
suggestion_id="suggestion-location",
)
draft = draft.add_suggestion(
field_name="title",
value="The first laugh",
source="echo",
suggestion_id="suggestion-title",
)
draft = draft.accept(
"suggestion-location",
reviewer="parent",
value="Seoul Forest",
)
draft = draft.reject_suggestion(
"suggestion-title",
reviewer="parent",
)
draft = draft.approve(
reviewer="parent",
)
memory = vault.finalize_draft(draft)
assert vault.verify(memory)
assert memory.metadata.location == "Seoul Forest"
for revision in draft.revision_history():
print(revision.action)
exporter = VaultExporter(vault)
chunks = exporter.to_rag_chunks()
print(chunks[0].metadata["review"])
Testing the trust boundary
Version 0.4.0 is covered by 122 tests.
The test suite includes:
Review state transitions
Deep nested immutability
Invalid deserialization
Draft persistence and reload
Draft filtering and deletion
Finalization requirements
Single finalization guarantees
Finalization after process restart
Derived revision history
Approval aware RAG exports
Approval aware generic JSONL
Conversation history provenance
Knowledge graph provenance
Reviewed memory export counts
Compatibility with existing memory features
The package passes Ruff checks, runs both public examples, and builds versioned source and wheel distributions.
Honest boundaries
This is still Alpha software.
The SDK does not:
Call an AI model
Decide whether a suggestion is true
Provide independently trusted timestamps
Establish legal authorship
Synchronize data through a cloud service
Integrate directly with the production DiaryVault mobile application
Guarantee that an exported copy can later be revoked
Drafts are local application records, while finalized memories pass through the SDK’s hashing, encryption, signing, and storage pipeline.
Applications should treat the entire storage directory as sensitive and require explicit approval before exporting personal data.
Why I built this
I am building DiaryVault around a simple idea:
Personal AI should help people remember without quietly rewriting their history.
AI can organize memories.
It can suggest context.
It can identify patterns that would otherwise disappear across thousands of photographs and journal entries.
But the final authority should remain human.
That principle should not live only in interface copy or a confirmation button.
It should exist in the underlying data model, persistence rules, integrity checks, revision history, and export formats.
That is what version 0.4.0 is designed to provide.
Try the project
The project is open source under the MIT license:
DiaryVault Memory Layer on GitHub
git clone https://github.com/DiaryVault/diaryvault-memory-layer.git
cd diaryvault-memory-layer
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
python -m ruff check sdk tests examples
python -m pytest tests/ -q
python examples/review_workflow.py
Personal AI will remember more about us.
The infrastructure underneath it should ensure that we remain the ones who decide what becomes part of the record.
Top comments (1)
Interesting take on personal AI memory! I've been working on secure memory handling in GPU environments, and the idea of explicit user control over what's retained is crucial—especially when dealing with sensitive data. It aligns well with how we handle memory isolation in VoltageGPU for confidential computing.