DEV Community

Tang Weigang
Tang Weigang

Posted on • Originally published at doramagic.ai

The First Letta Run Should Test Memory Isolation, Not Just Recall

#ai #security #opensource #tooling

Disclosure: This article was prepared with AI assistance and checked against the linked Doramagic project page, manual, and upstream README. It is an independent engineering interpretation, not official Letta documentation.

An agent remembering a detail is a demo. The first useful Letta test asks where that detail was stored, which session can read it, what tools share the boundary, and how the state can be removed.

There is a version trap before the first command. The upstream README says this repository contains the legacy Letta server and that active development moved to letta-ai/letta-code. The current CLI path is Node.js 22.19+ plus:

mkdir -p /tmp/letta-trial
cd /tmp/letta-trial
node --version
npm install -g @letta-ai/letta-code
letta --help
Enter fullscreen mode Exit fullscreen mode

That is not the same installation as the Python repository snapshot, whose pyproject.toml reports letta 0.16.8 and Python >=3.11,<3.14. The Agent SDK is a third surface and can target cloud, local, or self-hosted backends. I record the exact surface and backend before interpreting any result.

My first run uses synthetic or disposable data and one test identity. I capture the provider/model, agent and session identifiers, memory write/read behavior, tool permissions, sandbox mode, and what remains after cleanup. A coherent answer is not evidence that memory is isolated. A successful tool call is not evidence that the same policy applies across the CLI, SDK, and server.

The Doramagic manual separates provider compatibility, hierarchical memory/data, agents/tools/sandbox, and server/REST API. That is a useful debugging order. Provider failures belong to one gate; cross-session state leakage belongs to another; MCP refresh or tool configuration belongs to another. Repeating prompts until the transcript looks right only hides which gate failed.

The project page keeps community evidence visible: reports involving encrypted Bedrock or VLLM credentials, cross-session memory poisoning, MCP schema/refresh behavior, context compaction, and provider error handling. These are review inputs, not a blanket security verdict. They are enough to justify a disposable first run before real data or a primary configuration directory enters the system.

My acceptance packet contains the install command, runtime/version, exact product surface, provider/model, test data location, raw output, memory boundary, failure log, and rollback action. “It remembers me” is the start of the test, not the end.

Manual: https://doramagic.ai/en/projects/letta/manual/

Upstream: https://github.com/letta-ai/letta

Top comments (0)