DEV Community

Cover image for The Part Most Agent Demos Skip: Acting as a Specific User with Real Memory
Greg Mate
Greg Mate

Posted on

The Part Most Agent Demos Skip: Acting as a Specific User with Real Memory

#ai #agents #python #docker

Most agent demos connect to a CRM and update a record. Impressive in a presentation. Broken the moment a second user shows up.

The hard part is not the tool call. It is acting as the right person, remembering only their context, and making sure nothing leaks across users. This is the part that gets hand-waved in demos because it is genuinely annoying to get right.

We ran into this building a reference implementation for the Scalekit x Actian x Render Agents in Production Hackathon in San Francisco on June 27. Here is what we learned.

The problem with agent memory in multi-user systems

When you build an agent that acts on behalf of a user, there are two separate scoping problems:

  1. What is the agent allowed to do on their behalf (identity, permissions, tokens)
  2. What does the agent remember about them (context, history, prior decisions)

Most implementations treat these as the same problem. They are not. Scalekit handles the first one well. VectorAI DB handles the second. But getting them to agree on who the current user is requires deliberate wiring.

The naive approach, a single shared vector index for all users, fails quietly. Alice's agent starts pulling context that belongs to Bob. Nothing crashes. No errors. The output just gets subtly wrong in ways that are hard to debug.

One collection per user

VectorAI DB does not have a native multi-tenancy API. There are no user-scoped namespaces, no per-user tokens, no RBAC on individual collections. The Community Edition ships one isolation primitive: collections.

So the pattern is simple. One collection per user, named after the same identifier your auth layer already uses:

from actian_vectorai import VectorAIClient, VectorParams, Distance, CollectionExistsError

client = VectorAIClient("localhost:6574")

def get_or_create_user_collection(user_id: str, dim: int = 384):
    name = f"user-{user_id}-memories"
    try:
        client.collections.create(
            name,
            vectors_config=VectorParams(size=dim, distance=Distance.Cosine),
        )
    except CollectionExistsError:
        pass
    return name
Enter fullscreen mode Exit fullscreen mode

The key detail: whatever string Scalekit uses as the user identifier for its connected account is the string you pass here. One source of truth, no mapping table, no sync to maintain.

Two things you cannot change after collection creation: vector dimension and distance metric. Pick your embedding model before you create any collections. Changing either later requires deleting the collection and losing all its data.

Getting it running locally 

pip install actian-vectorai-client

docker pull actian/vectorai:latest
docker run -d --name vectorai \
  -v ./local_data:/var/lib/actian-vectorai \
  -p 6573-6575:6573-6575 \
  -e ACTIAN_VECTORAI_ACCEPT_EULA=YES \
  actian/vectorai:latest
Enter fullscreen mode Exit fullscreen mode

The container will not start without ACTIAN_VECTORAI_ACCEPT_EULA=YES. No error, just an immediate exit with code 1.

One thing that will trip you up: the pip package is actian-vectorai-client but the import is actian_vectorai. Different strings. It will fail at import time if you use the package name.

from actian_vectorai import VectorAIClient, VectorParams, Distance, PointStruct

client = VectorAIClient("localhost:6574")
Enter fullscreen mode Exit fullscreen mode

The dependency conflict nobody warns you about

If you are combining this with scalekit-sdk-python, you will hit a dependency conflict that is not version-specific and not obvious from the error message.

scalekit-sdk-python==2.12.0 pins protobuf<7.0.0. actian-vectorai-client needs protobuf>=6.31.1. When pip resolves this, it downgrades protobuf, and then actian_vectorai fails at import time with:

google.protobuf.runtime_version.VersionError: Detected incompatible Protobuf 
Gencode/Runtime versions when loading actian_vectorai_common.proto: 
gencode 6.31.1 runtime 5.29.6.
Enter fullscreen mode Exit fullscreen mode

The fix:

# Install everything except scalekit normally
grep -v scalekit-sdk-python requirements.txt > /tmp/req.txt
pip install -r /tmp/req.txt

# Then install scalekit without its dependency resolution
pip install scalekit-sdk-python==2.12.0 --no-deps

# Explicitly reinstate the versions actian-vectorai-client needs
pip install "protobuf>=6.31.1" "grpcio-status>=1.67.0"
Enter fullscreen mode Exit fullscreen mode

This works because scalekit's <1.67 grpcio-status constraint is stale metadata. At runtime, the newer versions are compatible. The --no-deps flag skips the constraint check. Not a blessed install path from Scalekit's side, but it works and the combination has been stable.

The capacity behavior you should know before demo day

Community Edition caps at 5,000 vectors total, across all your collections combined. That is not the surprising part.

The surprising part: the cap is enforced asynchronously. Writes succeed past the limit. About 30 seconds later, a background enforcement task runs and blocks further writes:

CapacityExceededError: Vector capacity exceeded: 5,005 vectors stored, 
limit is 5,000. Delete vectors or upgrade your licence to continue.
Enter fullscreen mode Exit fullscreen mode

During a demo, this means your inserts can succeed, your reads can succeed, and then your next write silently fails half a minute later with no indication of why at the point of the call. Worth knowing before you have 10 people watching.

The 30-day trial unlocks 1 million vectors. Get that set up before the day if you are planning anything beyond a few users.

Deploying to Render

VectorAI DB is Docker-only right now, which Render handles natively. Pull actian/vectorai:latest directly as a private Docker service, no custom Dockerfile needed.

Two things the service needs:

  • ACTIAN_VECTORAI_ACCEPT_EULA=YES as an environment variable
  • A persistent disk mounted at /var/lib/actian-vectorai, or you lose all data on every redeploy. This cannot be set via render.yaml on an existing service. It has to be added manually through the Render dashboard.

Keep the VectorAI DB service private, not public-facing. Your agent app connects to it over Render's internal network at vectorai-db:6574.

Build this at the hackathon

On June 27 in San Francisco, Scalekit, Actian, and Render are running a build day focused on agents that act as real users with real permissions. If this is the problem you want to work on, register here.

Our team will be on-site all day. The participant guide is live here with the install commands, the per-user pattern, and everything else in this post in a format you can keep open during the build.

Top comments (0)