I had the product definition, user stories, and backlog.
Now it was time to design the system.
I started with the main responsibilities of Second-Memory and came up with my first architecture draft:
flowchart
W[Web Client] --> G[API Gateway / BFF]
M[Mobile Client] --> G
G --> A[Auth]
G --> MS[Memory Service]
G --> AS[Ask Service]
MS --> D[(Database)]
MS --> V[(Vector DB)]
AS --> L[LLM Provider]
AS --> MSThe diagram itself is fairly simple, but there were several design principles behind it.
1. BFF for the clients
Both the web and mobile clients communicate through the API Gateway / BFF.
I chose the Backend for Frontend (BFF) pattern so that the clients don't need to communicate directly with the internal services.
The BFF provides a client-facing API while hiding the internal architecture.
It also gives me a place to handle things such as authentication, routing, and client-specific requirements.
This means the clients don't need to know whether a request is handled by the Memory Service, Ask Service, or something else.
The internal architecture can evolve without making the clients aware of every change.
2. Each service owns its data
One principle I wanted to follow was:
Don't share databases between services.
A service owns the data that belongs to its responsibility.
In this architecture, the Memory Service owns both the database and vector database.
If another service needs memory data, it doesn't connect directly to either database.
Instead, it calls the Memory Service through an internal API.
For example:
flowchart
AS[Ask Service] -->|Internal API| MS
MS[Memory Service] --> D[(Database)]
MS --> V[(Vector DB)]This keeps the data ownership clear and prevents other services from becoming coupled to the Memory Service's storage implementation.
If the storage implementation changes later, the other service shouldn't need to know.
3. Separation of concerns
Each component has a specific responsibility.
The BFF deals with client-facing communication.
The Memory Service deals with memory.
The Ask Service deals with the AI interaction.
The LLM generates the response.
The databases provide persistence.
This separation makes the architecture easier to reason about.
It also means that AI-specific logic doesn't have to spread throughout the rest of the application.
4. DDD and bounded contexts
The separation between Memory Service and Ask Service wasn't just about splitting the code into smaller services.
I was thinking about the different domains and responsibilities involved.
The Memory Service represents the memory domain:
creating memories
retrieving memories
searching memories
managing memory data
The Ask Service represents the AI interaction:
receiving a question
deciding what information is needed
requesting relevant memories
constructing context
calling the LLM
generating the answer
They are closely related, but they are not the same responsibility.
This is where the idea of bounded contexts from Domain-Driven Design helped me think about the boundaries.
The Ask Service uses the Memory Service, but it doesn't become responsible for managing memory itself.
5. Encapsulation and API contracts
The service boundary also creates an important form of encapsulation.
The Ask Service doesn't need to know how the Memory Service stores or searches memories.
It only needs to know what the Memory Service provides through its API.
For example, the Ask Service might ask for relevant memories, without knowing whether the Memory Service uses PostgreSQL, pgvector, a different vector store, or some other implementation in the future.
The API becomes the contract between the services.
This keeps implementation details inside the service that owns them.
These principles gave me the initial structure of Second-Memory.
It wasn't a complicated architecture, and that was intentional.
I wanted clear boundaries without creating unnecessary complexity.
With the boundaries in place, I could start looking at the individual technical decisions.
Top comments (0)