I built a Transactional NoSQL Database in Go (and now it has AI Agents!)

Gerardo Recinto — Fri, 05 Dec 2025 19:14:06 +0000

I am thrilled to announce the General Public Availability (GPA) of Scalable Objects Persistence (SOP) V2, a native Golang code library that embeds a high-performance, ACID-compliant storage engine directly into your application.

SOP is not just another wrapper around an existing database. It is a raw storage engine built from the ground up, bringing the power of B-Tree indexing, transaction management, and distributed coordination directly to your fingertips.

Why I Built SOP: The "Database-as-a-Library" Paradigm

In traditional architectures, the database is a monolithic external service. Your application talks to it over the wire, dealing with latency, serialization overhead, and "impedance mismatch."

SOP changes the game by turning your application cluster into the database cluster.

No External Dependencies (Standalone Mode): Build powerful desktop apps, CLI tools, or local AI models that store data in robust B-Trees on the local filesystem. No Docker containers or DB processes required.
Cluster Native: In distributed mode, SOP uses Redis for lightweight coordination (locking) and the filesystem for storage. This turns your microservices into a masterless, horizontally scalable storage cluster.

Key Features

1. Raw Performance & Control

SOP gives you direct access to the storage muscle.

B-Tree Indexing: A custom, modernized B-Tree implementation optimized for high node utilization (62%-75%+).
Direct I/O: Bypasses layers of abstraction to talk directly to the disk.
"NoCheck" Mode: For build-once-read-many scenarios, skip conflict checks entirely for raw, unbridled read speed.

2. Full ACID Transactions

Don't compromise on data integrity. SOP provides full ACID (Atomicity, Consistency, Isolation, Durability) support with Two-Phase Commit (2PC). You can seamlessly integrate SOP transactions with other data sources in your ecosystem.

3. AI & Vector Search Ready 🤖

SOP V2 is uniquely positioned for the AI era.

Partitioned Vector Search: Store high-dimensional vectors directly in B-Trees. By partitioning data (e.g., by UserID or DocumentID), SOP outperforms specialized vector stores for hybrid search workloads.
Streaming Data: Handle multi-gigabyte objects (video, large models) with the StreamingDataStore, supporting chunked uploads, downloads, and partial updates within ACID transactions.

Show Me The Code!

Go Example: Storing Users

import "github.com/sharedcode/sop/database"

// 1. Initialize (Standalone or Clustered)
db := database.NewDatabase(database.Standalone, "/var/lib/sop")

// 2. Start a Transaction
tx, _ := db.BeginTransaction(ctx, sop.ForWriting)

// 3. Open a Store
users, _ := db.NewBtree(ctx, "users", tx)

// 4. Add Data
users.Add(ctx, "user1", User{Name: "John Doe", Email: "john@example.com"})

// 5. Commit
tx.Commit(ctx)

Python Example: AI Vector Search

We recently released sop4py, allowing you to use SOP's power in Python for AI workflows.

from sop.ai import Database, DBType

# 1. Initialize
db = Database(ctx, storage_path="data/my_db", db_type=DBType.Standalone)

with db.begin_transaction(ctx) as tx:
    # 2. Open Vector Store
    vs = db.open_vector_store(ctx, tx, "products")

    # 3. Search for similar items
    results = vs.search(ctx, query_vector, limit=5)

    for item in results:
        print(f"Found: {item.id} (Score: {item.score})")

The "Moat": Swarm Computing & Auto-Merge

SOP introduces a capability unmatched by traditional RDBMS: Automatic Transaction Commit Merging.

In a distributed cluster, multiple machines can commit transactions concurrently. SOP automatically detects non-conflicting changes and merges them into a single cohesive state. This makes parallel programming trivial—transactions are natively thread-safe and machine-safe.

Get Involved!

SOP is open source and we are just getting started.

GitHub Repo: github.com/SharedCode/sop
Python Package: pip install sop4py
Discussions: Join us on GitHub Discussions!

I'd love to hear your thoughts on this architecture. Does "Database-as-a-Library" fit your use case? Let me know in the comments! 👇

parallels

Gerardo Recinto — Sun, 02 Jun 2019 01:32:31 +0000

Help wanted!
I'm developing a library for abstraction of software system's interactions & task distribution, coordination for parallel execution.

Tech stack: golang, cassandra, redis to begin with

Learn more here: https://github.com/SharedCode/parallels

If this interests you, please feel free to provide feedback and perhaps, join! the project. :)

It is in the first few days of incubation, thus, there is not head way that had been done yet. So, it is a good time to join and collaborate.

Thank you, sincerely,
-Gerard

DEV Community: Gerardo Recinto