DEV Community: Ivan Savu

Achieving scalability with custom locks

Ivan Savu — Tue, 12 Sep 2023 08:41:30 +0000

Scaling software to optimally utilize resources on a multi-processor system can be challenging. In this article, I will discuss how we used custom locks to achieve good scaling in VeloxDB.

Before going into the details of this article, let me briefly introduce VeloxDB. Chances are, this is the first time you're encountering this database, but prior knowledge of VeloxDB is not necessary to understand the concepts discussed here. Cache line contentions and the locking mechanisms explained can be applied effectively in other projects too.

VeloxDB is an open-source, fast, object-oriented, in-memory database designed for C#, with a strong emphasis on correctness. It achieves a remarkable rate of 2.5 million write transactions per second while ensuring strict serializability. Interacting with the database in C# is similar to working with an object-relational mapper (ORM). If you want to learn more about VeloxDB, I suggest you check out my introductory article.

CPU Cache

First, let's cover some basics of how CPU cache works because it is important for multicore scalability.

As modern CPUs have become faster, the speed at which they can access memory has not kept pace. This discrepancy in speed is commonly referred to as the CPU-Memory gap. To address this gap, CPUs employ multiple layers of caches positioned between themselves and the main memory. Each layer of cache closer to the CPU is smaller, faster, and more expensive than the previous layer. A modern x86 CPU has three layers of cache called L1, L2, and L3, respectively. L1 and L2 are per core caches, while L3 is shared. For example, the Intel i7 13700K (a 2022 processor) has 80k of L1 cache, 2MB of L2 cache and 30MB of L3 cache.

When the CPU requests the data from the memory, it first checks L1 cache to see if the requested data is there, and then L2 and L3 in the end if all caches fail, it goes to the memory for data. The fetched data is then stored in all caches for subsequent access. The data is always fetched in chunks known as a cache line, even if cpu has requested less data, the whole cache line is brought into the cache. The most common cache line size is 64 bytes.

The CPU cache has become much more complex with the introduction of multicore CPUs. Multicore CPUs have per-CPU caches (L1 and L2 are on the core, while L3 is shared). If multiple cores fetch the same cache line and one of them updates it, this change needs to be synchronized across multiple cores. The smallest unit of synchronization is a cache line. When multiple cores access the same cache line, this can cause significant performance degradation and is called cache line contention.

Here is a small C# program demonstrating cache line contentions:

static void Count(int index)
{
    Stopwatch sw = Stopwatch.StartNew();
    while(sw.ElapsedMilliseconds < timeLimit)
    {
        counters[index]++;
    }
}

private static void Main(string[] args)
{
    Thread[] threads = new Thread[Environment.ProcessorCount];
    counters = new long[threads.Length];
    for (int i = 0; i < threads.Length; i++)
    {
        threads[i] = StartThread(i);
    }
    Wait(threads);
    Console.WriteLine(counters.Sum() / 1000000.0);
}

Let's create a straightforward example to illustrate this concept. Imagine a program designed to increment a number as rapidly as possible. To make optimal use of modern multicore CPUs, we will assign each core the task of incrementing its own counter. Once the threads have completed their tasks, we will aggregate the results.

We'll test the program's scalability by running it on 32 core machine (AWS EC2 c6a.8xlarge) with a varying number of threads, ranging from 1 to 32.

Here are our results, and you can see that they are less than satisfactory. While the code does show some scaling, it achieves only a 3.5x scaling with 32 cores. Additionally, you can observe some unusual behavior on the chart where, by adding more cores, we are actually experiencing a decline in performance.

What's occurring here is cache line contention. Remember that a cache line consists of 64 bytes and can hold 8 64-bit integer values. In our array, every set of 8 integers shares a single cache line, which leads to frequent synchronization among the 8 cores. Essentially, you have 4 groups of 8 cores working concurrently, fighting with one another for access to a single cache line. This accounts for the scaling factor we observe, approximately 3.5x.

Let's attempt to address this issue. We can make our array more sparse, giving each counter 64 byte chunk of an array so that each counter gets a dedicated cache line.

And this is what we get:

We've achieved a scaling factor of 8.71x, which is more than twice the improvement over the initial version. However, it still seems like a 32-core CPU should be capable of even better performance. What's even more indicative of an issue here is the fact that we still observe irregularities in the graph—instances where adding more cores actually leads to a performance drop. These are challenging problems to troubleshoot, and there is no single approach to solving them. In my case, I identified the issue by disassembling the code and carefully analyzing what was happening. Profiling with a sampling profiler might also help because it can show code spending an increased amount of time at memory access points where contention occurs.

It turns out that C# is at fault here, and our previous understanding of how an array is structured in memory isn't entirely accurate. C# prefixes each array with its length, which is a common practice in high-level languages.

This length information is then used to check for index out-of-bounds errors each time the array is accessed. This is where our contention issue arises. The array's length is located on the same cache line as the first counter. The first core continuously updates its counter, while all other cores access the array length, which results in cache line synchronization across all cores.

To resolve this, we will simply skip the first 8 integers in the array and start placing our counters after them. This way, we ensure that array length has its own cache line.

And these are the results we obtain:

A 30x scaling, which represents almost a 4x improvement over the previous iteration. The graph also no longer exhibits performance drops as seen previously.

The purpose of this example was to illustrate how fickle scaling on multi-core processors can be. Even when your threads do not share data, accessing adjacent memory can lead to significant performance drops. This highlights the importance of testing how your code scales.

VeloxDB's HashTable

Now that we understand the cache line contentions, I'll take a quick detour to showcase VeloxDB's main data structure. This is important because it provides the context in which these locks operate and offers a better understanding of the problems we were trying to solve with them.

Each database table is represented by a hash table that maps a 64-bit object ID to the corresponding object. The hash table is implemented using unsafe C# and stores data in unmanaged memory. This was done to achieve better control over memory layout, especially for alignment, and to reduce the pressure on the garbage collector. The hash table needs to be thread-safe because it is accessed from multiple threads.
It also needs to be scalable, meaning that adding more cores should improve performance.

Multiprocessor RW Lock

So let’s first look at a lock specifically designed for multiprocessor systems. We use it for resizing the hash table. Resizing is a costly operation that must be performed under lock. Usually, when you need to resize a hashtable, you create a bigger storage array for buckets and then transfer items from the old buckets to the new, larger array. While .NET's ReaderWriterLockSlim is an option, we can improve upon it.

Hash table resizing is relatively rare and expensive, while accessing the hash map is common and fast. In multiprocessor systems, as we have seen, minimizing shared data across threads is ideal, as synchronizing data across cores incurs penalties. Using a standard reader-writers lock would create contention where none existed before. To avoid this, we implement a custom lock consisting of multiple locks, with each lock assigned to a specific processor and occupying a single cache line. The number of locks corresponds to the number of logical processors. For read lock (access), only the lock assigned to the current processor is locked, avoiding contention with other processors. For write lock (resizing), all locks are acquired. This approach allows fast hash map access by avoiding contention for read lock acquisition, while the slower write lock acquisition is insignificant compared to the resizing operation.

Here is a comparison between our Multiprocessor RW lock and .NET's ReaderWriterLockSlim. The benchmark measures the number of times a lock can be acquired in millions of acquisitions per second. The benchmark is repeated with varying numbers of threads. Notice how our lock's read performance scales and improves with each additional core, while the standard RW lock becomes slower. Conversely, the write lock is slower than the standard library lock and exhibits similar scaling behavior. For our use case, the benefits of improved read lock performance far outweigh the performance impact of the write lock.

If you want to take a deeper look, you can find our implementation of this lock here. Note that this implementation does a bit more than just the basic locking described here.

0-bit spinlock

Now, I'd like to introduce another lock that we employ: the 0-bit spin lock. To achieve effective parallelism, we aim to use fine-grained locks, locking only the minimal data for the shortest possible time. We utilize this lock for accessing the hash table's buckets, which means that only hash collisions or concurrent access to the same row can lead to lock contention. As we employ one lock per bucket, the number of locks scales proportionally with the number of rows in the database.

In .NET, any object can serve as a lock, but there's a 24-byte overhead associated with each object on the x64 platform. This doesn’t scale well to our number of locks, it would introduce significant memory overhead and increase strain on the garbage collector, to avoid avoid this, we've implemented our own spinlock that effectively doesn’t consume memory.

void EnterLock()
{
    while(true)
    observedState = state;
    if (GetBit(observedState, LOCK_BIT) != LOCKED)
    {
        // Calculate lock state with lock bit set
        lockedState = SetBit(observedState, LOCK_BIT, LOCKED)
        if (CompareAndExchange(ref state, lockedState, observedState) == observedState):
            return; // Lock acquired
    }
}

When reading from a bucket or adding a new element, we lock the bucket to prevent concurrent modifications. Here is the pseudocode illustrating the lock's function. The lock relies on the CompareAndExchange atomic operation. It updates the state value with the lockedState value if the state is equal to the observedState. This operation is provided by the platform and is guaranteed to be atomic. Our lock continuously spins until it observes the lock in an unlocked state. It then attempts to set the bit with the compare-and-exchange operation. If it succeeds, we have the lock. This is a spinlock, which actively utilizes the CPU during the waiting period. However, since lock contentions are expected to be infrequent and the tasks performed within the lock are minimal, it is acceptable to use it in this situation. Now, this lock obviously uses a single bit, although I consistently refer to it as a "0-bit" lock. We employ an additional trick here. Because our data resides in unmanaged memory, giving us control over how everything is allocated, and since we allocate buckets at 8-byte boundaries, it guarantees that the lowest 3 bits of a bucket’s pointer are always set to 0. We utilize one of these bits for the lock, effectively amortizing the memory cost of the lock and rendering it effectively a "0-bit" lock.

Takeaways

In this article, I've shown techniques used in VeloxDB for improving parallelism. Additionally, I'd like to point out something that may not have been apparent from the article: not all of our code is this heavily optimized. We don't employ unmanaged code in all cases, nor do we bit-pack every variable. A substantial portion of VeloxDB resembles typical C# code.

In the context of low-level optimization, two critical heuristics should be kept in mind: first, prioritize hotspots and avoid spreading complexity. If the complexity increase from optimization spreads beyond the targeted hot spots, it can lead to unintended consequences, such as decreased maintainability, increased risk of errors, and degraded performance. Therefore, it's crucial to manage the complexity increase carefully and avoid introducing complexity where it's not needed.

With the disclaimer out of the way, here's my advice:

Don't assume that you can't do better than the standard library. While the standard library is undoubtedly useful, it's important to remember that it's designed to handle a wide variety of use cases. As such, your specific use case may offer some unique performance opportunities that you can capitalize on.
When building code that needs to scale, it's important to keep contentions in mind.
Ideally, scalable code should share nothing, but this is rarely possible in practice. Therefore, it's crucial to minimize contentions to the best of your ability.
Relieve the pressure of the garbage collector in performance-critical paths of your code. Moving data to unmanaged memory might be overkill for most applications, but there are other techniques, like object pooling.

If you find this article interesting, check out my other articles. Also make sure to visit VeloxDB on GitHub.

Understanding VeloxDB's Transaction Model and Consistency Guarantees

Ivan Savu — Sun, 16 Apr 2023 08:21:44 +0000

Introduction

I'd like to take some time to explain VeloxDB's transaction model and consistency guarantees. This is important because many databases cut corners in this area, and it is one of the primary strengths of VeloxDB.

VeloxDB is a fast, in-memory (but with persistance) object-oriented (OO) database. It provides exceptional performance while offering an object-oriented interface. In VeloxDB, everything is done using C#, from model definition to data manipulation. The experience is similar to using a traditional ORM. You define the model with C# classes and access it with C# code. Database access is organized through database operations—C# methods executed within the database itself, similar to stored procedures in SQL databases. If you'd like to learn more about VeloxDB, please refer to my other article, Introducing VeloxDB. For now, I'll provide an example of a VeloxDB database operation:

[DbAPIOperation]
public long CreateBook(ObjectModel om, string title, string author)
{
    Book newBook = om.CreateObject<Book>();
    newBook.Title = title;
    newBook.Author = author;
    return newBook.Id;
}

Here is a simple database operation that creates a book using the provided parameters. It is a standard C# method that accepts an ObjectModel as the first argument, while the remaining arguments can be anything the user desires. The ObjectModel is a class that grants access to the database itself. By utilizing the object model, database operations can read data from the database and create, update, or delete objects within it.

The ACID Transaction Model

Each database operation represents a single ACID transaction. Let's break down each component of ACID and discuss what it means in the context of VeloxDB.

Atomicity - Each database operation is executed atomically. This means that either all changes are applied to the database or none; there are no partial executions. If the method executes successfully, the changes are applied. If it fails due to an exception being thrown or the user manually calling rollback, all changes made are rolled back, leaving the database as if the operation never occurred.

Consistency - VeloxDB ensures that each transaction leaves the database in a consistent state. VeloxDB validates references and user-specified model constraints, such as reference cardinality, nullability, and hash index uniqueness guarantees, among others. Transactions that violate these constraints will not be committed.

Isolation - Each transaction is isolated from other transactions. In other words, each transaction appears as if it is the only one running on the system. In reality, various databases implement isolation differently, and not all isolation levels provide full isolation. VeloxDB offers strict serializability, the gold standard of transaction isolation. We will revisit this topic later in the article.

Durability - Transactions are persisted on the hard drive. Once a transaction is complete, all its data is safely stored on the drive.

These four properties comprise the "secret sauce" of a database, making them incredibly convenient to work with.

The Importance of Transactions

Many bugs encountered in everyday work could be resolved by utilizing transactions. For instance, if every try/catch block in C# were atomic, it would significantly reduce the number of bugs created. Writing multithreaded software would also be much easier with transactions. Although there have been attempts to integrate transactions into common programming languages, such as software transactional memory and transactional memory, transactions are typically found only in databases for now.

A clear example of how transactions simplify work can be seen in the standard architecture of web applications. These applications inherently exhibit a high degree of concurrency, as countless users—potentially in the thousands or even millions—simultaneously access and modify shared data. Interestingly, the average web developer seldom contemplates concurrency. This lack of focus on concurrency is actually beneficial, as addressing concurrent issues for each new feature added to a web application would be unsustainable.

In the typical web application, the entirety of the state resides in a database, and all operations are carried out as transactions. This arrangement entrusts the database with the responsibility of managing concurrency, allowing developers to concentrate their efforts on crafting the business logic.

Concurrency is the challenging aspect; if tasks were executed in a purely serial fashion instead of concurrently, the situation would be markedly more straightforward. However, modern systems have multiple cores, and it would be inefficient for a database to use only one core. If multiple transactions are available, it is logical to use multiple cores for processing them.

Even in cases where only one core is available, transactions frequently encounter delays while waiting on disk or network resources. During these periods, the lone core could be more effectively engaged in processing other transactions. As a result, transactions are executed concurrently for the sake of performance.

Concurrent execution of transactions, however, necessitates the implementation of isolation to ensure that transactions do not conflict with one another, thereby jeopardizing the integrity of the database. The primary function of isolation is to segregate transactions from each other. In an ideal scenario, each transaction would be as isolated as if it were the sole transaction operating within the database. This degree of isolation is referred to as serializable isolation.

Nevertheless, serializable isolation is not the highest possible isolation level. Its definition lacks a time component, and it allows the database to rearrange transactions as it sees fit. By introducing an additional criterion - namely, that if transaction T2 is executed after transaction T1 completes execution, T2 must see the results of T1 - we attain the highest isolation level: strict serializable isolation. This is the isolation level employed by VeloxDB.

With strict serializable isolation, your system behaves as if it is executing each transaction sequentially. As a developer, you don't need to worry about any anomalies and can simply focus on writing your business logic. This isolation level should be the default, as specified in the SQL 92 standard. However, many databases opt for weaker isolation levels for performance reasons.

The problem with weaker isolation levels is that they can exhibit various anomalies in a concurrent environment. These anomalies include dirty reads, non-repeatable reads, phantom reads, lost updates, and write skew. These issues are not just theoretical - they can and do happen on real systems running in production.

For example, in 2017, researchers from Stanford analyzed 12 popular eCommerce solutions for exploits based on weak isolation anomalies. They identified 22 critical attacks that allowed attackers to corrupt store inventory, overspend gift cards, and steal inventory (ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications). This is a serious concern, which is why VeloxDB uses strict serialization as its default isolation level.

It's worth noting that all other isolation levels exhibit various anomalies, which you have to watch for and design for. As your code base and the number of people working on it grow, this becomes increasingly difficult to do. Therefore, I strongly recommend that you use strict serializable isolation as your default isolation level.

I'm not arguing that lower isolation levels shouldn't exist. It's perfectly acceptable to lower the isolation in specific performance bottlenecks after you have thoroughly analyzed the consequences. However, weaker isolation levels should not be the default. By defaulting to strict serializable isolation, you'll avoid many potential issues and make your system more secure and reliable.

VeloxDB Transaction implementation

Although VeloxDB ensures transactions are executed as if they were sequential, the database engine runs them in parallel for better performance. In order to execute transactions in parallel without compromising on serializability isolation, VeloxDB relies on multiversion concurrency control (MVCC). In this section, we'll dive into how MVCC enables VeloxDB to achieve this

The core principle of MVCC is that updates are processed by inserting new objects with updated versions. Consequently, an MVCC database can store multiple versions of a single object, which is why it is called "multiversion."

Let's consider a sequence of transactions using our book model as an example:

T1: AddBook("The Great Gatsby", "F. Scott Fitzegarld")
T2: AddBook("To Kill a Mockingbird", "Harper Lee")
T3: AddBook("One Hundred Years of Solitude", "Gabriel Garcia Marquez")
T4: UpdateBookAuthor(1, "F. Scott Fitzegarld")

The resulting Book table would appear as follows:

Id	Title	Author	Version
1	The Great Gatsby	F. Scott Fitzegarld	1
1	The Great Gatsby	F. Scott Fitzgerald	4
2	To Kill a Mockingbird	Harper Lee	2
3	One Hundred Years of Solitude	Gabriel Garcia Marquez	3

You may notice that the book with ID 1, "The Great Gatsby," appears twice in the table with different versions. This is because the book was updated to correct a misspelling of the author's surname. The version number represents the ID of the transaction that changed or inserted the record. Transaction IDs are assigned upon commit and are incremental.

Since objects are immutable, it is trivial to provide a snapshot of the database at a particular version. For example, if a user opened a read transaction at version 3 before "The Great Gatsby" was updated, to provide the user with the snapshot, we would simply filter the table with the criteria "Version ≤ 3." The read transaction's snapshot would remain unchanged even after the update since the filter excludes any newer versions of the object. The version at which the transaction sees its snapshot of the database is called the "read version". The read version is the version the transaction observed when it was started.

Conflict resolution

It is evident that this method offers simple read transaction isolation. However, let us now explore read-write transactions. In the absence of conflicts, MVCC essentially functions in the same way for multiple read-write transactions as it does for read transactions. For example, suppose transactions T1, T2, and T3 from the above sequence were all running concurrently. This means they all began with read version 0, and their snapshot was an empty database. Each transaction added a new book, and all of them were able to commit successfully without any conflicts.

However, conflicts may arise when multiple read-write transactions try to modify the same object concurrently. To handle such conflicts, VeloxDB employs optimistic concurrency control (OCC) as a conflict resolution strategy. In optimistic concurrency control, read-write transactions are permitted to proceed without any locks. If a conflict arises with another transaction running concurrently, the transaction is aborted and must be retried. This indicates that two transactions were unable to run in parallel, and therefore one had to be aborted. By retrying the aborted transaction, you essentially serialize them.

Here is a list of situations that can cause conflicts:

Two transactions modify the same object
A transaction modifies an object that another transaction has read
A transaction scans a table that another transaction has inserted a new object into

This list is not exhaustive for simplicity reasons, as there are other situations that can cause transactions to be in conflict. Only transactions whose executions overlap can be in conflict. Transactions that have not run concurrently will never be in conflict.

There is a separate garbage collection process that periodically cleans unnecessary records from memory. Unnecessary records are those that exist at a newer version and have a lower version than the lowest read version of all executing transactions. This means that these records will never be read and can safely be removed from memory.

Final words

Strict serialization transactions are a powerful tool that, in my opinion, is too easily discarded these days. Many modern databases offer weaker isolation levels, and they often obscure the exact guarantees they provide. Although weaker isolation levels can improve performance and scalability, they introduce problems that are challenging to diagnose and rectify.

In conclusion, we have examined the transaction-handling capabilities of VeloxDB. With its strict serialization approach, combined with a sleek ORM-like interface and cutting-edge performance, VeloxDB is a powerful tool.

For more information, check out these helpful resources:
Website: https://www.vlxdb.com
User Guide: https://vlxdb.com/guide/introduction.html
GitHub: https://github.com/VeloxDB/VeloxDB
Twitter: https://twitter.com/VeloxDB

VeloxDB + ASP.NET + Blazor WASM

Ivan Savu — Mon, 13 Feb 2023 11:28:43 +0000

Introduction

At long last, I get to unleash my greatest dream upon the world: my very own web stack. In a sea of LAMPs and MERNs and countless other acronyms, it's finally my turn to make my mark.
This stack is unlike any other for it is built upon the foundation of a single language: C#. From front-end to back-end, from database to the code itself, C# shall be the thread that binds all together.

In this guide, I'll walk you through building a simple web application using the VAB stack. For the sake of convenience, I will refer to the full stack as VAB, but feel free to suggest a more fitting name in the comments.

What is VAB?

The VAB stack is a tool for crafting modern, high-performing web applications. To understand the stack, let's examine each of its components and the role they play.

Blazor WASM is Microsoft's framework for writing web application frontends in C#. With Blazor WASM, the C# code is compiled to WebAssembly and actually executed in the browser – a truly remarkable feat. In our web app, we will leverage Blazor WASM to build the client-side portion of the application, eliminating the need for JavaScript.

VeloxDB is a new object-oriented database. Instead of using SQL, you use C# to create models and query data. It has a similar feel to using an ORM, but with the added bonus of exceptional speed. I am proud to say that I am one of the authors of VeloxDB. If you want to learn more about VeloxDB, be sure to check out my article on the subject. In our web app, we will use VeloxDB as our persistence layer and also implement our business logic within VeloxDB.

Finally, ASP.NET Core is Microsoft's solution for building web applications using .NET. Unlike Blazor WASM, ASP.NET Core operates on the server side. In our web app, we will use ASP.NET Core mainly to expose our VeloxDB operations to the client.

Getting Started

To get started, you'll need Microsoft .NET 7 installed. You can use any IDE of your choice, whether it be Visual Studio, VSCode, or something else entirely. This tutorial will primarily utilize Visual Studio.

With your environment prepared, it's time to start creating projects. The tutorial project can be found on GitHub. Each step of the tutorial is committed as an individual commit within the repository, which serves as a handy reference in case you encounter any difficulties.

Creating Projects

We will begin by creating a new Blazor WASM project. Fire up Visual Studio and navigate to the New Project dialog. From there, select the Blazor WebAssembly Empty template and click Next.

Now, enter a name for your project, such as "vabApp", and click Next once again.

In the Additional Information section, make sure to check the ASP.NET Core Hosted checkbox.

This will cause Visual Studio to generate both a Blazor WebAssembly project and an ASP.NET Core project, which will serve as the host for your Blazor app and API. If you neglect to check this box, you will only receive a Blazor WebAssembly project.

If you're utilizing the dotnet console tool, you can easily create a solution with the following command:
dotnet new blazorwasm-empty --hosted -o vabApp

Now you should have a solution containing the following projects:

vabApp.Client - This is where the Blazor WASM project resides and serves as the client.
vabApp.Server - The ASP.NET server project.
vabApp.Shared - A shared ClassLibrary, where you can place code that will be utilized by both the client and server.

To test your app, simply hit F5 and run. You should be presented with something similar to the following:

Given that we're building a VeloxDB app, it's important to note that our architecture is a bit different from the norm. VeloxDB plays a larger role than a traditional database, while ASP.NET Server has a more limited role. To reflect this, we will rename vabApp.Server to vabApp.WebAPI. Keep in mind, this will only change the name of the .csproj file, leaving the folder names intact. For consistency's sake, I suggest renaming the folder from "vabApp\Server" to "vabApp\WebAPI" and then removing the project and re-adding it to the solution.

Next, let's create another class library project and name it "Server". Once created, I recommend renaming it to "vabApp.Server" to maintain consistency with the other projects in the solution. This project will house our model and business logic.

At this point, your solution should look similar to this:

Before proceeding, open the properties for vabApp.Server and under Build, set the Platform target to x64.

We must now add the appropriate references to our projects. To add NuGet references to "vabApp.WebAPI", follow these steps:

Right-click on the project in the Solution Explorer and select Manage NuGet Packages.
In the NuGet Package Manager window, select the Browse tab.
Search for "VeloxDB.Protocol" and click on it.
Select the latest version and click the Install button.
Repeat the steps for "VeloxDB.AspNet" and "Swashbuckle.AspNetCore".

For "vabApp.Server", follow these steps:

Add references to VeloxDB and VeloxDB.Protocol NuGet packages.
To add a project reference, right-click on the project in the Solution Explorer and select Add then Project Reference.
In the Reference Manager window, select Projects from the left-hand side.
Check the box next to "vabApp.Shared" and click OK.

The VeloxDB package provides the core functionality for working with the VeloxDB database. The VeloxDB.Protocol package provides the necessary protocols for connecting to the VeloxDB server. Finally, the vabApp.Shared project reference provides the necessary shared code between projects.

To run the vabApp.Server project, follow these steps:

Right-click on the "vabApp.Server" project in the Solution Explorer.
Select Debug and then click Start without debugging.

This will start VeloxDB with an empty model.

Creating the Model

For our demonstration, we will be constructing a straightforward contacts app. This will consist of a single class model with basic information about a contact.

To get started, add a file named "Contact.cs" to the "vabApp.Server" project with the following content:

using VeloxDB.ObjectInterface;

namespace vabApp.Server
{
    [DatabaseClass]
    public abstract class Contact : DatabaseObject
    {
        [DatabaseProperty]
        public abstract string FirstName { get; set; }

        [DatabaseProperty]
        public abstract string LastName { get; set; }

        [DatabaseProperty]
        public abstract string Email { get; set; }

        [DatabaseProperty]
        public abstract string PhoneNumber { get; set; }        

    }
}

You can now run the server to ensure everything is functioning properly. If it starts up without any errors, you can proceed to the next step.

Creating CRUD API

The next step is to create and expose basic CRUD API.

We'll start by adding a DTO class for our Contact to the "vabApp.Shared" project. This DTO will be used for moving data between the Server and the WebAPI, and then again between the WebAPI and the Blazor client. To achieve this, simply create a new "Contact.cs" file within the "vabApp.Shared" project and include the following content:

namespace vabApp.Shared
{
    public class Contact
    {
        public long Id { get; set; }
        public string? FirstName { get; set; }
        public string? LastName { get; set; }
        public string? Email { get; set; }
        public string? PhoneNumber { get; set; }
    }
}

To streamline the process of moving data between the database and DTOs, VeloxDB comes equipped with a built-in automapper. Update the "Contact" class in "vabApp.Server" as follows:

[DatabaseClass]
public abstract partial class Contact : DatabaseObject
{
    [DatabaseProperty]
    public abstract string FirstName { get; set; }

    [DatabaseProperty]
    public abstract string LastName { get; set; }

    [DatabaseProperty]
    public abstract string Email { get; set; }

    [DatabaseProperty]
    public abstract string PhoneNumber { get; set; }

    public partial vabApp.Shared.Contact ToDTO();
    public static partial Contact FromDTO(ObjectModel om, vabApp.Shared.Contact dto, bool allowUpdate = false);
}

The "Contact" class is now marked partial, and there are two new methods: ToDTO and FromDTO. The automapper source generator will fill these methods automatically.

It's now time to create our API. In "vabApp.Server", create a new class and call it "ContactsAPI". Place the following content within it:

using VeloxDB.ObjectInterface;
using VeloxDB.Protocol;
using DTO = vabApp.Shared;

namespace vabApp.Server
{
    [DbAPI]
    public class ContactsAPI
    {
        [DbAPIOperation]
        public long Add(ObjectModel om, DTO.Contact toAdd)
        {
            // Creates new contact in the database using the data in toAdd DTO
            Contact newContact = Contact.FromDTO(om, toAdd);
            return newContact.Id;
        }

        [DbAPIOperation]
        public List<DTO.Contact> GetContacts(ObjectModel om)
        {
            // Gets all contacts in the database, converts them to DTOs and returns them
            return new List<DTO.Contact>(om.GetAllObjects<Contact>().Select(c=>c.ToDTO()));
        }

        [DbAPIOperation]
        public DTO.Contact? GetContact(ObjectModel om, long id)
        {
            Contact? c = om.GetObject<Contact>(id);

            // GetObject returns null if the object does not exist in the database
            if (c == null)
                return null;

            return c.ToDTO();
        }

        [DbAPIOperation]
        public bool Update(ObjectModel om, DTO.Contact toUpdate)
        {
            // Checks if the object with given id exists in the database
            if(om.GetObject<Contact>(toUpdate.Id) == null)
                return false;

            // FromDTO can update an object if it's 3rd argument (allowUpdate) is set to true.
            // It will fetch an object from the db and update all its properties with values from the dto.
            Contact.FromDTO(om, toUpdate, true);
            return true;
        }

        [DbAPIOperation]
        public bool Delete(ObjectModel om, long id)
        {
            Contact? c = om.GetObject<Contact>(id);

            // If the object with given id does not exist in the database, return false
            if (c == null)
                return false;

            // Delete the object
            c.Delete();
            return true;
        }
    }
}

Here, we've defined our database operations which are now accessible through VeloxDB's protocol. We have implemented Add, Update, Delete, and Get operations, and with the aid of the comments provided, the code should be straightforward to understand.

The first argument in every database operation is ObjectModel. ObjectModel is an interface that provides access to VeloxDB. It enables you to retrieve objects, create new ones, and perform queries on the indexes.

Each DbAPI operation is a database transaction. If the method completes successfully, the transaction is committed. If the method fails (an exception is thrown), the transaction is rolled back. VeloxDB is an ACID-compliant database with strict serializability guarantees.

To ensure high performance, VeloxDB uses an internal binary protocol to access these operations. However, web browsers can only communicate using the HTTP protocol, and as VeloxDB server doesn't support HTTP, a WebAPI needs to be created to allow web clients to access VeloxDB operations.

WebAPI

Let me start by addressing why VeloxDB doesn't have native support for a REST API. As a database engineered for maximum performance, it seemed like a waste of resources to have the database spend time parsing JSON. That's why VeloxDB only supports the most performant protocol available – our own, specifically engineered for the task. I will write a separate article on VeloxDB's RPC protocol in the future.
The additional benefits of having a separate JSON translation layer include:

The WebAPI layer can be easily scaled.
It can host simple validation logic, such as checks for input validity that don't require accessing the database.
VeloxDB's client library also performs batching of calls, so having a WebAPI server in front of VeloxDB would cause calls from clients to arrive in batches at VeloxDB, which would further relieve pressure on the database.

Let's create the interface for the ContactAPI in the "vabApp.WebAPI" project by creating a new class called "ContactsAPI". Copy and paste the following code into it:

using vabApp.Shared;
using VeloxDB.Client;
using VeloxDB.Protocol;

namespace vabApp.WebAPI
{
    [DbAPI(Name = "vabApp.Server.ContactsAPI")]
    public interface ContactsAPI
    {
        [DbAPIOperation]
        DatabaseTask<long> Add(Contact toAdd);

        [DbAPIOperation]
        DatabaseTask<List<Contact>> GetContacts();

        [DbAPIOperation]
        DatabaseTask<Contact?> GetContact(long id);

        [DbAPIOperation]
        DatabaseTask<bool> Update(Contact toUpdate);

        [DbAPIOperation]
        DatabaseTask<bool> Delete(long id);
    }
}

This is the interface of our ContactsAPI, which we will use to connect to the server. Note that ObjectModel is not present as the first argument, as it is provided by the database. Additionally, all return values are wrapped in DatabaseTask. This is because we want an asynchronous interface. If we had omitted DatabaseTask, we would have a synchronous interface, which would work fine, but it's recommended to use an asynchronous interface for web applications.

I noticed while writing this tutorial that controllers for WebAPI are a real chore to write. All they do is call a method from a contract interface. To alleviate this, I created a SourceGenerator that generates controllers from DbAPI. This generator is available through the "VeloxDB.AspNet" NuGet package. The generator is still quite simple, but it serves its purpose for this example.

The generator identifies all the DbAPIAttribute interfaces in the project and generates an appropriate controller for each. You can use attributes to guide the generator on what attributes to include in the generated code. If you want to write the controller yourself, you can use the DoNotGenerateControllerAttribute to prevent the generator from generating a controller for a particular interface. This can be useful if you want to do more with the controller, like validate arguments.

Let's first decorate our interface with ASP.NET attributes to better specify the desired controller. Update the interface as follows:

using Microsoft.AspNetCore.Mvc;
using vabApp.Shared;
using VeloxDB.AspNet;
using VeloxDB.Client;
using VeloxDB.Protocol;

namespace vabApp.WebAPI
{
    [Forward(typeof(RouteAttribute), "/api/Contacts/[action]")]
    [DbAPI(Name = "vabApp.Server.ContactsAPI")]
    public interface ContactsAPI
    {

        [DbAPIOperation]
        DatabaseTask<long> Add(Contact toAdd);

        [HttpGet]
        [DbAPIOperation]
        DatabaseTask<List<Contact>> GetContacts();

        [HttpGet]
        [DbAPIOperation]
        DatabaseTask<Contact?> GetContact(long id);

        [DbAPIOperation]
        DatabaseTask<bool> Update(Contact toUpdate);

        [HttpDelete]
        [DbAPIOperation]
        DatabaseTask<bool> Delete(long id);
    }
}

Pay heed to the Forward attribute – this attribute enables you to pass along any attribute to the generated controller. This is critical, as ASP.NET attributes cannot be applied to an interface. On the other hand, method attributes do not have these limitations, so we utilize them directly.

In the vabApp.Shared project, add a file named LongToStringConverter.cs with the following content:

using System.Text.Json.Serialization;
using System.Text.Json;

namespace vabApp.Shared
{
    public class LongToStringConverter : JsonConverter<long>
    {
        public override long Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            if (reader.TokenType == JsonTokenType.String)
            {
                return Convert.ToInt64(reader.GetString());
            }

            throw new NotSupportedException();
        }

        public override void Write(Utf8JsonWriter writer, long value, JsonSerializerOptions options)
        {
            writer.WriteStringValue(value.ToString());
        }
    }
}

I'll explain why we added this piece of code in a moment. Now open the Program.cs file in the vabApp.WebAPI project and modify it like this:

using Microsoft.OpenApi.Models;
using VeloxDB.AspNet.Extensions;
using vabApp.Shared;
using Microsoft.AspNetCore.Mvc.Formatters;

internal class Program
{
    private static void Main(string[] args)
    {
        var builder = WebApplication.CreateBuilder(args);

        builder.Services.AddControllersWithViews().AddJsonOptions((options) =>
        {
            options.JsonSerializerOptions.Converters.Add(new LongToStringConverter());
        });

        builder.Services.AddControllers(options =>
        {
            options.OutputFormatters.RemoveType<HttpNoContentOutputFormatter>();
        });

        builder.Services.AddRazorPages();
        builder.Services.AddSwaggerGen(options => options.MapType<long>(() => new OpenApiSchema { Type = "string" }));
        builder.Services.AddVeloxDBConnectionProvider("address=localhost:7568;");

        var app = builder.Build();

        // Configure the HTTP request pipeline.
        if (app.Environment.IsDevelopment())
        {
            app.UseWebAssemblyDebugging();
            app.UseSwagger();
            app.UseSwaggerUI();
        }
        else
        {
            // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
            app.UseHsts();
        }

        app.UseHttpsRedirection();
        app.UseBlazorFrameworkFiles();
        app.UseStaticFiles();

        app.UseRouting();

        app.MapRazorPages();
        app.MapControllers();
        app.MapFallbackToFile("index.html");

        app.Run();
    }
}

There are two things we have done here. Firstly, we've registered the VeloxDB connection provider using AddVeloxDBConnectionProvider. The VeloxDBConnection provider is a simple service that provides connections to VeloxDB. Our controller leverages it to connect to VeloxDB. For this sample application, I've hardcoded the connection string, but in a real-world scenario, it would be wise to put it in a config file or user secret.

Secondly, we've updated the code to configure JSON serialization to always serialize long values as strings. JavaScript handles all numbers as double floating types, and a 64-bit integer can have more numbers than a double type can handle. To avoid truncation, we use strings for our long values.

You should now be able to run the application. Start both the vabApp.WebAPI and vabApp.Server projects. You can start the first one using a regular run, and for the second project, go to the solution explorer, right-click on the project, select Debug, and choose Start Without Debugging.

Once the server starts and your browser opens, go to the "/Swagger" address. The Swagger UI will load, and you will see our Controller API. You can now try executing these methods from the UI, and they should work. Try adding, updating, and deleting contacts.

Now that the database is running and the WebAPI is exposing its interface to the web, we can move on to the final step.

Writing the Client

Finally, it's time to write the client. I don't want to get bogged down in too many details here – there are plenty of fantastic Blazor WebAssembly tutorials available. The primary goal of this article is to demonstrate how VeloxDB can be used in the creation of modern web apps. I opted for Blazor because I felt that its C#-based architecture is a great match for VeloxDB. But any web framework would do the trick.

With that in mind, let's tweak the Index.razor file within the vab.WebAPI project.

@page "/"
@using System.Text.Json;
@using vabApp.Shared;
@inject HttpClient Client;

<h1>Contacts</h1>

<a href="/add">Add Contact</a>

@if(message != null)
{
<p style="color:red;">
    @message
</p>
}

@if(contacts == null || contacts.Length == 0)
{
    <p>No contacts found!</p>
}
else
{
    <table>
        <thead>
            <tr>
                <th>First name</th>
                <th>Last name</th>
                <th>E-Mail</th>
                <th>Phone</th>
                <td></td>
                <td></td>
            </tr>
        </thead>

        @foreach (Contact contact in contacts)
        {           
            <tr>
                <td>@contact.FirstName</td>
                <td>@contact.LastName</td>
                <td>@contact.Email</td>
                <td>@contact.PhoneNumber</td>
                <td><a href="/Edit/@contact.Id">Edit</a></td>
                <td><a href="" @onclick="()=>Delete(contact.Id)" @onclick:preventDefault >Delete</a></td>
            </tr>
        }
    </table>
}

@code{
    private Contact[]? contacts = null;
    private string? message = null;

    JsonSerializerOptions options = new JsonSerializerOptions(JsonSerializerDefaults.Web)
    {
        Converters = { new LongToStringConverter() }
    };

    private async Task Delete(long id)
    {
        message = null;
        try
        {
            var response = await Client.DeleteAsync($"/api/Contacts/Delete?id={id}");
            response.EnsureSuccessStatusCode();
            await LoadContacts();
        }
        catch(HttpRequestException)
        {
            message = "Failed to delete contact";
        }
    }

    protected override async Task OnInitializedAsync()
    {
        await LoadContacts();
    }

    private async Task LoadContacts()
    {
        message = null;
        try
        {
            contacts = await Client.GetFromJsonAsync<Contact[]>("api/Contacts/GetContacts", options);
        }
        catch (HttpRequestException)
        {
            message = "Error fetching contacts";
        }
    }
}

This is a straightforward Blazor WASM page targeting a regular old WebAPI. Nothing special, just the usual suspects – fetch data, display it, catch errors.

Up next, let's create an Add.razor file with this inside:

@page "/Add"
@using System.Text.Json;
@using vabApp.Shared;
@using Microsoft.AspNetCore.Components.Forms
@inject HttpClient Client;
@inject NavigationManager NavManager

<h1>Add Contact</h1>

@if (errorVisible)
{
    <p style="color:red;">
        Error adding contact.
    </p>
}

<EditForm Model="@contact" OnSubmit="@HandleSubmit">
    <p>
        <label>
            First Name:
            <InputText id="firstName" @bind-Value="contact.FirstName" />
        </label>
    </p>
    <p>
        <label>
            Last Name:
            <InputText id="lastName" @bind-Value="contact.LastName" />
        </label>
    </p>
    <p>
        <label>
            E-Mail
            <InputText id="email" @bind-Value="contact.Email" />
        </label>
    </p>
    <p>
        <label>
            Phone Number:
            <InputText id="phoneNumber" @bind-Value="contact.PhoneNumber" />
        </label>
    </p>
    <button type="submit">Submit</button>
</EditForm>

@code {
    private Contact contact = new();
    private bool errorVisible = false;

    JsonSerializerOptions options = new JsonSerializerOptions(JsonSerializerDefaults.Web)
    {
        Converters = { new LongToStringConverter() }
    };

    private async Task HandleSubmit()
    {
        errorVisible = false;
        try
        {
            var response = await Client.PostAsJsonAsync<Contact>("/api/Contacts/Add", contact, options);
            response.EnsureSuccessStatusCode();

            NavManager.NavigateTo("/");
        }catch(HttpRequestException)
        {
            errorVisible = true;
        }
    }
}

And edit.razor page:

@page "/Edit/{Id}"
@using System.Text.Json;
@using vabApp.Shared;
@using Microsoft.AspNetCore.Components.Forms
@inject HttpClient Client;
@inject NavigationManager NavManager

<h1>Edit Contact</h1>

@if (message != null)
{
    <p style="color:red;">
        @message
    </p>
}
else
{
    <EditForm Model="@contact" OnSubmit="@HandleSubmit">

        <p>
            <label>
                First Name:
                <InputText id="firstName" @bind-Value="contact.FirstName" />
            </label>
        </p>
        <p>
            <label>
                Last Name:
                <InputText id="lastName" @bind-Value="contact.LastName" />
            </label>
        </p>
        <p>
            <label>
                E-Mail
                <InputText id="email" @bind-Value="contact.Email" />
            </label>
        </p>
        <p>
            <label>
                Phone Number:
                <InputText id="phoneNumber" @bind-Value="contact.PhoneNumber" />
            </label>
        </p>
        <button type="submit">Submit</button>

    </EditForm>
}

@code {
    private Contact contact = new();
    private string? message = null;

    [Parameter] public string? Id { get; set; }

    JsonSerializerOptions options = new JsonSerializerOptions(JsonSerializerDefaults.Web)
    {
        Converters = { new LongToStringConverter() }
    };

    protected override async Task OnInitializedAsync()
    {
        if(Id == null || !long.TryParse(Id, out _))
        {
            message = "Invalid contact Id";
            return;
        }

        Contact? newContact = null;
        try
        {
            newContact = await Client.GetFromJsonAsync<Contact>($"/api/Contacts/GetContact?Id={Id}", options);
        }
        catch (HttpRequestException)
        {
            message = "Error fetching contact";
        }

        if (newContact != null)
        {
            contact = newContact;           
        }
        else
        {
            message = "Contact not found";  
        }

    }

    private async Task HandleSubmit()
    {
        message = null;
        try
        {
            var response = await Client.PostAsJsonAsync<Contact>("/api/Contacts/Update", contact, options);
            response.EnsureSuccessStatusCode();

            NavManager.NavigateTo("/");
        }catch(HttpRequestException)
        {
            message = "Error updating contact";
        }
    }
}

And now, it's finally time to run the app. You're looking at a basic contacts web page with the ability to create, update, or delete contacts. All in C#, from the frontend to the very heart of the database. Marvelous, isn't it?

What's Next

VeloxDB is still in its beta phase but there's a ton of great things to come. We're talking SQL Select, Disk-based classes, Time-series classes, BTree index, Spatial index, and Full-text search. And as for the ASP.NET package, well, it's just a demo for now. But, I've got plans to add gRPC support, give you better control over controller generation, and throw in some ASP.NET features like authentication. If you want to stay in the loop, follow us on GitHub or Twitter.

Building a strong and supportive community around VeloxDB and VAB is a major priority for the future. I want to encourage developers to contribute their own ideas and feedback, which will play a crucial role in guiding the future direction of VeloxDB and VAB.

Ultimately, my goal is to make VeloxDB and VAB the go-to solution for building high-performance, scalable web applications using the power of C#. Stay tuned for exciting developments in the near future!

Introducing VeloxDB

Ivan Savu — Thu, 19 Jan 2023 08:42:28 +0000

This is a project I have been working on with some friends for some time. In March, we decided to leave our jobs and pursue it full-time, and now we have something to show.

We built a database called VeloxDB (Velox means speed in Latin). True to its name, VeloxDB is a fast database, with performance being its main focus. However, we have not sacrificed correctness for performance. Many high-performing databases make compromises in consistency and correctness, but VeloxDB is both fast and accurate. While we have made some other tradeoffs compared to traditional and NoSQL databases, correctness is not one of them.

What is VeloxDB

VeloxDB is a fast, object-oriented, open source in-memory database for C# with an emphasis on correctness.

Let me give a quick breakdown of all the terms involved:

"Fast": The whole database is built with performance in mind. We achieve 2.5 million ACID write transactions with an AWS c6a.8xlarge compute instance.
"Object-oriented": The DB offers an object-oriented interface, giving you an ORM-like experience. Your code is deployed within the database, reducing the overhead of communicating with the database to a minimum.
"Open source": Source code for community version of VeloxDB is available under MIT License.
"In-memory": The entire dataset has to fit in memory, so if you have a 30GB database, you need 30GB of RAM. The data is still persisted on disk, so there is no data loss if the server crashes.
"For C#": The ORM API is primarily designed for C#, although it should work with any .NET language (F#/VB).
"Emphasis on correctness": The database is fully ACID, with no eventual consistency.

What is it good for?

VeloxDB is suitable for a wide range of use cases that require high performance, low latency and a strong emphasis on consistency and correctness. Some of the most common use cases include:

High-performance transactional applications: VeloxDB's in-memory architecture and strict serializability guarantees make it a great fit for applications that require high levels of read and write throughput.
Real-time data processing: Its low latency and high performance make it an ideal choice for applications that require real-time data processing, such as financial trading systems, gaming servers, and IoT applications.
Graph-like queries: Its object-oriented approach and specialized inverse reference index make it well-suited for applications that require complex, graph-like queries, such as social networks, recommendation systems, and fraud detection.

It's important to note that as an in-memory database, VeloxDB requires that the entire dataset fits into memory. While this allows for very fast access to data, it also means that the amount of available memory must be taken into consideration when designing and scaling a VeloxDB application.

The Code

Let's take a closer look at the code required to build something using VeloxDB. Keep in mind that this is just a brief overview of the process. The goal is to give you a better understanding of what working with VeloxDB entails. I will be omitting certain details, but if you are interested in trying to build something yourself, there is a comprehensive getting started guide available on our website.

To start, we will create a new C# class library and add the VeloxDB and VeloxDB.Protocol NuGet packages to it. Then, we will begin coding.

As an example, let's say we are building a simple blog application that includes a collection of posts. We will first define our model for these posts.

[DatabaseClass]
public abstract class Post : DatabaseObject
{
    [DatabaseProperty]
    public abstract string Title { get; set; }

    [DatabaseProperty]
    public abstract string Content { get; set; }
}

As you can see, models in VeloxDB are represented as classes, similar to how most ORMs function. To utilize the model, we create a DbAPI for it. DbAPI serves as a publicly accessible API through VeloxDB's protocol. Below is an example of a simple DbAPI that adds a new post to the database.

[DbAPI]
public class BlogApi
{
    [DbAPIOperation]
    public long CreatePost(ObjectModel om, PostDTO post)
    {
        Post newPost = om.CreateObject<Post>();
        newPost.Title = post.Title;
        newPost.Content = post.Content;
        return newPost.Id;
    }
}

Here we define a method, CreatePost, that can be called by a client. The method accepts PostDTO, which is a Data Transfer Object version of our Post class. There is nothing particularly noteworthy about it, so I have omitted it here. The ObjectModel is provided by VeloxDB and is used to access the database.

We now have a class library that defines a model and a simple operation to work with it. The easiest way to run it is by simply running the project. The VeloxDB NuGet package comes with a bundled VeloxDB server, which is used to run the project.

Behind the scenes

The VeloxDB server loads the compiled class library (dll)
The server scans the dll for any classes marked with the DbAPI and DatabaseClass attributes.
The server updates its internal data model based on the classes it found in the dll and makes the DbAPI methods publicly accessible through its protocol.
When a client calls a method in a DbAPI class, VeloxDB opens a new transaction, creates an ObjectModel and executes the DbAPI method.
The results of the method are then returned to the client.

Conclusion

The use of compiled dlls in VeloxDB allows for minimal overhead when executing DbAPI methods. This approach is similar to how stored procedures work in traditional SQL databases, but instead of using SQL procedures, VeloxDB utilizes .NET dlls.

Compared to traditional ORM-Relational database setups, VeloxDB offers a similar user experience with less overhead.

The object-oriented approach is also well-suited for graph-like queries. Algorithms such as graph searches, graph coloring, and shortest path queries are easier to implement with an OO database than with a traditional SQL database.

However, it is worth noting that SQL excels in certain use cases, such as ad-hoc queries, ease of use, and set operations. For this reason, we plan to add SQL support to VeloxDB in the future.

If this article has sparked your interest, please visit our GitHub page and our website for more information.

The database is still in beta, and any feedback is most welcome.