A better way to work with AWS DynamoDB and .NET

Background

One of the challenges that we faced when we started building our platform a few years ago was the .NET tooling around DynamoDB. Most of our developers had been using .NET full-fledged and micro ORMs such as Entity Framework and Dapper with relational databases and expected the same experience with DynamoDB, our key-value and document database of choice.

Originally, we began using the .NET DynamoDB SDK and it worked well for us. We quickly found ourselves creating extensions and wrappers around the SDK to remove the low level duplicated code, mainly aimed to be more productive, until we stumbled upon a new tool, which was exactly what we have been looking for.

Introducing PocoDynamo

PocoDynamo is a Typed .NET client which extends ServiceStack's Simple POCO life by enabling re-use of your code-first data models with Amazon's industrial strength and highly-scalable NoSQL DynamoDB. It enhances and improves AWSSDK's low-level client, with rich, native support for intuitively mapping your re-usable code-first POCO Data models into DynamoDB Data Types. Thus improving the overall developer experience and productivity and makes working with DynamoDB, a joy!

Getting Started

In this section we will discuss how to get started using PocoDynamo in a real-world application using some of the best practices we learned.

First we will need to add the NuGet package:

> dotnet add package ServiceStack.Aws

Next we'll need to create an instance of AmazonDynamoDBClient with the AWS credentials and Region info:

public static IAmazonDynamoDB CreateAmazonDynamoDb(string serviceUrl)
{
            var clientConfig = new AmazonDynamoDBConfig {RegionEndpoint = RegionEndpoint.EUWest1};

            if (!string.IsNullOrEmpty(serviceUrl))
            {
                clientConfig.ServiceURL = serviceUrl;
            }

            var dynamoClient = new AmazonDynamoDBClient(clientConfig);

            return dynamoClient;
}

We can now create an instance of PocoDynamo. We can register the instance through dependency injection as the clients are Thread-Safe.

services.AddSingleton<IAmazonDynamoDB>(x => CreateAmazonDynamoDb(options.ServiceUrl));
services.AddSingleton<IPocoDynamo, PocoDynamo>();

Next we will configure PocoDynamo by adding it to our Startup class. The purpose of this configuration is to initialize the schema which we will discuss in the following sections in more details.

app.ConfigureDynamoDb();

And that's it. Now we are ready to add PocoDynamo as a dependency by injecting it in the constructor:

public EmployeeController(IPocoDynamo db)
{
   _db = db;
}

Design Approaches

When it comes to designing your data model in DynamoDB, there are two distinct design approaches multi-table or single-table. We are not going to discuss the difference between the two approaches, you can find plenty of articles about that. We will merely show how we can use the library in both.

Multi-Table

In a multi-table approach, we have one table per each entity and each item maps to a single instance of the entity providing consistency across attributes.

First we define an entity such as Employee

public class Employee
{
    [HashKey]
    public string Id { get; set; }
    public string Name { get; set; }
    public string DepartmentId { get; set; }

    public Employee() {}

    public Employee(string name, string departmentId)
    {
        Id = Guid.NewGuid().ToString("D");
        Name = name;
        DepartmentId = departmentId;
    }
}

Next we register the table for the employee type

var employeeType = typeof(Employee);
// This is how we set the name of the table
employeeType.AddAttributes(new AliasAttribute("employee"));

To add a new Employee to the table, it is very simple

var employee = new Employee(name, departmentId);
_db.PutItem(employee);

And to read the employee from the table

var employee = _db.GetItem<Employee>(id);

In a multi-table approach, working with PocoDynamo is very straightforward and doesn't require any additional considerations unlike a single-table approach.

Single-Table

In a single-table approach, one table holds multiple types of entities within it. Each item has different attributes set on it depending on its entity type. While this approach might be less common, it has lots of advantages when querying "related" data such as in a one to many relationship. Let's see an example.

First we will create two entities. We will define a generic combination of HashKey/RangeKey and we will create a Type attribute to hold the type name.

public class Customer : IRecord
{
     ...

     public Customer(string name)
     {
         CustomerId = Guid.NewGuid().ToString("D");
         Name = name;
         HashKey = $"CUSTOMER#{CustomerId}";
         RangeKey = $"CUSTOMER#{CustomerId}";
         Type = "CUSTOMER";
     }
}

public class Order : IRecord
{
    ...

    public Order(string customerId, double orderTotal)
    {
        OrderId = Guid.NewGuid().ToString("D");
        OrderTotal = orderTotal;
        CreatedDate = DateTime.Now;
        HashKey = $"CUSTOMER#{customerId}";
        RangeKey = $"ORDER#{OrderId}";
        Type = "ORDER";
    }
}

Next we are going to register both entities to map to the same table.

var customerType = typeof(Customer);
customerType.AddAttributes(new AliasAttribute("customer-orders"));

var orderType = typeof(Order);
orderType.AddAttributes(new AliasAttribute("customer-orders"));

db.RegisterTable(customerType);
db.RegisterTable(orderType);

Creating Customers and Orders would look similar to how we do it in a single-table approach so I am going to omit it. What we will focus on how we can query data.

The first example we will look at is how to query a single customer by customer id. Notice the use of the same Hash key and Range key to indicate that we are querying a customer here and not customer and orders.

var customer = _db
                .FromQuery<Order>()
                .KeyCondition($"HashKey = :customerId  and RangeKey = :orderId", new Dictionary<string, string>()
                {
                    {"customerId", $"CUSTOMER#{id}"},
                    {"orderId", $"CUSTOMER#{id}"}
                })
                .Exec().SingleOrDefault()

The second example we will look at is how to query the orders of a customer (in a case of one-to-many relationship). Notice the use of begins_with in the expression as we would like to query only orders and not to include the customer data since it will break the mapping.

var orders = _db
                .FromQuery<Order>()
                .KeyCondition($"HashKey = :customerId  and begins_with(RangeKey, :order)", new Dictionary<string, string>()
                {
                    {"customerId", $"CUSTOMER#{id}"},
                    {"order", $"ORDER"}
                })
                .Exec().ToList()

In a single table approach, you will need to use the right combination of Hash / Range key to get exactly the data that you need.

Considerations

PocoDynamo has a 10 Tables free-quota usage limit which can be unlocked with a commercial license key.

Conclusion

In this article we've looked at ServiceStack's PocoDynamo and how it can help us be more productive when working with DynamoDB. If you are okay with buying a commercial license, it is well worth it. If you know of any similar tools, free or open-source, I'd be glad to hear about it.

You can find the source code on Github