Getting Started With Dokklib-DB for DynamoDB Single Table

DynamoDB Single Table

Dokklib-DB is a Python library for the DynamoDB single table pattern. If you can anticipate your query patterns, DynamoDB is a particularly good fit for serverless applications, as it has minimal operational overhead and stateless connections. (Read more about DynamoDB vs SQL for serverless).

With the single table pattern, you put all your data that you want to perform relational queries on into one DynamoDB table. The key to a successful single table design is being able to anticipate query patterns, as you have to structure the data accordingly. Let's see how this works in practice with Dokklib-DB!

Install Dokklib-DB

Install Dokklib-DB and Boto3 with Pip (you need Python 3.6 or later):

pip install "boto3>=1.10.34,<2" dokklib-db

Table Setup

We need a table with a generic composite primary key. We will call the partition key PK and the sort key SK. The generic key names allow us to store different entity types in the same table. We also a need a global secondary index that is the inverse of the primary index (ie. where the partition key is SK and the sort key is PK). We will use this inverse index for many-to-many relational queries. You can read more about the single table setup in the Dokklib-DB docs or see a complete CloudFormation template here.

Define Entities

Suppose that we want to be able to add our users to groups and make many-to-many relational queries to fetch a user's groups or the members of a group. First, we define the user and group entities:

import dokklib_db as db


class User(db.EntityName):
    """User entity name.

    Key value: unique user name, eg. 'alice'.
    Example key: 'USER#alice'.

    """


class Group(db.EntityName):
    """Group entity name.

    Key value: unique group name, eg. 'my-group'.
    Example key: 'GROUP#my-group'.

    """

These classes don't hold any data, they just provide type-safety and documentation for our entities.

Insert Data

Let's see how we can add a user to a group:

table = db.Table('SingleTable')

# Construct entity keys

pk_alice = db.PartitionKey(User, 'alice')
print(str(pk_alice))
# USER#alice

sk_group_1 = db.SortKey(Group, '1')
print(str(sk_group_1))
# GROUP#1

# Add Alice to group one
table.insert(pk_alice, sk_group_1)

Suppose that we added another user and a second group and we have the following data in the table:

PK	SK
USER#alice	GROUP#1
USER#alice	GROUP#2
USER#bob	GROUP#2

Many-to-Many Queries

Now we can use the primary index to query all the groups of a user:

# Get all the groups that Alice belongs to

pk_alice = db.PartitionKey(User, 'alice')
print(str(pk_alice))
# USER#alice

group_prefix = db.PrefixSortKey(Group)
print(str(group_prefix))
# GROUP#

alice_groups = table.query_prefix(pk_alice, group_prefix)
print(alice_groups)
# [{'SK': '1'}, {'SK': '2'}]

And the inverse index to query all the members of a group:

# Get all users in group two

pk_group_2 = db.PartitionKey(Group, '2')
print(str(pk_group_2))
# GROUP#2

user_prefix = db.PrefixSortKey(User)
print(str(user_prefix))
# USER#

inverse_index = db.InversePrimaryIndex()

members = table.query_prefix(pk_group_2, user_prefix, 
    global_index=inverse_index)
print(members)
# [{'PK': 'alice'}, {'PK': 'bob'}]

This wraps up our short intro. I encourage you to continue exploring in the Dokklib-DB docs and to provide feedback/feature requests on Github.

Comments

[Seth Centerbar]

Agost, this library is exactly what I've been looking for. Your implementation looks super clean, installing it now!

Edit: Already loving it! Heads up, your website is down.