Single-table design for DynamoDB has gained popularity due to the increased ability to query relational data in No-SQL databases.
The TypeDORM is an Object Relational Mapper library running in Node,js and written in Typescript. TypeDORM is a feature-rich DynamoDB ORM to help you write high-quality, loosely coupled, scalable, and maintainable applications with highly relational data in the most productive way.
What this article is about
In this article, we will be looking at different challenges that may arise while developing with DynamoDB using a single-table-design pattern and how TypeDORM can help us avoid them.
What this article is NOT about
We will not be covering why and who should use DynamoDB single-table design as Alex DeBrie has done an excellent job at explaining it here.
Once you understand how single-table design works with DynamoDB, you would want to try it out with a real-life example. There is already a very popular article by Forrest Brazeal that can be found here, which you should first look at before continuing with this article.
What you should know before moving on
Once you are familiar with the single-table design pattern and have also looked at how it can be used to model highly relational data in DynamoDB, let's talk about why it isn't a very viable way to work with data and what can we do to make it better.
When first introduced to single-table design pattern, there are some common questions and concerns developers start to ponder:
- I can't use a single-table design, I need operation time "types safety".
- All the attributes and their names just don't make sense anymore, how do I reliably interact with them?
- As I keep adding more indexes, how do I make sure that all the attributes created for modeling don't end up returning to clients?
- If I choose to keep the original attribute value and its index-referenced attribute separate, how do I make sure they are always in sync?
- How do I keep the key schema up to date with a matching pattern for each entity that I add?
- How do I confidently interact with all entities without having to remember to provide correct condition expressions for common operations (i.e for unique attributes, making sure to always include condition expressions with attribute_not_exists)?
- With a single table design, I can't use data mapper libraries.
- The resulting code is hard to read as there are too many attributes/symbols representing different meanings.
- My colleges may know how to work with the DynamoDB but this particular way is very unfamiliar.
- I want to use the single-table design but I don't want to manage each entity schemas in a separate spreadsheet.
There might be many more questions but most of them are pointing to one concern in particular:
"Do I really want to trade off development ease, developer experience, code quality over this new way of working with DynamoDB?"
The answer is: you don't have to. There is a better option.
Let me introduce you to TypeDORM: Strongly typed ORM for DynamoDB - Built for the single-table-design pattern. Let's first have a quick look at what it is and how it can help us to solve all the problems stated earlier.
Later, we will also have a look at how we can leverage TypeDORM to better structure entities (continuing from the example from this great article).
Introduction to TypeDORM
TypeDORM is an ORM built from the ground up using TypeScript to provide an easy entry point when doing complex, highly relational data modeling in DynamoDB. Its syntax might be familiar since the decorator-based syntax is highly influenced by TypeORM - one of the very popular ORM for working with SQL databases.
Let's take a quick look at some of the many features that TypeDORM provides:
- Single table design first-class support
- DataMapper development pattern
- Full type safety enabled using advanced typescript generics
- Declarative relational schema
- Multiple connections support
- Operation based managers per connection, includes Entity manager, Transaction manager, Batch manager, and Scan Manager
- Code follows all possible best practices and provides first-class support for some of the very popular patterns used in the single-table-design pattern
- Additional higher-level features like, such as the ability to declare non-key attribute as unique, automatic retries
- Auto-Generated values for attributes
- Auto-update attributes and its referenced indexes on UPDATE operations
- Dynamic & Static default values for attributes
- Complex update, key condition, and condition expression all made easy to work with
- Powerful expression builder to auto-generate expressions from typesafe input
Getting Started with TypeDORM
Installing
Let's first install all the modules that we will need.
npm install @typedorm/common @typedorm/core --save
We will also need to install reflect-metadata shim that TypeDORM relies on and configure it so that metadata's are correctly detected by TypeDORM
npm install reflect-metadata --save
Enable emitting metadata at runtime
If you are using TypeDORM with TypeScript, make sure you also have the below options enabled in your tsconfig.json
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
Once this is done we need aws-sdk which TypeDORM uses to communicate with the DynamoDB table.
npm install aws-sdk --save
We are all set now, next we will need to configure the connection. If you have ever worked with TypeORM, this setup will look very familiar.
In the next section, we will have a look at the relational data set that we will be working with. You can always refer back to Forrest Brazeal's article any time you feel lost.
Getting to know our data
Here are some access patterns that Brazeal has listed:
- Get employee by employee ID
- Get direct reports for an employee
- Get discontinued products
- List all orders of a given product
- Get the most recent 25 orders
- Get shippers by company name
- Get customers by contact name
- List all products included in an order
- Get suppliers by country and region
Entities that we will be working with
These are all the entities we will need to create with a given signature to fulfill our access patterns. Here is a slightly modified version of the Entity chart.
The process of selecting what goes under PK and SK remains the same as described here.
If you were following this closely, you will notice few differences between the above entity chart and one presented by Brazeal, Which are:
- Each attribute is only ever referenced in a single index
- Each entity PK now has a prefix of STATIC entity type - this helps us better identify items where they are all put together in the same table as well as allows us to use incrementing ids
Our access patterns
Here is the list of all access patterns and what attributes we will be querying on. In case you didn't follow this article and came up with different access patterns, that's perfectly okay too.
Hereafter building on top of Brazeal's access patterns, this is what it looks like now.
You will still have to follow all the steps required in a single-table design pattern, with one slight change being that all the entities need to have a SQL-like entity structure (of course, you can still add more attributes to entities later).
Understanding how to build an entity schema would be a different article on its own - our main focus is how we can reliably work with the above entities using TypeDORM.
Developing with TypeDORM
This section will focus on getting connections, entities, tables configured for TypeDORM that can effectively parse data.
Declare a Table
Since we are doing a single-table design we really only need a single DynamoDB table with a single Global Secondary Index (GSI). We declare that by creating a new instance of Table:
Define entities
Now it's time to covert our entity chart into an actual TypeDORM entity, we do this by annotating our model with @Entity decorator, exported by @typedorm/common. Product and Employee entities are the only ones listed here, but you got the overall idea.
Note:TypeDORM exposes various decorators to be able to declaratively define an entity. For those not listed here, check out the TypeDORM API specification for more up-to-date APIs.
Product Entity
Employee Entity
You will need to define all entities in a similar way to the above examples.
Establishing a connection
Once we have all the entities in, the next thing is to initialize a DynamoDB connection. We do this by creating a connection instance:
If all of your entities are grouped under the same directory, you can also provide a relative path to entities option and TypeDORM will auto load all the entities from a given path.
Now since we have everything in place, we are ready to work with a single-table designed DynamoDB table in a type-safe way.
Querying records
Once everything is set up, it's will be super easy to perform all kinds of operations like Create, Delete, Put, Update, all in a type-safe way.
For example, let's have a look at what our queries will look like for the first two access patterns:
P.S. You can provide additional options to filter/log queries.
For the most up-to-date list of available options and entity have a look at API Spec.
Even though we have only covered querying items with TypeDORM here, there are many more use-cases where TypDORM proves its worth. Experiment with different TypeDORM APIs and tell us what you think in the comments.
More about TypeDORM
TypeDORM is committed to delivering new features every month until all the major document client APIs are covered, meanwhile if you want to access the feature before the release we also publish a beta version.
To find out more about what new features are on the road map, check out the TypeDORM public roadmap.
If there are any other features you would like to be included, open a new issue here.
Thanks for reading! If you liked this article, hit that clap button below👏 . It means a lot to us and it helps others easily find this article. For more updates follow us on Medium or follow me on Github.
Special thanks to Zak Barbuto for proofreading this article.
Top comments (2)
don't forget to star ⭐️ the repo. It helps others to more easily find it.
Glad, that you liked it. 😃