Forem

David Israel for Uclusion

Posted on • Edited on

1 1

Everything You’ve Always Wanted to Know About DynamoDB But Didn’t Know to Ask

What are the benefits to DynamoDB early in a project?

  1. Instant setup and near instant ability to create new tables and add columns

  2. No charge unless break out of the generous free tier

  3. No need to pay attention to scaling — the relatively new pay per request billing mode handles everything

  4. DynamoDB streams are very powerful compared to a relational DB’s triggers

What are the costs of DynamoDB early in a project?

  1. Eventually consistent requires thinking about long term scaling problems immediately. You can use consistent reads but they are much slower and not available on global secondary indexes (GSI, an index other than the main key structure of your table).

So for instance say you do this query (written in PynamoDB):

comments = CommentModel.market_id_index.query(hash_key=market_id, 
attributes_to_get=[CommentModel.id,CommentModel.investible_id,                                                                                     
CommentModel.comment_type,CommentModel.created_by,                                                                                
CommentModel.updated_at,CommentModel.resolved,                                                                        
CommentModel.version,CommentModel.uploaded_files])
Enter fullscreen mode Exit fullscreen mode

That won’t get you all the comments — just all the comments currently in the market_id_index GSI. Eventually all comments will be in that GSI but not until DynamoDB has propagated the latest changes to all nodes.

  1. Using DynamoDB streams means you will early on tackle having asynchronous APIs. Plus streams are not actually FIFO but just FIFO per shard and shards are determined by your hash keys.

  2. Speaking of keys you will need to choose your keys carefully. You get a hash key and a range key that must combine to something unique and random enough to guarantee even distribution across shards. Unlike a relational database you also want to maximize data stored in a table — so long as your keys are useful you can have your columns be sparsely populated. Then the table will “heat up” and so be warm for retrieval. See adjacency graph patterns.

What are the benefits to DynamoDB as your usage scales?

  1. The simple SQL queries that were coded ignoring overall usage of the relational database and data growth of any one table will eventually require an entire operational team to simulate the out of the box behavior of DynamoDB.

  2. The push for eventually consistent and asynchronous processing now pays off as you don’t have APIs promising synchronous results that increasingly you cannot deliver.

  3. Upgrades are much easier as you don’t have to play tricks replicating data between database instances and unless DynamoDB transactions were used your tables are likely more independent. A SQL “join” is the opposite of a services based solution.

  4. Your transactional and reporting databases are probably already separate so you don’t need to painstakingly split them. Using NoSQL and DynamoDB streams you most likely copied the data over into an entirely different reporting or search database.

We will be covering how Uclusion coded for eventual consistency and asynchronous APIs in later blogs.

Why is MySQL still so much more popular than DynamoDB?

See for instance this discussion, I believe that MySQL remains the devil developers know. Pay per request, transactions, encryption at rest and point in time recovery for DynamoDB only came out in 2018. Like a lot of AWS the decision to use DynamoDB relies as much on where the technology is going as where it is now. In this sense there is really no such thing as avoiding database vendor lock in — your choice of database technology will determine a lot of your application’s current architecture and your roadmap and your database’s roadmap will be on the same map.

Image of Datadog

How to Diagram Your Cloud Architecture

Cloud architecture diagrams provide critical visibility into the resources in your environment and how they’re connected. In our latest eBook, AWS Solution Architects Jason Mimick and James Wenzel walk through best practices on how to build effective and professional diagrams.

Download the Free eBook

Top comments (0)

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay