DEV Community

Cover image for DynamoDB physical partitions
DynoTable
DynoTable

Posted on • Edited on • Originally published at dynotable.com

DynamoDB physical partitions

A physical partition is the unit DynamoDB actually stores your data on: a slice
of SSD, replicated across Availability Zones, holding one slice of your key space.
Your table is a logical thing. Partitions are where the bytes — and the throughput
limits — really live.

How do DynamoDB partitions work?

DynamoDB stores your table across physical partitions — SSD slices replicated across Availability Zones. Each one caps at ~10 GB, 3,000 read units/sec, and 1,000 write units/sec. The hash of your partition key decides which partition an item lands on, and DynamoDB splits partitions automatically as they grow or get hot.

  • Every partition caps at ~10 GB of storage, 3,000 read units/sec, and 1,000 write units/sec. Those ceilings are per partition, not per table.
  • The hash of your partition key picks the partition. Items with the same key land together; a single hot hot item — or a monotonic sort key — is what pins one partition.
  • DynamoDB splits partitions for you — on size, and on sustained heat — including splitting one key's item collection at a sort-key boundary, unless an LSI or an ever-increasing sort key blocks it.
  • Throttling with capacity to spare is the tell. A ProvisionedThroughputExceeded error while your table sits at 5% usage means a single partition is maxed out.

How an item finds its partition

DynamoDB feeds your partition-key value through an internal hash function. The hash
output picks the physical partition. Same key in, same partition out — every time.

Coming from SQL, there's no analog. There's no index B-tree you tune, no shard key
you assign by hand. The placement is a hash you don't control and never see.

Items sharing a partition key form an item collection, stored together and
sorted by sort key. That's what makes a Query on one key cheap — it reads one
contiguous run on one partition. (See Query vs Scan.)

Take a match-event store for a game. The table keys are arenaId (partition) and
eventKey (sort):

Example Notes
# Item
arenaId = "ARENA#7f3a"
eventKey = "EVT#1719100800#a91c"
playerTag = "Nightjar"
dmgDealt = 412

Every event for arena 7f3a hashes to the same partition and stacks in sort-key
order. Great for "read this match's timeline." A liability if that one arena gets
all the traffic.

The three ceilings every partition enforces

A single partition is designed to deliver at most:

Limit Per partition Counted as
Storage ~10 GB raw item bytes
Read capacity 3,000 read units/sec 1 RU = one 4 KB strongly-consistent read
Write capacity 1,000 write units/sec 1 WU = one 1 KB write

Source: the AWS Best practices for designing partition keys guide.

Item size scales the math. A 20 KB item costs 5 read units per strongly-consistent
read, so one partition serves ~600 such reads/sec before it throttles — not 3,000.
Round write cost up per 1 KB, read cost up per 4 KB.

The trap: these are partition limits, not table limits. Your table can be
provisioned for 40,000 WCU and still throttle, because all the writes are pounding
one partition that tops out at 1,000.

How partitions split

DynamoDB adds partitions automatically in two cases. You never run a command.

Split on size. When a partition fills toward ~10 GB, DynamoDB splits its key
range in two and moves half the items to a new partition. Storage grows
transparently; your reads and writes keep working throughout.

Split for heat. When a partition takes sustained traffic near its throughput
ceiling, DynamoDB splits the hot key range so each half lands on its own partition.
AWS calls this the split-for-heat mechanism. Short throttling bursts that stop
on their own often mean split-for-heat kicked in — though brief spikes can also just
be burst capacity running dry.

DynamoDB physical partitions

Splitting buys room across many keys, and split-for-heat can even carve one key's
item collection at a sort-key cut. What it can't spread is a single hot item, an
ever-increasing sort key, or a collection pinned by an LSI.

Why a hot key beats the splitter

Here's the footgun. Splitting redistributes ranges of partition keys. If your
traffic concentrates on one key value, every request hashes to the same partition,
and there's no range left to divide.

If arena 7f3a is a tournament final pulling 4,000 writes/sec while every other
arena is idle, you'll throttle at 1,000 — and split-for-heat can't rescue it here,
because the timestamp-prefixed eventKey is monotonic, so every new write lands at
the leading edge of one narrow sort-key range with nothing to carve off. The newer
KeyRangeThroughputExceeded throttle reason names exactly this: one partition's key
range, not the table, is over its limit.

The fix is in the data model, not the capacity slider. Write-shard the hot key:
append a small suffix so one logical arena spreads across N physical partitions.

arenaId = "ARENA#7f3a#3"   # shard 0..9, chosen per write
Enter fullscreen mode Exit fullscreen mode

Reads then fan out across the shards and merge client-side. You can prototype the
key shapes and the Query for each shard with the
DynamoDB Expression Builder before you touch
a line of application code.

One nuance: the LSI exception

There's one case where storage is capped per partition key. Without a Local Secondary Index, an item collection splits across as many partitions as it
needs to serve both its stored bytes and its throughput — billions of sort-key values are fine.

Add an LSI, and the whole collection for one partition key must fit in a single
10 GB partition, because the LSI shares it. That's the per-PK cliff covered in
GSI vs LSI — another reason most teams reach for GSIs.

Designing so partitions stay cool

The lever you actually control is the partition key. Pick one with many distinct
values relative to row count, so traffic spreads evenly. (More patterns in
single-table design.)

  • High-cardinality key. A per-user or per-tenant key beats a per-day or per-status key that everyone hammers at once.
  • Watch for known hot keys. A "current tournament" or "today" value is a concentration risk before you ship, not after.
  • Shard the unavoidable hot key. When one key must take outsized traffic, a suffix is the standard escape hatch.

Throttling with capacity to spare is your signal that one partition is hot. Inspect
the skewed item collection and rehearse a sharded key layout in
DynoTable — point it at your own table, GROUP BY the partition key in
the SQL Workbench to see which keys dominate, and model the fix before it pages you.

Top comments (0)