Part of my AWS learning journey — transitioning from Systems Engineer to Cloud/DevOps. This session completes the database picture — Aurora's read/write architecture, ElastiCache caching strategies, and DynamoDB from table creation to production-ready query patterns.
📋 Topics Covered
| # | Topic | Type |
|---|---|---|
| 1 | Aurora Endpoints — Writer vs Reader | Concept + Interview |
| 2 | What Happens When the Aurora Writer Fails | Concept + Cert |
| 3 | ElastiCache Caching Patterns — Lazy Loading, Write Through, Session Store | Concept + Interview |
| 4 | Cache Invalidation | Concept + Interview |
| 5 | DynamoDB — What It Is and When to Use It | Concept + Interview |
| 6 | DynamoDB Table Creation — Keys and Settings | Concept + Lab |
| 7 | Table Classes — Standard vs Standard-IA | Concept + Cert |
| 8 | Capacity Modes — On-Demand vs Provisioned | Concept + Cert |
| 9 | Warm Throughput | Concept + Cert |
| 10 | DynamoDB Items & Attributes — CRUD Operations | Concept + Lab |
| 11 | Query vs Scan — The Critical Difference | Concept + Interview |
| 12 | Local Secondary Index (LSI) vs Global Secondary Index (GSI) | Concept + Cert |
| 13 | Bonus Concepts — Streams, DAX, Consistency, Transactions | Concept + Interview |
| 14 | Interview Questions | Interview |
| 15 | Practice Tasks | Practice |
Aurora Endpoints — Writer vs Reader
Aurora doesn't give you just one database endpoint — it gives you two, each serving a different purpose and routing to different parts of the cluster.
Writer Endpoint (Primary Endpoint): Always points to the current primary/writer instance. All write operations (INSERT, UPDATE, DELETE) go here. If a failover happens and a replica is promoted, Aurora automatically redirects this endpoint to the new writer — your application's configuration never needs to change.
Reader Endpoint: A load-balanced endpoint that distributes read-only queries (SELECT) across all available Aurora Replicas. You don't manage which replica serves each query — Aurora handles the routing, spreading read traffic evenly across however many replicas exist.
Why this architecture matters:
In a typical application, reads far outnumber writes — a product page might be read thousands of times per second but updated once a day. Without a reader endpoint, all reads and writes compete for the same primary instance. With Aurora's reader endpoint, reads are offloaded to replicas — the writer handles only writes, and read capacity scales horizontally by adding more replicas.
🎯 Interview Q1: Why do we need a Reader Endpoint?
To distribute read-only queries across multiple Aurora Replicas. This reduces load on the writer instance, improves response times for reads, and lets you scale read capacity horizontally by adding replicas — all without any change to the application connection string.
🎯 Interview Q2: What happens if the Aurora writer fails?
Aurora automatically promotes one of the existing replicas to become the new writer. The Writer Endpoint automatically redirects to the new primary. Because the endpoint itself doesn't change (only what it resolves to), the application needs zero configuration changes. Aurora's failover is typically faster than standard RDS Multi-AZ — often under 30 seconds.
Connection to last session: This is exactly the same principle as RDS using a DNS endpoint instead of a static IP — the endpoint is a stable address that AWS reroutes behind the scenes. Your application always connects to the same string, and AWS handles where that string points.
ElastiCache Caching Patterns
ElastiCache isn't just "put stuff in cache" — how you cache data matters significantly for consistency, performance, and staleness. There are three standard patterns, each designed for a different use case.
Pattern Comparison
| Pattern | How it works | Stale data? | Best for |
|---|---|---|---|
| Lazy Loading | Cache data only on the first read (cache miss triggers DB query + cache write) | Yes (possible) | Product catalog, blogs, non-critical reads |
| Write Through | Update cache and database together on every write | No (always in sync) | Banking, inventory, user profiles |
| Session Store | Store temporary session data in cache with a TTL | N/A (time-bound) | Shopping carts, login sessions, user state |
Lazy Loading (Cache-Aside)
Application checks cache → Cache Hit: return data immediately → Cache Miss: query the database, write result to cache, return data to caller.
The good: Only data that's actually requested gets cached — no wasted cache memory on data nobody reads. Very simple to implement.
The trade-off: On a cache miss, the call takes longer because it hits the database AND writes to the cache. And there's a window where the cache can serve stale data if the database was updated without invalidating the cache entry.
How stale data is handled in Lazy Loading:
When a database record is updated, the corresponding cache entry is explicitly deleted (invalidated). On the next read, a cache miss occurs, the latest data is fetched from the database, and the cache is refreshed with the fresh value. A TTL (Time To Live) is often set on cache entries as a backup — even if something isn't explicitly invalidated, the cache automatically expires it after the TTL elapses.
Write Through
Every database write also updates the cache at the same time — both happen together, never independently.
The good: The cache is always synchronized with the database. No stale data ever. When an application reads from cache, it always gets the current value.
The trade-off: Every write incurs extra latency because two writes happen (DB + cache) instead of one. Cache memory may be used for data that's written often but rarely read.
🎯 When to choose which: If consistency is critical (banking, inventory, anything financial) → Write Through. If reads dominate and occasional staleness is acceptable (product listings, blog content) → Lazy Loading. If you're storing temporary state that shouldn't live in the database at all → Session Store.
Session Store
Temporary session data (login tokens, shopping cart contents, user preferences) is stored in ElastiCache with a TTL. When the TTL expires, the session data is automatically deleted.
Why cache, not the database: Session data is read and written on every request, changes frequently, and has a natural expiry — it doesn't need the durability guarantees of a relational database. Storing it in a distributed cache means any application server can retrieve any user's session without sticky sessions.
Analogy: Session store is like a coat check at a restaurant. You hand over your coat (session data) when you arrive, get a token (session ID), and the coat is held temporarily. If you don't come back within closing time (TTL), the restaurant donates it. You never permanently owned the storage — it was always meant to be temporary.
DynamoDB — What It Is and When to Use It
Amazon DynamoDB is a fully managed, serverless, key-value and document NoSQL database. Unlike RDS, there are no servers to provision, no capacity planning for OS or engine, no connection limits to manage — you define a table, put data in it, and DynamoDB scales automatically.
When to choose DynamoDB over RDS:
| Choose RDS when | Choose DynamoDB when |
|---|---|
| Data has complex relationships, needs JOINs | Data access patterns are simple and predictable |
| Strong ACID transactions across many tables | Ultra-low latency at massive scale |
| Team is already SQL-native | Flexible or evolving schema (semi-structured data) |
| Reporting and analytics queries | Serverless, no schema migration headaches |
What DynamoDB is NOT for: Complex analytical queries, ad-hoc SQL reporting, or workloads where you don't know your access patterns ahead of time. DynamoDB is optimized for known, repetitive access patterns — "give me user 1234's orders" not "give me the average order value grouped by region for Q4."
DynamoDB Table Creation — Keys and Settings
The Primary Key — Two Flavors
Every item in a DynamoDB table must have a unique primary key. You define this at table creation and cannot change it later.
Option 1 — Partition Key only (Simple Primary Key)
The partition key alone uniquely identifies every item. DynamoDB uses this key to determine which physical partition stores the item (via internal hashing). No two items can share the same partition key value.
Good for: user profiles (user_id is unique per user), product catalog (product_id is unique per product).
Option 2 — Partition Key + Sort Key (Composite Primary Key)
Two attributes together form a unique identity. Items with the same partition key are grouped together and sorted by the sort key. This allows related items to live close together physically, making range queries efficient.
Good for: orders (partition key = user_id, sort key = order_timestamp → "give me all orders for user 1234, sorted by time"), messages (partition key = chat_id, sort key = message_timestamp → "give me all messages in this chat, newest first").
🎯 Interview tip: The partition key is the access gateway — your most common query pattern must be expressible using it. If you can't answer "what's my partition key?" you haven't finished designing your table.
Table Settings — What Each Option Does
Table Class:
| Class | Use case | Cost note |
|---|---|---|
| Standard | Frequently accessed data | Higher storage cost, optimized for read/write throughput |
| Standard-IA (Infrequent Access) | Data accessed rarely | Lower storage cost (~60% cheaper), higher per-read/write cost |
Capacity Mode — the most important config decision:
| On-Demand | Provisioned | |
|---|---|---|
| How it works | DynamoDB auto-scales instantly per request | You specify exact RCUs and WCUs upfront |
| Traffic pattern | Unpredictable, spiky, or new workloads | Predictable, steady, well-known workloads |
| Cost model | Pay per request (higher per-unit cost) | Pay for reserved capacity (lower cost if fully utilized) |
| Throttling risk | None — handles any traffic instantly | Yes — excess traffic is throttled |
| Typical use | Dev/test, early-stage products, variable traffic | Production workloads with known baselines |
RCU and WCU — what they measure:
One Read Capacity Unit (RCU) = one strongly consistent read per second for an item up to 4 KB. One Write Capacity Unit (WCU) = one write per second for an item up to 1 KB.
The Capacity Calculator in the console estimates how many RCUs and WCUs your workload needs — you input item size, read/write rate, and consistency requirement. Eventually consistent reads cost half a WCU each.
Warm Throughput
When DynamoDB allocates capacity to a new table, it starts conservatively. If a sudden burst of traffic hits immediately — say, a product launch — DynamoDB may throttle requests until it has time to scale up internally.
Warm Throughput pre-warms the table at creation time, telling DynamoDB to allocate higher baseline capacity from the start, so that sudden traffic spikes on day one don't cause throttling.
Think of it as pre-heating an oven before putting food in — you don't wait for it to warm up slowly after the food is already in there. You configure the starting temperature so it's ready when you need it.
DynamoDB Items & Attributes — CRUD Operations
Key Terms
Table: the container for your data (like a spreadsheet)
Item: a single record in the table (like a row)
Attribute: a field on an item (like a column) — but unlike SQL, different items in the same table can have different attributes (schema-flexible)
Example — two items in the same table with different attributes:
{ "user_id": "u-1234", "name": "Tejas", "email": "tejas@example.com", "city": "Nashik" }
{ "user_id": "u-5678", "name": "Rahul", "phone": "+91-9876543210" }
No schema migration needed — DynamoDB doesn't enforce that every item has the same attributes.
CRUD in the DynamoDB Console
Create: Use "Create item" → add attribute name and value → can switch between form view and JSON view → save.
Read: Use Query or Scan (see next section) to retrieve items, or open an item directly from the results list.
Update: Select an item → edit any attribute value → save. The partition key and sort key cannot be changed — they define the item's identity.
Delete: Select an item → delete. This permanently removes it.
The JSON view is particularly useful for seeing exactly how DynamoDB stores your data internally — every attribute has a type tag (S for string, N for number, BOOL for boolean, L for list, M for map).
Query vs Scan — The Critical Difference
This is one of the most important performance decisions in DynamoDB, and it comes up in almost every DynamoDB interview question.
| Query | Scan | |
|---|---|---|
| How it works | Retrieves items using the Partition Key (+ optional Sort Key) | Reads the entire table, then filters |
| Efficiency | Very efficient — reads only the relevant partition | Very inefficient — reads everything regardless |
| Cost | Low — charged only for data actually returned | High — charged for the entire table read |
| When to use | Production workloads — always prefer this | Occasional admin tasks, small tables, migrations |
🎯 The rule: In production, you should almost never Scan. If you find yourself needing to Scan frequently, it's a signal that your table's primary key design doesn't match your access patterns — and you need either a redesign or an additional index (LSI/GSI).
Concrete example:
You have an Orders table with partition key = user_id and sort key = order_date.
Query: "Give me all orders for user_id = 'u-1234' placed after 2026-01-01" → DynamoDB goes directly to that partition, reads only those items. Fast and cheap.
Scan: "Give me all orders where order total > 5000" → DynamoDB reads every single order in the table across all partitions, then filters. Slow and expensive — the filter happens after reading everything.
Filters with Query: You can add filter expressions to a Query result, but filters are applied after items are read from the partition — they reduce what you see, not what DynamoDB reads and charges for. Sort key conditions, however, genuinely reduce what is read.
Local Secondary Index (LSI) vs Global Secondary Index (GSI)
Sometimes your query patterns require filtering or sorting by an attribute that isn't your primary key. Indexes let you do this efficiently without falling back to Scan.
Analogy: Indexes in DynamoDB are like the index at the back of a book — instead of reading every page to find mentions of "VPC", you flip to the index and jump directly to the relevant pages. DynamoDB maintains this alternative lookup structure automatically.
Local Secondary Index (LSI)
Same Partition Key as the base table, but a different Sort Key. Allows you to sort or filter items within the same partition using a different attribute.
Example: Orders table has partition key = user_id, sort key = order_date. You also need to query orders sorted by order_total for the same user. Create an LSI with sort key = order_total.
Constraints:
- Must be created at table creation time — cannot add later
- Maximum 5 LSIs per table
- Shares provisioned throughput with the base table
- Only queries within a single partition (same partition key as base table)
Global Secondary Index (GSI)
A completely independent index with its own Partition Key and optional Sort Key — totally different from the base table's keys.
Example: Orders table has partition key = user_id. You want "all orders with status = 'PENDING' sorted by order_date." Create a GSI with partition key = status and sort key = order_date.
Constraints:
- Can be created at any time — much more flexible than LSI
- Maximum 20 GSIs per table
- Has its own separate provisioned throughput (additional cost)
- Queries can span all partitions of the base table (truly global)
LSI vs GSI Comparison
| LSI | GSI | |
|---|---|---|
| Partition Key | Same as base table | Different (you define it) |
| Sort Key | Different from base table | Optional, you define it |
| When created | At table creation only | Any time |
| Throughput | Shared with base table | Separate (extra cost) |
| Query scope | Single partition | Entire table |
| Max per table | 5 | 20 |
| Use when | Need different sort within same partition | Need to query by a completely different attribute |
🎯 Interview trap: "Can you add an LSI after creating a DynamoDB table?" → No. LSIs must be defined at creation. If you realize you need one later, you'd have to recreate the table and migrate data. GSIs can be added any time. Always think through your access patterns before creating a DynamoDB table.
Bonus Concepts — Worth Knowing
DynamoDB Streams
DynamoDB can stream a record of every change (INSERT, UPDATE, DELETE) to items in a table — useful for event-driven architectures.
Common pattern: DynamoDB item changes → Stream captures the event → Lambda is triggered → processes the change (send a notification, update a search index, replicate to another system, populate a cache).
DAX — DynamoDB Accelerator
DynamoDB's own in-memory caching layer, built specifically for DynamoDB. Reduces read latency from milliseconds to microseconds. Unlike ElastiCache (which is general-purpose), DAX is DynamoDB-native and requires no application code changes — it's a drop-in cache using the exact same DynamoDB API.
🎯 DAX vs ElastiCache for DynamoDB: If your application already uses DynamoDB and just needs faster reads, DAX is simpler (same API, no code changes). ElastiCache gives more flexibility if you need to cache data from multiple sources.
Eventually Consistent vs Strongly Consistent Reads
DynamoDB replicates data across multiple AZs. On a write, data propagates across replicas within roughly one second.
| Eventually Consistent (default) | Strongly Consistent | |
|---|---|---|
| Data freshness | Might be slightly stale | Always the most up-to-date |
| RCU cost | 0.5 RCU per 4 KB | 1 full RCU per 4 KB |
| Latency | Lower | Slightly higher |
| Use when | Most general reads | Must read your own write immediately |
When it matters: For most use cases (product catalog, session data), eventual consistency is fine. For financial transactions or reservation systems where you must immediately read your own write, use strongly consistent reads.
DynamoDB Transactions
DynamoDB supports ACID transactions via TransactGetItems and TransactWriteItems — you can atomically read or write multiple items across multiple tables, either all succeeding or all failing together. This addresses the common misconception that NoSQL databases can't handle transactions.
⚡ Quick Revision
Aurora Endpoints
- Writer Endpoint → always points to current primary → handles all writes
- Reader Endpoint → load-balances reads across all replicas
- On writer failure → replica promoted automatically → Writer Endpoint reroutes → zero app config changes
Caching Patterns
| Pattern | Stale data? | Use for |
|---|---|---|
| Lazy Loading | Yes (until invalidated or TTL expires) | Read-heavy, occasional stale ok |
| Write Through | No | Consistency-critical (banking, inventory) |
| Session Store | N/A (TTL-controlled) | Login sessions, shopping carts |
Cache invalidation in Lazy Loading: delete cache entry on DB update → next read repopulates.
DynamoDB Keys
- Partition Key only → must be unique per item
- Partition Key + Sort Key → partition groups related items, sort key orders them within a partition
Capacity Modes
- On-Demand: pay per request, no throttle, higher unit cost → unpredictable workloads
- Provisioned: fixed RCUs/WCUs, lower cost if utilized, throttles on excess → predictable production
Query vs Scan
- Query: uses Partition Key, reads only relevant data → fast, cheap → always prefer
- Scan: reads entire table → slow, expensive → admin tasks only
Indexes
- LSI: same partition key, different sort key, create at table creation only, max 5
- GSI: own partition key, create any time, max 20, separate throughput cost
Bonus
- DAX: DynamoDB-native microsecond cache, no code changes needed
- Streams: captures item-level changes for Lambda/event-driven patterns
- Strongly Consistent Read: 2× the RCU cost vs eventually consistent, guaranteed fresh data
- Transactions: ACID across multiple items/tables — DynamoDB does support transactions
💼 Interview Questions
Q1: What is the difference between the Writer Endpoint and the Reader Endpoint in Aurora?
The Writer Endpoint always points to the current primary instance and handles all write operations. The Reader Endpoint load-balances read queries across all Aurora Replicas. If the writer fails, Aurora promotes a replica and automatically redirects the Writer Endpoint to it — the application needs no configuration changes.
Q2: What are the three ElastiCache caching patterns and when would you use each?
Lazy Loading caches data only on a cache miss — good for read-heavy workloads where some staleness is acceptable. Write Through updates both the cache and database on every write — good when consistency is critical like banking or inventory. Session Store uses the cache with a TTL to hold temporary session data like login sessions and shopping carts — not persisted to a database at all.
Q3: How do you handle stale data in a Lazy Loading cache?
Explicitly delete the cache entry whenever the underlying database record is updated. The next read triggers a cache miss, fetches the latest data from the database, and refreshes the cache. A TTL on cache entries provides automatic backup expiry even if explicit invalidation is missed.
Q4: What is the difference between a Query and a Scan in DynamoDB?
A Query uses the Partition Key to efficiently retrieve only the relevant items — fast, cheap, and the right choice for production. A Scan reads the entire table across all partitions before filtering — slow, expensive, and charged for all data read regardless of what the filter returns. In production, Scans should almost never be used.
Q5: What is the difference between an LSI and a GSI?
An LSI uses the same Partition Key as the base table but a different Sort Key — it allows alternative sorting within a single partition and must be created at table creation time. A GSI has its own completely independent Partition Key (and optional Sort Key), can query across all partitions using a different attribute, and can be created at any time after the table exists. GSIs have separate throughput (additional cost); LSIs share the base table's throughput.
Q6: Can you add a Local Secondary Index after a DynamoDB table is created?
No. LSIs must be defined at table creation time. If you need one after the fact, you'd have to create a new table with the LSI defined and migrate the data. GSIs can be added at any time, making them far more flexible when requirements change.
Q7: When would you choose On-Demand capacity mode over Provisioned for DynamoDB?
On-Demand is best for unpredictable or spiky traffic, new products where read/write patterns aren't yet known, and development/testing environments. Provisioned capacity is better for steady, predictable production workloads where you know the baseline RCU/WCU requirements — it's cheaper per unit when fully utilized, but throttles if traffic exceeds the provisioned amount.
Q8: What is the difference between DAX and ElastiCache for DynamoDB caching?
DAX is a DynamoDB-native in-memory cache that uses the same API as DynamoDB and requires no application code changes, reducing latency from milliseconds to microseconds. ElastiCache is a general-purpose caching layer (Redis/Valkey/Memcached) that can cache data from DynamoDB or any other source, but requires application-level code to check the cache and handle misses. DAX is simpler for pure DynamoDB acceleration; ElastiCache offers more flexibility for multi-source caching.
🔬 Practice Tasks
Aurora Endpoints lab: Create an Aurora MySQL cluster with 2 replicas. Connect to the Writer Endpoint and insert some rows. Connect to the Reader Endpoint and SELECT the same rows — confirm it serves reads. Manually trigger a failover and verify the Writer Endpoint automatically redirects to the new primary within ~30 seconds.
Caching pattern implementation: Using Python and boto3, implement Lazy Loading against a DynamoDB table with an ElastiCache Redis cluster. Log every cache hit and miss. Observe the cache hit rate improve as the same keys are read repeatedly.
DynamoDB design exercise: You're building a messaging app. Users send messages to other users. Design the DynamoDB table: what's your partition key? Sort key? What LSI or GSI would you add to support "show all messages I received, sorted by timestamp"?
Query vs Scan cost experiment: Create a DynamoDB table with 1000 items. Run a Query for a specific partition key — note the consumed RCUs. Run a Scan with a filter returning the same single item — note the consumed RCUs. Compare. The Scan should show RCU consumption for all 1000 items, not just the one that matched.
GSI creation on existing table: Create a DynamoDB table with partition key =
user_id. Add items. After creation, add a GSI with partition key =city. Query by city and confirm it works without touching the base table's key structure.
AWS Session 12 — Aurora, ElastiCache Patterns & DynamoDB | Cloud + DevOps learning journey — Systems Engineer → Cloud/DevOps Engineer
Top comments (0)