DEV Community: Iwan Setiawan

Bare Metal vs. AWS RDS: A Deep Dive into NUMA-Aware Tuning and PostgreSQL Performance (Part 2)

Iwan Setiawan — Mon, 16 Mar 2026 06:52:44 +0000

Bare Metal vs. AWS RDS: CPU/NUMA Pinning and HugePages — How We Beat Aurora on Write Throughput (Part 2)

In Part 1, we established storage baselines — Local SSD vs Longhorn vs AWS managed PostgreSQL. This article goes deeper: CPU/NUMA pinning and HugePages push bare metal write performance past Aurora IO-Optimized at every concurrency level.

In Part 1, we ended with CNPG Local SSD — bare metal with direct-attached storage and AWS-matched PostgreSQL config. Already leading Aurora on write TPS at baseline. The question was: how much further can we push it without adding hardware?

Two steps. Significant results.

Setup Recap

Same constraint as Part 1: 2 vCPU / 8 GB RAM, single instance, no HA. Same PostgreSQL config matched to AWS defaults. Same benchmark: pgbench · scale factor 100 · 60s per run · 39 runs · ap-southeast-3.

Where we left off — CNPG Local SSD (Baseline):

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	749	1.34 ms	134	7.48 ms	99	10.10 ms
10	7,675	1.30 ms	1,425	7.02 ms	1,031	9.70 ms
25	6,788	3.68 ms	1,560	16.02 ms	1,073	23.30 ms
50	6,430	7.78 ms	1,550	32.27 ms	996	50.18 ms
100	6,092	16.41 ms	1,464	68.32 ms	902	110.92 ms

The 3-Layer Tuning Stack

Most PostgreSQL performance articles stop at database config. This one goes deeper.

The performance gains in this article come from tuning at three layers simultaneously — bare metal KVM hypervisor, VM/OS, and Kubernetes pod spec. Each layer is required for the next to work correctly.

Layer 1: KVM Hypervisor (Bare Metal Host)

<domain type='kvm'>
  <!-- NUMA: all VM memory from NUMA node 1 only -->
  <numatune>
    <memory mode='strict' nodeset='1'/>
  </numatune>

  <!-- CPU pinning: each vCPU mapped to specific physical core -->
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <!-- cores 8-13, 28-29 — all on NUMA node 1 -->
  </cputune>

  <!-- HugePages: VM uses host HugePages, memory locked (no swap) -->
  <memoryBacking>
    <hugepages/>
    <locked/>
  </memoryBacking>

  <!-- host-passthrough: CPU features exposed directly to VM -->
  <cpu mode='host-passthrough' check='none'/>

  <!-- Disable memory ballooning: hypervisor cannot steal VM memory -->
  <memballoon model='none'/>
</domain>

What this achieves:

mode='strict' nodeset='1' — zero cross-NUMA memory access. PostgreSQL shared buffer pool and its pinned CPU cores are on the same NUMA node. This is the primary driver of the 7.48ms → 1.81ms write latency drop at 1 client.
<locked/> — VM memory is non-swappable. Shared buffer pool stays in RAM permanently.
host-passthrough — VM inherits host CPU instruction set, hardware prefetcher, and cache optimization directly.
memballoon model='none' — hypervisor cannot reclaim memory from this VM for other VMs. Fixed allocation.

Layer 2: VM / OS Level

# /etc/sysctl.conf on the VM
echo "vm.nr_hugepages = 8192" >> /etc/sysctl.conf
sysctl -p

# CPU governor
cpupower frequency-set -g performance

HugePages must be pre-allocated at OS boot before PostgreSQL starts — they cannot be allocated on-demand. 8192 × 2MB = 16GB pre-allocated, enough to cover the 8Gi hugepages requested by the pod with headroom. The performance governor eliminates clock speed throttling for bursty query patterns.

Layer 3: Kubernetes Pod Spec

resources:
  limits:
    cpu: '2'
    hugepages-2Mi: 8Gi    # request HugePages from OS pool
    memory: 4Gi            # regular memory (non-huge) — separate accounting
  requests:
    cpu: '2'               # requests = limits = Guaranteed QoS class
    hugepages-2Mi: 8Gi
    memory: 4Gi

What this achieves:

requests = limits → Guaranteed QoS class — Kubernetes will not evict this pod under memory pressure. Other pods die first.
hugepages-2Mi: 8Gi as a separate resource → HugePages are tracked independently from regular memory. The 6 GB shared_buffers fits within the 8Gi hugepages allocation with headroom.
cpu: requests = limits → enables CPU Manager static policy — Kubernetes pins the pod to exclusive physical cores, which is what enables NUMA affinity at Layer 1 to be effective.

Why All Three Layers Matter

Remove any one layer and performance degrades:

Remove	Impact
KVM NUMA pinning	Cross-NUMA memory access → +3-4ms write latency per hop
`<locked/>`	Memory swappable → latency spikes under memory pressure
CPU governor	Clock throttling → latency spikes on short transactions
Kubernetes Guaranteed QoS	Pod can be evicted or CPU throttled under node pressure
HugePages	TLB pressure → higher latency at high concurrency

This is why the benchmark results are reproducible but not trivially so — you need all three layers configured correctly.

PostgreSQL was allocated 2 vCPU with no CPU affinity — running on whatever cores the kernel scheduled, potentially crossing NUMA boundaries on every memory access, with clock speed throttled by the default powersave governor.

Three changes applied simultaneously:

1. Dedicated CPU cores (Kubernetes CPU Manager: static policy)
Pins the PostgreSQL pod to specific physical cores. Eliminates context switching with other workloads.

2. CPU governor: powersave → performance

cpupower frequency-set -g performance

Default governor throttles clock speed at low load. Every short transaction pays a ramp-up penalty.

3. NUMA pinning
PostgreSQL process pinned to cores on the same NUMA node as its memory allocation. Cross-NUMA memory access adds 30–40% latency on NUMA-enabled systems — our 32-core host is NUMA-aware.

Tuning 1 Results

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	2,480	0.40 ms	552	1.81 ms	380	2.63 ms
10	8,066	1.24 ms	1,909	5.24 ms	1,265	7.91 ms
25	7,770	3.22 ms	1,902	13.14 ms	1,233	20.27 ms
50	7,384	6.77 ms	1,786	27.99 ms	1,173	42.62 ms
100	6,939	14.41 ms	1,657	60.36 ms	1,065	93.87 ms

vs Baseline:

Metric	Baseline	Tuning 1	Delta
Avg RO TPS	6,111	6,896	+12.8%
Avg RW TPS	1,355	1,659	+22.4%
Avg RW Lat	30.02 ms	25.44 ms	-15.3%
RW Lat (1c)	7.48 ms	1.81 ms	-75.8%

The single-client write latency drop from 7.48ms to 1.81ms is the most dramatic — this is the NUMA penalty being eliminated. Short transactions no longer wait for cross-NUMA memory access.

Tuning 2: HugePages

HugePages reduce TLB (Translation Lookaside Buffer) pressure by mapping PostgreSQL's shared buffer pool with 2 MB pages instead of the default 4 KB. Fewer TLB entries = fewer TLB misses under concurrent access.

Enabled at three levels:

# 1. VM OS — pre-allocate HugePages at boot
echo "vm.nr_hugepages = 8192" >> /etc/sysctl.conf
sysctl -p

# 2. Pod Resources — request HugePages as dedicated Kubernetes resource
resources:
  limits:
    cpu: '2'
    hugepages-2Mi: 8Gi     # request HugePages from OS pool
    memory: 4Gi            # regular memory (non-huge) — separate accounting
  requests:
    cpu: '2'               # requests = limits = Guaranteed QoS class
    hugepages-2Mi: 8Gi
    memory: 4Gi

# 3. PostgreSQL
huge_pages = on

Why requests = limits? This gives the pod Guaranteed QoS class — Kubernetes will not evict or throttle it under resource pressure. It also enables CPU Manager static policy to pin exclusive physical cores to this pod, which is what makes NUMA affinity effective.

Tuning 2 Results

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	2,558	0.39 ms	562	1.78 ms	386	2.59 ms
10	8,325	1.20 ms	1,903	5.25 ms	1,276	7.84 ms
25	8,205	3.05 ms	1,954	12.79 ms	1,254	19.94 ms
50	7,892	6.34 ms	1,875	26.67 ms	1,215	41.16 ms
100	7,485	13.36 ms	1,725	57.97 ms	1,111	90.01 ms

vs Tuning 1:

Metric	Tuning 1	Tuning 2	Delta
Avg RO TPS	6,896	7,232	+4.9%
Avg RW TPS	1,659	1,706	+2.8%
Avg RW Lat	25.44 ms	24.44 ms	-3.9%

Incremental improvement — HugePages reduce TLB contention at high concurrency. The impact is smaller than NUMA pinning but consistent across all workload types.

Full Tuning Journey

Step	Key Change	Avg RO TPS	Avg RW TPS	Avg RW Lat	Overall Avg
Baseline (Local SSD)	AWS-matched config	6,111	1,355	30.02 ms	2,796
Tuning 1	CPU/NUMA + perf governor	6,896	1,659	25.44 ms	3,214
Tuning 2	HugePages	7,232	1,706	24.44 ms	3,351

Write latency progression (1 client):

Baseline  7.48 ms  ████████████████████████████████████████
Tuning 1  1.81 ms  ████████
Tuning 2  1.78 ms  ████████

75% write latency reduction from Baseline → Tuning 2. Same hardware, same config.

Final Comparison: CNPG Tuning 2 vs AWS

Average RW TPS — All Environments

Environment	Avg RO TPS	Avg RW TPS	Avg RW Lat	Overall Avg
AWS RDS Standard	10,724	2,250	17.30 ms	4,826
AWS Aurora IO-Prov	8,370	1,234	29.72 ms	3,480
CNPG Tuning 2 (Final)	7,232	1,706	24.44 ms	3,351
AWS Aurora Standard	8,039	1,162	31.45 ms	3,326
CNPG Tuning 1	6,896	1,659	25.44 ms	3,214
CNPG Local SSD (Baseline)	6,111	1,355	30.02 ms	2,796

CNPG Tuning 2 Overall Avg (3,351) nearly matches Aurora IO-Optimized (3,480) — just -3.7% difference.
On write TPS: CNPG Tuning 2 (1,706) beats Aurora IO-Optimized (1,234) by +38%.
On write latency: CNPG Tuning 2 (24.44ms) beats Aurora IO-Optimized (29.72ms) by -17.7%.

The honest picture: With 2 vCPU and mid-range SAS SSD, CNPG Tuning 2 matches Aurora IO-Optimized on overall throughput (-3.7%) while beating it by 38% on write TPS. Aurora leads on reads (~14% higher Avg RO TPS) — this reflects its distributed read cache architecture, not a config gap. We verified by pushing PostgreSQL to its limit (shared_buffers=6GB, random_page_cost=1.1, effective_io_concurrency=200) and the read ceiling held. For write-intensive OLTP, bare metal wins. For read-heavy analytical workloads, Aurora's distributed cache is worth paying for.

⚠️ The RDS Standard Caveat: Burstable CPU

RDS Standard (t3.large) leads the benchmark at 4,826 overall avg — but this number requires an important caveat.

t3 instances use a CPU credit system:

Baseline CPU utilization: 30% (for t3.large)
Above baseline = consuming burst credits
When credits are exhausted: performance drops to the 30% baseline

Each benchmark run is 60 seconds, with the full test suite taking ~50 minutes total — within the burst window for t3.large. Our results therefore reflect peak burst performance, which is valid for this benchmark duration.

However, in a production workload running continuously 24/7, RDS Standard t3.large performance will drop once CPU credits are depleted:

t3.large burst performance:    ~4,826 avg TPS  ← what our ~50 min benchmark measured
t3.large baseline CPU:         30% of full capacity
t3.large sustained (24/7):     significantly lower once credits exhaust

If you need sustained, predictable performance on AWS, consider:

Option	vCPU	RAM	Key Difference
RDS t3.large	2	8 GB	Burstable — our benchmark used this
RDS m6i.large	2	8 GB	Non-burstable, dedicated CPU, consistent performance
RDS m7g.large	2	8 GB	Graviton3, non-burstable, better price/performance
Aurora Serverless v2	2 ACU	—	Auto-scales, consistent, higher baseline cost

For a truly fair comparison against bare metal with consistent, non-burstable performance, RDS m6i.large or m7g.large would be the appropriate AWS counterpart — not t3.large.

Bottom line: Our benchmark results for RDS Standard are valid — the ~50 minute test ran within the burst window. But if your production workload runs continuously 24/7, RDS t3.large will eventually underperform these numbers once CPU credits exhaust. CNPG Tuning 2's 3,351 overall avg is consistent regardless of duration — no burst credits, no performance cliffs.

Per-Client Write TPS Breakdown

Clients	Aurora IO-Prov	CNPG Tuning 1	Aurora Standard	RDS Standard
1	285	562 🥇	191	253
10	984	1,903 🥇	922	1,881
25	1,278	1,954 🥇	1,179	2,839 🥇
50	1,472	1,875 🥇	1,384	2,620
100	1,623	1,725 🥇	1,557	2,585

CNPG Tuning 2 beats Aurora IO-Optimized on RW TPS at every concurrency level. RDS Standard leads at 25–100 clients due to t3 burst credits.

Per-Client Write Latency

Clients	Aurora IO-Prov	CNPG Tuning 1	Aurora Standard
1	3.51 ms	1.78 ms 🥇	5.23 ms
10	10.16 ms	5.25 ms 🥇	10.85 ms
25	19.57 ms	12.79 ms 🥇	21.20 ms
50	33.96 ms	26.67 ms 🥇	36.13 ms
100	61.63 ms	57.97 ms 🥇	64.22 ms

Write latency: bare metal wins at every concurrency level vs Aurora.

Why Aurora Loses on Write Latency

Aurora replicates every write to its distributed storage fleet before acknowledging:

PostgreSQL → WAL → Aurora storage network → 2/3 replicas ack → done

On bare metal with NUMA-pinned CPUs and local SSD:

PostgreSQL → WAL buffer (HugePages) → local SSD → done

At 1 client: Aurora write path = 3.51ms, bare metal = 1.78ms — nearly 2× faster. At 100 clients, the gap narrows as both are network/IO bound, but bare metal still leads.

Platform Selection Guide

Workload	Best Choice	Reason
Write-intensive OLTP	CNPG Tuning 1	Best write TPS and latency vs Aurora
Read-heavy (API, reporting)	Aurora IO-Opt	47% higher Avg RO TPS vs bare metal tuned
Burst/unpredictable load	RDS Standard	t3 burst credits handle spikes
Cost-sensitive, stable load	CNPG Tuning 1	Aurora-level write perf at fraction of cost
Managed simplicity	Aurora Standard	Competitive overall, no ops overhead

Key Takeaways

NUMA pinning is the biggest single tuning lever. Tuning 1 (CPU/NUMA + performance governor) delivered +22% Avg RW TPS and -76% write latency at 1 client — more impact than any PostgreSQL config change.
HugePages: consistent but incremental. Tuning 2 added +2.8% Avg RW TPS on top of Tuning 1. Worth enabling for latency stability at high concurrency.
Bare metal beats Aurora IO-Optimized on writes — with mid-range SAS SSD. +38% Avg RW TPS and -18% write latency. Not NVMe. Not enterprise flash. Samsung SM863a SAS in RAID 1.
Aurora's read advantage is architectural, not a config gap. We maxed PostgreSQL config (6 GB shared_buffers, random_page_cost=1.1, effective_io_concurrency=200, maintenance_work_mem=512MB) and reached 86% of Aurora IO-Prov read throughput. The remaining 14% gap comes from Aurora's distributed read cache — a genuine architectural advantage for read-heavy workloads, not something tunable away.
"2 vCPU" is not equal. Same allocation on NUMA-aware 32-core bare metal with pinned cores outperforms hypervisor-backed t3.large on write-sensitive workloads.
RDS Standard t3.large benchmark results are valid — but context matters. Our ~50 minute benchmark ran within the burst window, so results accurately reflect t3 burst performance. However, in 24/7 production workloads, performance will drop once CPU credits exhaust (baseline CPU = 30%). For sustained production comparison, m6i.large or m7g.large (non-burstable) is more appropriate than t3.
Choose based on workload, not hype. Write-intensive OLTP → bare metal wins on both performance and cost. Read-heavy analytical → Aurora's distributed cache is worth paying for.

Environment Details

CloudNativePG: v1.24 on Kubernetes 1.31
Host: Bare Metal 32-Core (16 Physical / 16 HT), NUMA-Aware, 32 GB RAM
Storage: Samsung SM863a Enterprise SSD RAID 1 (SAS Interface) — mid-range enterprise SSD
PostgreSQL config (Tuning 2): shared_buffers=6GB, huge_pages=on, work_mem=4MB, max_connections=200, wal_buffers=64MB, random_page_cost=1.1, effective_io_concurrency=200, maintenance_work_mem=512MB, checkpoint_completion_target=0.9
Deployment: Single instance — no HA, no read replicas, no connection pooling
AWS Region: ap-southeast-3 (Indonesia)
Instance class: t3.large (2 vCPU, 8 GB) for all AWS environments
Scale Factor: 100 (~10M rows, ~1.5 GB table)
Benchmark runner: Kubernetes-native pgbench Job — source on GitHub

← Part 1: Storage Baseline — Longhorn vs Local SSD vs Managed Cloud

— Iwan Setiawan, Hybrid Cloud & Platform Architect · portfolio.kangservice.cloud

Redis + AOF + Distributed Storage: A Cautionary Benchmark

Iwan Setiawan — Sat, 07 Mar 2026 23:13:17 +0000

We put AOF persistence through 9 configurations across local SSD SAS and Longhorn. The results are definitive.

When designing a caching layer for a production migration to bare metal Kubernetes, we faced a question that sounds simple but turned out to have an expensive answer: should Redis AOF persistence live on Longhorn distributed storage?

The Redis documentation hints at the answer. But intuition and documentation are not the same as production data. So we ran redis-benchmark across nine configurations — varying storage backend, persistence settings, and dataset size — and measured the impact empirically.

The results are unambiguous, and one number in particular should give any architect pause.

Test Configuration

All tests used the same parameters throughout:

requests:    50,000
clients:     20 parallel
payload:     180,000 bytes (~180 KB)
pipeline:    keep-alive=1
thread:      single-threaded

The 180 KB payload is intentional — it reflects realistic cache object sizes for the production workload being benchmarked, not the micro-payload tests commonly seen in vendor benchmarks.

Nine environments tested:

Label	Storage	AOF	RDB	Dataset
Local · AOF off	Local SSD SAS	No	Thresholds	Empty
Local · AOF on (baseline)	Local SSD SAS	Yes	Thresholds	Empty
Local · AOF on (tuning 1)	Local SSD SAS	Yes	Thresholds	Empty
Local · AOF on (tuning 2)	Local SSD SAS	Yes	Thresholds	Empty
Local · AOF on (t2 + data)	Local SSD SAS	Yes	Thresholds	375,795 keys
Longhorn · AOF on (empty)	Longhorn	Yes	Thresholds	Empty
Longhorn · AOF on (data)	Longhorn	Yes	Thresholds	375,795 keys

SET Throughput: The Core Finding

The most important metric for a write-capable cache is SET throughput under load. Here are the results:

Configuration	SET RPS	SET avg latency	SET p99 latency
Local · AOF off	7,696	1.47 ms	5.12 ms
Local · AOF on (baseline)	1,275	14.39 ms	102.53 ms
Local · AOF on (tuning 1)	1,251	15.03 ms	105.92 ms
Local · AOF on (tuning 2)	1,248	15.03 ms	112.38 ms
Local · AOF on (t2 + 375K keys)	1,212	15.85 ms	121.15 ms
Longhorn · AOF on (empty)	577	33.56 ms	225.66 ms
Longhorn · AOF on (375K keys)	537	36.17 ms	201.86 ms

Let that sink in. Local SSD SAS with AOF disabled: 7,696 SET RPS, p99 = 5 ms. Longhorn with AOF enabled: 537 SET RPS, p99 = 202 ms.

That is a 14.3x throughput difference and a 39x p99 latency difference — on the same application code, same Redis version, same client.

The worst-case single SET operation on Longhorn reached 903 ms. For a cache layer.

The AOF Wall on Local Storage

Before we get to Longhorn, it's worth understanding what AOF persistence costs even on fast local SSD SAS.

Disabling AOF (keeping only RDB snapshot thresholds) delivers:

SET p99: 3.8–5.1 ms
Average SET latency: ~1.5 ms

Enabling AOF on the same local storage:

SET p99: 102–121 ms
Average SET latency: 14–16 ms

That's roughly a 20x p99 latency penalty just from AOF on local SSD SAS. And critically — tuning doesn't help. Across three tuning iterations (different appendfsync settings, no-appendfsync-on-rewrite toggles, and RDB threshold adjustments), the p99 numbers barely moved:

Baseline:  102.5 ms p99
Tuning 1:  105.9 ms p99
Tuning 2:  112.4 ms p99  ← actually got worse

The reason is fundamental: AOF with appendfsync everysec must call fsync() at least once per second. On an otherwise busy single-threaded Redis instance processing 180 KB payloads, this fsync stall dominates. You cannot tune your way past it.

Why Longhorn Makes AOF Catastrophic

Longhorn is a distributed block storage system for Kubernetes. It replicates data across nodes for durability. This is excellent for stateful workloads like databases with controlled write patterns.

Redis AOF is not that.

AOF appends to a log file on every write operation (or at least every second with everysec). The write pattern is continuous, small, and latency-sensitive. When this write pattern hits Longhorn:

Each AOF append crosses the network to the Longhorn controller
The controller replicates to N replicas before acknowledging
Only then does Redis get its fsync confirmation
Redis is single-threaded — it waits

The result: every SET operation pays the cost of network round-trip + multi-replica write confirmation. At 180 KB payload size, this stacks badly.

Redis's own documentation says:

"Avoid storing AOF/RDB files on storage that has network latency in the I/O path, such as NFS mounts."

Longhorn is effectively that — a network-replicated volume. The documentation warning is correct. Our benchmark puts a number on it: 903 ms max latency, 202 ms p99.

GET Performance

One important nuance: GET performance is much less affected by persistence settings.

Configuration	GET RPS	GET avg latency
Local · AOF off	8,027	1.47 ms
Local · AOF on (baseline)	2,537	4.29 ms
Longhorn · AOF on (375K keys)	2,522	4.21 ms

Longhorn doesn't significantly degrade GET performance compared to AOF-on local storage. This makes sense — reads don't write to the AOF log. The Longhorn penalty only appears when Redis needs to persist.

PING Latency: The Baseline

PING throughput gives a sense of the overhead without persistence in the picture:

Configuration	PING RPS	PING avg latency
Local · AOF off	~37,000	0.32 ms
Local · AOF on (baseline)	~11,000–18,000	0.84–1.50 ms
Longhorn · AOF on	~19,000–21,000	0.74–0.83 ms

Interestingly, PING performance on Longhorn is better than AOF-on-local at baseline. The Longhorn overhead only materializes when Redis actually needs to write to the AOF log — confirming that the bottleneck is specifically the persistence write path, not general Longhorn I/O overhead.

The Recommended Architecture

Based on these results, the right architecture for this workload is a split-persistence design:

Hot path (primary):

Redis with AOF disabled
RDB snapshots only, with generous thresholds (e.g., save 3600 1)
Local-path storage on SSD SAS
Result: 7,600+ SET RPS, sub-5 ms p99

Recovery path (replica):

Redis replica of the primary
RDB-only snapshots to persistent storage (Longhorn acceptable here — snapshot writes are infrequent and bursty)
Not in the hot write path

This gives you sub-5 ms p99 at full throughput on the write path, while maintaining durability guarantees through the replica's periodic snapshots. If the primary fails, you lose at most one RDB snapshot interval of data — which for most cache workloads is acceptable.

If true durability for every write is required (it rarely is for a cache), the right answer is a different tool — not Redis with AOF on distributed storage.

Summary

Question	Answer
Can AOF on local SSD SAS achieve good SET latency?	No. p99 stays above 100 ms regardless of tuning.
Can AOF on Longhorn achieve acceptable SET latency?	No. p99 reaches 202 ms, max 903 ms.
Does Longhorn affect GET performance with AOF?	Minimally — GETs don't write to AOF.
What's the right architecture for high-throughput caching?	AOF disabled on hot path, RDB replica for recovery.
Is the Redis documentation warning about network storage accurate?	Definitively yes. Our data confirms it.

The 14x throughput gap between AOF-on-Longhorn and AOF-off-local is not a configuration problem. It is an architectural mismatch. Building a fast cache on slow persistence is a contradiction — and these numbers prove it.

Environment Details

Redis version: 7.x
Storage backends: Local-path provisioner (SSD SAS) and Longhorn 1.6 on Kubernetes 1.31
redis-benchmark parameters: -n 50000 -c 20 -d 180000 --keepalive 1
Single-threaded mode throughout (no --threads flag)
Dataset: Empty at baseline; 375,795 keys for loaded tests

Questions about Redis architecture on Kubernetes? Leave a comment below.

— Iwan Setiawan, Hybrid Cloud & Platform Architect · portfolio.kangservice.cloud

Bare Metal vs. AWS RDS: A Deep Dive into NUMA-Aware Tuning and PostgreSQL Performance (Part 1)

Iwan Setiawan — Sat, 07 Mar 2026 23:00:09 +0000

Bare Metal vs. AWS RDS: Storage Baseline — Longhorn vs Local SSD vs Managed Cloud

Before tuning anything, we needed to answer a simpler question first: does storage backend matter more than the platform itself?

This is Part 1 of a 2-part series. This article establishes bare metal storage baselines across four configurations and compares them against AWS managed PostgreSQL. Part 2 covers CPU/NUMA pinning and HugePages, where bare metal overtakes Aurora on write throughput.

Most PostgreSQL performance comparisons jump straight to config tuning. We didn't.

Before touching CPU governors or HugePages, we needed to answer a more fundamental question: how much does storage backend affect performance on bare metal Kubernetes? We ran four configurations — and the results reveal exactly where the bottleneck lives.

The Setup

All environments: 2 vCPU / 8 GB RAM throughout. Our bare metal node is a 32-core NUMA-aware host with Samsung SM863a Enterprise SSD in RAID 1 (SAS). AWS environments run on t3.large in ap-southeast-3.

Single-instance comparison throughout — one CNPG pod vs one RDS instance vs one Aurora instance. No read replicas, no Multi-AZ, no connection pooling.

PostgreSQL config — intentionally matched to AWS defaults:

shared_buffers       = ~1.9 GB
effective_cache_size = ~3.8 GB
work_mem             = 4 MB
max_connections      = 839
wal_buffers          = ~60 MB
maintenance_work_mem = 128 MB

By using the same config across all environments, performance differences come purely from platform and storage architecture — not tuning.

Benchmark: pgbench · Scale factor 100 (~10M rows) · 60s per run · 39 runs per environment

Four bare metal storage configurations tested:

Label	Storage	Replicas	Disk
CNPG Local SSD	Direct-attached Samsung SM863a SAS	—	Dedicated
CNPG Longhorn 1R	Longhorn distributed storage	1 replica	Dedicated
CNPG Longhorn 2R	Longhorn distributed storage	2 replicas	Dedicated
CNPG Longhorn 2R+shared	Longhorn distributed storage	2 replicas	Shared with OS/worker

Results: AWS RDS Standard (t3.large)

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	1,677	0.60 ms	253	3.95 ms	178	5.63 ms
10	13,955	0.72 ms	1,881	5.32 ms	1,460	6.85 ms
25	12,859	1.94 ms	2,839	8.80 ms	1,864	13.41 ms
50	10,397	4.81 ms	2,620	19.09 ms	1,646	30.37 ms
100	10,627	9.41 ms	2,585	38.68 ms	1,623	61.61 ms

Results: AWS Aurora IO-Optimized (t3.large)

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	2,607	0.38 ms	285	3.51 ms	218	4.58 ms
10	10,928	0.92 ms	984	10.16 ms	739	13.53 ms
25	9,265	2.70 ms	1,278	19.57 ms	880	28.42 ms
50	8,163	6.12 ms	1,472	33.96 ms	990	50.49 ms
100	7,783	12.85 ms	1,623	61.63 ms	1,027	97.41 ms

Results: AWS Aurora Standard (t3.large)

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	1,540	0.65 ms	191	5.23 ms	150	6.66 ms
10	10,020	1.00 ms	922	10.85 ms	690	14.48 ms
25	9,189	2.72 ms	1,179	21.20 ms	800	31.23 ms
50	8,014	6.24 ms	1,384	36.13 ms	897	55.77 ms
100	7,665	13.05 ms	1,557	64.22 ms	970	103.10 ms

Results: CNPG Local SSD

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	749	1.34 ms	134	7.48 ms	99	10.10 ms
10	7,675	1.30 ms	1,425	7.02 ms	1,031	9.70 ms
25	6,788	3.68 ms	1,560	16.02 ms	1,073	23.30 ms
50	6,430	7.78 ms	1,550	32.27 ms	996	50.18 ms
100	6,092	16.41 ms	1,464	68.32 ms	902	110.92 ms

Results: CNPG Longhorn 1 Replica (Dedicated Disk)

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	754	1.33 ms	119	8.43 ms	90	11.12 ms
10	7,713	1.30 ms	940	10.64 ms	748	13.37 ms
25	7,311	3.42 ms	1,254	19.93 ms	1,015	24.64 ms
50	6,587	7.59 ms	1,384	36.12 ms	1,064	46.98 ms
100	6,109	16.37 ms	1,453	68.83 ms	1,009	99.14 ms

Results: CNPG Longhorn 2 Replicas (Dedicated Disk)

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	752	1.33 ms	103	9.71 ms	81	12.39 ms
10	7,655	1.31 ms	712	14.04 ms	607	16.47 ms
25	7,379	3.39 ms	996	25.11 ms	835	29.93 ms
50	6,699	7.46 ms	1,144	43.71 ms	908	55.07 ms
100	6,063	16.49 ms	1,269	78.78 ms	852	117.39 ms

Results: CNPG Longhorn 2 Replicas (Shared Disk)

Clients	RO TPS	RO Lat	RW TPS	RW Lat	TPC-B TPS	TPC-B Lat
1	741	1.35 ms	97	10.32 ms	76	13.08 ms
10	8,399	1.19 ms	681	14.69 ms	598	16.72 ms
25	8,279	3.02 ms	957	26.11 ms	802	31.17 ms
50	7,406	6.75 ms	1,081	46.27 ms	873	57.29 ms
100	6,697	14.93 ms	1,206	82.89 ms	829	120.67 ms

The Combined Average Summary

Averaged across all 13 client/thread combinations per workload type:

Config	Avg RO TPS	Avg RW TPS	Avg RW Lat	Overall Avg TPS
AWS RDS Standard	10,724	2,250	17.30 ms	4,826
AWS Aurora IO-Prov	8,370	1,234	29.72 ms	3,480
AWS Aurora Standard	8,039	1,162	31.45 ms	3,326
CNPG Local SSD	6,111	1,355	30.02 ms	2,796
CNPG Longhorn 1R	6,376	1,152	32.46 ms	2,797
CNPG Longhorn 2R	6,356	935	39.35 ms	2,671
CNPG Longhorn 2R+shared	7,052	892	40.95 ms	2,885

What The Data Tells Us

Finding 1: Overall average is misleading for storage comparison.
Overall avg TPS (RO+RW+TPCB combined) shows all bare metal configs at ~2,670–2,885 — nearly identical. This is because read tests dominate by volume and read performance is unaffected by storage. Always disaggregate by workload type.

Finding 2: Read performance — storage backend is irrelevant.
All four bare metal configs produce 6,111–7,052 Avg RO TPS — variation is within normal test noise. Aurora leads on reads (8,039–8,370) due to its distributed read-optimized storage layer.

Finding 3: Write performance — Local SSD wins clearly.
Local SSD delivers 1,355 Avg RW TPS vs Longhorn 2R's 892 — a 52% advantage. Every Longhorn replica adds ~3–4 ms write latency (network round-trip for replication acknowledgment).

Finding 4: Dedicated vs shared disk makes almost no difference.
Longhorn 2R dedicated (935 Avg RW TPS) vs Longhorn 2R shared (892) — only 4.6% difference. The bottleneck is network replication, not disk I/O contention.

Finding 5: Bare metal Local SSD write TPS beats Aurora IO-Optimized.
Local SSD Avg RW TPS (1,355) vs Aurora IO-Prov (1,234) — +9.8% advantage at baseline, before any CPU or kernel tuning. Aurora's write path pays network replication overhead just like Longhorn.

Finding 6: RDS Standard leads overall — but it's burstable.
RDS Standard's 4,826 overall avg and 2,250 Avg RW TPS comes from t3 CPU burst credits. Once credits are exhausted in sustained workloads, performance drops significantly.

Recommendations

Workload	Recommendation
Write-intensive OLTP	Local SSD — 52% higher write TPS vs Longhorn 2R
Read-heavy (API, reporting)	Longhorn is fine — zero read overhead vs local SSD
HA with block-level durability	Longhorn 2R — accept write penalty, gain replication
Best write + HA	Local SSD + CNPG streaming replication — no storage network in write path
Managed simplicity	Aurora Standard — competitive write TPS, no ops overhead

**→ Part 2 :** CPU/NUMA pinning + HugePages — pushing bare metal write performance even further past Aurora.

Environment Details

CloudNativePG: v1.24 on Kubernetes 1.31
Host: Bare Metal 32-Core (16 Physical / 16 HT), NUMA-Aware
Storage: Samsung SM863a Enterprise SSD RAID 1 (SAS Interface)
PostgreSQL config: Intentionally matched to AWS t3.large defaults for fair comparison
Deployment: Single instance — no HA, no read replicas, no connection pooling
AWS Region: ap-southeast-3 (Indonesia)
Scale Factor: 100 (~10M rows, ~1.5 GB table)
Benchmark runner: Kubernetes-native pgbench Job — source on GitHub

— Iwan Setiawan, Hybrid Cloud & Platform Architect · portfolio.kangservice.cloud