How I Cut API Response Times by 60% with Go Caching Strategies

REST APIs that don't cache are burning CPU and database connections on identical work. Once traffic picks up, you feel it fast — latency climbs, the database starts choking, and horizontal scaling just defers the problem. Here's a walkthrough of three caching patterns I applied to a Go service that brought p99 response time from ~420ms down to ~165ms.

The Problem: Too Many Identical Database Round-Trips

Every endpoint was going to the database even when nothing had changed. Cache hit rate was 0%. Profiling was unambiguous: 70% of request time was in SQL queries for rows that hadn't changed in hours. The fix wasn't a faster database — it was not hitting the database at all for identical reads.

Pattern 1: In-Process TTL Cache for Hot Read Paths

Start simple. A generic in-process cache with TTL covers most cases and has zero external dependencies.

package cache

import (
    "sync"
    "time"
)

type entry[V any] struct {
    value     V
    expiresAt time.Time
}

type TTL[K comparable, V any] struct {
    mu    sync.RWMutex
    items map[K]entry[V]
    ttl   time.Duration
}

func New[K comparable, V any](ttl time.Duration) *TTL[K, V] {
    c := &TTL[K, V]{
        items: make(map[K]entry[V]),
        ttl:   ttl,
    }
    go c.runEviction()
    return c
}

func (c *TTL[K, V]) Get(key K) (V, bool) {
    c.mu.RLock()
    defer c.mu.RUnlock()
    e, ok := c.items[key]
    if !ok || time.Now().After(e.expiresAt) {
        var zero V
        return zero, false
    }
    return e.value, true
}

func (c *TTL[K, V]) Set(key K, value V) {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.items[key] = entry[V]{value: value, expiresAt: time.Now().Add(c.ttl)}
}

func (c *TTL[K, V]) runEviction() {
    ticker := time.NewTicker(30 * time.Second)
    for range ticker.C {
        c.mu.Lock()
        now := time.Now()
        for k, e := range c.items {
            if now.After(e.expiresAt) {
                delete(c.items, k)
            }
        }
        c.mu.Unlock()
    }
}

Plug this into any handler layer. For product catalogs, config data, or user preferences where a few seconds of staleness is acceptable, this alone cut database query volume by 45% in my service. The generics make it type-safe without any interface{} juggling.

Pattern 2: singleflight to Prevent Cache Stampedes

In-process TTL caches have a well-known failure mode: when a key expires under burst traffic, dozens of goroutines simultaneously miss the cache and pile into the database. This is a cache stampede, and it can take a service down just when it's under the most pressure.

The standard Go fix is golang.org/x/sync/singleflight:

package handler

import (
    "context"
    "encoding/json"
    "time"

    "golang.org/x/sync/singleflight"

    "yourapp/cache"
    "yourapp/db"
)

var (
    productCache = cache.New[string, []byte](5 * time.Minute)
    sfGroup      singleflight.Group
)

func GetProductsByCategory(ctx context.Context, categoryID string) ([]Product, error) {
    if cached, ok := productCache.Get(categoryID); ok {
        var products []Product
        if err := json.Unmarshal(cached, &products); err == nil {
            return products, nil
        }
    }

    result, err, _ := sfGroup.Do(categoryID, func() (interface{}, error) {
        products, err := db.FetchProductsByCategory(ctx, categoryID)
        if err != nil {
            return nil, err
        }
        if data, marshalErr := json.Marshal(products); marshalErr == nil {
            productCache.Set(categoryID, data)
        }
        return products, nil
    })

    if err != nil {
        return nil, err
    }
    return result.([]Product), nil
}

sfGroup.Do deduplicates concurrent calls for the same key. All waiting goroutines get the result of the single in-flight fetch. Under a traffic spike, this turns hundreds of parallel DB queries into one. The third return value (shared bool) tells you whether the result was actually shared — useful for metrics.

Pattern 3: Tiered Caching with Redis for Multi-Instance Deployments

In-process caches don't survive restarts and don't share state across instances. For data that's expensive to generate and shared across users — aggregations, dashboards, leaderboard rankings — a Redis tier makes sense.

The access pattern is: check in-process cache → check Redis → fetch from DB → write back to both.

package cache

import (
    "context"
    "encoding/json"
    "time"

    "github.com/redis/go-redis/v9"
)

type Tiered[V any] struct {
    local  *TTL[string, V]
    remote *redis.Client
    prefix string
}

func NewTiered[V any](local *TTL[string, V], rdb *redis.Client, prefix string) *Tiered[V] {
    return &Tiered[V]{local: local, remote: rdb, prefix: prefix}
}

func (t *Tiered[V]) Get(ctx context.Context, key string) (V, bool) {
    if v, ok := t.local.Get(key); ok {
        return v, true
    }

    data, err := t.remote.Get(ctx, t.prefix+":"+key).Bytes()
    if err == nil {
        var v V
        if json.Unmarshal(data, &v) == nil {
            t.local.Set(key, v)
            return v, true
        }
    }

    var zero V
    return zero, false
}

func (t *Tiered[V]) Set(ctx context.Context, key string, value V, ttl time.Duration) error {
    t.local.Set(key, value)
    data, err := json.Marshal(value)
    if err != nil {
        return err
    }
    return t.remote.Set(ctx, t.prefix+":"+key, data, ttl).Err()
}

The local layer absorbs the hot tail. Redis handles cold hits and cross-instance consistency. You get sub-millisecond local hits on the most-requested keys without making each instance an island.

What Not to Cache

A few anti-patterns worth naming:

User-specific write-heavy data. Cache invalidation becomes a correctness problem. If a user updates their profile, your cache has to know about it. The logic complexity often outweighs the gain.

Paginated results backed by changing data. You'll serve inconsistent pages — page 1 reflects state at T0, page 2 reflects state at T0+5min.

Anything requiring real-time accuracy. Inventory counts, account balances, seat availability. Cache TTL should express how much staleness is acceptable, not how fast you want things to feel. A 5-minute TTL on data that must be accurate to the second is a bug.

Security Note on Caching

One thing that's easy to overlook: cached responses can expose data to the wrong user if your authorization layer isn't checked on every request. Caching at the handler level can bypass per-user access checks if you're not careful about cache key design. Always include user or tenant identifiers in cache keys for user-scoped data. I keep a security hardening checklist for Go APIs that covers these patterns if you want a reference.

The Takeaway

The 60% improvement came from three layers applied in sequence:

1. In-process TTL cache on the hottest read endpoints — immediate, zero dependencies, visible impact.
2. singleflight to collapse burst cache misses — critical under traffic spikes.
3. Redis tiered cache for shared aggregations — consistent across instances, survives restarts.

None of this required a rewrite. Profile first to confirm where the time actually goes, then apply the simplest pattern that solves it.

---

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.