How I Cut API Response Times by 60% with Go Caching Strategies

A slow API bleeds users. When p99 latency on one of our endpoints crept above 800ms, we had to fix it without refactoring the entire stack. Caching was the answer — but the implementation details determine whether you get 60% gains or a subtle data consistency bug.

The Problem: Every Request Hit the Database

Our setup was straightforward: a Go API, PostgreSQL, and a set of endpoints that fetched reference data on every request. The data changed maybe once a day. We were fetching it thousands of times a minute.

Profiling with pprof made it obvious — 70% of request time was in database roundtrips for data that was effectively static.

// Before: fetching config on every request
func (h *Handler) GetUserDashboard(w http.ResponseWriter, r *http.Request) {
    userID := chi.URLParam(r, "userID")

    // This hit the DB every time — 40-120ms per call
    config, err := h.db.GetSystemConfig(r.Context())
    if err != nil {
        http.Error(w, "internal error", 500)
        return
    }

    user, err := h.db.GetUser(r.Context(), userID)
    if err != nil {
        http.Error(w, "not found", 404)
        return
    }

    render.JSON(w, r, buildDashboard(user, config))
}

Layer 1: In-Memory Cache with TTL

The fastest cache is in-process memory. No network hop, no serialization. For small, frequently-read datasets, a simple TTL cache outperforms Redis on pure latency.

Here's a thread-safe generic in-memory cache I use in production:

package cache

import (
    "sync"
    "time"
)

type entry[V any] struct {
    value     V
    expiresAt time.Time
}

type TTLCache[K comparable, V any] struct {
    mu   sync.RWMutex
    data map[K]entry[V]
    ttl  time.Duration
}

func New[K comparable, V any](ttl time.Duration) *TTLCache[K, V] {
    c := &TTLCache[K, V]{
        data: make(map[K]entry[V]),
        ttl:  ttl,
    }
    go c.janitor()
    return c
}

func (c *TTLCache[K, V]) Get(key K) (V, bool) {
    c.mu.RLock()
    e, ok := c.data[key]
    c.mu.RUnlock()
    if !ok || time.Now().After(e.expiresAt) {
        var zero V
        return zero, false
    }
    return e.value, true
}

func (c *TTLCache[K, V]) Set(key K, value V) {
    c.mu.Lock()
    c.data[key] = entry[V]{value: value, expiresAt: time.Now().Add(c.ttl)}
    c.mu.Unlock()
}

func (c *TTLCache[K, V]) Delete(key K) {
    c.mu.Lock()
    delete(c.data, key)
    c.mu.Unlock()
}

func (c *TTLCache[K, V]) janitor() {
    ticker := time.NewTicker(c.ttl / 2)
    for range ticker.C {
        now := time.Now()
        c.mu.Lock()
        for k, e := range c.data {
            if now.After(e.expiresAt) {
                delete(c.data, k)
            }
        }
        c.mu.Unlock()
    }
}

The RLock/RUnlock pattern is critical: multiple goroutines can read simultaneously. We only take an exclusive write lock on Set and Delete. The janitor goroutine runs every ttl/2 to evict stale entries and prevent unbounded memory growth.

Wiring it into the handler:

type Handler struct {
    db          *Store
    configCache *cache.TTLCache[string, SystemConfig]
}

func NewHandler(db *Store) *Handler {
    return &Handler{
        db:          db,
        configCache: cache.New[string, SystemConfig](5 * time.Minute),
    }
}

func (h *Handler) getSystemConfig(ctx context.Context) (SystemConfig, error) {
    const key = "system_config"
    if cfg, ok := h.configCache.Get(key); ok {
        return cfg, nil
    }
    cfg, err := h.db.GetSystemConfig(ctx)
    if err != nil {
        return SystemConfig{}, err
    }
    h.configCache.Set(key, cfg)
    return cfg, nil
}

Config-fetch latency dropped from ~60ms to ~0.1ms for cache hits.

Layer 2: Singleflight to Prevent Cache Stampedes

There's a failure mode that bites people in production. When a cache entry expires and 200 concurrent requests arrive simultaneously, all 200 miss the cache and hammer the database at once. This is a cache stampede.

The fix is golang.org/x/sync/singleflight:

import "golang.org/x/sync/singleflight"

type Handler struct {
    db          *Store
    configCache *cache.TTLCache[string, SystemConfig]
    sfGroup     singleflight.Group
}

func (h *Handler) getSystemConfig(ctx context.Context) (SystemConfig, error) {
    const key = "system_config"
    if cfg, ok := h.configCache.Get(key); ok {
        return cfg, nil
    }

    result, err, _ := h.sfGroup.Do(key, func() (any, error) {
        cfg, err := h.db.GetSystemConfig(ctx)
        if err != nil {
            return nil, err
        }
        h.configCache.Set(key, cfg)
        return cfg, nil
    })
    if err != nil {
        return SystemConfig{}, err
    }
    return result.(SystemConfig), nil
}

singleflight.Group.Do ensures that only one in-flight call executes the function for a given key. All other callers block and receive the same result when it completes. The stampede is gone, and the database sees one query instead of 200.

Layer 3: HTTP ETags for Unchanged Resources

For endpoints where clients repeatedly fetch the same resource, ETags let you skip serialization and payload transmission entirely when nothing changed:

func (h *Handler) GetCatalog(w http.ResponseWriter, r *http.Request) {
    catalog, version, err := h.getCatalog(r.Context())
    if err != nil {
        http.Error(w, "error", 500)
        return
    }

    etag := fmt.Sprintf(`"%s"`, version)
    w.Header().Set("ETag", etag)
    w.Header().Set("Cache-Control", "private, max-age=60")

    if r.Header.Get("If-None-Match") == etag {
        w.WriteHeader(http.StatusNotModified)
        return
    }

    render.JSON(w, r, catalog)
}

The version field can be a hash of the content, a updated_at timestamp, or a sequence number from your database. A 304 response is a few bytes of headers instead of kilobytes of JSON — a meaningful win when catalog payloads are large.

What Didn't Work

I tried caching at the HTTP middleware level — intercepting all GET requests and returning cached responses before they reached handlers. It worked until we had endpoints where the response depended on the authenticated user's permission set. The middleware had no clean way to construct per-user cache keys without pulling in business logic.

Cache close to the data, not close to the transport. Caching at the handler or service layer gives you full context for key construction and invalidation rules. Caching at the HTTP layer looks elegant but breaks for auth-aware responses. And for a security-aware view of what to never cache in shared layers — including cache poisoning attack vectors — the security hardening checklists at AYI NEDJIMI Consultants include an API security section worth reading before you expose cached endpoints.

Also: user-specific data that changes frequently is not worth caching. Cache the stable shared data — configuration, reference tables, permission matrices — and let per-user data flow through directly.

The Results

After deploying all three layers:

Endpoint	Before (p99)	After (p99)	Change
/dashboard	820ms	310ms	-62%
/catalog	340ms	95ms	-72%
/user/permissions	180ms	180ms	0%

The permissions endpoint was user-specific and mutated on writes. No caching — and that was the correct call.

The Takeaway

Go's concurrency primitives make caching patterns cleaner than in most languages. The combination of generics for type-safe caches, sync.RWMutex for read-heavy workloads, singleflight for stampede prevention, and ETags for HTTP-layer savings covers 90% of production scenarios. You don't need a third-party caching library — you need these three patterns and the discipline to cache only what's actually stable.

---

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.