DEV Community

Cover image for Building a Local-First 3D Dashboard with IndexedDB: 89% Cache Hit Rate
Emmanuel
Emmanuel

Posted on

Building a Local-First 3D Dashboard with IndexedDB: 89% Cache Hit Rate

#angular #pwa #typescript #database

How we achieved sub-100ms data loads and offline resilience with IndexedDB

Problem: Our 3D globe dashboard made API calls on every interaction, causing 500-2000ms delays and complete failure when offline.

Solution: Implemented a local-first architecture with 4 IndexedDB databases, cloud sync, and a cache version manager.

Result: 89% cache hit rate, <50ms data loads, full offline functionality, and seamless cloud backup.

The Problem: Network-Dependent 3D Visualization

Our 3D Global Dashboard displays bird migration paths sourced from the GBIF API (2.5 billion observations). Users can also play geography quizzes, discover countries, and unlock achievements.

// ❌ Original approach: Always fetch from API
async getMigrationData(speciesId: string): Promise<MigrationPath[]> {
  const response = await fetch(`/api/gbif/species/${speciesId}/migrations`);
  return response.json();
}
Enter fullscreen mode Exit fullscreen mode

The problems:

  • 500-2000ms latency for every species lookup
  • Complete failure offline — no airplane mode support
  • Redundant requests — same data fetched repeatedly
  • API rate limiting — GBIF has request limits
  • Poor UX — loading spinners everywhere

The Solution: Local-First Architecture

Local-first means treating the network as an optimization, not a dependency. All IndexedDB access, cache eviction, and cloud synchronization are owned by dedicated services, giving us a single place to reason about data lifecycle, performance, and consistency.

User Action → Local State → IndexedDB → UI Update (instant)
                              ↓
                        Cloud Sync (async, debounced)
Enter fullscreen mode Exit fullscreen mode

Benefits:

  • Instant UI updates — no waiting for network
  • Offline resilience — everything works without internet
  • Reduced API calls — cache serves most requests
  • Better UX — responsive interactions, background sync

Architecture: 4 Specialized Databases

We split data across 4 IndexedDB databases based on data characteristics:

Database Purpose TTL Sync to Cloud
quiz-stats-db Game sessions & scores Permanent Yes
country-discoveries-db Country interactions Permanent Yes
achievements-db Gamification progress Permanent Yes
cache-version-manager Tracks cache versions Permanent No


Chrome DevTools Application tab showing the 4 specialized databases

Why separate databases?

  • Single responsibility (stats, discoveries, achievements)
  • Independent versioning and migrations
  • Easier debugging and maintenance
  • Clear ownership per service

Implementation: The idb Library

We use the idb library—a thin Promise wrapper around IndexedDB:

pnpm add idb
Enter fullscreen mode Exit fullscreen mode

Database Schema Definition 

// src/app/core/services/country-discovery.service.ts

import { openDB, IDBPDatabase, DBSchema } from 'idb';

// Define the schema with TypeScript
interface CountryDiscoveryDB extends DBSchema {
  discoveries: {
    key: string;  // countryCode
    value: CountryDiscovery;
    indexes: {
      'by-method': string;
      'by-first-discovered': Date;
      'by-last-interacted': Date;
    };
  };
}

interface CountryDiscovery {
  countryCode: string;
  countryName: string;
  discoveryMethod: 'click' | 'hover' | 'search' | 'quiz' | 'migration';
  firstDiscoveredAt: Date;
  lastInteractedAt: Date;
  interactionCount: number;
}
Enter fullscreen mode Exit fullscreen mode

Database Initialization 

// src/app/core/services/country-discovery.service.ts

@Injectable({ providedIn: 'root' })
export class CountryDiscoveryService {
  private readonly DB_NAME = 'country-discoveries-db';
  private readonly DB_VERSION = 1;
  private db: IDBPDatabase<CountryDiscoveryDB> | null = null;

  async initialize(): Promise<void> {
    this.db = await openDB<CountryDiscoveryDB>(
      this.DB_NAME,
      this.DB_VERSION,
      {
        upgrade(db, oldVersion, newVersion, transaction) {
          // Create object store if it doesn't exist
          if (!db.objectStoreNames.contains('discoveries')) {
            const store = db.createObjectStore('discoveries', {
              keyPath: 'countryCode',
            });

            // Create indexes for efficient queries
            store.createIndex('by-method', 'discoveryMethod');
            store.createIndex('by-first-discovered', 'firstDiscoveredAt');
            store.createIndex('by-last-interacted', 'lastInteractedAt');
          }
        },
      }
    );
  }
}
Enter fullscreen mode Exit fullscreen mode

Persistent Local State

User Data (Permanent Storage)

User-generated data (quiz scores, discoveries, achievements) lives permanently in IndexedDB and syncs to Supabase:

// src/app/core/services/user-stats.service.ts

import { openDB, IDBPDatabase, DBSchema } from 'idb';

interface QuizStatsDB extends DBSchema {
  sessions: {
    key: string;
    value: GameSession;
    indexes: {
      'by-mode': string;
      'by-date': number;
      'by-completed': boolean;
    };
  };
  meta: {
    key: string;
    value: UserStatsV1;
  };
}

@Injectable({ providedIn: 'root' })
export class UserStatsService {
  private readonly DB_NAME = 'quiz-stats-db';
  private readonly DB_VERSION = 1;
  private db: IDBPDatabase<QuizStatsDB> | null = null;

  // Reactive state with signals
  private readonly _stats = signal<UserStatsV1 | null>(null);
  private readonly _recentSessions = signal<GameSession[]>([]);

  // Public readonly signals for components
  readonly stats = this._stats.asReadonly();
  readonly recentSessions = this._recentSessions.asReadonly();

  /**
   * Save a completed game session
   */
  async saveSession(session: GameSession): Promise<void> {
    if (!this.db) await this.initialize();

    // Atomic transaction: save session AND update aggregated stats
    const tx = this.db!.transaction(['sessions', 'meta'], 'readwrite');

    try {
      // Save the session
      await tx.objectStore('sessions').put(session);

      // Update aggregated stats
      const currentStats = await tx.objectStore('meta').get('user_stats_v1');
      const updatedStats = this.calculateUpdatedStats(currentStats, session);
      await tx.objectStore('meta').put(updatedStats);

      await tx.done;

      // Update signals (triggers UI updates)
      this._stats.set(updatedStats);
      this._recentSessions.update(sessions => [session, ...sessions].slice(0, 20));

      // Queue cloud sync (debounced)
      this.cloudSyncService.queueSync();

    } catch (error) {
      tx.abort();
      throw error;
    }
  }

  // Recent sessions are queried via IndexedDB cursors to avoid loading large datasets into memory.
}
Enter fullscreen mode Exit fullscreen mode

API Cache with TTL (7-Day Expiration)

External API responses are cached with automatic expiration:

// src/app/features/bird-migration/services/gbif-adapter.service.ts

import { openDB, IDBPDatabase, DBSchema } from 'idb';

interface GbifCacheDB extends DBSchema {
  occurrences: {
    key: string;
    value: CacheEntry<GbifSearchResponse>;
    indexes: {
      'by-timestamp': number;
      'by-expires': number;
    };
  };
  species: {
    key: string;
    value: CacheEntry<GbifSpeciesSearchResult[]>;
    indexes: {
      'by-timestamp': number;
      'by-expires': number;
    };
  };
}

interface CacheEntry<T> {
  cacheKey: string;
  data: T;
  timestamp: number;
  expiresAt: number;
}

@Injectable({ providedIn: 'root' })
export class GbifAdapterService {
  private readonly DB_NAME = 'gbif-cache';
  private readonly DB_VERSION = 1;
  private readonly DEFAULT_TTL = 7 * 24 * 60 * 60 * 1000;  // 7 days in ms
  private readonly MAX_ENTRIES = 500;

  private db: IDBPDatabase<GbifCacheDB> | null = null;

  /**
   * Get data from cache, checking TTL
   */
  private async getFromCache<T>(
    storeName: 'occurrences' | 'species',
    cacheKey: string,
  ): Promise<T | null> {
    if (!this.db) await this.initialize();

    const entry = await this.db!.get(storeName, cacheKey);

    if (!entry) return null;  // Cache miss

    // Check if expired
    if (Date.now() > entry.expiresAt) {
      await this.db!.delete(storeName, cacheKey);
      return null;  // Treat as cache miss
    }

    return entry.data as T;  // Cache hit
  }

  /**
   * Store data in cache with TTL
   */
  private async storeInCache<T>(
    storeName: 'occurrences' | 'species',
    cacheKey: string,
    data: T,
    ttl: number = this.DEFAULT_TTL,
  ): Promise<void> {
    if (!this.db) await this.initialize();

    const entry: CacheEntry<T> = {
      cacheKey,
      data,
      timestamp: Date.now(),
      expiresAt: Date.now() + ttl,
    };

    await this.db!.put(storeName, entry);

    // Cleanup if too many entries
    await this.enforceMaxEntries(storeName);
  }

  /**
   * Main data fetching method with cache-first strategy
   */
  async searchOccurrences(
    speciesKey: number,
    options: SearchOptions = {},
  ): Promise<GbifSearchResponse> {
    const cacheKey = this.generateCacheKey('occurrences', {
      speciesKey,
      ...options,
    });

    // 1. Try cache first
    const cached = await this.getFromCache<GbifSearchResponse>(
      'occurrences',
      cacheKey,
    );

    if (cached) {
      this.logger.debug(`Cache hit for ${cacheKey}`);
      return cached;  // Return immediately, no network request
    }

    // 2. Cache miss: fetch from API
    this.logger.debug(`Cache miss for ${cacheKey}, fetching from API`);
    const response = await this.fetchFromGbifApi(speciesKey, options);

    // 3. Store in cache for future requests
    await this.storeInCache('occurrences', cacheKey, response);

    return response;
  }

  /**
   * Clean up old entries when cache grows too large
   */
  private async enforceMaxEntries(
    storeName: 'occurrences' | 'species',
  ): Promise<void> {
    const count = await this.db!.count(storeName);
    if (count <= this.MAX_ENTRIES) return;

    // Delete oldest entries using IndexedDB cursor
    const tx = this.db!.transaction(storeName, 'readwrite');
    const index = tx.objectStore(storeName).index('by-timestamp');

    let cursor = await index.openCursor();
    let deleted = 0;
    const toDelete = count - this.MAX_ENTRIES + 50;

    while (cursor && deleted < toDelete) {
      await cursor.delete();
      deleted++;
      cursor = await cursor.continue();
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Data Freshness & Consistency

Cache Version Management

When your app updates, you may need to invalidate caches. We built a central version manager:

// src/app/core/services/cache-version.service.ts

import { openDB, IDBPDatabase } from 'idb';

interface CacheDatabase {
  name: string;
  clearOnVersionChange: boolean;
}

@Injectable({ providedIn: 'root' })
export class CacheVersionService {
  private readonly VERSION_DB_NAME = 'cache-version-manager';
  private readonly CURRENT_VERSION = '1.2.0';  // Bump this to invalidate caches

  // User data databases (preserved on version change)
  private readonly databases: CacheDatabase[] = [
    { name: 'quiz-stats-db', clearOnVersionChange: false },
    { name: 'country-discoveries-db', clearOnVersionChange: false },
    { name: 'achievements-db', clearOnVersionChange: false },
  ];

  /**
   * Check version and migrate if needed (called on app init)
   */
  async checkAndMigrate(): Promise<void> {
    const storedVersion = await this.getStoredVersion();

    if (storedVersion !== this.CURRENT_VERSION) {
      this.logger.info(
        `Cache version changed: ${storedVersion} → ${this.CURRENT_VERSION}`
      );

      await this.migrateToNewVersion(storedVersion);
      await this.setStoredVersion(this.CURRENT_VERSION);
    }
  }

  private async migrateToNewVersion(oldVersion: string | null): Promise<void> {
    // Clear API caches (they'll be refetched with fresh data)
    for (const db of this.databases) {
      if (db.clearOnVersionChange) {
        await this.clearDatabase(db.name);
        this.logger.info(`Cleared cache: ${db.name}`);
      }
    }
    // User data is preserved—no action needed
  }

  /**
   * Clear only API caches (useful for "refresh data" button)
   */
  async clearApiCaches(): Promise<void> {
    for (const db of this.databases) {
      if (db.clearOnVersionChange) {
        await this.clearDatabase(db.name);
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Register it as an app initializer:

// src/app/app.config.ts

import { APP_INITIALIZER } from '@angular/core';
import { CacheVersionService } from './core/services/cache-version.service';

export const appConfig: ApplicationConfig = {
  providers: [
    {
      provide: APP_INITIALIZER,
      useFactory: (cacheVersion: CacheVersionService) => {
        return () => cacheVersion.checkAndMigrate();
      },
      deps: [CacheVersionService],
      multi: true,
    },
  ],
};
Enter fullscreen mode Exit fullscreen mode

Sync & Performance Strategy

Cloud Sync with Debouncing

User data syncs to Supabase with a 5-second debounce to batch rapid changes:

// src/app/core/services/cloud-sync.service.ts

@Injectable({ providedIn: 'root' })
export class CloudSyncService {
  private readonly supabase = inject(SupabaseService);

  // Sync state (reactive)
  readonly syncStatus = signal<'idle' | 'syncing' | 'synced' | 'error'>('idle');
  readonly lastSyncTime = signal<Date | null>(null);

  private syncDebounceTimer: ReturnType<typeof setTimeout> | null = null;
  private readonly SYNC_DEBOUNCE_MS = 5000;  // 5 seconds

  constructor() {
    // Auto-sync when user logs in
    effect(() => {
      const isAuthenticated = this.supabase.isAuthenticated();
      if (isAuthenticated) {
        this.syncFromCloud();  // Pull latest cloud data
      }
    });
  }

  /**
   * Queue a sync (debounced to batch rapid changes)
   */
  queueSync(): void {
    if (!this.supabase.isAuthenticated()) return;

    if (this.syncDebounceTimer) {
      clearTimeout(this.syncDebounceTimer);
    }

    this.syncDebounceTimer = setTimeout(() => {
      this.syncToCloud();
    }, this.SYNC_DEBOUNCE_MS);
  }

  /**
   * Push local data to cloud
   */
  async syncToCloud(): Promise<void> {
    if (!this.supabase.isAuthenticated()) return;

    this.syncStatus.set('syncing');

    try {
      const [sessions, stats, discoveries, achievements] = await Promise.all([
        this.userStatsService.getRecentSessions(1000),
        this.userStatsService.getStats(),
        this.discoveryService.getAllDiscoveries(),
        this.achievementsService.getAllAchievements(),
      ]);

      await Promise.all([
        this.supabase.uploadQuizSessions(sessions),
        this.supabase.uploadUserStats(stats),
        this.supabase.uploadDiscoveries(discoveries),
        this.supabase.uploadAchievements(achievements),
      ]);

      this.syncStatus.set('synced');
      this.lastSyncTime.set(new Date());

    } catch (error) {
      this.syncStatus.set('error');
      this.logger.error('Cloud sync failed', 'CloudSyncService', error);
    }
  }

  // Cloud data is merged using a simple "newest wins" strategy based on timestamps,
  // keeping conflict resolution predictable and debuggable.
}
Enter fullscreen mode Exit fullscreen mode

Multi-Layer Caching for Three.js

For Three.js assets, we add an in-memory cache on top of IndexedDB and the browser's HTTP cache. This avoids repeated GPU uploads and keeps texture loads effectively instant after the first render.

// In-memory texture cache (fastest layer)
private textureCache = new Map<string, Texture>();
Enter fullscreen mode Exit fullscreen mode

The cache hierarchy: Memory (instant) → Browser HTTP Cache (fast) → Network (slow)

Browser Console Helpers

For debugging, we expose helpers to the browser console:

// src/app/dev-helpers.ts

export function setupDevHelpers(injector: Injector): void {
  const cacheVersion = injector.get(CacheVersionService);
  const cloudSync = injector.get(CloudSyncService);

  (window as any).cacheVersion = {
    check: () => cacheVersion.getStoredVersion(),
    getDatabases: () => cacheVersion.getDatabases(),
    clearApiCaches: () => cacheVersion.clearApiCaches(),
  };

  (window as any).cloudSync = {
    status: () => cloudSync.syncStatus(),
    syncNow: () => cloudSync.forceSyncNow(),
  };

  // Deletes all IndexedDB databases for a clean slate
  (window as any).clearLocalData = async () => { /* ... */ };
}
Enter fullscreen mode Exit fullscreen mode

Performance Results

Cache Hit Rates

Data Type Hit Rate Avg Load Time (Cache) Avg Load Time (Network)
Country data 95% <10ms 200ms
Quiz sessions 100% <5ms N/A (local-only)
Bird migrations 89% <50ms 800ms
Species lookups 92% <20ms 400ms
Textures 99% <1ms 150ms

Overall Impact

Metric Before After Improvement
First interaction delay 500-2000ms <50ms 90% faster
Offline functionality None Full 100% offline
API requests per session ~200 ~20 90% reduction
Data freshness Always current 7-day max stale Acceptable

Why Not Service Workers?

We intentionally avoided Service Workers for caching application data. IndexedDB gives us explicit control over TTLs, migrations, and conflict resolution, while keeping logic in the same mental model as our Angular services. Service Workers are excellent for static assets, but harder to reason about for evolving domain data.

Tradeoffs and Gotchas

When Local-First Makes Sense

  • Read-heavy workloads — Same data accessed multiple times
  • Offline requirements — Users need to work without internet
  • API rate limits — External APIs restrict request frequency
  • Latency-sensitive UX — Users expect instant feedback

When to Skip Local-First

  • Real-time collaborative data — Stale data causes conflicts
  • Sensitive data — IndexedDB is readable by browser extensions
  • Small datasets — Network is fast enough, caching adds complexity
  • Frequently changing data — 7-day TTL would serve stale data

Common Mistakes

1.Not handling quota exceeded errors

try {
  await db.put(storeName, data);
} catch (error) {
  if (error.name === 'QuotaExceededError') {
    await this.evictOldEntries();
    await db.put(storeName, data);
  }
}
Enter fullscreen mode Exit fullscreen mode

2.Forgetting to close database connections

// Good: close when done
db.close();
Enter fullscreen mode Exit fullscreen mode

3.Blocking the main thread with large reads

// Good: use cursors for large datasets
let cursor = await db.transaction('items').store.openCursor();
while (cursor && items.length < 100) {
  items.push(cursor.value);
  cursor = await cursor.continue();
}
Enter fullscreen mode Exit fullscreen mode

4.Not versioning cache entries — Always include timestamp and expiresAt

Key Takeaways

  1. Separate databases by data lifecycle — Permanent user data vs. expiring caches 2.Always check TTL before returning cached data — Stale data is worse than slow data
  2. Debounce cloud sync — Batch rapid local changes (we use 5 seconds)
  3. Newest wins for conflict resolution — Simple and predictable
  4. Handle quota errors gracefully — IndexedDB has storage limits

The 3D Global Dashboard now loads data in <50ms, works completely offline, and seamlessly syncs to the cloud when connected. Users on spotty connections finally have a reliable experience.

GlobePlay - Interactive Geography, Bird Migration & Quiz

Explore 241 countries, trace bird migration paths, and test your geography knowledge on an interactive 3D globe. Built with Three.js and Angular.

favicon globeplay.world

Top comments (0)