Geolocation Optimization: Caching and Local Database Lookup

This post discusses optimizing geolocation lookups in the devlog-ist/landing project, which enhances user experience by personalizing content based on location.

The Challenge

Initially, the application relied on an external HTTP service (ip-api.com) to determine a visitor's country based on their IP address. This approach introduced significant latency, adding approximately 1 second to each page load. Furthermore, relying on an external service meant potential rate limits and service disruptions.

The Solution

To address these issues, we implemented two key optimizations:

1. Local Database Lookup: Replaced the external HTTP calls with lookups against a local MaxMind GeoLite2-City database. This eliminates network latency and reliance on an external service.
2. Caching: Introduced a caching layer to minimize repeat lookups. Resolved country codes are cached for 7 days, while unresolvable IPs are cached for 1 hour.

Implementation Details

The local database lookup is performed using the maxminddb package in Go. Here's an example:

package main

import (
    "fmt"
    "net"

    "github.com/oschwald/geoip2/maxminddb"
)

func main() {
    db, err := maxminddb.Open("GeoLite2-City.mmdb")
    if err != nil {
        panic(err)
    }
    defer db.Close()

    ip := net.ParseIP("8.8.8.8")

    var record struct {
        Country struct {
            IsoCode string `json:"iso_code"`
        } `json:"country"`
    }


    err = db.Lookup(ip, &record)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Country code: %v\n", record.Country.IsoCode)
}

This code snippet demonstrates how to open the MaxMind database, perform a lookup for a given IP address, and extract the country code. A similar function integrates with a caching layer (e.g., Redis) to store the results. When an IP address needs to be geolocated, the cache is checked first; if the result is not in the cache, the local database lookup is performed, and the result is then stored in the cache with the appropriate expiration time.

To simplify the process of keeping the GeoLite2-City database up-to-date, an artisan command was added to download the latest version.

The Result

By switching to a local database and implementing caching, we significantly reduced the latency associated with geolocation lookups. This resulted in faster page load times and an improved user experience. The application is also more resilient to external service outages and rate limits.

The Takeaway

Optimizing geolocation lookups can have a significant impact on application performance and reliability. By leveraging local databases and caching, developers can minimize latency and improve the user experience. Regularly updating the local database is crucial to ensure accuracy.