Cross-domain Cookies: Building Less Annoying Consent Solutions

When I worked at IBM, we had a cookie preferences pop-up, as every company should:

But there was a problem: whenever the user clicked a link that opened a page in another IBM-owned domain, that new domain had a different set of cookies and "forgot" what the user chose in their cookie dialogue. While it may seem like a minor annoyance, it ultimately did factor into bounce rates. Even a small distraction can discourage potential clients from researching products!

What didn't work

My initial solution used a third-party cookie to track what level of privacy the user chose in a given domain. While this worked for the majority of users for several months, I eventually noticed bounce rates increasing again. Another engineer on my team discovered we had an increasing share of users who were using cookie-blocking browsers - Firefox, Brave, and extensions on Chrome dedicated to doing so. Anyone who used these browsers would have their pop-ups reappear on domains.

The new solution

I still needed an external service to manage cookie preference levels between domains, but recognized that third-party cookies weren't going to cut it.

So I turned to Redis.

With Redis, I created a lightning-fast in-memory mapping of users' IP address to their cookie preference level. This had the disadvantage of losing preferences whenever the user moved networks, but this rarely happened in the middle of their browsing sessions. I stored the level as a first-party cookie as well so that we didn't lose this information in the event their IP address did change.

Here's what the backend setup looks like in Node.js:

import express from "express";
import { createClient } from "redis";

const app = express();
app.use(express.json());

// Initialize Redis client
const redis = createClient({ url: "redis://localhost:6379" });

redis.on("error", (err) => console.error("Redis Client Error", err));
await redis.connect();

// TTL = 30 days (in seconds)
const TTL_SECONDS = 30 * 24 * 60 * 60;

/**
 * GET /cache
 * Retrieves cached notice_preferences for the client's IP
 */
app.get("/cache", async (req, res) => {
  const ip = req.ip;

  try {
    const value = await redis.get(ip);
    if (value === null) {
      return res.status(404).json({ message: "No value found for this IP" });
    }
    res.json({ ip, notice_preferences: value });
  } catch (err) {
    console.error("Error getting Redis value:", err);
    res.status(500).json({ error: "Failed to get value from Redis" });
  }
});

/**
 * POST /cache
 * Sets notice_preferences for the client's IP
 * Body: { data: { notice_preferences: "0" } }
 */
app.post("/cache", async (req, res) => {
  const ip = req.ip;
  const notice_preferences = req.body?.data?.notice_preferences;

  if (typeof notice_preferences === "undefined") {
    return res.status(400).json({
      error: "Missing 'data.notice_preferences' in request body",
    });
  }

  try {
    await redis.set(ip, notice_preferences, { EX: TTL_SECONDS });
    res.json({
      message: "Value set successfully",
      ip,
      notice_preferences,
      expires_in_days: 30,
    });
  } catch (err) {
    console.error("Error setting Redis value:", err);
    res.status(500).json({ error: "Failed to set value in Redis" });
  }
});

app.listen(3000, () => console.log("Server running on http://localhost:3000"));

In order to test this, I wanted to simulate a live environment. So I tested it in production.

Now, before you tear your hair out, I want to be clear. I tested in prod, but I did not use it in prod. I kept the third-party cookie implementation while also hitting the QA instance of my new backend.

The point? To see how real user traffic would interact with the new system.

I observed the key value stores in Redis and noticed something incredibly odd. I expected only public IP addresses in Redis, but found many private IPs, and their preference levels were constantly changing.

That's when I realized people at various companies used their own VPN, and they were sending over the proxy's IP address and not the client's. Had I integrated it into the core logic, many people would accidentally be setting each other's cookie preferences. Oops!

I corrected it by checking the X-Forwarded-For header instead of the provided IP. Express.js makes this easy.

// Trust proxy headers for accurate client IP detection
app.set("trust proxy", true);

In the end, caching IP addresses to cookie preference settings wasn't a perfect solution. But for our purposes, it got the job of smoothing out the user's journey across domains.

Takeaways

• Use data to understand your users. Introducing the cookie banner correlated with higher bounce rates. Minimizing the need to interact with it lowered it.
• Continuously verify your results. We saw initial success with third-party cookies, but as the browser landscape shifted, the problem demanded a new solution.
• Testing in prod is okay... Sometimes. Ideally, simulate without user engagement, but the most important thing is to not break your user experience.