DEV Community: felipecarrillo100

CloudFront A/B Testing Without Cache Fragmentation: The Shadow Origin Pattern

felipecarrillo100 — Tue, 17 Mar 2026 04:11:09 +0000

In AWS architectures, there is a silent trade-off that often goes unnoticed—until your AWS bill spikes or your latency graphs degrade:

Personalization vs. Cache Efficiency

The common approach to A/B testing—executing logic in a viewer-request hook and varying the cache key using cookies—seems convenient, but it is an architectural trap.

By introducing variants into the cache key, you:

Fragment your global edge cache
Destroy your Cache Hit Ratio (CHR)
Increase S3 origin load and egress costs
Reintroduce cold-start latency for users

There is a better way.

🧠 The Solution: The Shadow Origin Pattern

The Shadow Origin Pattern is a technique where the public request URI remains unchanged, while the origin fetch path is internally rewritten after a cache miss. Instead of exposing variants to the CDN cache key, we move the decision behind the cache layer.

1. The Architecture: Performance-First Routing

The goal is simple: Do not fragment the cache key at the edge.

How it works

Public Request (Cache Key): /shop
Internal Origin Fetch (after cache miss): /variants/B/shop

This is achieved using a Lambda@Edge origin-request hook, which runs only after CloudFront determines the object is not in cache.

Why this is powerful

The cache key remains clean and stable.
Variants are resolved internally.
Each variant is cached efficiently once fetched.
No unnecessary duplication of edge cache entries.

2. The Implementation: The “Shadow” Hook

This Lambda@Edge function acts as a precise traffic controller.

/**
 * Lambda@Edge: Origin Request Trigger
 * Goal: Internal URI mutation for high-CHR A/B Testing
 */
'use strict';

// Remove this line in AWS production
exports.hookType = 'origin-request';

exports.handler = async (event) => {
    const request = event.Records[0].cf.request;
    const { headers } = request;

    // 1. Determine User Segment
    let variant = 'control';

    if (headers.cookie) {
        for (let i = 0; i < headers.cookie.length; i++) {
            if (headers.cookie[i].value.includes('X-Variant=B')) {
                variant = 'B';
                break;
            }
        }
    }

    // 2. Internal Path Mutation
    if (variant === 'B') {
        request.uri = `/variants/B${request.uri}`;
    }

    // 3. S3 OAC Normalization
    if (request.uri.endsWith('/')) {
        request.uri += 'index.html';
    }

    return request;
};

💡 Save this file as `ab-testing.js`

⚠️ Critical AWS Configuration (Do Not Skip)

This function runs at the origin-request stage. Unlike viewer-request, cookies are not automatically available. You must configure CloudFront to forward them:

Cache Policy: Cookies -> Include (or whitelist X-Variant).
Origin Request Policy: Cookies -> Include (or whitelist X-Variant).

If missed, your logic silently defaults to variant = 'control', and your A/B test will appear broken.

3. The Fidelity Gap: Why Testing is Critical

The biggest barrier to adopting this pattern is what we call the Fidelity Gap. Everything happens inside AWS infrastructure; your browser "lies" to you because it only sees the public URI.

This leads to the classic (and painful) workflow: Deploy → Wait 20 minutes → Discover a 403 → Repeat.

Closing the Gap with Local Simulation

To eliminate this blind spot, we can simulate the environment locally using CloudFrontize.

Step 1: Mock the S3 Origin

Recreate your production structure in a local /public folder:

/public
  ├── index.html
  └── variants/
      └── B/
          └── index.html

Step 2: Run the CloudFrontize simulator

Emulate Lambda@Edge behavior locally with the --debug flag to see the "Invisible" rewrite:

cloudfrontize ./public --edge ./ab-testing.js --debug

Step 3: Verify the Invisible Rewrite

Open http://localhost:3000/.
Set the variant cookie in the browser console:

document.cookie = "X-Variant=B";
location.reload();

What you should see in the terminal:

[origin-request] Original URI: /
[Debug] Rewriting URI: / -> /variants/B/index.html

Your browser still shows /, but the content has changed. The Shadow Origin Pattern is working.

4. Professional Gotchas (Hard Lessons)

No Environment Variables: Lambda@Edge does not support process.env. Your function must be self-contained.
The 40KB Limit: Large cookies can break your function unexpectedly.
Cache Invalidation: Invalidating /shop does not invalidate /variants/B/shop. You must invalidate both.
Regex Latency: On high-traffic sites, complex regex adds latency. Prefer simple string operations.

🧾 Summary

High-performance edge architectures require discipline: keep the cache key clean and move logic behind the cache layer. By applying the Shadow Origin Pattern and validating it locally with CloudFrontize, you eliminate the deployment "black box" and gain full control over your edge behavior.

Next Step: Are you ready to bridge the fidelity gap? Check out the CloudFrontize GitHub Repository for more edge patterns.

Why is my CDN slow? Bypassing the 10MB Compression Limit on AWS CloudFront

felipecarrillo100 — Thu, 05 Mar 2026 23:19:35 +0000

The "Silent" Performance Killer

You’ve deployed your data-heavy app to AWS CloudFront. You checked the "Compress Objects Automatically" box. You assume your 12MB GeoJSON or a 15MB JavaScript module is being zipped down to a manageable size.

It isn't. AWS CloudFront has a strict, hard-coded 10MB limit for automatic compression. If your file is 10.1MB, CloudFront serves the full-fat, uncompressed version. While you're testing on high-speed office Wi-Fi, your users on 4G/LTE suffer waiting 20 seconds or more for the app to load, and your Lighthouse scores are in the basement.

To fix this, you have to take matters into your own hands: Pre-compress your assets and use Lambda@Edge to serve them.

🛠️ Step 1: Pre-Compressing the "Monsters"

Since CloudFront won't touch files over 10MB, we compress them ourselves. We can do this manually but it is much more optimal if we do the work at build time.

Using Vite, we can generate high-efficiency compression (Gzip and Brotli) for only the files that exceed that 10MB threshold. We can target any asset type: js,css, json, geojson, etc

export default defineConfig({
    plugins: [
        ...otherStuff,
        // Pass 1: Gzip
        viteCompression({
            algorithm: 'gzip',
            ext: '.gz',
            filter: /\.(js|mjs|json|css|html|geojson)$/i,
            // ONLY compress files that CloudFront will refuse to touch (>10MB)
            threshold: 10485760,
            verbose: true
        }),
        // Pass 2: Brotli
        viteCompression({
            algorithm: 'brotliCompress',
            ext: '.br',
            filter: /\.(js|mjs|json|css|html|geojson)$/i,
            // SAME threshold: 10MB
            threshold: 10485760,
            verbose: true
        }),
    ],
    publicDir: 'public'
});

After building, your ./dist folder contains both the original asset plus the tiny .br and .gz versions of any file larger than 10MB, these are typically 7 to 10 times smaller.

⚡ Step 2: The Logic (Lambda@Edge)

AWS Cloudfront will not take the compressed files automatically. We need configure the logic using a Lambda@Edge function.

Think of this as a "Traffic Controller" at the edge. Its job is to check if the browser supports high-efficiency compression and swap the file path of the large asset(s) NYBuildings.geojson for its compressed version before it reaches the origin.

'use strict';

// This line tells the simulator to run this on 'origin-request'. Remove this line before deploying to AWS
exports.hookType = 'origin-request';

exports.handler = (event, context, callback) => {
    const request = event.Records[0].cf.request;
    const headers = request.headers;
    const uri = request.uri;

    // Targeted Match: Define a match pattern for the pre-compressed files
    // This example ONLY rewrites NYBuildings.geojson 
    if (uri === '/NYBuildings.geojson') {
        const aeHeader = headers['accept-encoding'];
        const acceptEncoding = (aeHeader && aeHeader.length > 0) ? aeHeader[0].value : '';

        // Priority: Brotli (.br) > Gzip (.gz)
        if (acceptEncoding.includes('br')) {
            request.uri += '.br';
        } else if (acceptEncoding.includes('gzip')) {
            request.uri += '.gz';
        }

        // Log the rewrite for CloudWatch debugging
        console.log(`[Lambda@Edge] Rewriting ${uri} to ${request.uri}`);
    }

    // All other files (JS, CSS, etc.) pass through normally.
    // CloudFront will use its "Automatic Compression" for them if enabled.
    callback(null, request);
};

🛠️ Step 3: Test the solution before deploying to AWS

Developing Lambda@Edge is notoriously frustrating. Usually, every small change requires a full AWS deployment and 15 minutes of waiting for global propagation just to see if your code works. If you have a typo, you get a 502 Bad Gateway and no explanation.

Rather than running your development code in an AWS environment—where every error costs time and money—you can run and debug your Lambda@Edge code directly on your PC using a simulator.

This is where CloudFrontize changes the game. It is a local simulator that creates a high-fidelity AWS Edge environment on your machine. You don't even need an AWS account to start developing.

Verify your logic instantly:

cloudfrontize ./dist --edge ./lambdaCompress.js

Instant Feedback: Refresh your browser and see the 12MB file instantly shrink to 1.3MB in the Network Tab.
Catch Errors Early: CloudFrontize enforces strict AWS limits (like the 40KB body limit) locally. If your code is going to crash in production, it crashes in your terminal first.

The cloudfrontize simulator acts as a static file server with edge super powers. Your files at ./dist are served by default to port 3000. Open your browser at:

http://localhost:3000/

In your browser, inspect the network panel. You will see that the large geojson file is serve compressed in brotli format.

You can also verify the compression and size using curl

curl -I -H "Accept-Encoding: br" http://localhost:3000/NYBuildings.geojson

Notice the encoding is br and the size is only 1316586 (1.3MB).

HTTP/1.1 200 OK
content-encoding: br
host: localhost:3000
user-agent: curl/8.13.0
accept: */*
accept-encoding: br
Content-Length: 1316586
Content-Disposition: inline; filename="NYBuildings.geojson.br"
Accept-Ranges: bytes
ETag: "0adb23e9ab344696b54737fe4e4bfa430f2ddf3c"
Access-Control-Allow-Origin: *
Date: Thu, 05 Mar 2026 23:10:32 GMT
Connection: keep-alive
Keep-Alive: timeout=5

🚀 Step 4: Confident Deployment

Once your cloudfrontize simulation proves the logic is bulletproof, you can deploy to your live AWS distribution using the standard AWS console.

💎 Step 5: The "Pro" Solution – CloudFront Functions (CFF)

For conditional rewrites, CloudFront Functions (CFF) are superior to Lambda@Edge in both speed and economy. CFF is specifically designed for lightweight tasks, delivering results faster while being 6x cheaper. Unless your logic requires complex external integrations or body access, CFF is the more efficient choice.

The Logic

Save the following snippet. Note that we use strictly JavaScript ES 5.1 syntax (using var and indexOf) to comply with the restricted CloudFront Functions runtime. Using modern ES6+ features like const or .includes() will cause the function to fail in production.

function handler(event) {
    var request = event.request;
    var uri = request.uri;

    if (uri === '/NYBuildings.geojson') {
        var headers = request.headers;
        var acceptEncoding = headers['accept-encoding'] ? headers['accept-encoding'].value : '';

        if (acceptEncoding.indexOf('br') !== -1) {
            request.uri = uri + '.br';
        } else if (acceptEncoding.indexOf('gzip') !== -1) {
            request.uri = uri + '.gz';
        }
    }
    return request;
}

To run this in the 'cloudfrontize' , save the code as viewer-request-compression.js. The viewer-request prefix is mandatory; it tells the simulator exactly which CFF trigger point to execute.

 cloudfrontize ./dist --cff ./viewer-request-compression.js --strict -d

👉 Serving Precompressed Assets Correctly (Important!!)

The example above works “as-is” for JSON files, since browsers are generally less strict when handling JSON responses. However, particular care is required when serving assets such as .js and .css.

Modern web browsers rely on correct HTTP headers to properly interpret and execute resources. If you serve a compressed file (for example, file.js.br) as file.js without the appropriate headers, the browser will fail to process it correctly..

This is not due to the compression itself, but because the browser is unaware of the encoding or receives an incorrect content type.

When using CloudFront Functions (CFF) and Lambda@Edge Functions, keep in mind that you can modify request and response headers, but you must ensure that compressed assets are returned with the correct metadata or browser will discard them.

You can configure the required headers in multiple places:

At the S3 bucket level by setting object metadata
In a Lambda@Edge function (response phase)
In a CloudFront Function (viewer-response)

For JavaScript files compressed using brotli, ensure the following headers are present:

Content-Encoding: br
Content-Type: application/javascript

or if you use gzip compression:

Content-Encoding: gz
Content-Type: application/javascript

Failing to set these headers correctly will prevent the browser from interpreting the file as valid JavaScript/CSS.

Caution: Brotli requires a secure context (HTTPS or localhost); always offer gzip as a fallback.

📊 The Results

By bypassing the 10MB limit and using a local-first development workflow, the numbers speak for themselves:

Metric	CloudFront (Uncompressed)	Lambda@Edge + brotli pre-compression
Payload Size	12.15 MB	1.31 MB
Bandwidth Saving	0%	~89%

🚀 Try it Yourself

I’ve put together a complete repository featuring a 12MB NYC Buildings dataset in GeoJSON format. Using the Brotli pre-compression strategy, we shrink this file to just 1.3MB during the build.

The repository includes:

Vite Setup: Pre-configured with the compression plugin and the 10MB threshold.
The "Traffic Controller": The complete Lambda@Edge function for the URI rewrite.
One-Click Simulator: Everything you need to see the 90% size reduction in action in seconds.

GitHub: vite-brotli-cloudfront-sample