DEV Community

loading...
Cover image for Building Image CDN with Firebase

Building Image CDN with Firebase

dbanisimov profile image Denis Anisimov ・5 min read

Delivering a perfect image on the web is not that easy of a task as it seems. Long gone the days where you can just put 'image.jpg' in your '/var/www/dist' and call it a day. You want your website to load images quickly, in the right format and with a perfect size based on the user's device. And you want that without blowing up your cloud storage bills and ideally without tedious work of resizing and tweaking your images manually.

Luckily for us Image CDNs exist. These are specialized types of Content Delivery Networks (CDN) that can take a source image, perform the conversions and transformations on the fly and cache the result to be delivered globally in a matter of milliseconds. Some examples are Cloudinary and imgix.

In this post we will build our own image CDN - efficient, fast, customizable, living on our own domain, and free using Firebase.

What we are going to build

In a nutshell we want to:

  1. Upload an image image.jpg to a storage bucket.
  2. Use a specially formatted URL to retrieve that image in the right size, e.g /cdn/image/width=100,height=50/image.jpg.
  3. That URL should work fast.
  4. We want to get the image in the webp format if our browser supports that.

Overall architecture

Firebase Image CDN Architecture Diagram

Btw, this image is served from Cloudinary Image CDN

Some important details about the proposed design:

Firebase Hosting

The Firebase Hosting part is where the automatic caching magic happens. By design Hosting caches all the responses from Functions that have Cache-control HTTP header set to public with an appropriate expiration time. That means that subsequent requests to the same image size, even from different browsers, will be served super fast and without firing another Function execution which saves us money.

Note that Firebase Hosting rewrite rules allow us to co-host our image CDN on the same domain as our main website.

Firebase Function

The Firebase Function contains the image processing logic. It is also responsible for setting the appropriate response HTTP headers for Firebase Hosting to cache processed images correctly.

Firebase Storage

The main storage for original images is Firebase Storage and we identify images by their key in the bucket. Using Firebase Storage is good for the speed of access and works on the free tier since we aren't making outbound requests from our functions.

Browser

Modern browsers that support webp format send a special Accept HTTP header with a value image/webp (at least Firefox and Chrome do that). We will use that to automatically detect the support and convert the image.

Note that since the desired image response depends on the HTTP header we also need to include it in the Function's response Vary header. This ensures that Firebase Hosting cache won't result in serving webp images to the browsers that don't support it.

The image transformation function

For image processing we are going to use sharp JavaScript library. This is super handy for us as we don't need to spawn any additional processes. Also sharp is faster than ImageMagic, so win-win.

You can find the full source code here. But without a single listing that wouldn't be a serious DEV post, right? So here is the snippet of the function implementation:

// Full code is here: https://github.com/dbanisimov/firebase-image-cdn

// Run the image transformation on Http requests.
// To modify memory and CPU allowance use .runWith({...}) method
export const imageTransform = functions.https.onRequest((request, response) => {
  let sourceUrl;
  let options;
  try {
    const [optionsStr, sourceUrlStr] = tokenizeUrl(request.url);
    sourceUrl = new URL(sourceUrlStr);
    options = parseOptions(optionsStr);
  } catch (error) {
    response.status(400).send();
    return;
  }

  // Modern browsers that support WebP format will send an appropriate Accept header
  const acceptHeader = request.header('Accept');
  const webpAccepted =
    !!acceptHeader && acceptHeader.indexOf('image/webp') !== -1;

  // If one of the dimensions is undefined the automatic sizing
  // preserving the aspect ratio will be applied
  const transform = sharp()
    .resize(
      options.width ? Number(options.width) : undefined,
      options.height ? Number(options.height) : undefined,
      {
        fit: 'cover'
      }
    )
    .webp({ force: webpAccepted, lossless: !!options.lossless });

  // Set cache control headers. This lets Firebase Hosting CDN to cache
  // the converted image and serve it from cache on subsequent requests.
  // We need to Vary on Accept header to correctly handle WebP support detection.
  const responsePipe = response
    .set('Cache-Control', `public, max-age=${cacheMaxAge}`)
    .set('Vary', 'Accept');

  // The built-in node https works here
  https.get(sourceUrl, res => res.pipe(transform).pipe(responsePipe));
});
Enter fullscreen mode Exit fullscreen mode

And also firebase.json rewrites section, which transparently forwards requests to the Function:

{
...
  "rewrites": [
    {
      "source": "/cdn/image/**",
      "function": "imageTransform"
    },
    {
      "source": "**",
      "destination": "/index.html"
    }
  ]
...
}
Enter fullscreen mode Exit fullscreen mode

The end result

You can see the live demo here: https://fir-image-cdn.web.app/.

The cat's photo is resized on-the-fly and cached, you can check that by opening it again in a private browsing mode - the result will be noticeably fast. Another way to verify that we are seeing content served by the CDN cache is to check for the x-cache: HIT header in the response.

How fast is it?

Cached responses - low 10ms for small images which should stay pretty consistent globally.

Cache miss - 500ms+ for small images, up to 1s for medium images and seconds (!) for large images. Whom to blame here is an open question - quick experiment with changing the Firebase Function type to a faster one hasn't shown noticeable difference, so it may not be due to the compute power, but rather Functions outbound networking or CDN cache fill performance.

Caveats

Cold starts. The first Function execution takes significantly longer than subsequent executions, which may affect the user's experience. One way to avoid that is to use Cloud Run containers, which allow concurrent requests to the same instance. The great thing is that Cloud Run is supported by Firebase Hosting as the request forwarding destination.

Low cache hit ratio. In our approach we vary the cached response based on the Accept header in order to correctly support webp detection. Every browser may send different value for that header which may result in very low cache hit ratio. We may avoid that by disabling the automatic detection and request webp version explicitly. Modern browsers allow to do that with a combination of <picture> and <source type="image/webp"> HTML elements.

How much does it cost?

Great news everyone! The combo described above works perfectly under the Firebase free Spark plan. You roughly get 5GB of stored source images and 100K image transformations per month.

It also seems like the cached responses served by CDN count towards the total Hosting download allowance of 10GB per month.

Next steps

If you want to take the self-hosted Image CDN idea to the next level then take a look at Thumbor. It has a ton of features and could be easily run inside of a Cloud Run container.

You may get better overall performance (and cost of operations) by manually combining Cloud CDN, Load Balancer, multi-regional Cloud Storage and Cloud Functions deployed in multiple regions. But should you?

What else to read


PSA: If you're organizing events or running a community check out Introwise 🎈

Stay sharp!

Discussion (6)

Collapse
najibghadri profile image
Najib Ghadri

Hi Denis! I like the post, nice architecture, but. I loaded the demo and I don't understand why the initial request takes so long. Say you upload an image, immediately we could start processing each different size and upload it to cdn globally, so no user has to wait for resize processing just download. I mean I compared it to loading instagram.com and that is faster than these small cat images. Am I missing something? Thank you.

Collapse
najibghadri profile image
Najib Ghadri

Oh, or is it because the cdn cache is very short lived i.e. couple of minutes only? If so is it something you can define in the firebase.json or console?

Collapse
najibghadri profile image
Najib Ghadri

Okey, bruh, got it. You set both cdn and browser cache max-age to 300s. If I wanted to set a different cache for CDN there is s-maxage header. Docs: firebase.google.com/docs/hosting/m.... Peace ✌️

Collapse
santiagosj profile image
Santiago Spinetto Jung

Hi Denis! very interesting topic and nice to follow tutorial, I wonder how can I use firebase as CDN to provide images in GatsbyJS? Should I'll make an Image component and set the transformations in it as a props? depending if in the url's origin there is a string fragment like "firebasestorage.googleapis.com/" ?
Regards!

Collapse
darwiin profile image
darwiin

Hi Denis,

Great post ! But I have a question, and tried many things without success to use it with subdirectories on my bucket. Do you have any advice ?

Collapse
spock123 profile image
Lars Rye Jeppesen

Very cool stuff, will implement tonight

Forem Open with the Forem app