DEV Community: Rahul Nanwani

Optimize and resize images in AWS S3 in real-time with ImageKit

Rahul Nanwani — Thu, 02 Jul 2020 13:11:13 +0000

AWS Simple Storage Service or S3 is one of the most commonly used cloud-based storage across the globe. It provides infinite storage with high uptime, immediate access, and scales up as per your requirements.

A common use of AWS S3 storage is to store large files, like images. Images are often huge and have to be stored for really long times. Storing them in S3 takes away the pain of managing the storage on your own or continuously adding space on your server.

Features missing for image optimization natively on AWS S3

While S3 is excellent for image storage, it does not provide the following features, which are now fundamental for every website that is serious about delivering an exceptional experience to its users -

The ability to be able to compress the images
The ability to be able to convert the images to a newer, better formats like WebP
The ability to resize and crop the images for use on different pages of our application

And maybe a more advanced requirement could be to be able to these and more advanced manipulations like watermarking those images, creating smart thumbnails in real-time without having to write or maintain complex solutions for it. More often than not, managing images is not your core business, and you do not want to be spending a lot of time on it.

A large and unoptimized image in S3

What is ImageKit, and how does it optimize images in S3?

ImageKit.io is a cloud-based image CDN with real-time image optimization and transformation features that help you deliver perfectly optimized images across all devices.

Just by delivering through ImageKit, your images get optimized. You can then start resizing the images as well.

It comes integrated with AWS CloudFront for image delivery (and other non-image files too). Over 15,000 developers and companies already serve billions of optimized images via ImageKit and its CDN every day. So, you do not have to pay for a separate CDN if you are using ImageKit.

You can attach your AWS S3 bucket to ImageKit and start delivering optimized images in just a few minutes. Without writing a single line of code.

Setup of your AWS S3 storage with ImageKit

Image Optimization on S3 + ImageKit — How does it work?

Let’s look at how a request gets handled when you optimize your S3 images via ImageKit.

When a request comes to ImageKit, the image is delivered from the CDN cache.
If the image is not there in the CDN cache, ImageKit tries to deliver the image from its internal caches. Usually, images that are transformed once will stay in ImageKit’s caches for some time.
If the image is not found in this internal cache as well, then ImageKit gets the image from your AWS S3 bucket attached to ImageKit, optimizes it, and transforms it in real-time and sends it back to the user.
After this first time processing that takes about 200ms on an average, the image gets cached on the CDN and ImageKit’s internal caches, bringing down the response time for almost 98% of image requests to under 50ms.

The flow of request with ImageKit’s CDN, Processing Server and your AWS S3 storage

Setting up ImageKit and AWS S3 for image optimization and resizing

Let’s set up AWS S3 with ImageKit and start delivering optimized images. It will take a few minutes if you follow the steps here.

1. Creating a new ImageKit account

The first step is to create a new ImageKit account if you have not done so already. You get 20GB free output bandwidth every month forever without adding any credit card details.

A new ImageKit account that has been setup for this demo

ImageKit has six regions for processing the images. Please select a region that is the same as your S3 bucket or is close to it.

2. Creating a new user in AWS

ImageKit only needs a read-only or, more specifically, the GetObject access to your S3 bucket. It is recommended that you create a separate user in AWS for use in ImageKit.

You can create a new user by going to the AWS IAM console. For this demo, we have created a user called imagekit-s3-demo-integration with Programmatic access, i.e. access using keys, enabled.

Creating a new user in AWS IAM console with Programmatic access enabled

You just need to create a new user as of now. Don’t worry about the access you grant to this user. We will be doing that in the next section.

Once the user is created, copy its access key and secret key as we will use it later to integrate with ImageKit.

Once the new user is created copy the access key ID and the Secret access key

3. Assigning access to the bucket

We have a bucket named imagekit-s3-demo-integration for this demo with a couple of images in it.

A demo S3 bucket that has been for this guide

The above bucket is private, but the steps here will be the same, even for a public bucket. We need to provide GetObject access on this bucket to the newly created user.

a. To do this, click on the “Add Permissions” button for this user. Select “Attach existing policies directly” and then click on “Create Policy.”

Click on “Add Permissions” for the newly created user to get started

Create a new policy to be attached to this user

b. Select the JSON creator option for the policy and use the following JSON to create the policy that provides the GetObject access to our demo bucket. Note, that you need to change the bucket name to that of your bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::imagekit-s3-demo-integration",
                "arn:aws:s3:::imagekit-s3-demo-integration/*"
            ]
        }
    ]
}

We named our policy ImageKitS3DemoIntegration. Once you save the policy, it looks like this -

c. Attach this newly created policy to the user and confirm that the user in IAM panel has this policy attached.

Once the policy is attached, this is how the User in IAM panel looks like

4. Adding access in ImageKit

We will now use the access and secret keys for the new user that we created in AWS to enable ImageKit to access the S3 bucket. The documentation for the same can also be found here — Documentation.

In ImageKit, go to the Origins section under External Storage and then click on the “Add Origin” button. Change the “Origin type” to “Amazon S3 Bucket” and add the values in the corresponding fields.

Once we save this, if this is the first origin that you added in your ImageKit account, it will automatically get mapped against a URL endpoint. If this does not happen automatically, you can always go to the URL endpoints section and add this new S3 origin in the origin preference sequence.

AWS S3 Bucket gets mapped to the URL Endpoint

Let’s access the optimized image via ImageKit

Our sample bucket has an image at the path monuments/eiffel-tower.jpeg. The image eiffel-tower.jpeg is 938kb initially in size.

As indicated under the URL Endpoint inside your ImageKit dashboard, to access this image using ImageKit, you need to append the URL endpoint from your ImageKit dashboard with the path of the image in the bucket.

[https://ik.imagekit.io/s3demoik/monuments/eiffel-tower.jpeg](https://ik.imagekit.io/s3demoik/monuments/eiffel-tower.jpeg)

And voila!

Without adding any other parameter to the URL, the image is now down to 583KB in size and is also automatically converted to WebP image format in supported browsers.

Accessing the image via ImageKit optimizes the image and converts it to the right format too

Resizing and other transforms on AWS S3 images with ImageKit

Not just automatic optimization, you can also resize and transform the images in your S3 bucket in real-time. You just need to specify the right transformation in your image URL

Here is the above image resized to a width of 300px -

https://ik.imagekit.io/s3demoik/monuments/eiffel-tower.jpeg?tr=w-300

We can also specify both height and width to get a square thumbnail of this image.

https://ik.imagekit.io/s3demoik/monuments/eiffel-tower.jpeg?tr=w-300,h-300

There are many other transformations in ImageKit. All of them can be used with the images in the AWS S3 bucket. You can use the smart crop feature to center the image’s subject in the output thumbnail automatically. Or you could use overlays to create dynamic banners, control how thee image aspect ratio and cropping are handled, and so on.

Here is a video demonstrating the use of some of these transformations on the image.

Real-time image optimization for AWS S3 without writing any code

And just like that, optimizing and resizing images in your AWS S3 bucket becomes really simple. Easy integration and you start delivering optimized images in minutes.

If you use some metrics like Google Lighthouse to monitor the images on your website, then you would be able to check off all the image-related points from that list using ImageKit with minimal effort.

Get started with ImageKit now!

ImageKit comes with a forever free account that gives you access to all the image optimization and transformation related features for use with your S3 bucket. And you get 20GB free delivery bandwidth every month.

Originally published at https://imagekit.io on June 30, 2020.

What is a Content Delivery Network (CDN) — A Beginner’s Guide

Rahul Nanwani — Thu, 18 Jun 2020 07:25:13 +0000

A Content Delivery Network, or a CDN as it is commonly called, is an essential part of any modern website and application. The content that you view on your phones today, on any website or app, videos or images, or any other kind of content, is very likely to be delivered using a content delivery network.

A CDN is also an essential part of our image CDN offering at ImageKit. And given the lack of example-led resources for beginners about CDNs on the internet, we decided to write this short guide to help understand what a CDN is and how does it work, using some common scenarios that you would relate with.

What is a CDN — the theoretical definition

A Content Delivery Network or a CDN is a globally-distributed network of servers that helps provide high-availability, faster performance, and security to websites distributing their content via it.

Understanding CDN with an example

One of the most common use case of a CDN is to improve page load time. Let’s use an e-commerce store as an example to understand how a CDN helps improve page load time.

Consider that you run an e-commerce store in the US. You have developed a fantastic website and host it on a server located in a city on the US east coast, let’s say, North Virginia. All the files needed on your website are stored in this server. So, when a user accesses your website, everything that loads on that website comes from this server in North Virginia.

Now, a user in California, on the west coast, is trying to access the website. For every resource that loads on the website — the textual content, javascript files, stylesheets, and images — a request goes from the user’s device to your servers, where the files are stored. These two locations, the user in California and the server in North Virginia are over 2000 miles apart.

Negating all the other factors, this distance between the customer and the server is responsible for adding a few hundred milliseconds in load time of a resource. Imagine those extra milliseconds getting added up over hundreds of resources that load on your website, and you end up with a slow page load time.

Greater the geographical distance, higher the latency

And no one likes a slow website. It frustrates your users, hits your sales, and even impacts how you rank on search engines.

How does a CDN improve page load time?

As mentioned earlier, it is a globally distributed network of servers that store (commonly referred to as “cache”) and deliver some or all of your website’s content. Each of these servers in the CDN’s network is called a Point of Presence (PoP) or an edge server.

Instead of delivering your website resources directly from your website server, you deliver them via a CDN’s PoPs or edges.

CDNs cut down the geographical distance between the user and the content, thereby reducing load time

In the above image, we are using a CDN along with our server in North Virginia. This CDN has PoPs present in multiple locations across the US, including the west coast as well.

Now, when the user accesses your website, instead of getting the resources from your website server on the east coast, the user gets them from the CDN server that is closer to him on the west coast. Geographically, the user and the CDN server that responds to the user’s request are now just a few miles apart, which reduces the time taken to load the resource significantly.

The page loads faster, your users are happy, and your sales start looking up.

What are the other functions of a CDN?

As mentioned earlier, improving load time by delivering your content via a content delivery network is the most common use case for any CDN.

But there are other use cases as well, some that are implicit to using a CDN and some that are used by slightly larger and technologically advanced organizations.

1. Increasing Availability

This is an automatic result of using any CDN.

For simplicity, availability can be considered a simple measure of how long your website and its functions remain accessible in a given period.

Usually, when you are serving content from your servers, you need to add more servers as your traffic goes up. If there is an unexpected issue with your server or a database, it could take the application down.

With a CDN coming into the picture, it does two things. One, a lot of traffic doesn’t even come to your servers. The edge server of the CDN serves a lot of content from its cache. So, you need a slightly fewer number of servers.

Second, as long as the content is available in the CDNs cache, even if your actual servers are not working, the CDN will keep serving the content. This gives you some buffer time to fix issues on your servers while the CDN serves whatever content it can from its cache.

The CDN delivers content from its own cache while the origin server is down.

2. Website Security

This is a more advanced use of CDNs that is generally used by larger companies.

Since the CDN PoP or edge server is now the first layer in the system which accepts incoming traffic, it also becomes the first line of defence against attacks on your website.

Now, if a CDN can isolate bad traffic from good traffic, it can stop all the bad traffic from coming to your servers. Your servers only respond to the “good” requests coming from actual users.

CDN blocks the request from a malicious user while serving good requests as usual

Website security is a very vast topic in itself and beyond the scope of this blog. But, there are certain features like blocking access on non-HTTP ports, which are a standard feature in all CDNs, and help provide basic security. Such features are accessible to everyone.

Then there are more advanced features like Bot Protection, Web Application Firewall (WAF), DDoS protection, etc. that are available as add-ons in certain CDNs. Such add-ons are usually expensive, and configuring them takes time and effort too. Therefore, they are used by a select few companies who face such challenges and can afford to deploy more expensive customized solutions.

What kind of content can be delivered through a CDN?

Theoretically, you can use a CDN to cache and deliver your entire website. How long can you cache it on the CDN, or can you or should you cache it at all, depends on the type of content.

Let’s look at an example.

If you are selling Nike shoes on your website, and two users are looking at that product page — the first is a male from California, and the second is a female from New York.

It’s a black running shoe, and both see the same image for the product.

All users see the same product image on the website

Such content that does not change on a per-user basis is a great candidate for serving from the CDN cache. Had you been using your server directly, that server would have also sent out the same image. Content like this image, that does not change or remains “static” for users is called static content. Javascript, which affects the interactions on your website, and the CSS, which affects how your website looks, also remains the same for all users and are also classified as static content.

But your website can have different discounts or shipping rates for different regions within the country. You might want to tune product recommendations differently for your male and female audience. Or you may have an offer that is valid for only the next hour in New York. So, the actual website content, the text, the offers, and the APIs that get the product recommendations can vary for the two users.

The image remains the same but different offers are presented to each user

Such content is called dynamic content. It can change on a per-user basis (like recommendations), location-basis (discounts and shipping), or on a time-basis (like a discount that is available till midnight). It becomes difficult, if not impossible, to have such content stored in a CDN’s cache for a long time. Imagine an offer that was supposed to expire at 1 pm, continues to be stored and delivered from the CDN server till 3 pm. This would only result in confusion for your users and a drop in sales.

Maybe you can keep shipping rates cached for some hours on the CDN because they don’t change very often. However, content like recommendations for a user is likely to change frequently as the user navigates through other products, making it non-cacheable. And if you cannot store it on the CDN, then should you be using a CDN for such content at all.

Note: This is a simple example. There are certain cases, as a breaking news item on a high-traffic news website, where even a short cache time of 1 or 2 minutes can be useful for reducing the stress on servers while accelerating content delivery. A lot of websites do that in practice. Plus, CDNs can still act as the first layer of security, and therefore it makes sense to use it even if you are not caching any content on them.

The most common use case of a CDN is to cache content and deliver it to the end-user, reducing the page load time. This means that the content should be cached on the CDN edge as long as possible. The longer it stays, the longer you get the benefit of the fast load time. Google PageSpeed, for example, penalizes you for not using a long cache time for your static content.

PageSpeed giving a warning related to Cache Policy

However, you should be able to control how long the content stays on the CDN and how do you force it to refresh if the content on your server has changed.

For example, your CDN has stored, on its edge servers, a copy of the black Nike running shoe that we talked about earlier. Even if you change the image on your origin server, the file cached on the CDN won’t change automatically.

For the same product image, the CDN cache has a different version than the origin server. The user keeps getting the image from the CDN cache.

There are some standard cache control headers and best practices for updating your resources (and their URLs), which, when combined, ensure that the content on the CDN remains up-to-date and in sync with the updates on your servers. These techniques have been discussed in detail in this guide — The Ultimate Guide To Caching Static Assets and would require some technical know-how about HTTP requests.

Wrapping Up

This guide was intended to give you a gentle introduction of what a CDN is and how does it work. It was intentionally built as a light read, avoiding the technical jargon associated with Content Delivery Networks as much as possible.

We would ourselves follow this up with a more detailed, technical guide on how CDNs work. You are now equipped to go out on the internet and dive deeper into the functioning of a CDN.

Originally published at https://imagekit.io on June 9, 2020.

What is an Image CDN - The Complete Guide

Rahul Nanwani — Wed, 10 Jun 2020 04:27:22 +0000

If you are reading this guide, you are probably trying to learn more about an image CDN, and how does it work. Or you are a bit further down the line ‚ trying to evaluate what an image CDN should do or understanding the essential features of an image CDN that can help you with faster page load time, better image delivery, improved SEO, and simpler integration.

You have come to the right place!

This guide not only covers the basics but also includes why a particular feature is essential for an image CDN. Having this understanding would enable you to map your requirements to what an image CDN offers.

Let's quickly get started. We take a quick look at a regular content delivery network (CDN) and then gradually move on to the specifics of an image CDN.

What is a Content Delivery Network (CDN), in general

A Content Delivery Network forms the basis of what we will cover in the rest of this guide. Let's understand how it works with an example.

Consider that you run an e-commerce store in the US hosted on a server located in a city on the US east coast, let's say, North Virginia. So, when a user accesses your website, everything that loads on that website comes from this server in North Virginia.

But, your users access it from across all of the United States. And maybe even in other countries, which means that they could be a few thousand miles away from your server. And, the farther they are, the longer it would take for the content to get transferred from your server to their devices. Your users get frustrated because of the slow website and drop off without converting to a paid customer.

Users geographically far away from the origin server experience slow load time

This is where a Content Delivery Network comes into the picture. While your server sits in one location, a CDN consists of globally distributed servers that can store your website's content (or commonly referred to as "cache" your website's content) and deliver it to your users. So now, if users sitting in California access your website, instead of downloading the content from North Virginia, they can get it from one of the servers of the CDN, a few miles from their location. Shorter distance, super-fast load time, happier users, and more sales for you.

With a CDN the geographical distance, and hence the load time, decreases

If you want to read more about how a general CDN works, we have written a separate guide that gives you a gentle introduction to the same. You can access it from here - Content Delivery Network - A Beginner's Guide.

What is an image CDN?

Images present a slightly complicated case when it comes to delivering the perfect variation on each device using a CDN. You need to consider the image format, image dimensions, image compression, aspect ratio, and a lot more, while balancing the image's visual quality. Therefore, we need to have a CDN tuned for image delivery.

An image CDN is a regular content delivery network topped with a set of software enhancements to enhance the underlying CDNs functionality for optimizing and transforming images in real-time, thereby making it more suitable for image delivery.

This is the most important thing to understand about all the image CDNs out there. They are not an entirely different content delivery network; they are built on top of existing ones with software enhancements required for image optimization.

Let's look at different parts of the process to understand how are image CDNs built. There are three of them.

The Delivery

Delivery means how does an image actually gets to the user's device and how quickly we can get it there. This is done using the Content Delivery Networks we talked about earlier.

There are already massive, well-performing general content delivery networks out there. They have their network spread across hundreds of countries, already tuned for fast load times. Some of the leading ones are AWS CloudFront, Akamai, and Fastly.

Almost all good image CDNs would offer one of these leading CDN providers as a part of their service. Eventually, it is this underlying Content Delivery Network that is responsible for delivering the images to the users.

The Processing

As mentioned earlier, we need to add to the content delivery network the unique functionality required for modifying and optimizing the images. This processing functionality would allow you to resize images, crop them, watermark them, and apply visual effects to them.

This functionality is performed via dedicated "processing servers," which are separate from the servers in the content delivery network. The processing servers are the ones doing the heavy lifting in the image CDN setup. These would always be far fewer in number than the servers in a content delivery network and localized in a few regions.

The Storage

Now, this is a part that may vary across different image CDNs.

A good image CDN should be able to deliver all the image optimizations and transformations on any image, even the ones that are outside of the image CDN's system.

But, the leading ones take it a step further. They come with a storage or a media library for users to upload and manage their images. These uploaded images can be accessed via the CDN, and can be processed by the processing part of the image CDN. This media library makes it simple to get started with using the service apart from simplifying image upload and management.

Media Library in ImageKit.io

The overall structure

Let's look at ImageKit.io, which is an image CDN, as an example. It comes with an image storage called the Media Library (the "Storage" part). The bundled CDN is AWS CloudFront (the "Delivery" part), an extensive, general content delivery network with over 200 delivery nodes worldwide. The image processing servers are present in 6 regions across the globe (the "Processing" part).

Here is how the structure looks like when we combine the three components mentioned above

The three components of an Image CDN, taking ImageKit.io as an example

The combination of these three different components collectively makes an image CDN.

How does an image CDN work?

The flow of a user request and response in an Image CDN (CDN + Processing Server + Storage = Image CDN)

Now that we know that an image CDN is a combination of three distinct elements, let's see how do these come together to deliver the intended functionality.

When a user requests an image, it first goes to the delivery part of the image CDN, i.e., the underlying general CDN.
If the CDN does not have the image cached on its servers, it goes to the processing servers requesting the same.
The processing server then has two ways of fulfilling the request. Usually, it first attempts to send back an existing copy of the image that is requested.
If that is not possible for any reason, the processing server gets the original image from the storage. It can come integrated with the image CDN as shown above (like the Media Library in ImageKit), or it may be some external image storage. The server then does all the optimizations and transformations requested on the image in real-time and sends it back to the CDN. It also stores a copy of this processed image for subsequent requests.
The CDN delivers the image to the user and also caches it on its servers for subsequent requests.
Next time a request comes for the same image, the CDN would deliver it from its cache.

There might be some variations in the above steps that would vary across different image CDN services, but on a high-level, all image CDNs work in the same way.

**Note: **Regardless of the setup, an image CDN should be able to modify images in real-time. There are a few other services that let you do a one-time image processing and store them in some storage for you to use later, but, for the context of this article, and even otherwise, those are not image CDNs. They are great for a simple, one-time use case, but lag significantly in terms of functionality compared to real-time image CDNs.

Important infrastructure-level features of an image CDN

We have now covered how an image CDN is set up, and how does it work.

We now need to look at the features of an image CDN that will be help you deliver perfect images on your websites and apps with minimum effort.

Let's first look at some of the essential infrastructure-level features of an image CDN.

1. Integration with existing storages and servers

This is by far the most crucial feature that you should be looking for in an image CDN - more than any other infrastructure-level feature.

If you have been running a website for some time, you would already have quite a lot of images being used and stored in a particular location on your server or a storage service like AWS S3. Your processes to upload images to your storage, your content management systems, and your teams using it are also set and mature by that time.

Very often, in this scenario, all that you are looking for is a service that can just integrate with your existing systems and help you deliver optimized and resized images across devices. You do not want to opt for an image CDN where you have to move all your images to a designated storage to use its features. Or make tons of changes to your content management systems just to integrate it with your process.

2. Choice of the CDN (Bonus: Third-party CDN integrations)

As mentioned earlier, building your delivery network is a non-trivial task. Most image CDNs would rely on an established Content Delivery Network for the final image delivery.

Therefore, the first thing one should look for when evaluating an image CDN is the content delivery network used to serve the images. Good image CDNs are built on the best general CDNs available out there - Amazon CloudFront, Akamai, Fastly, Google CDN, among others. These CDNs would ensure that you get the fastest possible content delivery across the globe.

Bonus: Heavy users, those running a website with several million page views every month, might have their preferred CDN. This preference could be because of contractual obligations, use cases, or the strength of a particular CDN in the region they operate. In either case, the image CDN should allow integration with other CDNs apart from the one that comes bundled with it.

3. Location of the image processing regions

As mentioned earlier, the processing part of the image CDN is different from its delivery part. The processing servers are far fewer in number as compared to the delivery nodes of the underlying CDN and are present in a few geographical regions.

If you are using the image CDN with your storage or server, there is some time (a few milliseconds) that is needed to get the image from your storage to the image CDNs processing server. The shorter this distance geographically, the shorter this time to retrieve the original image and hence faster the image load. Similarly, any image request that is not delivered from the CDN cache goes the processing server for retrieval or fresh processing. The higher the distance between the CDN cache and the processing server, the higher the time for fulfilling this request.

Therefore, you should opt for image CDNs that can provide processing regions close to either your end-users or your existing infrastructure.

4. HTTPS and HTTP/2 (Bonus - With Custom Domain Names)

HTTPS is a protocol for securely transmitting data from a user's browser and the website's server. Google, the biggest search engine in the world, has always maintained that HTTPS-enabled websites get a ranking boost in search results. Modern browsers like Chrome, also give subtle preferences to secure websites.

Google uses a padlock in the address bar to indicate secure websites

HTTP/2, on the other hand, is an upgrade to HTTP/1.1 protocol, that allows for faster data transfer and page load. A faster page load time not just improves the user experience but is of great importance for SEO as well.

HTTP/2 on the left with all requests getting fired in parallel. HTTP/1.1 limiting the requests in parallel. Source: https://imagekit.io/blog/http2-vs-http1-performance/

Given the importance of HTTPS and HTTP/2 for securing websites, SEO, and faster page loads, your image CDN should support them, without fail.

Custom Domain Names: While most third-party image CDNs do support these out of the box, providing them over custom domain names like images.yourwebsite.com, is often not supported or is an expensive add-on. While the importance of custom domain names for SEO is debatable, they do help to white-label the image CDN and present the image URLs as if they are coming from your service, instead of a third-party. You should look for an image CDN provider with inexpensive custom domain name integrations.

5. Availability of an integrated media library

Unlike the first three points, this might be non-essential in most cases. But, used correctly, it can prove to be a great add-on or help you completely overhaul your image management systems and make them more efficient.

In companies with large fragmented teams and multiple stakeholders working on the images that go on a website or an app, having a central media library for storing, searching, and managing images can prove to be useful.

Media Library with Search and image uploads

In the previous section, we looked at the essential features of an image CDN from an infrastructure perspective.

Image-related features of an image CDN

We now get to the actual optimization and transformation capabilities that you should look for in an image CDN - the processing part. While the infrastructure helps you accelerate delivery, the processing features help you adapt the image to your requirements.

1. Automatic format optimization including WebP support

We have already talked about this earlier - different image formats are suitable for different kinds of use cases. For example, while JPG is excellent for photograph-like images, PNGs are suitable for graphics and logos.

Always JPG image

WebP or JPG image (depending on browser support). Image by Fabian

Good image CDNs should be able to convert your images to the correct image format, automatically taking into account factors like browser support and original image content.

Do note that, most image CDNs would just default to converting every image to WebP, whenever possible. However, there are a few cases, like the one illustrated below, where a lossy-compressed PNG is smaller than the WebP variant and looks the same. Therefore, image format optimization should not be just about converting the image to WebP but about converting it to the "right" format for that particular image.

This logo image is always in PNG even in WebP-supported browsers (1.6KB), as it is smaller than WebP (2.1KB)

2. Controllable image compression (Bonus: Network-based image compression)

The next step in image optimization is compression. And while lossy compression does lead to visual degradation, if done correctly and up to a certain level, the human eye is not able to perceive those visual differences.

For example, the two images below at different quality levels look nearly identical, though the one that is compressed is almost one-fifth in terms of byte size.

At Quality 100, JPG image 107KB

At Quality 90, JPG image 20.3KB. Image by Fabian

Different kinds of businesses have different quality requirements. For example, an e-commerce business selling clothes with a fine texture would prefer to deliver images at a higher quality because the visual quality of those details is what would help sell the product. On the other hand, news and content websites can get away with slightly lower quality for photographs. Or there might be individual sections of the website, like the part where you present a zoomed-in image, where you need a higher quality image. Therefore, the quality level or the compression should be controllable on a per-URL basis.

**Bonus: **Mobile network connections are still flaky. The download speed on a user's device can vary. To provide a seamless experience to users on different network speeds, a right image CDN should adapt the image compression to different network levels. A more compressed, lighter image can download faster on a slow user's device resulting in improved user experience.

3. Real-time transformations for responsive images

Any image CDN should come with fundamental image transformations like resizing, cropping (with different crop modes), DPR variation, and aspect ratio modification. These transformations, combined with specifications like responsive image tags, Client Hints (though an outgoing spec), or client-side javascript, help adapt the images for that particular device.

Varying layout across devices requires different image sizes

Bonus: Smart cropping of images

When transforming images for smaller sizes, cropping usually becomes unavoidable. If the subject of the image is not in the center, it can get cropped out of the thumbnail. An image CDN that provides smart crop (not just face crop, but identifies other important elements of the image), helps you overcome this problem.

From left to right - the original image, center cropped, and smart cropped image. Smart cropping gives a much better result.

4. Support for complex transformations and compressions

Just simple resize and crop transformations may not be enough for adapting your images for web and mobile. For example, you might want to overlay an image with your brand's logo in the top-right corner. With hundreds of thousands of images in your product catalog, it can be quite a task if you were to do it manually. Thus, it is a feature that you should look for in an image CDN.

Image with watermark added on the top dynamically using the image URL. URL here.

Overlays should not be just for watermarking, but good image CDNs would even provide you with text overlays. One can combine them with other image transformations like background modification to convert your images into personalized banners for your users. You can use these personalized banners for sending targeted emails, or on the website to help boost your conversion rates.

Image with text added and styled dynamically directly from the image URL. URL here.

In addition to the above examples, features like progressive images, Brotli compression for text-based image formats like SVG, support for conversion to animated WebP, are some of the other advanced features that can be useful in an image CDN.

5. URL usability and security

Real-time image transformations are fantastic. Just change a parameter, and you get a new image that fits your requirement perfectly. But this capability is like a double-edged sword.

While it makes it easier for someone to create a new image variant, it can pose problems for your technology team and make the URLs prone to plagiarism.

For example,

*a. Usability *- It becomes difficult for your technology team to remember the transformations that need to be applied to an image to adapt it to different parts of the website. For transforming an image to 400x300 size with a watermark, what is the exact transformation string? There should be a more natural way to refer to these transformations so that your development team can speak a common language.

*b. Security *- The transformation parameters can be removed or changed. Any unauthorized entity would then be able to access your original images, or download a modified variant and use it themselves. The image CDN should, therefore, come with features like URL-modification prevention, allowing access to only specific transformations, URLs with time-based expiry, and others to reduce the unauthorized use of your image URLs.

6. SDKs and Support

While this may not seem to be directly related to images, but excellent technical support and SDKs for use with different frameworks and platforms can provide a significant boost to your image optimization efforts.

The reason for this is that image optimization is essential, yet using an image CDN to its full potential and adopting the best practices related to image-loading is non-trivial.

You need your image CDN to be able to provide you with reliable technical advice, guides, and solid SDKs to make it simpler to implement the best practices in your code.

Identifying improvement areas for images on your website

We have now covered most of the image CDN and the different features that come along with it that can be useful for you. But do you need one? How is the current state of images on your website? And if it is terrible, then what are the areas that you should be looking to improve first?

The following tools can help you answer this question.

1. Google PageSpeed Insights

Built on top of Lighthouse, the new industry-standard performance measurement tool, PageSpeed insights, gives you a complete report of your desktop and mobile web page performance.

Within that detailed report, five parameters are related to image delivery and optimization - offscreen images, optimize images, properly size images, images in next-gen formats, and efficient cache-control.

PageSpeed Insights report of ImageKit.io

2. ImageKit's Website Analyzer and Performance Center

ImageKit is a complete image CDN that comes with two tools to help you understand and solve image optimization related problems. The website analyzer provides you with a complete report for a single image URL on both desktop and mobile, providing you with an image-by-image detailed report of how that particular image can be optimized.

ImageKit Website Analyzer with an image-by-image report for optimizing your images

The Performance Center, though available only to the paid users of ImageKit.io, takes the website analyzer report a step further. It automatically analyzes the critical pages on your website for image optimization every day. It then presents a visual trend specifically for the size and scope of optimization for the images on your website. This trend helps your team to stay on top of image optimization and delivery as you release new pages and layout variations.

The Performance Center shows that the image payload has gone up around Dec 25.

Wrapping it Up

Images are an essential part of all websites and apps - not just in terms of visual presentation but also in terms of the size that good quality images take on a website. Optimizing them, therefore, should be the number one priority for any website or app owner, who wants to deliver a great experience to the users.

And selecting the right image CDN that integrates well with your existing setup and provides the essential transformation capabilities, makes it simple to deliver perfect images across devices.

One such image CDN is ImageKit.io. It automatically optimizes your images to the right format and compression level and comes with tons of transformations to deliver the perfect image. It comes with AWS CloudFront and a media library built-in. Plus, you are in good company as over 15,000 companies and developers use it in 40+ countries. There is a generous free plan that gives you up to 20GB of bandwidth, absolutely FREE, every month!

Don't compromise on the images on your website. Start optimizing them today!

Originally published at https://imagekit.io on June 9, 2020.

The Ultimate Guide To HTTP Caching For Static Assets

Rahul Nanwani — Sat, 16 May 2020 12:00:46 +0000

According to HTTP Archive, among the top 300,000 sites, the browser can cache nearly half of all the downloaded responses. Undoubtedly this is a massive saving for repeat page views and visits. It reduces the time it takes the visitor to interact with the page (e.g., see the images or start using filters) and, at the same time, reduces the load on your server. Not to mention it reduces the cost of data transfer from your server (or CDN) to the end-user.

How Caching Works?

Suppose you open a webpage https://www.example.com and the server returns below HTML:

...
<link type="text/css" href="https://www.example.com/app.css" rel="stylesheet">
...
<!-- Rest of the HTML -->

When the browser parses this HTML, it identifies that a CSS resource needs to be loaded from https://www.example.com/app.css. The browser issues a request to the server, the server returns the image and tells the browser to cache it for 30 days.

GET app.css, no object found in local cache

Now, let's say you open the same page again after a few hours. The browser again parses the HTML and come across the resource at https://www.example.com/app.css. Since the browser has this particular resource available in its local cache, it won't even go to the server. The network request is avoided in this case, and the styles are applied very quickly.

GET app.css, object found in the local cache

Caching Static Assets On CDN

Now you understand how browser caches the resource, let's talk about taking it a step further. Let's say users across different physical locations access your website millions of times in a month. And every page load would load the same image logo-white.png (and many other static assets like JS and CSS). All these requests are coming to your server for the first time for each user. This unnecessarily strains your web server. To avoid this, you can use a Content Delivery Network (CDN) and cache the static resources on CDN nodes itself.

Resource loading without CDN

Resources are downloaded from the origin server

Resource loading with CDN

Resources are downloaded from the nearest CDN edge

Best Practices For Caching

Leveraging the cache is crucial, and there are few things to keep in mind while you set up your server, CDN, and application.

Choose The Right Cache Header Directive

Values of following HTTP headers in the response controls who can cache the response, under which conditions, and for how long.

Expires header
Last-modified header
Cache-control header (recommended)

The Cache-Control header is defined as part of the HTTP/1.1 specification, and it supersedes the other caching headers. All modern browsers support Cache-Control, and you can forget about the other two headers.

Set Optimal Cache Lifetime

The cache lifetime depends on what kind of resource you are trying to cache. For example, frequently updating content like avatars or script loaders can have a shorter cache lifetime. However, in almost all cases, static assets like JS, CSS, and images can be cached for a much longer duration.

You can use the table below to set the appropriate cache lifetime based on the type of resource:

Resource type	Optimial cache-control value	Description
JS, images, CSS	public, max-age=15552000	This tells that the resource can be cached by the browser and any intermediary caches for up to 6 months. Note: Static assets can be safely cached for a longer duration, like six months or even one year.
HTML	no-cache, no-store	Don’t cache HTML on the browser so that you can quickly push updates to the client-side. Note: You might want to cache HTML content on CDN but never cache HTML content on intermediate proxies and browsers. Different CDN has a different way of setting this cache privately without setting browser cache.

You can learn more about HTTP caching and different values for the cache-control header.

Ensure That The Server Adds A Validation Token (ETag)

Let's say you have specified a cache lifetime of 300 seconds, and now a page issues a new request for the same resource. Assuming 300 seconds have passed since the first request, the browser can't use the cached response. Now browser can issue a new request to the server. But, it would be inefficient because if the resource on the server hasn't changed, then it doesn't make sense to download the same resource we have in the local cache.

The validation token like ETag is here to solve this problem. The server generates this token, and it is typically a hash (or some other fingerprint) of the content, which means if the content changes, then this token change.

So the browser sends the value of this ETag in If-None-Match request header. The server checks this token against the latest resource. If the token hasn't changed, then a 304 - Not modified response is generated. 304 - Not modified response tells the browser that the resource in its local cache hasn't changed on the server, and its cache lifetime can be renewed for another 300 seconds. This saves time and bandwidth.

First download of the resource

First download of the resource

Subsequent If-None-Match request when object in local cache is expired

Subsequent If-None-Match request when object in local cache is expired

You need to ensure that the server is providing the ETag tokens by making sure the necessary flags are set. Use these sample server configuration to confirm if your server is configured correctly.

Embed fingerprints in the URLs of images, JS, CSS, and other static assets

The browser looks up the resources in its local cache based on the URL. You can force the client side to download the newer file by changing the URL of that resource. Even changing the query parameters is considered as changing the resource URL. Ever noticed file names like below?

<script src="/bundles/app.bundle.d587bbd6e38337f5accd.js" type="text/javascript"></script>
...
<div class="website-logo">
  <img src="https://www.example.com/logo-white_B43Kdsf1.png">
</div>
<div class="product-list">
  <img src="https://www.example.com/product1.jpg?v=93jdje93">
  <img src="https://www.example.com/product1.jpg?v=kdj39djd">
  ...
</div>
<!-- Rest of the HTML -->

In the above code, snippet B43Kdsf1, d587bbd6e38337f5accd, 93jdje93, and kdj39djd are essentially the fingerprint (or hash) of the content of the file. If you change the content, the fingerprint changes, hence the whole URL changes and browser's local cache for that resource is ignored.

You don't have to manually embed fingerprints in all references to static resources in your codebase. Based on your application and build tools, this can be automated.

One very popular tool WebPack can do this and much more for us. You can use long-term caching by embedding content hash in the file name using html-webpack-plugin:

plugins: [
  new HtmlWebpackPlugin({
    filename: 'index.[contenthash].html'
  })
]

Avoid embedding current timestamps in the file names because this forces the client-side to download a new file even if the content is the same. Fingerprint should be calculated based on the content of the file and should only change when the content changes.

You can also use gulp to automatic this using gulp-cache-bust module. Example setup:

var cachebust = require('gulp-cache-bust');

gulp.src('./dist/*/*.html')
    .pipe(cachebust({
        type: 'MD5'
    }))
    .pipe(gulp.dest('./dist'));

Aim for a higher hit ratio on CDN

A "cache hit" occurs when a file is requested from a CDN, and the CDN can fulfill that request from its cache. A cache miss is when the CDN cache does not contain the requested content.

Cache hit ratio is a measurement of how many requests a cache can fulfill successfully, compared to how many requests it received.

The formula for a cache hit ratio is (Number of Cache Hits) divided by the (Number of Cache Hits + Number of Cache Misses)

When using a CDN, we should aim for a high cache hit ratio. When serving static assets using CDN, it is easy to get a cache hit ratio between 95-99% range. Anything below 80% for static assets delivery is considered bad.

Factors that can reduce cache hit ratio

Use of inconsistent URLs - If you serve the same content on different URLs, then that content is fetched and stored multiple times. Also, note that URLs are case sensitive.

For example, the following two URL point to the same resource.

https://ik.imagekit.io/demo/medium_cafe_B1iTdD0C.jpg?tr=w-100,h-100

https://ik.imagekit.io/demo/medium_cafe_B1iTdD0C.jpg?tr=h-100,w-100

But the URLs are different, and hence, two different requests are issued to the server. And two different copies are maintained on the CDN cache.
Use of timestamps in URLs - If you are embedding current timestamps in URLs, then this changes the URL and response won't be returned from the cache. For example - https://www.example.com/static/dist/js/app.bundle.dn238dj2.js?v=1578556395255
A high number of image variations - If you have implemented responsive images and have too many variations for every possible DPR value and screen size. It could lead to a low cache hit ratio. If a user gets a tailored image dimension from your servers based on their screen width and DPR, then they might be the first user to request this specific image from your CDN layer.

A rule of thumb is to check popular viewport sizes and device types from your Google Analytics and optimize your application for them.
Using shorter cache lifetime - Static assets can be cached for a longer duration as long as you have a mechanism to push updates like embedding fingerprints in the URL. Setting a short cache duration increases the chances of a cache miss.

Use Vary Header Responsibly

By default, every CDN looks up objects in its cache based on the path and host header value. The Vary header tells any HTTP cache (intermediate proxies and CDN), which request header to take into account when trying to find the right object. This mechanism is called content negotiation and is widely used to serve WebP images in supported browsers and leveraging Brotli compression.

For example, if the client supports Brotli compression, then it adds br in the value of the Accept-Encoding request header, and the server can use Brotli compression. If the client-side doesn't support Brotli, then br won't be present in this header. In this case, the server can use gzip compression.

Since the response varies on the value of the Accept-Encoding header received from the client while sending the response, the server should add the following header to indicate the same.

Vary: Accept-Encoding

Or you can serve WebP images if the value of Accept header has a string webp. The server should add the following header in the image response:

Vary: Accept

This allows us to serve & cache different content on the same URL. However, you should use the Vary header responsibly as this can unnecessarily create multiple versions of the same resource in caches.

You should never vary responses based on User-Agent. For every unique value of User-Agent, a separate object is cached. This reduces the cache hit ratio.

Normalize request header, if possible. Most CDNs normalizes the Accept-Encoding header before sending it to the origin. For example, if we are only interested in the value of the Accept-Encoding header has the string br or not, then we don't need to cache separate copy of the resource for each unique value of Accept-Encoding header.

Cache Invalidation - Different Methods And Caveats

If you need to update the content, then there are a couple of ways to go about it. However, you should keep in the mind that:

Browser cache can't be purged unless you change the URL, which means you changed the resource. You will have to wait till the resource expires in the local cache.
Purging the resource on CDN doesn't guarantee that your visitor will see updated content because intermediate proxies (and browser) could still serve a cached response.

The above limitation leaves us with the following options:

Change the fingerprint in the resource URL

Always embed fingerprints in the URL of static assets and do not cache HTML on the browser. This will allow you to push changes quickly and, at the same time, leverage long term cache. By changing the fingerprint in URL, we are essentially changing the URL of resource and forcing the client-side to download a new file.

Wait till resource expires in the local cache

If you have chosen your cache policy wisely based on how often you change the content, then you don't need to invalidate the resource from caches at all. Just wait till the resources expire in caches.

Use Service Workers To Manage Cache

You can cache files using service workers to manage local cache using the Cache interface.

This is useful when caching content, which often changes such as avatars or marketing banner images. However, you are responsible for implementing how your script (service worker) handles updates to the cache. All updates to items in the cache must be explicitly requested; items will not expire and must be deleted.

You can put items in the cache on network request:

self.addEventListener('fetch', function(event) {
  event.respondWith(
    caches.open('mysite-cache').then(function(cache) {
      return cache.match(event.request).then(function (response) {
        return response || fetch(event.request).then(function(response) {
          cache.put(event.request, response.clone());
          return response;
        });
      });
    })
  );
});

And then later serve it from the cache:

self.addEventListener('fetch', function(event) {

  event.respondWith(caches.match(event.request));

});

Purge From CDN

If you need to purge the cache from CDN and don't have any other option, most CDN providers have this option. You can integrate their cache purge API in your CMS so that if a resource is changed on the same URL, a cache purge request is submitted on the CDN.

Conclusion

Here is what you need to remember while caching static resources on CDN or local cache server:

Use Cache-control HTTP directive to control who can cache the response, under which conditions, and for how long.
Configure your server or application to send validation token Etag.
Do not cache HTML in the browser. Always set cache-control: no-store, no-cache before sending HTML response to the client-side.
Embed fingerprints in the URL of static resources like image, JS, CSS, and font files.
Safely cache static resources, i.e., images, JS, CSS, font files for a longer duration like six months.
Avoid embedding timestamps in URLs as this could quickly increase the variations of the same content on a local cache (and CDN), which will ultimately result in a lower cache hit ratio.
Use Vary header responsibly. Avoid using Vary: User-agent in the response header.
Consider implementing CDN purge API in your CMS if you need to purge content from the cache in an automated way.

Network-Based Image Optimization And Role Of Service Workers

Rahul Nanwani — Thu, 16 Apr 2020 08:34:57 +0000

Importance of Page Load Speed

47% of consumers expect a web page to load in 2 seconds or less, and a 1-second delay in page response can result in a 7% reduction in conversions!

Are we still debating about the importance of page load speed?

The many, many images added to your website to increase its visual, as well as contextual appeal, are assets weighing down your webpages.

Amongst all the resources that load on a webpage, images make up for most of the payload!

And when data travels from point A to Point B, it takes time, in turn, slowing down the page load speed, and therefore, affecting website performance.

Role Of Network Speed in Website Performance

While time is spent optimizing images for various viewports, and compressing and optimizing them to reduce their bandwidth consumption, the importance of optimizing their delivery is often discounted, or sometimes even entirely ignored.

Yes, using a CDN helps cut latency, but what about the latency at the user’s end?

The various bandwidth connections have a direct impact on your website’s load speed. With each asset consuming bandwidth, the weight of that asset becomes a critical factor, and more so if the network speed fluctuates during the session.

So, assuming a user is visiting a webpage at the network speed of 4G, there would hardly be a hiccup the website faces as the bandwidth is fast enough for the resources to consume, and load quickly.

But if the user is on a 2G network, the available bandwidth would be much less, and with heavy website assets, the webpages would take far too long to load.

The bounce rate multiplies for each passing second the website takes to load. Those are potential conversions you’ve lost due to resources on your websites that haven’t been optimized for slower bandwidth connections.

Additionally, network speeds aren’t always stable. A user may have started their session on your website at a 4G connection, but it can drop down to a 2G one during their time on your webpages. You need to prepare your websites for such instances as well.

Why Network-Based Image Optimization?

As high-quality images consume more bandwidth further slowing the page load speed, and with more and more users shifting to mobiles for internet browsing, network speeds at which your websites are visited are now consequential factors to consider in your image optimization strategy.

Poor data download speeds is not a concept limited to certain countries or data providers, it is an issue everyone faces. When one is camping in the woods or in the basement of their building parking their car, are some instances where mobile data download speed takes a hit.

Image optimization based on network speeds is a relatively new concept, but now a critical yet effective one.

Essentially, the image is compressed and served based on the user’s network speed. So, slower the bandwidth speed, more the compression that needs to be applied to the images for a faster website.

Even if you do take care of optimizing the images in general, optimizing them further for slower networks can reduce the overall page weight by almost 30%. This reduces their weight, making them easier and faster to load even on slow networks.

Though you can optimize other assets apart from images on the basis of network speed, and while doing that will actually have a bigger impact on website performance than optimizing images for network speeds.
However, optimizing images is just simpler to do and, additional compression of images doesn’t break the critical functionality of any application.

Yes, the image quality drops a bit to provide for a better user experience. But unlike stripping away Javascript, which would require much thought, compressing images is relatively straightforward.

And all this can be done with minimum effort using ImageKit.io and our service worker.

It can help select the right image quality to be delivered to the user, without affecting the perceived website performance, for their bandwidth connection. All this without any changes in your code or image URLs, and no additional requests made to the server origins. Hence, saving bandwidth costs as well.

So how does it work?

Amongst the many challenges faced by developers is rendering the appropriate quality of any image based on the bandwidth or network speed of their users. But it does not need to be a complicated process. It should be as simple as passing an additional parameter in the image URL when the browser triggers a load for it.

ImageKit.io service worker can help optimize and deliver your images based on the network speed of your users, that too, in a simple, straightforward manner.

Network speeds are broadly divided into four categories - 4G, 3G, 2G, and slow 2G. ImageKit.io allows you to set different image quality levels based on each respective network speed based on application, within their dashboard.

All that needs doing is enabling the setting for network-based image optimization in the ImageKit dashboard. You get a code snippet right there which needs to be added to an existing service worker on your website or to a new service worker.

The ImageKit.io code snippet is like a plugin meant for use in service workers. It intercepts the image requests originating from the user’s browser, detects the user’s network speed, and adds the necessary parameters to the image URL. These parameters can be understood by the ImageKit server to compress the image to the desired level and maintain efficient client-side caching.

With ImageKit.io you can get the image with the desired compression level for a particular network speed in a simple manner.

The compression and quality levels can be customized and are entirely upon your discretion within ImageKit.io.

All you have to do is pass the network type in the image URL, and with the network type to image quality settings already stated within the dashboard, ImageKit.io service worker understands the parameter, and optimizes and renders the image in the required quality.

Quality is directly proportional to image size, thus, higher the quality number, larger will be the resulting image.

Additionally, in the ImageKit.io dashboard, you can specify the image URLs (or patterns in URLs) that should not be optimized based on the network type. For example, you would want to present the same crisp logo of your brand to your users regardless of their network speed.

Network-Based Image Caching With Service Worker

ImageKit.io service worker manages the caching (and expiration) of images.

So as an added advantage of using ImageKit.io service worker, the plugin bypasses the browser cache in favor of network-based image cache in the browser. Bypassing the browser cache means that the service worker can maintain different caches for different network types and chooses the correct image from the cache or request a new one based on the user’s current network condition.

The service worker plugin automatically uses the cache-first technique to load the images and also implements a waterfall method to pick the right one from the cache. With this waterfall method, images at higher quality get preference over images at a lower quality.

What this means is if the user’s speed drops to 2G and he has a particular image cached from the time when he was experiencing good download speed on a 4G network, the service worker will use that cached 4G-level image for delivery instead of downloading the image over the 2G network.

But the reverse is not valid.

If the user is experiencing 4G network speeds, the service worker won’t pick up the 2G image from the cache, because it is possible to fetch a better quality image and the resources allow for it.

Basically, it intercepts all image requests to check for the right cache of images to be served based on the requesting bandwidth connection, or check whether a simple image URL modification would suffice.

ImageKit.io has made it quite easy for you to optimize your images based on bandwidth speeds. It requires almost zero configuration on your part while offering network-based optimizations in a simple format within the tool.

With the feature incorporated in the dashboard as a convenient and effortless toggle, you can just turn it off or on by a simple click of a button!

Analytics

Another advantage of network-based optimizations with ImageKit.io is that the tool provides analytics on your user’s observed network type within its dashboard!

It gives you an insight into the number of times network-based optimization gets triggered and what does the user distribution look like across different network types.

This distribution analysis can be helpful even for optimizing other resources on your website.

Things To Keep In Mind When Employing A Service Worker For Your Websites:

While using service workers for optimizing your images based on varying network speeds has its merits, there are some things that need to be put in order and kept in mind before you get started with them.

For security reasons, service workers are available to secure origins only. Hence, your web pages, and subsequently, your service worker script needs to be served via HTTPS for it to be functional.
The service worker should be placed in the root folder so it can access all the requests. This is imperative as the scope of the service worker is limited to the folder it is present in.

Final Thoughts

Any website should aim for an optimum user experience, and network-based image optimizations serve that very purpose.

With your images being cached and served at sizes most appropriate for your users' network speeds, ImageKit.io service worker aids more than just a faster website, it also saves on your users‘ bandwidths. And a good user experience is about more than just the website, isn't it?

Website speed testing – Are you doing it right?

Rahul Nanwani — Wed, 20 Feb 2019 13:31:18 +0000

If you can’t measure it, you can’t improve it. That is why website speed testing is of great interest to every webmaster. While there are many tools available to measure different metrics related to web page performance, relying on a particular metric alone often leads to inaccurate conclusions.

The aim of this guide is to highlight the bigger picture around website speed testing. It includes what to measure, how to measure it accurately and various pitfalls in the whole process. Let’s cover them one by one:

What to measure during website speed testing?

When it comes to website speed testing, we immediately start thinking about the total page load time. And no doubt about the importance of this metric but if you dig deeper you will know that caring too much about this particular metric, as reported by several tools, will often mislead you. The page load time alone is a poor indicator of the user experience of your visitors.

Let me show you why. For example, take a look at this image

Notice how third party calls starting with ipt are taking quite a lot of time to load and increasing the reported page load time. Even though these calls happen asynchronously in the background after all the important content have been loaded, they are still reported in the total page load time. If you are in a similar situation, then ask yourself this question

Do any of these third party tracking API calls or ad scripts actually affect the way your user interacts with the page?

If the answer to the above question is no then you need to stop taking decisions based on this metric alone.

In other words, you need to start measuring metrics that matter to your application and are specific to your use case & business.

For example, on an e-commerce website, the users should be able to use filters and see the products. So all the JS files, HTML templates and product images should load as quickly as possible. And this is exactly what you should be measuring. We will soon discuss how you can measure the loading time of different assets individually.

On the other hand, for a news website, your visitors should be able to see the text and the associated images before losing interest and bouncing off. Only after you quickly feed them the content by quickly loading your page, do the chances of getting more engagement, more ad impressions and more clicks increase.

Now we know that one size doesn’t fit all, as not all metrics carry same importance. Let’s discuss how we can measure these page performance metrics separately and accurately.

How to measure different performance metrics of a web page?

There are broadly two measurement techniques:

Synthetic testing with no actual user. This could be done using tools like CatchPoint or Pingdom
Real user monitoring in an actual user’s browser by injecting javascript to collect timing metrics

If you care about how your users are actually experiencing your applications, then Real User Monitoring(RUM) will provide the most accurate insights.

There are a few companies and open-source projects for RUM but you can build your own scripts too!

Skip the next section if you are already familiar with Resource Timing API and the related properties.

About Resource Timing API

The Resource Timing API is exposed through the performance property of the window object.window.performance.getEntries() provides an array of PerformanceResourceTiming object for every asset on the page. Each PerformanceResourceTiming object in this array contains following crucial timing information:

initiatorType represents the type of resource that initiated the performance event. It could be element’s localname, css or xmlhttprequest etc.
redirectStart is recorded when the first HTTP redirect starts.
redirectEnd is recorded when the last HTTP redirect is completed.
fetchStart is recorded when the browser is ready to fetch the document using an HTTP request. This moment is before the check to any application cache.
domainLookupStart is recorded immediately before the domain name lookup.
domainLookupEnd is recorded immediately after the domain name lookup is successfully done.
connectStart is recorded immediately before initiating the connection to the server.
connectEnd is recorded immediately after the connection to the server or the proxy is established.
secureConnectionStart is recorded when the handshake begins for securing the connection. It is used only if TLS or SSL is in use.
requestStart is recorded immediately before the device starts sending the request for the resource.
responseStart is recorded immediately after the device receives the first byte of the response.
responseEnd is recorded immediately after receiving the last byte of the response.

A typical request lifecycle in the browser starts by a DNS lookup, then a TCP connection and finally actual downloading.

TTFB = responseStart - requestStart

Download time = responseEnd - responseStart

Total download time = responseEnd - requestStart

Here are a few scripts that you can use to see how your website is performing.

Demo script to calculate the download time of a specific JS file on a page

Suppose we want to measure the download time for a compiled JS file called vendor.js

function measureTimings() {
  var timingInfo = window.performance.getEntriesByName("https://imagekit.io/static/build/js/vendor.js");
  if(timingInfo.length) {
    var duration = timingInfo[0].duration;
    var beacon = new Image();
    beacon.src = "//example.com/rum/result/log-timings?script=vendor&amp;duration=" + duration;
  }

}

Call measureTimings() after vendor.js is loaded.

Simply implement an end point on your backend, like the log-timings end point in the above example, to receive this duration data. You can then calculate data like average, median and percentile load times to learn how long does it take before your web page is useful for your visitors.

This data reflects what your real users are experiencing on your website. You might have more than one JS file or some other assets that are more important for your use case, go measure timings for those.

Instead of implementing your own backend for recording these timings, you can also leverage Google Analytics’s User Timings tracking. You can use send command like this:

ga('send', {
  hitType: 'timing',
  timingCategory: 'JS Dependency',
  timingVar: 'load',
  timingValue: 1249

});

I will write a detailed post about how we can collect more information like visitor’s IP and location to be able to generate much better insights about our page performance. Let me know if you want a heads up

Demo scripts to calculate the 90% percentile download time for all images on a page

To calculate how much time it takes before 90% of the images on a web page have been loaded, you can use something like this

function measureImageTimings() {
  var timings = window.performance.getEntries();
  var sum = 0;
  for(var i=0, l=entries.length; i &lt; l;i++) {
    // check on name property to check only for images or any other custom check you want
    if(entries[i].name.indexOf("images.example.com") != -1) {
      timings.push(entries[i].duration);
    }
  }
  timings.sort(function(a,b){ return a-b; });
  var len = timings.length, ninetieth, beacon;
  if(len) {
      ninetieth = timings[parseInt(len *.9)] || 0;
      beacon = new Image();
      beacon.src = "//example.com/rum/result/log-nineteenth-percentile?ninetieth="+ninetieth;
  }

}

Trigger measureImageTimings() a few seconds after the user has been on the page. Ideally, you would want to wait till most of the images have been loaded. You can also send timings.length to note how many images have been loaded at the time of this recording. Experiment with different wait times before triggering measureImageTimings() to increase this count. If you have a better idea for this, do share in the comments section.

Various pitfalls during website speed testing

Timing metrics not available

Not all timing metrics are exposed on every origin. Timing-Allow-Origin response header should contain the whitelisted origin on which you want to allow access to above timing information. In case this check fails, responseStart and many other metrics will simply report 0 and your whole analysis can go wrong. You can whitelist all origins like this to be safe side

Timing-Allow-Origin : *

While relying on RUM, you should always collect a lot of data to make up for any variance.
Median and percentiles should be given more importance than average.
Remove the outlier values when analysing the data.
Don’t get overwhelmed with the sheer amount of data and always remember what to measure. Remember, the page load time example above. Don’t get locked into a single metric.
Know the difference between duration field and actual download time

During a page load, there are many resources being downloaded in parallel and they all compete for a limited bandwidth. On top of it if your server is using HTTP/1 instead of HTTP/2 then browser puts a further restriction of only 6 TCP connections per origin at any point in time. This can result in queueing your requests for a much longer time than compared to the actual download time for that asset. The browser can postpone your request for other reasons as well like a lower priority request than critical resources (such as scripts/styles). This often happens with images. For example-

Now notice that the overall download time (TTFB + download time) is only around 95ms but since the request was queued for 1.4 seconds, the total download duration is reported as 1.5 seconds.

You need to make sure you are using a right field from the PerformanceResourceTiming object for your measurements.

For example, If you are comparing the performance of various CDNs, then you should care about TTFB and total download time. On the other hand, if you want to measure how much time it takes before your images are actually visible to the visitors or how much time it takes before your JS is downloaded (i.e. filters start working), then you should take into consideration the duration field. It will account for all the queuing, stalling, TCP handshakes, SSL handshake & DNS lookup times.

I hope you are all set to use these techniques for measuring the performance of your own website. Website speed testing is a continuous process and besides RUM, you should also use other tools available to monitor the overall performance of your websites like Google PageSpeed Inshgits, WebPageTest and Pingdom.

To gain deeper insights about your images health, try ImageKit website analyzer

If you see too many unoptimized images in the analyzer report, sign up on ImageKit now and give it a try. It is an intelligent global image CDN free to begin with and only takes a couple of minutes to get up and running.

Not sure why image optimization is important, read this?

Please share this guide with your team and leave your feedback & views in the comments section below.

Speed up your website with image optimization, Absolutely Free!

Rahul Nanwani — Tue, 29 Jan 2019 13:18:29 +0000

A picture can paint a thousand words is an adage that is true for all things online. Whether it is the image of a product or a place, having them on the website is an excellent way to communicate with your audience. Also, with many sites having similar products, ideas and services an image can leave a lasting impact, enhance user experience and driving traffic to the webpage.

To stay ahead of the competition, companies need to understand not just the value of visuals, but also how and why it should be optimized. Optimized images can be a great differentiator between the haves and the have-nots.

If you are already aware of the importance of image optimization, you can skip the next section and jump straight to how ImageKit can help you achieve it for free.

If you are not convinced, let us look at how image optimization can help you stay ahead of the curve.

Why do you need image optimization?

Simply put, image optimization is a process of quickly delivering images in the correct format, dimensions and compression level to each device while ensuring good visual quality.

For example, you cannot use a product photograph clicked using a DSLR camera directly on your website. That image would run into quite a few Megabytes and take forever to download for your user. You need to resize the image down to the exact dimensions required on your website, convert it to, let’s say, JPG format and compress it a bit to get the size down to a few Kilobytes which is more in line with what is acceptable on a website. And then you need to ensure that this image is delivered quickly to your user’s device regardless of their geographical location.

1. Faster page load for better conversions -

With optimized images, your page size goes down, and the images load quicker which reduces the total load time of your page. This reduction in load time results in an improved user experience helping improve retention and decreasing the bounce rate. Thus the chances of a user converting, either purchasing a product or signing up on your website, improve significantly with image optimization.

2. Better SEO for organic traffic -

Plus, search engines like Google also factor in the page load speed when ranking any webpage. Lower the page load time higher is the organic traffic you can get to your website.

3. Perfect image experience across devices -

The definition doesn’t stop here. Most of the users use their smartphones and other hand-held devices to access websites. Therefore, for a perfect visual experience, you need to ensure that the images are resized correctly irrespective of the user's device.

The stepping stone of any successful online business is using correctly optimized and resized images.

So why do these seemingly clear and straightforward tasks are in reality difficult to achieve? The lack of a simple, yet complete tool for image optimization? Something that is easy to set up for any website and doesn’t cost a bomb? Well, all of that now changes with ImageKit!

What is ImageKit and how can it help with image optimization?

ImageKit is an intelligent global image CDN which is a one-stop solution for all your image-related needs.

- Real-time URL-based image transformations like resizing and cropping
- Automatic best format selection including WebP support
- Automatic image compression
- Intelligent cropping with Smart Crop
- Image overlays (watermarks)
- A ready-to-use media storage and many more features...

All the optimizations and transformations are done in real-time and can be changed directly from the URL.

Combine this with AWS CloudFront Content Delivery Network, that comes by default with ImageKit and has 150+ delivery nodes across the globe, your images will not only be smaller in size but will load in a few milliseconds for all your users.

And it is now available for absolutely free!

At ImageKit, we understand that you don’t work with bottomless budgets and have to be sure about the investments you make. And something as basic as image optimization should be accessible to everyone regardless of the size of their online presence.

That is why we have launched our Forever Free plan that gives you access to all the image-related features mentioned above and up to 20GB bandwidth for delivering optimized images every month, which translates to close to a million files that you can load with ImageKit! And yes it is free, no credit card needed. Forever!

Whether it is a WordPress blog or a Magento or Shopify e-commerce store or any other website for that matter, ImageKit can work for you with just a few minutes of setup and a small URL change. And if you are migrating from a complex third-party image management system, our team can help you with that as well.

ImageKit’s Forever Free plan gets you to access a single world-class tool to address multiple image-related requirements and turn your users into a custodian of your brand.

Just don’t believe in what we say! Sign up for the free plan now and see how ImageKit can help you deliver a perfect visual experience on your website

How Smart Crop can help you deliver perfect responsive images on each device

Rahul Nanwani — Thu, 17 Jan 2019 08:40:07 +0000

Images are a critical piece of all websites and apps. Whether it is an e-commerce store, a travel website or a content website, a typical website would have hundreds of thousands of images. The number can be significantly higher if you have user-generated content on the website. With such a large number of images on a website, for a perfect responsive image experience on each user’s device, the images should also change according to the device that is requesting them. Which means that instead of resizing the image using CSS, the image that loads on a mobile device should be actually smaller in dimensions than the one that loads on a desktop device.

Why does regular cropping fail?

In such a scenario, where a variety of images have to be resized for different devices and different placeholders, cropping in an image is inevitable. And the default cropping strategy adopted for images is centre cropping – preserve the centre of the image and crop out from the edges. While this strategy works fine for a lot of cases, it can fail when the subject is not at the centre of the image. Here are some examples of poor thumbnails as a result of centre cropping –

Original Images (scaled down)	Image thumbnails 150px x 200px

You may get away with such thumbnails on your website, not all of your users may care about it. But if you are one of those companies, that want to deliver a perfect experience to your users, even if it is a thumbnail of a product, you would not want such improperly cropped thumbnails to be shown to your users.

What’s the solution then? How do we ensure that we are cropping the images correctly while creating thumbnails?

Enter ImageKit’s Smart Crop

Smart Crop is a content-aware cropping method available to all ImageKit users that analyses the image content and tries to ensure that the most important part of the image is always at the centre of the final thumbnail.

Let’s see how it works for the images where the centre crop failed.

Original Images (scaled down)	Centre-cropped image thumbnails 150px x 200px	Smart-cropped image thumbnails 150px x 200px

In each case, the smart crop was able to get the most important part of the image in the centre.

How to enable smart crop in the image URL?

Well, it’s really simple. Along with the height and width, you need to specify the focus mode for cropping and set it to auto. This is done using the fo-auto parameter in the transformation string in the URL. For example,

https://ik.imagekit.io/demo/img/tr:w-150,h-200,fo-auto/test_image.jpg

Practical applications of smart crop

Now that we have seen how ImageKit works, let’s take a look at how you can use it on your website.

1. For e-commerce websites

Creating perfect product thumbnails

A product needs to be displayed on multiple pages on your website or app – the listing page, the product detail page, as a small thumbnail in an image carousel etc. With smart crop, you can ensure that each product thumbnail is perfect and showcases your product the way it should be done.

Here is how it smart crop can help your e-commerce store. We are using scaled-down versions of images here for display, but it does give an idea about the power of smart crop.

Original Image

Without Smart Crop

Mobile Thumbnail 200px x 200px	Desktop Listing Page 350px x 300px	Desktop Product Carousel 400px x 600px (scaled to 60%)

With Smart Crop

Mobile Thumbnail 200px x 200px	Desktop Listing Page 350px x 300px	Desktop Product Carousel 400px x 600px (scaled to 60%)

2. Travel & content websites and user-generated images

Get the most important region in your thumbnails

ImageKit’s smart crop can automatically determine the most important part in an image, whether it’s a monument or a landscape or a car, and preserve it in the final thumbnail.

Here are a few examples of the smart crop in action for different kinds of images –

Original Image

Without Smart Crop

Mobile Thumbnail 200px x 200px	Desktop Cover Image 800px x 300px (scaled to 60%)

With Smart Crop

Mobile Thumbnail 200px x 200px	Desktop Cover Image 800px x 300px (scaled to 60%)

Face thumbnails with smart crop

Till now, we have seen how smart crop works great for bringing the subject to the centre of the image. But there is more to it.

The smart crop has another face-cropping mode that finds out the face from an image and extracts it to create a thumbnail. This would help in cases where you need to create profile pictures from a user’s photo.

Here are a few examples of the face-cropping mode in action

Original Image	Regular Smart Cropped Image	Face-cropped Image

How to get the face cropping mode for your images?

Again, like every other transformation in ImageKit, this one is really simple as well. You need to set the value of the focus parameter to face in your image URL. This is done using the fo-face parameter in the transformation string in the URL. For example

https://ik.imagekit.io/demo/img/tr:w-200,h-200,fo-face/test_image.jpg

Putting it all together with other transforms

Apart from the examples demonstrated above, you can combine smart cropping with other transforms in ImageKit to create really cool user interfaces.

Let’s say, just like Facebook, you want to create a user’s profile page with a user profile picture and a cover photo. Let’s take these two images for the same.

With the smart crop set to face-crop mode and the radius transformation, we can get a rounded profile picture of the user.

We can then use the regular smart crop and blur transformation to resize the cover photo as per the required size for desktop and mobile devices.

Combining these two pictures using HTML and CSS (or in an app), this is how the end result can look like on different devices

Conclusion

Using smart crop for your images requires a very small change in your image URLs but it can help you inch closer towards a perfect image experience across devices. Just add fo-auto for general smart crop and fo-face for the face-crop mode of smart crop. It is especially useful if you get a lot of user-generated content or the main subject in the photos is off-centre.

So, what are you waiting for? Give it a try for your images and start delivering a better experience to your users.

Lazy Loading Images - The Complete Guide

Rahul Nanwani — Fri, 23 Nov 2018 13:50:37 +0000

Images are critical for every website and application today. Whether it is marketing banners, product images or logos, it is impossible to imagine a website without images. Sadly though, images are large in size making them the single biggest contributor to the page size. As per the latest HTTP Archive data, the median page size on desktops is 1511 KB. Images make up for almost 650 KB of that size, roughly 45% of the total page size. Now, since we cannot do away with images, we need to make our web pages load really fast with them. In this guide, we will talk about lazy loading images, a technique that helps improve the page load time and reduce page size while still retaining all the images on the page.

A quick glance at what lazy loading achieves

Before moving further, here is a sample video demonstrating the working of lazy loading. Notice how when the page is scrolled, the grey placeholder is replaced with the actual image.

What is Lazy Loading Images?

Lazy Loading Images refers to a set of techniques in web and application development that defer the loading of images on a page to a later point in time – when those images are actually needed instead of loading them up front. These techniques help in improving performance, better utilization of the device’s resources and reducing associated costs.

The word “lazy” in the English language is often attributed to the act of avoiding work as long as possible.

Similarly, lazy loading defers the loading of resources on the page as long as they are not needed. Instead of loading these resources as soon as the page loads, which is what normally happens, we defer the load of these resources to a later time when they are actually needed.

The technique of lazy loading can be applied to almost all the resources on a page. For example, in a single page application, if a JS file is not needed until later, it is best not to load it initially. If an image is not needed up front, load it later when it is actually needed.

Here, we will stick to lazy loading images and how to do it well on your website.

Why go for lazy loading images at all?

Lazy Loading defers the loading of an image that is not needed on the page immediately. An image that is not visible to the user when the page loads, is loaded later when the user scrolls and the image actually become visible. If the user never scrolls, an image that is not visible to the user never gets loaded.

This has two main advantages.

1. Performance Improvement

The is the most important one for you as a website administrator – better performance and load time. With lazy loading, you are reducing the number of images that need to be loaded on the page initially. Lesser resource requests mean lesser bytes to download and lesser competition for the limited network bandwidth available to the user. This ensures that the device is able to download and process the remaining resources much faster. Hence, the page becomes usable much earlier compared to the case without lazy loading.

2. Cost reduction

The second benefit for you is in terms of delivery costs. Image delivery, or delivery of any other asset, is usually charged on the basis of the number of bytes transferred. As mentioned earlier, with lazy loading, if the image is not visible, it never gets loaded. Thus, you reduce the total bytes delivered on the page. Especially for users that bounce off the page or interact only with the top portion of the page. This reduction in bytes transferred from your delivery network reduces delivery costs. This will become more apparent as we explore lazy loading in the coming sections.

Which images can be lazy loaded?

The basic idea of lazy loading is simple – defer loading anything that is not needed right now. For images, it usually translates to, any image that is not visible to the user up front, can be lazy loaded. As the user scrolls down the page, the image placeholders start coming into viewport (visible part of the webpage). We trigger the load for these images when they become visible.

You can find out which images are a candidate for lazy loading and how many bytes you can save on the initial page load by using the Google Lighthouse audit tool. The audit performed by this tool has a section dedicated for offscreen images. You can also use ImageKit’s website analyzer to identify if your website uses lazy loading or not apart from other critical image related optimizations on your page.

Lazy loading is critical not only to good performance but also to deliver a good user experience to your users. Since combining performance and user experience with lazy loading is important and challenging, we will continue to address this topic in more detail throughout this guide after we have looked at different ways to lazy load images.

Lazy Loading Techniques for images

Images on a webpage can be loaded in two ways – using the tag or using the CSS background property. We will first look at the more common of the two, the tag and then move to CSS background images.

The general concept of lazy loading images in tag

Lazy loading images can be broken down into two steps

Step one is to prevent the image load up front. For images loaded using the <img /> tag, the browser uses the src attribute of the tag to trigger the image load. Doesn’t matter if it is the 1st or the 1000th image in your HTML and well off-screen. If the browser gets the src attribute, it would trigger the image load.

Thus, to lazyload such images, put the image URL in an attribute other than src. Let’s say we specify the image URL in the data-src attribute of the image tag. Now that src is empty, the browser doesn’t trigger the image load

Now that we stopped the upfront load, we need to tell the browser when to load the image. For this, we check that as soon as the image (i.e. its placeholder) enters the viewport, we trigger the load. To check when an image enters the viewport, there are two ways. Let’s look at both of them with working code samples.

Trigger image load using Javascript events

In this technique, we use event listeners on the scroll, resize and orientationChange events in the browser. The scroll event is an obvious one to check when the user scrolls the page. The resize and orientationChange events are equally important for lazy loading. The resize event occurs when the browser window size changes. The orientationChange event gets triggered when the device is rotated from landscape to portrait mode or vice versa. In such cases, the number of images that become visible on the screen will change. Thus, we would need to trigger a load for these images.

When either of these events occurs, we find all the images on the page that are to be lazy loaded and have not yet been loaded. From these images, we check which ones are now in the viewport. This is done using the image’s top offset, the current document scroll top and window height. If it has entered the viewport, we pick the URL from data-src attribute and put it in the src attribute. This triggers the image load. We also remove the class lazy that identifies the images to be lazily loaded for events that trigger later. Once all the images are loaded, we remove the event listeners.

When we scroll, the scroll event triggers multiple times rapidly. Thus, for performance, we add a small timeout that throttles the lazy loading function execution.

Here is a working example of this approach.

If you would have noticed, the first 3 images in the example are loaded up front. The URL is present directly in the src attribute instead of the data-src attribute. This is essential for a good user experience. Since these images are at the top of the page, they should be made visible as soon as possible. We must not wait for an event or JS execution to load them.

Using Intersection Observer API to trigger image loads

Intersection Observer API is a relatively new API in browsers. It makes it really simple to detect when an element enters the viewport and take an action when it does. In the previous method, we had to bind events, keep performance in mind and implement a way to calculate if the element was in the viewport or not. The Intersection Observer API makes this really simple, helps avoid the math and delivers great performance.

Below is an example of using the Intersection Observer API to lazy load images. We attach the observer on all the images to be lazy loaded. Once the API detects that the element has entered the viewport, using the isIntersecting property, we pick the URL from the data-src attribute and move it to the src attribute for the browser to trigger the image load. Once this is done, we remove the lazy class from the image and also remove the observer from that image.

If you compare the image loading times for the two methods, event listeners vs Intersection Observer, you would find that using the Intersection Observer API, the image load is triggered much quicker and yet the site doesn’t appear sluggish on scrolling. In the method involving event listeners, we had to add a timeout to make it performant, which has a slightly negative impact on the user experience as the image load is triggered with a slight delay.

However, like everything new in browsers, the support for Intersection Observer API is not available across all browsers. So, we need to fallback to the event listener method in browsers where the Intersection Observer API is not supported. We have taken this into account in the example above.

Lazy Loading CSS Background Images

After <img /> tags, background images are the most common way to load images on a webpage. For <img /> tags, the browser has a very simple approach – if the image URL is available, let’s load the image.

With CSS background images it is not that straightforward. To load CSS background images, the browser needs to build the DOM (Document Object Model) tree as well as the CSSOM (CSS Object Model) tree to decide if the CSS style applies to a DOM node in the current document. If the CSS rule specifying the background image does not apply to an element in the document, then the browser does not load the background image. If the CSS rule is applicable to an element in the current document, then the browser loads the image.

This may seem complex at first, but, this same behavior forms the basis of the technique for lazy loading background images. Simply put, we trick the browser into not applying the background-image CSS property to an element, till that element comes into the viewport.

Here is a working example that lazy loads a CSS background image.

One thing to note here is that the Javascript code for lazy loading is still the same. We are using the Intersection Observer API method with a fallback to the event listeners. The trick lies in the CSS.

The element with ID bg-image has a background-image specified in the CSS. However, when the class lazy is added to this element, in the CSS we override the background-image property and set it to none. Since the rule, combining #bg-image with .lazy class has a higher preference in CSS than just #bg-image, the browser applies the property background-image: none to the element initially. When we scroll down, the Intersection Observer (or event listeners) detects that the image is in the viewport, it removes the class lazy. This changes the applicable CSS and applies the actual background-image property to the element triggering the load of the background image.

Better user experience with lazy loading images

Lazy loading presents a great performance benefit. For an e-commerce company that loads hundreds of product images on a page, lazy loading can provide a significant improvement in initial page load time while decreasing the bandwidth consumption. However, a lot of companies do not opt for lazy loading, because they believe it goes against delivering a great user experience – the initial placeholder is ugly, the load times are slow etc. In this section, we will try to solve some concerns around user experience with lazy loading of images.

1. Using the right image placeholders

A placeholder is what appears in the container until the actual image is loaded. Normally, we see developers using a solid color placeholder for images or a single image as a placeholder for all images.

We used the same in our example code as well. A solid light grey color is used for all our image backgrounds. However, we can do better to provide a more pleasing user experience. Below are some examples of using better placeholders for our images.

a. Dominant color placeholder

Instead of using a fixed color for the image placeholder, we find the dominant color from the original image and use that as a placeholder. This technique has been used for quite some time in Google image search results and Pinterest.

Sample image picked from Manu.ninja

This might look complex to achieve. But, a very simple way of accomplishing this is to first scale down the image to a 1×1 pixel and then scale it up to the size of the placeholder – a very rough approximation but a simple, no-fuss way to get a single dominant color. Using ImageKit, the dominant color placeholder can be obtained using a chained transform in ImageKit as shown below.

Dominant color placeholder image URL example using ImageKit

The placeholder image is just 661 bytes in size compared to the original image that is 12700 bytes – 19x smaller. And it provides a nicer transition experience from placeholder to the actual image.

Here is a video demonstrating how this effect works for the user.

You can view the working example and code for using dominant color placeholder here.

b. Low quality image placeholder (LQIP)

We can extend the above idea of using a dominant color placeholder further. Instead of using a single color, we use a very low-quality, blurred version of the original image as the placeholder. Not only does it look better, it also gives the user some idea about what to expect in the actual image and gives the perception that the image load is in progress. This is great for improving the perceived loading experience. This technique has been utilized by the likes of Facebook and Medium.com for images on their websites and apps.

LQIP image URL example using ImageKit

The LQIP is 1300 bytes in size, still almost 10x smaller than the original image and a significant improvement in terms of visual experience over any other placeholder technique.

Here is a video demonstrating how this effect works for the user.

You can view the working example and code for using LQIP technique here.

It is clearly evident from the video samples of the two techniques above, that using dominant-color placeholders or using low-quality image placeholders provides a smoother transition from the placeholder to the actual image, gives the user an idea of what is to come in place of that placeholder and improves loading perception.

2. Adding some buffer time for image load

When we discussed different methods to trigger image load above, we checked for the point of time where the image enters the viewport i.e. image load is triggered when the top edge of the image placeholder, coincides with the bottom edge of the viewport.

The problem

Often, users scroll really fast through the page and the image will need some time to load and appear on the screen. In this scenario, combined with the fact that the load image event might be triggered with a delay because of throttling, you would often face the scenario where the placeholders come into the viewport, the user waits for a few milliseconds and then the image shows up. This delay makes for a poor user experience.

While using Intersection Observers to load the image or using low-quality image placeholders provides better loading performance and user experience, there is another simple trick that you can use to ensure that the images are always loaded completely when they enter the viewport – introduce a margin to the trigger point for images.

The solution

Instead of loading the image just when they exactly enter the viewport, load the images when they are, let’s say, 500px away from entering the viewport. This provides additional time, between the load trigger and the actual entry in the viewport, for the images to load.

With the Intersection Observer API, you can use the root parameter along with the rootMargin parameter (works as standard CSS margin rule), to increase the effective bounding box that is considered to find the “intersection”. With the event listener method, instead of checking for the difference between the image edge and the viewport edge to be 0, we can use a positive number to add some threshold.

The example here uses a 500px threshold to load images.

As evident from the video below (monitor the network requests closely appearing at the bottom), while scrolling, when the third image is in view, the 5th image gets loaded. When the 4th image comes into the view, the 6th image gets loaded. This way we are giving sufficient time for the images to load completely and in most cases, the user won’t see the placeholder at all.

And if you didn’t notice earlier, in all our examples, the third image (image3.jpg) is always loaded up front, even though it is outside the viewport. This was also done following the same principal – load slightly in advance instead of loading exactly at the threshold for better user experience.

3. Avoiding content shifting with lazy loading

This is another trivial point, which if solved, can help maintain a good user experience.

The problem

When there is no image, the browser doesn’t know the dimensions of the content that is to be displayed in the enclosing container. And if we do not specify it using CSS, the enclosing container would have no dimensions i.e. 0 x 0 pixels. Then, when the image gets loaded, the browser would resize the enclosing container to fit the image. This sudden change in the layout causes other elements to move around and it is called content shifting. As demonstrated in this content shifting article & video from Smashing Magazine, it is a rather unpleasant experience for a user, as the content moves suddenly when the image loads.

The solution

This can be avoided by specifying a height and/or width for your enclosing container so that the browser can paint the image container with a known height and width. Later, when the image loads, since the container size is already specified and the image fits into that perfectly, the rest of the content around that container does not move.

4. Do not lazy load all the images

This is another mistake that developers often commit – lazy load all the images on the page. This might reduce the initial page load, but would also result in bad user experience as a lot of images even at the top of the webpage won’t show up till the Javascript gets executed.

We can follow some general principles to identify which images should be lazy loaded.

a. Any image that is present in the viewport, or at the beginning of the webpage, should not be lazy loaded. This applies to any header image, marketing banner, logos etc. as the user should see them as soon as the page loads. Also, remember that mobile and desktop devices will have different screen sizes and hence a different number of images that will be visible on the screen initially. So you need to take into account the device type to decide which resources to load up front and which to lazy load.

b. Any image that is just slightly off the viewport should not be lazy loaded. This is going by the principle discussed above – load slightly in advance. So let’s say any image that is 500px or a single scroll from the bottom of the viewport can be loaded up front as well.

c. If the page isn’t lengthy, may be just a single scroll or a couple of scrolls, or if there are less than 5 images outside the viewport, then lazy loading can be avoided altogether. It would not provide any significant benefit to the end user in terms of performance. The additional JS that you load on the page to enable lazy loading will offset the gain from lazy loading such a small number of images.

Javascript dependency of Lazy Loading

The entire idea of lazy loading is dependent on the availability of Javascript execution capabilities in the user’s browser. While most of your users would have Javascript execution enabled in their browser, as it is essential for almost all websites these days, you may want to plan for users that do not allow javascript execution in the browser or use a browser that doesn’t support javascript at all.

You could either show them a message telling them why the images won’t load and that they need to switch to a modern browser or enable Javascript. Or you can use the noscript tag to create a usable experience for these users as well. Using the tag approach for such users has some gotchas. This question thread on Stack Overflow does a great job addressing these concerns and is a recommended read for anyone looking to address this set of users.

Popular Javascript libraries for lazy loading on your website
Since browser environments and implementation details can vary across browsers and devices, it is best to use a tried and tested library for lazy loading. Here is a list of popular libraries and platform specific plugins that will allow you to implement lazy loading with minimal effort

yall.js (Yet Another Lazy Loader) – Uses Intersection Observer and falls back to event-based lazy loading. Supports all major HTML element types but not background-images. Works on IE11+ as well.

lazysizes – Very popular and extensive functionality. Supports responsive images srcset and sizes attribute as well. High performance even without Intersection Observer.

jQuery Lazy – A simple, jquery based lazy loading library.

WeltPixel Lazy Loading Enhanced – A Magento 2 extension for lazy loading images

Magento Lazy Image Loader – A Magento 1.x extension for lazy loading images

Shopify Lazy Image Plugin – A Shopify extension for lazy loading images. It is paid though.

WordPress A3 Lazy Load – Image lazy loading plugin for WordPress

How to test if lazy loading is working?
Once you have implemented lazy loading, you would want to check if the behaviour of images on your website is as intended. The simplest way would be to open up developer tools in Chrome browser.

Go to Network Tab > Images. Here, when you refresh the page for the first time only the images that are to be loaded up front should get loaded. Then as you start scrolling down the page, other image load requests would get triggered and loaded. You can also notice the timings for image load in the waterfall column in this view. It would help you identify image loading issues if any or issues in triggering the image load.

Another way would be to run the Google Chrome Lighthouse audit report on your page after you have implemented the changes and look for suggestions under the “Offscreen images” section.

Conclusion

We have covered almost everything related to lazy loading images in this guide. Lazy loading if implemented correctly will significantly improve the loading performance of your web pages, reduce page size and delivery costs by cutting down on unnecessary resources loaded up front, while keeping the necessary content intact on the page. Your users would love it too when they get a great experience with the faster loading pages.

So, what are you waiting for? Get started with lazy loading images now!