DEV Community: Yishai Zehavi

Building Git Commit Linter Type in Typescript (Medium Difficulty)

Yishai Zehavi — Mon, 27 Nov 2023 13:43:02 +0000

Linting is the process of statically analyzing a piece of code and validating it against a set of rules.
Wouldn't it be cool to build a Git commit linter using Typescript types? Join me in this challenge, and let's build a linter together in Typescript!

In this challenge, we'll build a Typescript custom type to validate a commit message against a commit template and set of constraints.

Defining the challenge

Given three parameters:

A commit message.
A commit template.
A constraints object for the template's tokens.

Build a linter that validates the commit message against the template and returns either null if validation succeeded or an error message if validation failed.

Let's define these requirements in Typescript:

// Type definitions:
type CommitMessage = string;
type CommitTemplate = `${string}*${string}*${string}`;
type Constraints = Record<string, {
    required?: boolean;
    kind?: string|number; // union type
    minLength?: number;
    maxLength?: number;
}>

// The type we need to implement:
type CommitLinter<Message extends CommitMessage, Template extends CommitTemplate, C extends Constraints> = /* todo */;

Let's explain these types:

CommitTemplate should contain at least one token, where token is a placeholder for some text. To indicate that a string is a token we wrap it with asterisks.

Examples of commit templates:

type ValidTemplate<Template extends string> = Template extends CommitTemplate ? true : false;

type ValidTemplateResult1 = ValidTemplate<"[]">;
// => false

type ValidTemplateResult2 = ValidTemplate<"*message*">
// => true

type ValidTemplateResult3 = ValidTemplate<"*message* - <*author*>">;
// => true

Constraints is an object where each key corresponds to a token name in the template, and its value is an object with conditions that should apply to this token.

Now, we're ready to begin 💪

Stop here if you want to solve this challenge by yourself!

Thinking about a solution

Let's take a moment to think about how to approach this challenge. This is essentially a string matching problem: we need to match the parts of the template that are not tokens to the commit message in order. We also need to verify that each token in the commit message conforms to the constraints imposed by the Constraints object.

To help us complete these two tasks, we need a Token Parser.

What is a Token Parser?

A Parser is a tool (or a type, in our case) that takes a string and breaks it down into pieces called "tokens". Other tools can take these tokens and use them for various tasks (e.g., analyze the structure of the string). In our case, the parser will receive the commit template and return an array containing the token names (words wrapped in asterisks) along with the template parts preceding and following each token.

// Define toker parser signature:
type TokenParser<Template extends CommitTemplate> = /* todo */;

// Our parser should return these results:

type ParseResult1 = TokenParser<"*message*">;
// => [["message", ["", ""]]]

type ParseResult2 = TokenParser<"*message* - <*author*>">;
// => [["message", ["", " - <"]], ["author", ["", ">"]]]

type ParseResult3 = TokenParser<"[*type*] *message*">;
// => [["type", ["[", "] "]], ["message", ["", ""]]]

Don't worry if this looks a bit scary; let's break it down.

Our parser will return an array of TokenResults, where each TokenResult is a tuple containing two elements: token name and a sub-tuple containing the strings that surround the token:

type TokenResult = [TokenName, [Prefix, Suffix]]

// Where

type TokenName = string;
type Prefix = string;
type Suffix = string;

So, if the commit template looks like this:

"--*value*--"

The parser will return:

[                       <-- array of TokenResults
    [                   <-- TokenResult tuple
        "value",        <-- token name
        ["--", "--"]    <-- token's prefix and suffix
    ]
]

Now, let's analyze the examples from earlier:

TokenParser<"*message*"> // => ?

The token name is "message". No other strings appear before or after the token name, so the prefix and suffix are both empty strings:

TokenParser<"*message*"> // => [["message", ["", ""]]]

Continuing with the next example:

TokenParser<"*message* - <*author*>"> // => ?

In this example, we've got two tokens, so our parser will return two TokenResults.

The first token name is "message". There is no string before the token name, so its prefix is empty. What about the suffix? Well, we'll take the part of the template from the end of the token name until the beginning of the next token, and we get that the suffix is " - <".

The name of the second token is "author". What is the prefix of this token? The answer is nothing! Since we related the slice " - <" to the first token, no other string appears before this token's name, so the prefix is empty. The suffix will be the string following this token, ">".

In conclusion, we're getting the following answer:

TokenParser<"*message* - <*author*>">
// => [["message", ["", " - <"]], ["author", ["", ">"]]]

I'll leave the analysis of the last example for you as an exercise 😉

Phew, that was a lot of theory. Let's move to implementation.

Building the token parser

We'll start by defining the signature of the parser:

// `Tokens` parameter below is used to store the found tokens between calls.
type TokenParser<Template extends CommitTemplate, Tokens extends TokenResult[] = []> = /* todo */

Each TokenResult should contain the token name, its prefix, and suffix. So, we'll start by looking for the token name.
Remember, a token name is any string enclosed in asterisks:

type TokenParser<Template extends CommitTemplate, Tokens extends TokenResult[] = []> = 
    Template extends `${infer Prefix}*${infer Token}*${infer After}`
        ? /* todo */
        : never;

We split our Template type into three parts: first, the Prefix, which is the string slice that appears before the first asterisk in the template. After that comes the Token, which is the slice enclosed between the two asterisks. And last comes the rest of the string, which I named After. I didn't call it "Suffix" because this part of the string might contain another token (we'll check for that next).

Since the type CommitTemplate requires it to contain at least one token, We return never if it doesn't - to indicate that the provided template is invalid.

Extracting the token's suffix

We have the token's name and its prefix. All that is left is to extract its suffix from the After type. But how?

Let's think about it for a moment. Either the After type contains another token(s), or it doesn't. If it does, the string slice before that token is the suffix. If it doesn't - then the suffix is the entire string.

Let's begin by checking if the After type contains a token or not:

type TokenParser<Template extends CommitTemplate, Tokens extends TokenResult[] = []> = 
    Template extends `${infer Prefix}*${infer Token}*${infer After}`
        ? After extends `${infer Suffix}*${string}*${string}` // notice the asterisks!
            ? /* todo */
            : [...Tokens, [Token, [Prefix, After]]]
        : never;

We look for a token in the After type. If no token is found, we know that our suffix is the entire string. We append our token to the TokenResult array and return it.

If a token is found, we will extract the suffix from the string. To do that, we'll use a helper type, StripLeft. This type accepts two strings and "strips" the second string from the beginning of the first string:

type StripLeft<T extends string, U extends string> = T extends `${U}${infer V}` ? V : T;

Now we can implement the last branch of our TokenParser type:

type TokenParser<Template extends CommitTemplate, Tokens extends TokenResult[] = []> = 
    Template extends `${infer Prefix}*${infer Token}*${infer After}`
        ? After extends `${infer Suffix}*${string}*${string}` // notice the asterisks!
            ? TokenParser<StripLeft<After, Suffix>, [...Tokens, [Token, [Prefix, Suffix]]]>
            : [...Tokens, [Token, [Prefix, After]]]
        : never;

Let's run some tests:

type TokenParseResult1 = TokenParser<"*message*">;
// => [["message", ["", ""]]]

type TokenParseResult2 = TokenParser<"*message* - <*author*>">;
// => [["message", ["", " - <"]], ["author", ["", ">"]]]

type TokenParseResult3 = TokenParser<"[*type*] *message*">;
// => [["type", ["[", "] "]], ["message", ["", ""]]]

Great, our parser is ready! 🥳

Before constructing our linter, remember we mentioned having a Constraints type? We need a Validator type to validate our commit message tokens against these constraints. Let's build it next 💪

Enforcing constraints with a validator

Let's call this type ValidateValue and define its signature:

type ValidateValue<C extends Constraints, TokenName extends keyof C, Value extends string, Checks = C[TokenName]> = /* todo */

Our validator accepts a Constraints object, a token name, and the token's value in the commit message.

To better understand what Value represents, let's imagine our commit template is [*kind*] *message*, and the actual commit message is [feat] implemented sso login. For the first token, kind, its Value will be feat; for the second token, message, its Value will be implemented sso login.

Last, we have the Checks type, which is set to the specific constraints of TokenName.

We begin by looking for various constraints in the Checks type. If they match, we'll validate the provided Value against the appropriate validator:

type ValidateValue<C extends Constraints, TokenName extends keyof C, Value extends string, Checks = C[TokenName]> = 
    TokenName extends string
        ? Checks extends { required: boolean }
            ? /* todo: validate required value */
            : Checks extends { kind: string|number }
                ? /* todo: validate value's kind (i.e., enum type) */
                : Checks extends { minLength: number }
                    ? /* todo: validate value's min length */
                    : Checks extends { maxLength: number }
                        ? /* todo: validate value's max length */
                        : null // all checks pass
        : 'Invalid key';

First, we ensure that TokenName is a string (and not a symbol, for example). This will be useful when we need to print an error message.

Then, we compare the Checks object to each type of constraint. If it matches, we perform the check. The validator returns null if all the checks pass. Otherwise, it returns an error message.

We'll build a separate validator for each type of check, which we'll use in the respective branch in our ValidateValue type. We'll start with the RequiredValidator type:

type RequiredValidator<Value extends string, IsRequired extends boolean> = IsRequired | Value extends true | '' ? false : true;

This validator accepts a string value and an IsRequired boolean, denoting whether the value is required. The validator returns true only if the value is not empty or not required and false otherwise.

type RequiredValidatorResult1 = RequiredValidator<'', true>; // => false

type RequiredValidatorResult2 = RequiredValidator<'value', true>; // => true

type RequiredValidatorResult3 = RequiredValidator<'', false>; // => true

The next validator we'll look at is the KindValidator type:

type KindValidator<Value extends string, Options extends string|number> = Value extends Options ? true : false;

Options is a union that contains all the possible values for Value.

type KindValidatorResult1 = KindValidator<'debug', 'info'|'warning'|'error'>; // => false

type KindValidatorResult2 = KindValidator<'warning', 'info'|'warning'|'error'>; // => true

For the last two validators, MinLengthValidator and MaxLengthValidator, we'll first build a helper type, WordLength:

type WordLength<W extends string, Counter extends number[] = []> = W extends ''
    ? Counter['length']
    : W extends `${string}${infer T}`
        ? WordLength<T, [...Counter, 1]>
        : [...Counter, 1]['length'];

type WordLengthResult1 = WordLength<''>; // => 0
type WordLengthResult2 = WordLength<'typescript'>; // => 10

This type accepts a string and returns the length of that string.

Now, we're ready to build our last two validators. Let's start with MinLengthValidator:

type MinLengthValidator<Value extends string, Length extends number, Counter extends number[] = []> = Counter['length'] extends Length
    ? true
    : Counter['length'] extends WordLength<Value>
        ? false
        : MinLengthValidator<Value, Length, [...Counter, 1]>;

We increase the size of Counter every round until its length equals Length or the length of Value.

This validator returns true only if the counter length reaches the length of Value before it reaches Length or false otherwise.

type MinLengthResult1 = MinLengthValidator<'hello', 4>; // => true

type MinLengthResult2 = MinLengthValidator<'', 4>; // => false

type MinLengthResult3 = MinLengthValidator<'', 0>; // => true

For the last validator, MaxLengthValidator, instead of duplicating the code, we'll use MinLengthValidator and flip its result:

type MaxLengthValidator<Value extends string, Length extends number> = MinLengthValidator<Value, Length> extends true ? false : true;

Since we're reversing the result of MinLengthValidator there is an edge case we need to watch out for:

type MaxLengthResult = MaxLengthValidator<'hello', 5>; // => false???

Since the word "hello" is five letters long, MinLengthValidator has returned true - which our validator flipped, and we got false.

The solution is simple: first, check whether the length of Value equals Length. If it is - return true; otherwise, call MinLengthValidator and flip its result:

type MaxLengthValidator<Value extends string, Length extends number> = WordLength<Value> extends Length
    ? true
    : MinLengthValidator<Value, Length> extends true ? false : true;

type MaxLengthResult = MaxLengthValidator<'hello', 5>; // => true ✅

Now we can finish our ValidateValue type:

type ValidateValue<C extends Constraints, TokenName extends keyof C, Value extends string, Checks = C[TokenName]> = 
    TokenName extends string
        ? Checks extends { required: boolean }
            ? RequiredValidator<Value, Checks['required']> extends true
                ? ValidateValue<C, TokenName, Value, Omit<Checks, 'required'>>
                : `${TokenName} must not be empty`
            : Checks extends { kind: string|number }
                ? KindValidator<Value, Checks['kind']> extends true
                    ? ValidateValue<C, TokenName, Value, Omit<Checks, 'kind'>>
                    : `Invalid value "${Value}"`
                : Checks extends { minLength: number }
                    ? MinLengthValidator<Value, Checks['minLength']> extends true
                        ? ValidateValue<C, TokenName, Value, Omit<Checks, 'minLength'>>
                        : `${TokenName} should be at least ${Checks['minLength']} characters long.`
                    : Checks extends { maxLength: number }
                        ? MaxLengthValidator<Value, Checks['maxLength']> extends true
                            ? ValidateValue<C, TokenName, Value, Omit<Checks, 'maxLength'>>
                            : `${TokenName} cannot exceed ${Checks['maxLength']} characters.`
                        : null // all checks pass
        : 'Invalid key';

This code should make sense to you now. After performing a check, if the result is positive, we call ValidateValue again, removing this check from the Checks object. If the result is negative, we return immediately with an appropriate error message. Let's run some tests:

type DemoConstraints = {
    level: { required: true, kind: 'debug'|'info'|'warn'|'error' },
    message: { required: true, minLength: 10, maxLength: 30 }
}

type ValidationResult1 = ValidateValue<DemoConstraints, 'level', ''>;
// => "level must not be empty"

type ValidationResult2 = ValidateValue<DemoConstraints, 'level', 'critical'>;
// => "Invalid value 'critical'"

type ValidationResult3 = ValidateValue<DemoConstraints, 'message', 'test'>;
// => "message should be at least 10 characters long."

By the way, now you also understand why we had to ensure that TokenName is a string: we can only print it if it's a string or number, while the original type of it was string | number | symbol.

Assembling our linter

We have all the parts of our linter ready to be assembled. Let's start as usual by defining the signature of our linter:

type CommitLinter<Message extends CommitMessage, Template extends CommitTemplate, C extends Constraints, Tokens = TokenParser<Template>> = /* todo */;

Our linter inspects one token at a time and matches the beginning of Message to that token pattern. If all the tokens have been matched successfully (i.e., the Tokens array is empty), it returns a success value, null.

type CommitLinter<Message extends CommitMessage, Template extends CommitTemplate, C extends Constraints, Tokens = TokenParser<Template>> = 
    Tokens extends [infer Token, ...infer RestTokens]
        ? Token extends [infer TokenName extends string, [infer Prefix extends string, infer Suffix extends string]]
            ? /* todo */
            : 'Invalid token'
        : null;

We extract the first token's name, prefix, and suffix from the' Tokens' array. Then, we match Message to the pattern of this token:

type CommitLinter<Message extends CommitMessage, Template extends CommitTemplate, C extends Constraints, Tokens = TokenParser<Template>> = 
    Tokens extends [infer Token, ...infer RestTokens]
        ? Token extends [infer TokenName extends string, [infer Prefix extends string, infer Suffix extends string]]
            ? Message extends `${Prefix}${infer Value}${Suffix}${infer RestOfMessage}`
                ? /* todo */
                : 'Commit message does not match the template'
            : 'Invalid token'
        : null;

Notice the two infer keywords in the string we match to Message. This will become significant later on.

If the message does not match the token's pattern, we return an error message. If it does, we extract the token's value and the remaining of the commit message.

type CommitLinter<Message extends CommitMessage, Template extends CommitTemplate, C extends Constraints, Tokens = TokenParser<Template>> = 
    Tokens extends [infer Token, ...infer RestTokens]
        ? Token extends [infer TokenName extends string, [infer Prefix extends string, infer Suffix extends string]]
            ? Message extends `${Prefix}${infer Value}${Suffix}${infer RestOfMessage}`
                ? ValidateValue<C, TokenName, Value> extends `${infer Error}`
                    ? Error
                    : Linter<RestOfMessage, Template, C, RestTokens>
                : 'Commit message does not match the template'
            : 'Invalid token'
        : null;

We run the token's value through the ValidateValue validator. This validator returns either null or an error message. If it has produced an error message, we return it. Otherwise, we call Linter with the remainder of the commit message and the remainder of the tokens array.

It looks like our linter is ready, right? Let's test it:

type DemoConstraints = {
    change: { required: true, kind: 'feat'|'fix' },
    message: { required: true, minLength: 10, maxLength: 30 }
}

type CommitLinterResult1 = CommitLinter<'[feat] redesign login page.', '[*change*] *message*.', DemoConstraints>;
// => null

type CommitLinterResult2 = CommitLinter<'[foo] redesign login page.', '[*change*] *message*.', DemoConstraints>;
// => "Invalid value 'foo'"

type CommitLinterResult3 = CommitLinter<'redesign login page', '*message*', DemoConstraints>;
// => "message should be at least 10 characters long." ???

What happened in the third test? Why we're getting an error message? Our commit message is longer than ten characters!

Well, here's where the double infer keywords I mentioned earlier come into play 🤦

To understand the problem with two infer keywords, let's perform a little experiment:

type DoubleInfer<T extends string> = T extends `${infer Start}${infer End}` ? Start : never;

type Experiment1 = DoubleInfer<'abc'>; // => ['a', 'bc']
type Experiment2 = DoubleInfer<'a'>; // => ['a', '']

Where we match a string to a pattern that contains two or more infer keywords, Typescript tries to "slice" the string such that the inferred types will match one letter each, and the last inferred type will match the remainder of the string.

Now, let's pass our commit message to DoubleInfer and compare the result to the result of the match statement of our linter:

// Our experiment:
DoubleInfer<'redesign login page'>; // ['r', 'edesign login page']

// Our linter:
Message extends `${Prefix}${infer Value}${Suffix}${infer RestOfMessage}`
// => Prefix='', Value='r', Suffix='', RestOfMessage='edesign login page'

And now let's try to validate this Value:

ValidateValue<C, TokenName, Value /* only 'r'! */> extends `${infer Message}`
// => Validation error: "message should be at least 10 characters long."

Because the problem is rooted in the double infer keywords and only in cases where the entire string matches one token, the solution will be to match the entire string by using a single infer keyword.

Here's a suggested solution:

type CommitLinter<Message extends CommitMessage, Template extends CommitTemplate, C extends Constraints, Tokens = TokenParser<Template>> = 
    Tokens extends [infer Token, ...infer RestTokens]
        ? Token extends [infer TokenName extends string, [infer Prefix extends string, infer Suffix extends string]]
            ? Message extends `${Prefix}${infer Value}${Suffix}${infer RestOfMessage}`
                ? ValidateValue<C, TokenName, Value> extends `${infer Error}` // first validation
                    ? Message extends `${Prefix}${infer Value}${Suffix}`
                        ? ValidateValue<C, TokenName, Value> extends `${infer Error}` // second validation
                            ? Error
                            : CommitLinter<'', Template, C, RestTokens>
                        : Error
                    : CommitLinter<RestOfMessage, Template, C, RestTokens>
                : 'Commit message does not match the template'
            : 'Invalid token'
        : null;

Notice that if the first validation fails, we don't return the error message immediately; instead, we try to match the message to the token again, but this time, we remove the RestOfMessage inferred type, and we're left with only one inferred type which correctly matches to the entire message.

Let's run the previous tests again:

type DemoConstraints = {
    change: { required: true, kind: 'feat'|'fix' },
    message: { required: true, minLength: 10, maxLength: 30 }
}

type CommitLinterResult1 = CommitLinter<'[feat] redesign login page.', '[*change*] *message*.', DemoConstraints>;
// => null

type CommitLinterResult2 = CommitLinter<'[foo] redesign login page.', '[*change*] *message*.', DemoConstraints>;
// => "Invalid value 'foo'"

type CommitLinterResult3 = CommitLinter<'redesign login page', '*message*', DemoConstraints>;
// => null

This time, all the tests pass. 🥳

Congratulations, we've made it! All that's left is to incorporate that into Git hooks, but that's a topic for another post 😉

Thank you for reading this post. If you have any questions/suggestions/comments, please write them down in the comments. I'd love to hear your thoughts. 🤗

See you next article ✍️👋🏼

💻 7 Must-Know Tips for Responsive Images to Boost Your Site Score 🚀

Yishai Zehavi — Thu, 09 Nov 2023 15:35:10 +0000

We all use images on our websites, whether a hero image, our logo, or an icon on buttons and links. Using images and graphics is important, but we must be aware of some pitfalls when incorporating images into our websites.

Here's a list of must-know tips when using images in our code. It is not a comprehensive list, but a good starting point.

Let's jump right in.

🖼️ Choose the appropriate image type

When adding an image to a page, picking the appropriate type of image is essential. While no strict rules apply, here are some practical guidelines for choosing the proper image format:

Choose SVG format for logos, icons, decorations, or simple graphics. Its vector properties allow you to scale it up without damaging its quality, and its size is smaller than raster image formats like JPEG and PNG. It can also be manipulated and animated via CSS and JS and embedded directly into HTML documents.
GIF and WebP are the most popular choices for animated images (APNG is also a good pick).
Use WebP, AVIF, JPEG, or PNG for complex images and real-world photos. They are widely supported, have excellent compression algorithms, and support "alpha channel" transparency.

🖥️ Display different images according to screen size and resolution

Displaying a wide image on a page looks fine on desktop screens but might look too cramped when viewed on mobile screens.

Demo non-responsive website by Mozilla

Instead of providing a single image to all screen sizes, leverage the srcset and sizes attributes to offer multiple image sources and let the browser pick the most suitable one based on the user's device, screen size, and resolution.

Demo responsive website by Mozilla

🤌 Improve page load time by deferring image loading

Largest Contentful Paint (LCP) is a Core Web Vital performance metric. It measures the time it takes for the browser to render the most prominent content element (e.g., image, text block) visible within the viewport. When a page loads, all the content is downloaded and parsed, regardless of whether it's visible in the viewport. Images block the page render, so we want to delay image loading as late as possible.

A recommended best practice is to delay the loading of images that are not immediately visible to the user by setting the loading attribute of those images to lazy.

<img src="image.jpg" loading="lazy" alt="..." />

😵‍💫 Set `width` and `height` for images to avoid Cumulative Layout Shift

Cumulative Layout Shift (CLS) occurs when a page's content unexpectedly shifts due to dynamically added elements. This issue is often triggered by images or videos with unknown dimensions. After being loaded, these elements are inserted into the page, causing other elements around them to move to accommodate them.

To mitigate this issue and ensure a smoother user experience, set the width and height attributes on img tags to inform the browser ahead of time how much space it needs to reserve for each image.

<img src="image.jpg" width="640" height="360" alt="..." />

📟 Support old browsers by providing a fallback format

WebP and AVIF image formats are efficient and lightweight, making them excellent choices for web usage. However, they are not fully supported in all major browsers yet (WebP has better support than AVIF and JPEG XL).

Employ the picture and source tags to serve these formats to compatible browsers, and include an img tag with a more universally supported format as a fallback for older browsers.

<picture>
  <source type="image/webp" src="image.webp" /> <!-- Supported by browsers with WebP format compatibility -->
  <img src="image.jpg" alt="..."> <!-- Default fallback format for older browsers -->
</picture>

🦾 Utilize build tools to prepare your images for production

Incorporate build tools into your development workflow to automate image optimization for production. Optimizing your images ensures optimal rendering on various devices and saves user bandwidth.

There are variety of responsive image plugins like gulp-responsive for Gulp, image-minimizer-webpack-plugin for Webpack, or vite-imagetools for Vite.

💯 Verify best practices for images on your page

Proper usage of images on the web involves selecting image format, responsive design, and accessibility. Regularly audit your web pages to confirm you are following the best practices and providing users with a seamless experience.
After following the tips above, use a tool like Lighthouse or RespImageLint to test your pages for best practices. They give you insights about your page's performance and tips for improving it.

🔍 Bonus Tip: Set `Cache-Control` headers for your local images

When serving images from your server, set the Cache-Control header for your images. It'll save your users' bandwidth, reduce load on your server, and improve your site's loading time.

A good practice is to hash your image file names and set their Cache-Control header as follows:

Cache-Control: max-age=31536000, immutable

To learn more about caching your images and other assets check out my articles about caching and how to implement caching strategies in your code.

Implementing these seven tips in your code can significantly benefit your user's experience and your site's performance. Comment below if you have more tips and tricks you want to share.

Thanks for reading. See you in the next article! ✍️👋🏼

⏩ 1-minute concept: HTTP Cache-Control

Yishai Zehavi — Wed, 01 Nov 2023 14:26:34 +0000

HTTP Caching is an optional feature of the HTTP protocol that significantly reduces latency and network overhead by reusing response messages. It relies on a set of headers to dictate the behavior of local and intermediary cache stores.

As developers, we should utilize HTTP caching to enhance apps' and websites' performance and support a seamless user experience.

An example of configuring cache strategy for a route in Remix:

Check out my 2-part series here to learn more about the Cache-Control header and how to set it up in your application.

Cache-Control explained

Yishai Zehavi ・ Sep 12 '23

#http #webdev #cache

Implementing Cache-Control strategies in React, NodeJS, and Nginx

Yishai Zehavi — Wed, 20 Sep 2023 11:44:17 +0000

In part 1 of this series, we discussed the Cache-Control header and its directives. In this part of the series, we'll look at implementing cache control strategies in various React frameworks and on the server.

Remix

There are two ways in Remix to set headers depending on the type of the request. For page requests, you can export the headers hook from the route page:

    export const headers: HeadersFunction = () => ({
      "Cache-Control": "max-age=86400, s-maxage=604800, stale-while-revalidate=86400"
    });

For data requests, you can set headers directly on the response object:

export const action: ActionFunction = async ({ request }) => {
  const formData = await request.clone().formData();
  const data = await getData(formData);

  return json(
    data,
    {
      headers: {
        "Cache-Control": "max-age=0, must-revalidate, no-cache",
      },
    },
  );
};

Gatsby

Gatsby lets you configure headers in the gatsby-config.js file:

module.exports = {
  headers: [{
    source: "/about",
    headers: [
      {
        key: "cache-control",
        value: "max-age=86400",
      }
    ]
  }]
}

Gatsby also has an adapter for Netlify, gatsby-adapter-netlify, that takes care of setting up various configurations automatically, including HTTP headers.

For more information about Gatsby's recommended caching strategy, read here.

Express

Express's default Cache-Control header for static assets is: public, max-age=0.

You can change the default settings by passing an options block to the static middleware:

app.use(
  express.static("assets", {
    etag: true, // This is default
    immutable: false, // This is default
    lastModified: true, // This is default
    maxAge: 1000 * 60, // In MILLISECONDS, default is 0
    setHeaders: (response, path) => {
      // Set custom headers
    },
  }),
);

To set headers for a route request, use the setHeader method:

app.get("/", (req, res) => {
  const page = renderPage("Home");
  res.setHeader("Cache-Control", "max-age=86400, must-revalidate");
  res.send(page);
});

Nginx

If you use Nginx to serve static content, use the add_header directive to set headers:

server {
  listen 80;
  listen [::]:80;
  server_name mywebsite.com;
  root /etc/nginx/dist;
  index index.html;

  location / {
    add_header Cache-Control "max-age=86400";
  }

  location /images/ {
    add_header Cache-Control "max-age=31536000, immutable";
  }
}

If you use Nginx to serve dynamic content, you should configure Nginx's cache first:

proxy_cache_path /path/to/cache levels=1:2 keys_zone=choose_cache_name:10m max_size=4g inactive=180m use_temp_path=off;

Then, activate the cache:

proxy_cache our_cache_name;
proxy_cache_revalidate on;

proxy_cache instructs Nginx to use the specified cache for our request. proxy_cache_revalidate requires Nginx to revalidate its cache items with the origin server when they go stale. If we don't set this directive - Nginx will request a fresh response from the origin server whenever an item goes stale.

Here is a complete example of setting up a proxy cache:

proxy_cache_path /etc/nginx/cache levels=1:2 keys_zone=my_app_cache:10m max_size=4g inactive=180m use_temp_path=off;

server {
  listen 80;
  listen [::]:80;
  server_name dynamic_cache;

  proxy_cache my_app_cache;
  proxy_cache_revalidate on;

  location / {
    add_header X-Cache-Status $upstream_cache_status;
    proxy_pass http://my_app_upstream;
  }
}

Nginx cache status

Nginx provides the $upstream_cache_status embedded variable (via the ngx_http_upstream_module module). It holds the status of the cache response. The possible values are MISS, HIT, EXPIRED, or REVALIDATED. These values are explained in the following diagram.

Thanks for reading! See you in the next article. 👋🏼

Cache-Control explained

Yishai Zehavi — Tue, 12 Sep 2023 11:08:49 +0000

HTTP Caching is a mechanism that helps developers speed up web content loading. It uses a set of headers to instruct local and intermediate caches to store responses and serve them on subsequent requests without going to the origin server repeatedly. This post will focus on the Cache-Control header, one of the most essential headers associated with HTTP caching.

But first, why do we even need caching on the web?

Well, modern websites and web apps are heavy. They contain various assets: images, documents, audio files, CSS stylesheets, and client-side libraries. Frequent download of these files slows page loading, hurts SEO scoring, and increases the load on the origin server.

HTTP caching helps to mitigate these issues in two ways:

Caching resources saves the user a trip to the origin server, as the browser can serve those cached assets without needing to access the remote server.
Modern CDNs cache responses from remote servers on "edge servers" located close to the user's machine. This feature helps to reduce latency and improve page load times.

The Cache-Control Header

Cache-Control is a standard HTTP header that can appear on both the request and response objects. It contains one or more directives, separated by a comma. This header instructs the cache on how and when to store the response from the origin server and how to serve it for future requests.

Here's the structure of the Cache-Control header:

Cache-Control: directive, [directive, ...]

Before we dive into the technicality of cache control, let's review the terminology we'll use in this article:

Private vs Shared Cache: Private cache is the local cache on the user's machine. Shared cache, on the other hand, refers to intermediate caches (e.g., CDN, ISP cache, proxies, etc.).
Age: The time passed since a response was generated. It is measured in seconds.
Max Age: A measure that determines the "freshness" of a cache item. If the cache item's age is less than its max age, it is said to be "fresh". Otherwise, it is considered "stale".
Revalidation: When a cache item becomes stale, the cache will ask the origin server to confirm its validity before serving it to future users. This process is called "revalidation". It is done by sending a conditional request to the origin server requesting the most up-to-date version of the item. The origin server will either confirm the validity of the cached item or reply with the newer version.

Note: If the cache is disconnected from the origin server, it can return stale items until it reconnects.
Conditional Request: This type of request is sent during revalidation. It's a GET request that contains one or more validator headers. (usually, the If-Modified-Since or the If-None-Match header, in the case of cache revalidation).

The server will reply with a 304 Not Modified status if the version of the resource on the server is the same as the version provided in the request or with the new version otherwise.
Cache Busting: A technique where a static resource contains a unique hash/version in its filename. Whenever the content of the file changes, the unique hash/version in its name also changes. Using this technique "locks" the URL of a resource to a specific version of that resource (e.g., /static/react-16.0.3.js).

A common trap in HTTP caching is when the developer sets a long max-age for a file (we'll learn more about max-age in the next section), and during that time, he makes changes to the file, but since the old version in the cache is still fresh, users who request the file, get the old version from the cache.

The "Cache Busting" technique avoids this trap because when the file content changes - so does its URL. Users who want to get the newer version of the file - only need to request the new URL of the file. Since the cache associates each item to a URL and each file version has a "locked" URL - this guarantees that no URL will have two versions associated with it. Later, in the use cases section, we'll see other ways to handle this trap.

HTTP Cache-Control Flow

When a user requests a resource for the first time, the server responds with the resource and includes a Cache-Control header to indicate how and when to store the response and use it again. Intermediate caches, as well as the browser's local cache, store the response accordingly. The next time a user requests that resource, the browser will try to serve it from its local cache. If the resource has gone stale, the browser will dispatch a conditional request to intermediate caches. If their version has gone stale, they'll forward it to the remote server. The server will check the version of the resource provided in the request and compare it to its copy of the resource. If the version of the requested resource is the same as the server's version of that resource, it'll respond with a 304 Not Modified status, indicating to the caches that their version is still valid and they can turn it fresh again and use it. However, if the server has a newer version of the resource, it'll respond with a 200 OK status and the caches will update their copy.

Okay, now that the process of HTTP caching is clear, let's review the most common Cache-Control directives.

Directive	Description
`max-age`	The `max-age` directive instructs the cache for how long to keep the response fresh. The value for this directive is the time measured in seconds. After that period, the response will become stale, and the cache will try to revalidate the response with the origin server before using it again.
`s-maxage`	Similar to `max-age`, but shared caches (e.g., CDNs, proxy servers) will ignore `max-age` if this directive is also present. This directive can be used along with the `max-age` directive to give the response a different max age on shared vs. private caches.
`private`	This directive indicates that the response is intended for a single user (e.g., personal page) and can be stored only in private caches (i.e., browser cache).
`no-cache`	Allows caches to store the response, but they must revalidate it every time before using it. This has the benefit of always getting the most up-to-date response while not having to generate it every time (if the response has not been changed on the origin server, the cache will reply with its copy).
`no-store`	This directive prohibits caches from storing the response. They should forward the request to the origin server every time.
`must-revalidate`	This directive prohibits the caches from returning stale responses under any circumstances (see "revalidation"). If the origin server is unreachable, the cache will reply with a 504 Gateway Timeout status.
`immutable`	When a user clicks the reload button, the browser revalidates its local cache, even the fresh items. For a page containing many static files that do not change - this results in useless revalidation requests. To prevent that, add the `immutable` directive to instruct the cache never to revalidate these items while they're fresh. This directive is used when implementing the "Cache Busting" technique.
`stale-while-revalidate`	When a response becomes stale, the cache waits until the revalidation response returns from the origin server to resolve the request. This delay can negatively affect the user experience (especially if the `max-age` of the response is relatively short, leading to more frequent revalidations). This directive specifies a period of time after a response went stale, during which the cache is allowed to serve the stale response while revalidating it in the background for the next request.

Use Cases

Here, we'll look at common scenarios and what cache directives we should apply for each case.

Static Web Page Think of a web page showing similar content to all users, like a blog post. Such a page will not update frequently, and usually, when there is an update, it's okay to display an outdated version of the page for some time until the update propagates to all caches. We can take here the most naive approach:

  Cache-Control: max-age=604800;

We instruct the cache to store the page for a week (604800 seconds). If the page is updated during that time, the user will still see the old version until the cached page becomes stale, and then the cache will fetch the updated version from the origin server.

In this example, changes to the page will take up to a week to be shown to all users.

You can modify the number of seconds in the example above to cache the page for a shorter or longer time.

Sometimes, we want changes to propagate as soon as possible to the users. In this case, we can take a slightly different approach: We'll cache the page on the user's browser for a short time (e.g., 10 minutes) and cache it on the shared cache for a longer time (e.g., a week). When we want to update the page, we can deploy our change, clear the shared cache manually, and wait up to 10 minutes for the cached page on the user's browser cache to go stale. After 10 minutes, the users will get the most up-to-date version of our page from the origin server/shared cache.

  Cache-Control: max-age=600, s-maxage=604800, stale-while-revalidate=86400;

I added stale-while-revalidate to prevent users from "hanging" while the browser revalidates the page. See above for an explanation of this directive.

Real-Time Information Let's shift our focus to a news website or a weather report website. These websites update frequently and our users expect to see the most up-to-date information when they visit them. In this scenario, we can still cache the page, but we have to revalidate it every time with the server:

  Cache-Control: no-cache, max-age=0, must-revalidate;

The first directive allows the cache to store the page, but it has to revalidate the page every time with the origin server before serving it to users. The subsequent two directives are "fallback directives" for caches that do not support the first directive: We set the max age of the response to zero, and we tell the cache that it has to revalidate the response after it has gone stale (i.e., immediately after it was received).

Immutable Assets These assets include CSS stylesheets, JS libraries, images, audio and video files, documents, etc. Since these files are included in the page (e.g., via links) and not accessed directly by users, they should be versioned and immutable.

  Cache-Control: max-age=31536000, immutable;

We can cache these files for a year since their URL is fixed to their current version, and we add the immutable directive to prevent the browser from revalidating them.

FAQ

Here, I'll answer the most common questions about cache control:

What happens if no Cache-Control header is present on the response?
The behavior is unspecified, and the cache can decide whether to cache the response. To prevent caching of a response, specify it explicitly with:

Cache-Control: no-store.
What is the difference between no-store, no-cache, and must-revalidate?
no-store - don't save the response in the cache. The origin server needs to generate the response for every request.
no-cache - save the response in the cache, but revalidate it with the origin server for every incoming request.
must-revalidate - don't serve stale responses without revalidating them. If the origin server is not accessible, return a "504 Gateway Timeout" response instead.
What is the difference between the public and private, and when should I use each?
Both directives allow a cache to store a response, even if it's not cacheable by default. The private directive enables this behavior only to private caches (i.e., browser cache), and the public directive allows the response to be stored on private and shared caches.
I see a lot of examples where the public directive is used. Should I use it as well?
Probably no. The public directive allows shared caches to store responses that otherwise are not cacheable by shared caches. An example is a response to a request that contains the Authorization header. Such response is access controlled and thus not cacheable by shared caches by default.

Shared caches automatically cache a response if it contains the s-maxage or the must-revalidate directives.

From RFC9111 section 5.2.2.9:

The public response directive indicates that a cache may store the response even if it would otherwise be prohibited ... Note that it is unnecessary to add the public directive to a response that is already cacheable.

That's all for now. In the following article, we'll look at implementing the Cache-Control header in popular React frameworks and on the server.

Thanks for reading!

Follow for more content, see you in the next article 👋🏼

DEV Community: Yishai Zehavi

Building Git Commit Linter Type in Typescript (Medium Difficulty)

Defining the challenge

Thinking about a solution

What is a Token Parser?

Building the token parser

Extracting the token's suffix

Enforcing constraints with a validator

Assembling our linter

💻 7 Must-Know Tips for Responsive Images to Boost Your Site Score 🚀

🖼️ Choose the appropriate image type

🖥️ Display different images according to screen size and resolution

🤌 Improve page load time by deferring image loading

😵‍💫 Set width and height for images to avoid Cumulative Layout Shift

📟 Support old browsers by providing a fallback format

🦾 Utilize build tools to prepare your images for production

💯 Verify best practices for images on your page

🔍 Bonus Tip: Set Cache-Control headers for your local images

⏩ 1-minute concept: HTTP Cache-Control

Cache-Control explained

Yishai Zehavi ・ Sep 12 '23

Implementing Cache-Control strategies in React, NodeJS, and Nginx

Remix

Gatsby

Express

Nginx

Nginx cache status

Cache-Control explained

The Cache-Control Header

HTTP Cache-Control Flow

Use Cases

FAQ

😵‍💫 Set `width` and `height` for images to avoid Cumulative Layout Shift

🔍 Bonus Tip: Set `Cache-Control` headers for your local images