Less load on the server and less bandwidth usage for the same result? Where should I sign up? Nowhere, you just need to know the right headers.
Let's keep it simple - NodeJS, no dependencies. Build with me some endpoints, each using different headers, and find out how the browser behaves based on the headers received.
Go directly to the /no-headers endpoint or take a (very quick) look at the easiest server there is.
index.mj
import { createServer } from "http";
import noHeaders from "./src/index.mjs";
createServer((req, res) => {
switch (req.url) {
case "/no-headers":
return noHeaders(req, res);
}
}).listen(8000, "127.0.0.1", () =>
console.info("Exposed on http://127.0.0.1:8000")
);
src/utils.mjs
import fs from "fs/promises";
import path from "path";
export function to(promise) {
return promise.then((res) => [res, null]).catch((err) => [null, err]);
}
export async function getView(name) {
const filepath = path.resolve(
process.cwd(),
"src",
"views",
name + ".html"
);
return await to(fs.readFile(filepath, "utf-8"));
}
export async function getViewStats(name) {
const filepath = path.resolve(process.cwd(), "src", "views", name + ".html");
return await to(fs.stat(filepath));
}
Add an HTML file at src/views/index.html
. Its content is irrelevant.
No Headers - Endpoint
It simply reads the file and sends it to the requester. Apart from the Content-Type
, no caching-related header is added.
// src/no-headers.mjs
import { getView } from "./utils.mjs";
export default async (req, res) => {
res.setHeader("Content-Type", "text/html");
const [html, err] = await getView("index");
if (err) {
res.writeHead(500).end("Internal Server Error");
return;
}
res.writeHead(200).end(html);
};
Start the server (node index.mjs
), open /no-headers
, and check the developer tools > network tab. Enable preserver log and hit refresh a few times.
Open any of them, and check the Request Headers
- there is nothing related to caching, and the browser obeys.
HTTP/1.1 200 OK
Content-Type: text/html
Date: <date>
Connection: keep-alive
Keep-Alive: timeout=5
Transfer-Encoding: chunked
Last-Modified - Endpoint
Create a new endpoint (to be registered at the url /last-modified
). It reads the modification time of the file (mtime
) and adds it formatted as UTC under the Last-Modified
header.
// src/last-modified.mjs
import { getView, getViewStats } from "./utils.mjs";
export default async (req, res) => {
res.setHeader("Content-Type", "text/html");
const [stats, errStats] = await getViewStats("index");
if (errStats) {
res.writeHead(500).end("Internal Server Error");
return;
}
const lastModified = new Date(stats.mtime);
res.setHeader("Last-Modified", lastModified.toUTCString());
const [html, errGet] = await getView("index");
if (errGet) {
res.writeHead(500).end("Internal Server Error");
return;
}
res.writeHead(200).end(html);
};
In fact, among the response headers to /last-modified
, you find:
HTTP/1.1 200 OK
Last-Modified: Thu, 15 Nov 2023 19:18:46 GMT
Anyway, if you refresh the page, the entire resource is still downloaded.
Yet something changed - the browser found Last-Modified
, so it reuses the value for the If-Modified-Since
Request Header. The serve receives that value and, if the condition is found to be not true (not modified since), returns the status 304 Not Modified.
import { getView, getViewStats } from "./utils.mjs";
export default async (req, res) => {
res.setHeader("Content-Type", "text/html");
const [stats, _] = await getViewStats("index");
const lastModified = new Date(stats.mtime);
lastModified.setMilliseconds(0); // IMPORTANT
res.setHeader("Last-Modified", lastModified.toUTCString());
const ifModifiedSince = new Headers(req.headers).get("If-Modified-Since");
if (
ifModifiedSince &&
new Date(ifModifiedSince).getTime() >= lastModified.getTime()
) {
res.writeHead(304).end();
return;
}
// This is done ONLY IF it was not a 304!
const [html, _] = await getView("index");
res.writeHead(200, headers).end(html);
};
By spec Last-Modified
Note:
- The Response Header
Last-Modified
is always added, even in the case of304 Not Modified
. - The Request Header
if-modified-since
may not be present - definitely happens on the first call from a new client.
Most importantly, HTTP dates are always expressed in GMT, never in local time.
While formatting a date using toUTCString
, you may observe that the resulting string loses information about milliseconds. However mtime
retains millisecond precision - it may have a few milliseconds more than the value received from the client, which, after formatting, loses those milliseconds.
To ensure a valid comparison between the two values, it becomes necessary to remove the milliseconds from the mtime
before performing the comparison.
lastModified.setMilliseconds(0);
Finally, request the resource few times.
Now, just go and update the HTML file. Then ask the browser to refresh and expect to receive a 200 OK
Response.
It's essential to recognize that the 304 response is consistently more lightweight than the 200 response. Beyond just the reduced data payload, it contributes to a decrease in server load. This optimization extends beyond mere HTML file reads and can apply to any intricate or resource-intensive operation.
Last-Modified
is a weak caching header, as the browser applies a heuristic to determine whether to fetch the item from the cache or not. Heuristics vary between browsers.
Top comments (2)
There is a mistake in this line :
"Open any of them, and check the Request Headers - there is nothing related to caching, and the browser obeys."
You mean't Response Headers not Request Headers
:)
Very well explained, keep up the good work :)