DEV Community: Adarsh Madrecha

Caching S3 Signed URL Images in the Browser Using the Cache API

Adarsh Madrecha — Mon, 17 Mar 2025 18:14:07 +0000

You have images served via signed URLs from Amazon S3, and you want to reduce network costs by caching these images in the browser using the Cache API, specifically with methods like Cache.add() or Cache.put(). Signed URLs provide temporary access to private S3 objects and typically include query parameters like signatures and expiration times, which may change even for the same image. This tutorial will guide you through implementing a caching solution that accounts for these challenges, ensuring efficient caching and cost savings.

The key idea is to cache images based on their content rather than their potentially changing URLs. We'll use a service worker to intercept image requests, generate a consistent cache key using the image's S3 object path and ETag (a unique identifier for the content), and store the responses in the browser's Cache Storage. We'll also modify how signed URLs are generated on the server to include the ETag, enabling the client to identify the image content reliably.

Below is a detailed, step-by-step tutorial to achieve this.

Prerequisites

Basic knowledge of JavaScript, HTML, and web development.
An S3 bucket with images and the ability to generate signed URLs (e.g., via AWS SDK).
A web application where images are loaded via <img> tags or similar.
Node.js and AWS SDK installed on your server (if you're generating signed URLs server-side).

Step 1: Understanding the Problem

Signed URLs from S3 typically look like this:

https://your-bucket.s3.amazonaws.com/path/to/image.jpg?AWSAccessKeyId=...&Signature=...&Expires=...

Challenge: The query parameters (e.g., Signature, Expires) may change each time a signed URL is generated, even for the same image. The Cache API keys entries by the full URL, so a new URL means a cache miss, leading to repeated S3 fetches and increased costs.
Goal: Cache the image based on its content, not its transient URL, so that subsequent requests—regardless of URL changes—use the cached version unless the image itself is updated.
Solution: Use the S3 object’s ETag (a hash of the content) as part of a custom cache key, combined with the URL path. The ETag changes only when the image content changes, making it ideal for content-based caching.

Step 2: Modify Server-Side Signed URL Generation

To enable content-based caching, include the image’s ETag in the signed URL as a query parameter. The ETag is a unique identifier provided by S3 for each object version. Here’s how to do it using Node.js and the AWS SDK v3.

Server-Side Code (Node.js)

Create a function to generate a signed URL with the ETag appended:

const { S3Client, HeadObjectCommand, GetObjectCommand } = require("@aws-sdk/client-s3");
const { getSignedUrl } = require("@aws-sdk/s3-request-presigner");

const s3Client = new S3Client({ region: "your-region" }); // Replace with your AWS region

async function getSignedUrlWithEtag(bucket, key) {
  try {
    // Step 1: Get the ETag of the object
    const headCommand = new HeadObjectCommand({ Bucket: bucket, Key: key });
    const headResponse = await s3Client.send(headCommand);
    const etag = headResponse.ETag.replace(/"/g, ""); // Remove quotes from ETag (e.g., "abc123" -> abc123)

    // Step 2: Generate the signed URL
    const getCommand = new GetObjectCommand({ Bucket: bucket, Key: key });
    const signedUrl = await getSignedUrl(s3Client, getCommand, {
      expiresIn: 3600, // URL valid for 1 hour
    });

    // Step 3: Append the ETag as a query parameter
    const signedUrlWithEtag = signedUrl + "&etag=" + encodeURIComponent(etag);
    return signedUrlWithEtag;
  } catch (error) {
    console.error("Error generating signed URL:", error);
    throw error;
  }
}

// Example usage
(async () => {
  const url = await getSignedUrlWithEtag("your-bucket", "path/to/image.jpg");
  console.log("Signed URL with ETag:", url);
})();

Explanation

HeadObjectCommand: Retrieves metadata (including the ETag) for the S3 object without downloading it.
getSignedUrl: Generates a standard signed URL with an expiration time (e.g., 1 hour).
Appending ETag: The ETag is added as &etag=<value> after the signed URL is generated. S3 ignores extra query parameters for GET requests, so this doesn’t affect URL validity.
Output: A URL like https://your-bucket.s3.amazonaws.com/path/to/image.jpg?AWSAccessKeyId=...&Signature=...&Expires=...&etag=abc123.

Use this URL in your web application (e.g., in <img src="..."> tags) to load images.

Step 3: Set Up the Client-Side Web Application

In your web app, register a service worker to manage caching. The service worker will intercept image requests and use the Cache API.

Main JavaScript File (e.g., `app.js`)

Add this code to your main JavaScript file to register the service worker:

if ("serviceWorker" in navigator) {
  window.addEventListener("load", () => {
    navigator.serviceWorker
      .register("/service-worker.js")
      .then((registration) => {
        console.log("Service Worker registered with scope:", registration.scope);
      })
      .catch((error) => {
        console.error("Service Worker registration failed:", error);
      });
  });
}

HTML Example

Ensure your images use the signed URLs with ETags:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Caching S3 Images</title>
</head>
<body>
  <img src="https://your-bucket.s3.amazonaws.com/path/to/image.jpg?AWSAccessKeyId=...&Signature=...&etag=abc123" alt="Cached Image">
  <script src="app.js"></script>
</body>
</html>

Step 4: Implement the Service Worker

Create a file named service-worker.js in your project’s root directory. This script will intercept image requests, create a custom cache key, and manage caching.

Service Worker Code (`service-worker.js`)

self.addEventListener("fetch", (event) => {
  // Check if the request is for an image
  if (event.request.destination === "image") {
    const originalUrl = new URL(event.request.url);
    const etag = originalUrl.searchParams.get("etag");

    if (etag) {
      // Create a custom cache key: origin + pathname + ?etag=<value>
      const cacheKeyUrl = originalUrl.origin + originalUrl.pathname + "?etag=" + etag;
      const cacheKeyRequest = new Request(cacheKeyUrl);

      event.respondWith(
        caches.match(cacheKeyRequest).then((cachedResponse) => {
          if (cachedResponse) {
            // Serve from cache if found
            return cachedResponse;
          }
          // Fetch from network, cache it, then serve
          return fetch(event.request).then((response) => {
            return caches.open("image-cache").then((cache) => {
              cache.put(cacheKeyRequest, response.clone());
              return response;
            });
          });
        })
      );
    } else {
      // If no ETag, fetch normally without caching
      event.respondWith(fetch(event.request));
    }
  }
});

Explanation

Event Listener: Listens for fetch events, which occur for every resource request.
Image Check: event.request.destination === "image" ensures we only process image requests (e.g., from <img> tags).
ETag Extraction: Parses the request URL to get the etag query parameter.
Cache Key: Constructs a custom key (origin + pathname + ?etag=<etag>) that identifies the image by its path and content (ETag), ignoring signature parameters.
caches.match: Checks if the cache has an entry for this key.
- Hit: Returns the cached response.
- Miss: Fetches the image using the original signed URL, caches it with cache.put(), and returns the response.
Fallback: If no ETag is present, fetches the image without caching.

Step 5: Test the Implementation

Deploy Your App: Serve your web app via a local server (e.g., npx serve) or a production server with HTTPS (required for service workers outside localhost).
Generate a Signed URL: Use the server-side function to get a URL with an ETag and load it in an <img> tag.
Inspect Cache:
- Open your browser’s DevTools (e.g., Chrome).
- Go to the “Application” tab > “Cache Storage” > “image-cache”.
- Verify that cached entries use URLs like https://your-bucket.s3.amazonaws.com/path/to/image.jpg?etag=abc123.
Simulate Requests:
- Reload the page within the signed URL’s expiration time. Check the “Network” tab; the image should load from the service worker cache.
- Update the image on S3, generate a new signed URL with a different ETag, and reload. The new image should be fetched and cached.

How It Saves Network Costs

Initial Fetch: The first request fetches the image from S3 and caches it locally using the custom key (path + ETag).
Subsequent Requests: If the signed URL changes (e.g., new signature) but the ETag remains the same, the service worker serves the cached image, avoiding an S3 fetch.
Content Updates: When the image is updated on S3, the ETag changes, triggering a new fetch and cache update, ensuring freshness.
Result: Fewer S3 data transfer requests, reducing costs, especially for frequently accessed images.

Security and Considerations

Expiration: Cached images persist beyond the signed URL’s expiration. This is acceptable if users retaining local copies aligns with your security policy (since they already had access). For sensitive images, consider shorter cache durations or additional controls.
Cache Management: The cache grows with each unique image. Implement cache eviction (e.g., delete old entries) if storage becomes a concern.
Browser Support: The Cache API and Service Workers are widely supported in modern browsers (Chrome, Firefox, Safari, Edge).

Optional Enhancements

Cache Expiration: Add a max-age policy by checking timestamps or adding a custom header during cache.put().
Pre-Caching: Use the service worker’s install event to pre-cache known images.
Error Handling: Add try-catch blocks in the service worker to handle fetch failures gracefully.

This tutorial provides a robust solution to cache S3 signed URL images in the browser, minimizing network costs while handling the complexities of changing URLs and content updates. Implement it, test thoroughly, and adjust based on your specific needs!

Getting Started with Graphology

Adarsh Madrecha — Mon, 17 Mar 2025 18:13:05 +0000

Key Points

Graphology is a JavaScript library for creating and manipulating graph data structures, supporting various graph types like directed, undirected, or mixed, with or without self-loops and parallel edges.
It seems likely that you can install it via npm with npm install graphology, and it includes a standard library for additional graph algorithms.
The evidence leans toward it being used with sigma.js for interactive graph rendering, and it supports TypeScript with graphology-types.

Getting Started with Graphology

What is Graphology?

Graphology is a robust and versatile library for working with graphs in JavaScript and TypeScript. It allows you to create graphs that can be directed, undirected, or mixed, and supports features like self-loops and parallel edges. It’s often used with sigma.js for visualizing graphs interactively, making it great for network analysis or data visualization projects.

Installation

To get started, install Graphology using npm:

npm install graphology

For additional graph algorithms, install the standard library:

npm install graphology-library

If you’re using TypeScript, you might need to install type declarations:

npm install graphology-types

You can also use standalone builds from the releases page on GitHub (releases) if you prefer script tags.

Creating and Using a Graph

Here’s how to create a basic graph and add nodes and edges:

import Graph from 'graphology';

const graph = new Graph();
graph.addNode('node1', { label: 'Node One', color: 'blue' });
graph.addNode('node2', { label: 'Node Two', color: 'red' });
graph.addEdge('node1', 'node2', { weight: 5 });

You can check the number of nodes and edges:

console.log('Nodes:', graph.order); // Number of nodes
console.log('Edges:', graph.size); // Number of edges

Graphology also lets you set and get attributes for nodes and edges, like size or color, and traverse the graph to find connected nodes.

Unexpected Detail: Standard Library Features

Beyond the basics, Graphology’s standard library offers advanced features like graph density calculations, layout algorithms (e.g., ForceAtlas2), and community detection, which might be useful for more complex projects. For example, calculate density:

import * as metrics from 'graphology-library/metrics';
const density = metrics.graph.density(graph);

Resources for Further Learning

For more details, check the official website (Graphology) and the standard library documentation (standard-library).

Comprehensive Survey Note on Using Graphology

This survey note provides an in-depth exploration of Graphology, a robust and multipurpose Graph object for JavaScript and TypeScript, designed for creating and manipulating graph data structures. It covers installation, usage, and advanced features, drawing from official sources and examples to ensure a thorough understanding. The note is structured to mimic a professional article, offering detailed guidance for developers at all levels.

Introduction to Graphology

Graphology is a versatile library aimed at supporting various kinds of graphs under a unified interface. It can handle directed, undirected, or mixed graphs, with options for self-loops and parallel edges. This flexibility makes it suitable for a wide range of applications, from network analysis to interactive visualizations. Notably, it is used as the data backend for sigma.js, a WebGL-based graph renderer, enhancing its utility for browser-based interactive applications.

The library also includes a comprehensive standard library, offering graph theory algorithms, utilities, generators, layouts, and traversals. This makes Graphology not just a data structure but a full ecosystem for graph-related tasks. Additionally, Graphology graphs emit various events, which are ideal for building dynamic, interactive renderers.

Installation and Setup

To begin using Graphology, installation is straightforward via npm. The primary command is:

npm install graphology

For access to additional algorithms and utilities, install the standard library:

npm install graphology-library

For TypeScript users, ensure compatibility by installing type declarations if needed:

npm install graphology-types

For projects that cannot use npm, standalone builds are available in the repository’s releases (releases). These can be loaded via script tags, with graphology.min.js exposing the graphology global and graphology-library.min.js exposing graphologyLibrary.

Creating and Managing Graphs

Instantiating a Graph

To create a new graph, import the Graph class and instantiate it:

import Graph from 'graphology';

const graph = new Graph();

This creates a default undirected, simple graph. You can specify options like type (e.g., 'directed', 'undirected', 'mixed') and allowSelfLoops when creating the graph for more control.

Adding Nodes and Edges

Nodes and edges form the backbone of any graph. Add a node with:

graph.addNode('node1'); // Basic node
graph.addNode('node2', { label: 'Node Two', color: 'red', size: 10 }); // Node with attributes

Edges connect nodes, and can also have attributes:

graph.addEdge('node1', 'node2'); // Basic edge
graph.addEdge('node1', 'node2', { weight: 5, color: 'blue' }); // Edge with attributes

Attributes like label, color, and size are commonly used for visualization, especially when integrating with sigma.js.

Accessing Graph Properties

Graphology provides methods to query the graph’s structure. Get the number of nodes and edges:

console.log('Number of nodes:', graph.order);
console.log('Number of edges:', graph.size);

List all nodes and edges:

graph.nodes().forEach(node => console.log(node));
graph.edges().forEach(edge => console.log(edge));

These methods are essential for understanding the graph’s scale and structure.

Working with Node and Edge Attributes

Attributes enhance the graph’s data, allowing for richer representations. Set and get node attributes as follows:

graph.setNodeAttribute('node1', 'size', 10);
const size = graph.getNodeAttribute('node1', 'size');

Similarly, for edges:

graph.setEdgeAttribute('edge_id', 'weight', 3);
const weight = graph.getEdgeAttribute('edge_id', 'weight');

These attributes are crucial for visualizations, where properties like color or size can represent data like node importance or edge strength.

Traversing the Graph

Traversal is key for analyzing graph connectivity. For undirected graphs, use neighbors to find all connected nodes:

const neighbors = graph.neighbors('node1');

For directed graphs, Graphology offers outNeighbors for nodes the given node points to, and inNeighbors for nodes pointing to it:

const outNeighbors = graph.outNeighbors('node1');
const inNeighbors = graph.inNeighbors('node1');

These methods are particularly useful in directed graphs, such as social networks or dependency graphs, where direction matters.

Leveraging the Standard Library

Graphology’s standard library extends its functionality with various packages, each serving specific graph-related tasks. Below is a table summarizing key packages and their purposes, based on official documentation:

Package Name	Description
assertions	Miscellaneous assertions (same nodes, same edges etc.).
bipartite	Bipartite graph helper functions (coloring, projection etc.).
canvas	Canvas rendering routines for graphs.
communities-louvain	Louvain method for community detection.
components	Connected components (strong, weak etc.).
dag	Functions related to directed acyclic graphs (cycle detection, topological sorting etc.).
cores	Various utilities related to k-cores.
generators	Graph generators (random graphs, complete graphs etc.).
gexf	Parsers & writers for the GEXF file format.
graphml	Parsers & writers for the GRAPHML file format.
indices	Various specialized graph indices (neighborhood, louvain etc.)
layout	Basic graph layouts (random, circle etc.).
layout-force	Basic force layout algorithm.
layout-forceatlas2	ForceAtlas2 layout algorithm.
layout-noverlap	Noverlap anti-collision layout algorithm.
metrics	Modularity, density, centrality etc.
operators	Graph unary, binary & cast operators (reverse, union, intersection, conversion etc.)
shortest-path	Shortest path functions (Dijkstra, A* etc.).
simple-path	Simple path related functions (e.g. all paths between source & target)
svg	SVG export for graphs.
traversal	Traversal functions (DFS, BFS, etc.).
utils	Miscellaneous utils used by most of the other modules.

For example, to calculate graph density:

import * as metrics from 'graphology-library/metrics';
const density = metrics.graph.density(graph);

The standard library is accessible via graphology-library, with browser-specific versions available for certain packages like XML parsers and layouts with web workers. For interactive rendering, consider sigma.js (sigma.js).

Handling Events

Graphology graphs emit events, which are useful for building interactive applications. Listen to events like node addition:

graph.on('nodeAdded', (node, attributes) => {
  console.log(`Node ${node} added with attributes:`, attributes);
});

Other events include edge addition, node removal, and more, making it ideal for real-time updates in visualizations.

Advanced Considerations and Resources

While the core Graph class provides essential functionality, Graphology’s design allows for re-implementation for specialized needs, retaining the ability to use the standard library (implementing-graphology). This flexibility is particularly useful for performance optimization in specific use cases.

For further learning, the official website (Graphology) offers insights into design choices and implementation details. The changelog is available at (changelog), and citation information is provided for academic use, published on Zenodo with DOI (Zenodo DOI).

Conclusion

This essentials of getting started with Graphology, from installation to advanced usage with the standard library. It provides a foundation to build and manipulate graph data structures, with resources for deeper exploration. For those interested in visualization, integrating with sigma.js offers powerful rendering capabilities, enhancing the interactive experience.

Key Citations

Can I share GitHub copilot like Netflix account?

Adarsh Madrecha — Wed, 20 Jul 2022 11:53:24 +0000

Let's be honest, we all at some point in time have shared the Netflix account with Family or Friends.

So, I am wondering if the same can be done with GitHub Copilot too?

Use case: For a hobby project with my two friends, it may turn out to be helpful.

DEV Community: Adarsh Madrecha

Caching S3 Signed URL Images in the Browser Using the Cache API

Prerequisites

Step 1: Understanding the Problem

Step 2: Modify Server-Side Signed URL Generation

Server-Side Code (Node.js)

Explanation

Step 3: Set Up the Client-Side Web Application

Main JavaScript File (e.g., app.js)

HTML Example

Step 4: Implement the Service Worker

Service Worker Code (service-worker.js)

Explanation

Step 5: Test the Implementation

How It Saves Network Costs

Security and Considerations

Optional Enhancements

Getting Started with Graphology

Key Points

Getting Started with Graphology

Comprehensive Survey Note on Using Graphology

Introduction to Graphology

Installation and Setup

Creating and Managing Graphs

Instantiating a Graph

Adding Nodes and Edges

Accessing Graph Properties

Working with Node and Edge Attributes

Traversing the Graph

Leveraging the Standard Library

Handling Events

Advanced Considerations and Resources

Conclusion

Key Citations

Can I share GitHub copilot like Netflix account?

Main JavaScript File (e.g., `app.js`)

Service Worker Code (`service-worker.js`)