tq-bit

Posted on Aug 20, 2021 • Updated on Dec 2, 2022

How to monitor the progress of a Javascript fetch - request and cancel it on demand.

#javascript #webdev #http

TL: DR -> Take me to the code: https://github.com/tq-bit/fetch-progress

In an earlier post, I've already given an overview of how to interact with an API using fetch. In this article, I'd like to dig deeper into two more detailed use-cases:

Monitor the download progress while making an HTTP request.
Gracefully cancel a request by a user's input.

If you would like to follow along, you can use this Github branch to get started. It includes no Javascript, just some styles and HTML: https://github.com/tq-bit/fetch-progress/tree/get-started.

This is the UI we will start off with. The progress indicator will visualize the fetch - progress

So spin up your favorite code editor and let's dive in.

Create the basic fetch request

Before starting with the advanced stuff, let's build up a simple function. The task is to develop a piece of utility code that allows you to search for universities. Fortunately, Hipo has just the tool to build up upon.

I'm using this repository's hosted API as a starting place.
Its root URL is http://universities.hipolabs.com/.
I'd like to restrict my search to all universities in the USA with a query.
On the technical side, I'd like to keep my fetch logic inside a wrapper function.

That being said, let's start by adding the following code to the client.js file:

export default function http(rootUrl) {
  let loading = false;

  let chunks = [];
  let results = null;
  let error = null;


  // let controller = null; // We will get to this variable in a second

  const json = async (path, options,) => {
    loading = true

    try {
      const response = await fetch(rootUrl + path, { ...options });

      if (response.status >= 200 && response.status < 300) {
        results = await response.json();
        return results
      } else {
        throw new Error(response.statusText)
      }
    } catch (err) {
      error = err
      results = null
      return error
    } finally {
      loading = false
    }
  }

  return { json }
}

Next, let's import this function into the main.js file and initialize it:

// Import the fetch client and initalize it
import http from './client.js';
const { json } = http('http://universities.hipolabs.com/');

// Grab the DOM elements
const progressbutton = document.getElementById('fetch-button');

// Bind the fetch function to the button's click event
progressbutton.addEventListener('click', async () => {
  const universities = await json('search?country=United+States');
  console.log(universities);
});

Clicking on the Fetch - button will now print us the requested universities to our console:

Rebuild the .json() - method

To monitor progress, we need to rebuild a good part of the standard .json() method. It also implicates that we will also have to take care of assembling the response body, chunk by chunk.

I've written an article about handling Node.js streams earlier. The approach shown here is quite similar.

So let's add the following to the client.js file, right below the json function:

export default function http(rootUrl) { 

  // ... previous functions
  const _readBody = async (response) => {
    const reader = response.body.getReader();

    // Declare received as 0 initially
    let received = 0;

    // Loop through the response stream and extract data chunks
    while (loading) {
      const { done, value } = await reader.read();
      if (done) {
        // Finish loading 
        loading = false;
      } else {
        // Push values to the chunk array
        chunks.push(value);
      }
    }

    // Concat the chinks into a single array
    let body = new Uint8Array(received);
    let position = 0;

    // Order the chunks by their respective position
    for (let chunk of chunks) {
      body.set(chunk, position);
      position += chunk.length;
    }

    // Decode the response and return it
    return new TextDecoder('utf-8').decode(body);
  }
  return { json }
}

Next, let's replace response.json() as follows:

  // results = response.json();
  // return results;
  results = await _readBody(response)
  return JSON.parse(results)

The response in the browser is still the same as previously - a decoded JSON object. As the response's body itself is a readable stream, we can now monitor whenever a new piece of data is being read or whether the stream is closed yet.

Get the maximum and current data length

The two core numbers for progress monitoring are found here:

The content-length header from the response, the variable length.
The cumulated length of the received data chunks, variable received.

Note that this function does not work if the content-length header is not configured on the serverside.

As we already have the variable received available, let's add content-length to our _readBody function:

  const _readBody = async (response) => {
    const reader = response.body.getReader();

    // This header must be configured serverside
    const length = +response.headers.get('content-length'); 

    // Declare received as 0 initially
    let received = 0; 
  // ...
  if (done) {
      // Finish loading
      loading = false;
    } else {
      // Push values to the chunk array
      chunks.push(value);

      // Add on to the received length
      received += value.length; 
    }
  }

With that, we have all relevant indicator values available. What is missing is a way to emit them to the calling function. That can easily be done by using a Javascript framework's reactive features, like React Hooks or Vue's composition API. In this case, however, we'll stick with a builtin browser feature called CustomEvent.

Make fetch progress available with events

To wrap the monitoring feature up, let's create two custom events:

One for whenever a data chunk is read, event fetch-progress.
One for when the fetch request is finished, event fetch-finished.

Both events will be bound to the window object. Like this, they'll be available outside of the http - function's scope.

Inside the _readBody(), adjust the while... loop as follows:

  const _readBody = async (response) => {
    // ...

    // Loop through the response stream and extract data chunks
    while (loading) {
      const { done, value } = await reader.read();
      const payload = { detail: { received, length, loading } }
      const onProgress = new CustomEvent('fetch-progress', payload);
      const onFinished = new CustomEvent('fetch-finished', payload)

      if (done) {
        // Finish loading
        loading = false;

        // Fired when reading the response body finishes
        window.dispatchEvent(onFinished)
      } else {
        // Push values to the chunk array
        chunks.push(value);
        received += value.length;

        // Fired on each .read() - progress tick
        window.dispatchEvent(onProgress); 
      }
    }
    // ... 
  }

Display progress in the UI

The final step to take is catching both custom events and change the progress bar's value accordingly. Let's jump over to the main.js file and adjust it as follows:

Grab some relevant DOM elements
Add the event listener for fetch-progress
Add the event listener for fetch-finished
We can then access the progress values by destructuring from the e.detail property and adjust the progress bar value.

// Import the fetch client and initalize it
import http from './client.js';

// Grab the DOM elements
const progressbar = document.getElementById('progress-bar');
const progressbutton = document.getElementById('fetch-button');
const progresslabel = document.getElementById('progress-label');
const { json } = http('http://universities.hipolabs.com/');

const setProgressbarValue = (payload) => {
  const { received, length, loading } = payload;
  const value = ((received / length) * 100).toFixed(2);
  progresslabel.textContent = `Download progress: ${value}%`;
  progressbar.value = value;
};

// Bind the fetch function to the button's click event
progressbutton.addEventListener('click', async () => {
  const universities = await json('search?country=United+States');
  console.log(universities);
});

window.addEventListener('fetch-progress', (e) => {
  setProgressbarValue(e.detail);
});

window.addEventListener('fetch-finished', (e) => {
  setProgressbarValue(e.detail);
});

And there we have it - you can now monitor your fetch request's progress.

Still, there are some adjustments to be made:

Reset the scoped variables
Allow the user to cancel the request

If you've come this far with reading, stay with me for a few more lines.

Reset the scoped variables

This is as straightforward as it sounds and gives us a nice, reusable function.

Add the following right under the _readBody() - function in your client.js file:

const _resetLocals = () => {
  loading = false;

  chunks = [];
  results = null;
  error = null;

  controller = new AbortController();
}

Remeber that you must call resetLocals() in the json() function first.

export default function http(rootUrl) {
  let loading = false;

  let chunks = [];
  let results = null;
  let error = null;

  let controller = null; // Make sure to uncomment this variable
  const json = async (path, options,) => {
    _resetLocals();
    loading = true
  // ... rest of the json function
  }
// ... rest of the http function

With the above function, we also brought in a new object called AbortController. As the name suggests, we can use it to cut an active request.

Cancel an ongoing request

Using the created AbortController, we can now create a signal. It serves as a communication interface between the controller itself and the outgoing HTTP request. Imagine it like a built-in kill switch.

To set it up, modify your client.js file like this:

Create the signal & pass it into the fetch request options.
Create a new function that calls the controller's abort function.

const json = async (path, options,) => {
  _resetLocals();
  let signal = controller.signal; 
  loading = true

  try {
    const response = await fetch(rootUrl + path, { signal, ...options });
  // ... rest of the trycatch function
  }
// ... rest of the json function
}

// Cancel an ongoing fetch request
const cancel = () => {
  _resetLocals();
  controller.abort();
};

// Make sure to export cancel
return { json, cancel }

Finally, let's jump over to main.js and bind the event to our second button

// ... other variable declarations
const abortbutton = document.getElementById('abort-button');
const { json, cancel } = http('http://universities.hipolabs.com/');

// ... other functions and event listeners
abortbutton.addEventListener('click', () => {
  cancel()
  alert('Request has been cancelled')
})

If you now hit Fetch and Cancel Request right after, you will see an alert indicating that the request, even if it returns an HTTP status of 200, returns no data.

Update: Vue 3 composition function for fetch

I have recreated this functionality with Vue 3's Composition API. If you are looking to implement monitoring and cancelling fetch requests in your Vue app, you should have a look into this Gist:

https://gist.github.com/tq-bit/79d6ab61727ebf29ed0ff9ddc4deedca

What next?

Unfortunately, by the time I researched for this article, I could not find a common way to monitor upload progress. The official whatwg Github repository has an open issue on a feature named FetchObserver. However, it seems we'll have to be patient for it to be implemented. Perhaps, it will make the features described in this article easier as well. The future will tell.

~~https://github.com/whatwg/fetch/issues/607~~

UPDATE 01.12.2022 (Thank you @nickchomey for the hint):

Chrome supports streaming uploads starting from Version 105. The specification can be found here: https://chromestatus.com/feature/5274139738767360. Expect a followup article for this one

This post was originally published at https://blog.q-bit.me/monitoring-and-canceling-a-javascript-fetch-request/
Thank you for reading. If you enjoyed this article, let's stay in touch on Twitter 🐤 @qbitme

Top comments (6)

Johan Dahlborg • Feb 21 '23

The ok read-only property of the Response interface contains a Boolean stating whether the response was successful (status in the range 200-299) or not.
Replace (response.status >= 200 && response.status < 300) with (response.ok)

Ali Zangiabadi • Mar 28 '22

There is a problem here, if the server uses response compressing the content-length would be the compressed size and the body reader length would be the actual one.