React Infinite Scrolling

Overview

In this post we will be making an http request to OpenLibrary and fetch the name of book titles that match a query. Then, those results will be paginated and displayed. The next batch of documents will be fetched upon the user having scrolled to the final document, that is, when it is rendered on the screen.

Tools

We will make use of React hooks such as UseState, UseEffect, useRef, useCallback and a Custom hook that will form the logic for making our http request. We will also be using Axios which will help us simplify said logic.

Step 1 - Initializing

Let's go to CodeSandbox and initialize a new React project: react.new

Simple Enough.

Step 2 - Elements

For now, we can render an input field, some divs that will represent the book title and two h3 tags that will display a Loading message and an Error message.

import React from "react";
import "./styles.css";

export default function App() {
  return (
    <div className="App">
      <h1>React infinite scroll</h1>
      <input type="text" />
      <div>Book Title</div>
      <div>Book Title</div>
      <div>Book Title</div>
      <div>
        <h3>Loading...</h3>
      </div>
      <div>
        <h3>There seems to be an error</h3>
      </div>
    </div>
  );
}

That should give us the following layout:

For now, we will focus on functionality and add styling in a later post.

Step 3 - Making our http request

Let's create a file called useGetData.js inside src. Let's also install axios via npm i axios. Now, let's import UseState and UseEffect from react, as well as axios from axios.

import { useState, useEffect } from "react";
import axios from "axios";

That's all we need to import for our custom hook to work.

Now, let's define a function that will take in a query parameter and a pageNumber parameter and initialize 4 variables that represent the loading and error state, a books array containing all of our books and a hasMore variable that will determine when we have reached the end of our results and cease making API calls.

export default function useGetData(query, pageNumber) {
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(false);
  const [books, setBooks] = useState([]);
  const [hasMore, setHasMore] = useState([]);
}

Now let's use a UseEffect to make our API call only when either our query parameter changes or pageNumber does. Inside, we want to set loading to true and error to false.

useEffect(() => {
    setLoading(true);
    setError(false);
  }, [query, pageNumber]);

The meat of this program

Now, we will make an http request via axios. We will declare a cancel variable that axios uses to cancel a request. This is necessary because we don't really want to make a request every time our query changes because that means a request is made every time a new character is typed in our input field. Thus, resulting in inefficient code. The solution is only making a request once the user has finished typing. Axios makes it easy to determine whether such event has ocurred.

let cancel;
    axios({
      method: "GET",
      url: "https://openlibrary.org/search.json",
      params: { q: query, page: pageNumber },
      cancelToken: new axios.CancelToken((c) => (cancel = c))
    })
      .then((res) => {
        setBooks(prevBooks => {
          return [...new Set([...prevBooks, ...res.data.docs.map(b => b.title)])]
        })
        setHasMore(res.data.docs.length > 0)
        setLoading(false)
      })
      .catch((e) => {
        if (axios.isCancel(e)) return;
        setError(true)
      });
    return () => cancel();

As you can see, we need to pass an additional option called cancelToken inside the options parameter object after the param key. It returns a CancelToken that axios will use to cancel a request.

A key part of this is our mini useEffect:

 useEffect(() => {
    setBooks([])
   }, [query])

This snippet is required in order to reset the result list after the user creates a new query. Otherwise, we would infinitely append documents, never clearing the previous results.

Another key part of this functionality is our catch method:

catch((e) => {
        if (axios.isCancel(e)) return;
        setError(true)
      })

Notice how an if statement is triggered that evaluates whether axios.isCancel(e) is true or false. This is the equivalent of detecting whether a key change was detected and thus cancelling the request. If the request was processed and an error was received, we will use setError(true) to update our error state.

Yet another key part is our cleanup function: return () => cancel(). This functionality is provided by React's UseEffect hook and we can use it to execute the function returned by axios' CancelToken object. Now, the request will only be processed upon an uninterrupted fetch. Once, the user types again and triggers the state change, the request will be cancelled and preprocessed.

Still a little more meat

You might have noticed we skipped over the results of our http request and we will deal with that now, here is a successful call:

then((res) => {
        setBooks(prevBooks => {
          return [...new Set([...prevBooks, ...res.data.docs.map(b => b.title)])]
        })
        setHasMore(res.data.docs.length > 0)
        setLoading(false)
      })

Using the function version of setState, we declare a function that takes in the previous state and returns the new state. The state returned is a destructured Set of a destructured array of the previous books and a destructured array of the documents fetched after their respective book title field has been extracted. I know, a mouthful.

This is done in this manner for the reason that we may have repeating book titles and thus Set easily allows us to filter all the repeating values at the cost of mutating our array. Therefore, a shallow copy of this array is necessary to maintain it's integrity. The new state is now the previous book titles and our new results.

Once we have our results, it is time to check whether or not we have reached the end of the results. For that, setHasMore(res.data.docs.length > 0) will evaluate to true. How do we know this? Well, the data retrieved is an array of documents and if the length of that array is 0 we can assume we have reached the end.

A console.log(res.data) reveals our retrieved data.

Returning our variables

We can see that return {loading, error, books, hasMore} at the end of our custom hook will return all the necessary variables that our 'front end' needs to visualize the data.

This is our final useGetData.js:

import { useState, useEffect } from "react";
import axios from "axios";

export default function useGetData(query, pageNumber) {
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(false);
  const [books, setBooks] = useState([]);
  const [hasMore, setHasMore] = useState(false);

  useEffect(() => {
    setBooks([])
   }, [query])

  useEffect(() => {
    setLoading(true)
    setError(false)
    let cancel;
    axios({
      method: "GET",
      url: "https://openlibrary.org/search.json",
      params: { q: query, page: pageNumber },
      cancelToken: new axios.CancelToken((c) => (cancel = c))
    })
      .then((res) => {
        setBooks(prevBooks => {
          return [...new Set([...prevBooks, ...res.data.docs.map(b => b.title)])]
        })
        console.log(res.data)
        setHasMore(res.data.docs.length > 0)
        setLoading(false)
      })
      .catch((e) => {
        if (axios.isCancel(e)) return;
        setError(true)
      });
    return () => cancel();
  }, [query, pageNumber]);

  return {loading, error, books, hasMore};
}

Step 4 - Displaying our results

Let's return to our App.js and import the following:

import React, { useState, useRef, useCallback } from "react";
import useGetData from "./useGetData";
import "./styles.css";

Let's declare some variables:

const [query, setQuery] = useState("");
const [pageNumber, setPageNumber] = useState(1);
const { books, hasMore, loading, error } = useGetData(query, pageNumber);

Our query variable allows us to store the query state. Then, pageNumber is initialized to 1, which represents the first page. Finally, we declare a destructured object that represents the variables retrieved from our custom hook. Notice that we must pass in query and pageNumber in order for our hook to be processed correctly.

Now we will write the following code:

const observer = useRef();
  const lastBookElement = useCallback(
    (node) => {
      if (loading) return;
      if (observer.current) observer.current.disconnect();
      observer.current = new IntersectionObserver((entries) => {
        if (entries[0].isIntersecting && hasMore) {
          setPageNumber((prevPageNumber) => prevPageNumber + 1);
        }
      });
      if (node) observer.current.observe(node);
    },
    [loading, hasMore]
  );

As you can see, using const observer = useRef(); we can declare an observer that will be triggered when the last element of our results comes into view. The next function, our lastBookElement, employs a useCallBack to prevent it from being re-created unless we have changed our loading state or our hasMore flag changes, thus we added them as dependencies via [loading, hasMore].
Now, inside our useCallback hook we will received an HTML node element. Firstly, we must return if loading evaluates to true, meaning we do not want to detect the final node for now. The next evaluation, if (observer.current) observer.current.disconnect();, simply disconnects the observer to the current element, so that a new element will be connected once a new list of documents has been fetched.

Intersection Observer

The following code snippet allows use to determine whether our referenced node is displayed in our window as well as determines whether or not there are more search results.

observer.current = new IntersectionObserver((entries) => {
        if (entries[0].isIntersecting && hasMore) {
          setPageNumber((prevPageNumber) => prevPageNumber + 1);
        }
      });

We assign the observer a new IntersectionObserver which takes in a function as an argument, which takes an array of node entries, and returns various properties of those elements such as isIntersecting, which is the variable we need. Once we can visualize this node, let's udpate the page number to increment by 1.

Let's keep going

function handleSearch(e) {
    setQuery(e.target.value);
    setPageNumber(1);
  }

We now declare our handleSearch function that will update our query and pageNumber.

Finally, let's return our HTML components.

return (
    <div className="App">
      <input type="text" value={query} onChange={handleSearch}></input>
      {books.map((book, index) => {
        if (books.length === index + 1) {
          return (
            <div ref={lastBookElement} key={book}>
              {book}
            </div>
          );
        } else {
          return (
            <div key={book}>
              <h3>{book}</h3>
            </div>
          );
        }
      })}
      {loading && (
        <div>
          <h3>Loading...</h3>
        </div>
      )}
      {error && (
        <div>
          <h3>There seems to be an error</h3>
        </div>
      )}
    </div>
  );

First, lets update our input element to:

<input type="text" value={query} onChange={handleSearch}>

Now, it's value will be tracked and the onChange method attached.

Next, we will map through our results:

{books.map((book, index) => {
        if (books.length === index + 1) {
          return (
            <div ref={lastBookElement} key={book}>
              {book}
            </div>
          );
        } else {
          return (
            <div key={book}>
              {book}
            </div>
          );
        }
      })}

Notice how we attached the ref attribute exclusively when we are at the last element: (books.length === index + 1). Otherwise, return an element with no ref attribute.

We can now display our loading and error elements accordingly:

{loading && (
        <div>
          <h3>Loading...</h3>
        </div>
      )}
      {error && (
        <div>
          <h3>There seems to be an error</h3>
        </div>
      )}

This is our final App.js:

import React, { useState, useRef, useCallback } from "react";
import useGetData from "./useGetData";
import "./styles.css";

export default function App() {
  const [query, setQuery] = useState("");
  const [pageNumber, setPageNumber] = useState(1);
  const { books, hasMore, loading, error } = useGetData(query, pageNumber);

  const observer = useRef();
  const lastBookElement = useCallback(
    (node) => {
      if (loading) return;
      if (observer.current) observer.current.disconnect();
      observer.current = new IntersectionObserver((entries) => {
        if (entries[0].isIntersecting && hasMore) {
          setPageNumber((prevPageNumber) => prevPageNumber + 1);
        }
      });
      if (node) observer.current.observe(node);
    },
    [loading, hasMore]
  );

  function handleSearch(e) {
    setQuery(e.target.value);
    setPageNumber(1);
  }

  return (
    <div className="App">
      <input type="text" value={query} onChange={handleSearch}></input>
      {books.map((book, index) => {
        if (books.length === index + 1) {
          return (
            <div ref={lastBookElement} key={book}>
              {book}
            </div>
          );
        } else {
          return (
            <div key={book}>
              <h3>{book}</h3>
            </div>
          );
        }
      })}
      {loading && (
        <div>
          <h3>Loading...</h3>
        </div>
      )}
      {error && (
        <div>
          <h3>There seems to be an error</h3>
        </div>
      )}
    </div>
  );
}

Results

Fetching a query:

Reaching the end:

Functional Demo: React Infinite Scroll - Carlos Z.