How to implement Infinite Scroll in React

K Ramakrishna Vaidya — Sun, 22 Aug 2021 16:19:52 +0000

Infinite scrolling becoming more and more popular and we can see it in most of the applications, like LinkedIn, Facebook, Instagram, Youtube etc to name a few. So what exactly is "Infinite scrolling"? how to make an infinite scrolling view in react? Let's see.

I'm Ramakrishna and I'm a full stack developer. I love to know "how" part of the solutions rather than just building them.

I was taking a front end challenge and that challenge was about to build a simple react application, similar to Netflix (Without that fancy UI). But the catch was to include Lazy loading on page content. So as I scroll horizontally / vertically the content should be lazily loaded.

Why?

Lazy loading helps in application performance. User will be able to interact with the application much faster, as it loads only essential at the first render and will render other things as the user proceeds.
Browser load reduces. As the browser tries to load application in small small peices, it can render quickly and make the UX better. Coming back to the previous issue, so how to get the inifnite scrolling?

How Infinite Scroll works?

To implement something, we need to understand how it works under the hood.

So, as for the infinite scrolling, let's take FB as an example. An user might follow 1000s of friends and pages and might have millions of posts to watch. But loading that much posts will effect the performance. So for simplicity's sake, FB will load 10 posts on initial load. As user reaches the end of 10th post, it makes an async call to fetch next 10 posts. So as long as user scrolls, it fetches more and more posts.

React Implementation.

This implementation is done using a custom hook and IntersectionObserver. Let's dive into code.

Custom Hook to fetch posts as user scrolls.

import { useState, useEffect, useCallback } from "react";
import axios from "axios";

function useFetch(page, url = '') {
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(false);
  const [list, setList] = useState([]);
  const [formattedList, setFormattedList] = useState([]);

  const getPosts = useCallback(async () => {
    try {
      await setLoading(true);
      await setError(false);
      let res = []
      if(list && list.length === 0) {
      res = await axios.get(url);
      await setList(res.data)
      await setFormattedList([...res.data.slice(0, 10)])
     }
     else if (list.length > formattedList.length) {
        await setFormattedList([...formattedList, ...list.slice(formattedList.length, formattedList.length + 10)])
     }
      setLoading(false);
    } catch (err) {
      setError(err);
    }
  }, [page]);

  useEffect(() => {
    getPosts();
  }, [ getPosts, page]);

  return { loading, error, formattedList };
}

export default useFetch;

Above function is a custom hook to fetch data as per the scroll. Main thing to note here is, it's taking the url dynamically and makes a call only first time. And the url used here DOESN'T HAVE PAGINATION. So the hook is built in a way to handle splitting of result in progressive way. Find out more on useCallback here

Now, let's use the custom hook in our component. I have a custom component, which list's albums on each row, and each album will have multiple songs.

App.js component which uses custom hook for infinite scroll.

import './App.css';
import React, { Suspense, useState, useRef, useEffect, useCallback } from "react";
import useFetch from "./utils/customFetch";
import { AlbumList } from './components';

const App = () => {

const [page, setPage] = useState(1);
const { loading, error, formattedList = [] } = useFetch(page, 'https://jsonplaceholder.typicode.com/albums');

const row = useRef(null);

const handleObserver = useCallback((entries) => {
  const target = entries[0];
  if (target.isIntersecting) {
    setPage((prev) => prev + 1);
  }
}, []);

useEffect(() => {
  const option = {
    root: null,
    rootMargin: "20px",
    threshold: 0
  };
  const observer = new IntersectionObserver(handleObserver, option);
  if (row.current) observer.observe(row.current);
}, [handleObserver]);

  return (
    <div className="App">
      <Suspense fallback={<div>Loading</div>}>
        <AlbumList label="Component 1" albums={formattedList} />
      </Suspense>
      {loading && <p>Loading...</p>}
      {error && <p>Error!</p>}
      <div ref={row} />
    </div>
  );
}

export default App;

Here, I'm loading an AlbumList custom component which will fetch 10 albums on initial load. an empty div is put after AlbumList (To mark the end of current view / page). The ref is used inside IntersectionObserver to listen to when the scroll position reached this div. If the Observer detects intersection, it calls the customHook to fetch next 10 albums.
Like this, this custom hook can be used everywhere we need to make an infinite scrolling. Read more about IntersectionObserver here and here

My github repo link is here, which has the complete running version of the infinite scrolling.

Feel free to put feedback :)

References

Attach file stream / buffer in nodejs

K Ramakrishna Vaidya — Sun, 15 Aug 2021 08:03:56 +0000

Have you ever been in a situation where you wanted to attach a streaming file to an axios call from nodejs? Or you had base64 content of the file and wanted to store it in remote server by passing it down as an attachment? Then this post will guide you with that.

So in the past week I was also at the same situation, where I was getting a stream of pdf file (in which Content-Disposition: attachment; filename="filename.pdf" was set to make it exclusively downloadable) from a JAVA based service, which I had to convert to a file in node service and attach it as an attachment to another JAVA based service. But the problem is, I was getting a readstream and for attaching it I needed a file object.

I have gone through the stack overflow (like most of us) but couldn't find proper answers. So after some more experimenting, I figured out a way to achieve this.

1. Attaching a ReadStream to axios as a file

Here's the code sample to do this.

const fs = require('fs')
const Buffer = require('buffer').Buffer
const axios = require('axios')
const FormData = require('form-data')

const readFromStream = () => {
  /* this is a sample url, can be found here. https://www.learningcontainer.com/sample-pdf-files-for-testing/# */
 axios.get('https://www.learningcontainer.com/download/sample-pdf-file-for-testing/?wpdmdl=1566&refresh=60fd3a056fd6e1627208197')
    .then((data) => {
        let file = fs.writeFileSync('testfile.pdf', data.data )        

        let form = new FormData()

        form.append('files', fs.createReadStream('./testfile.pdf'), { filename: 'testfile.pdf' });  

        axios.post(`your-url-goes-here`, form, 
            {headers:{"Content-Type": "multipart/form-data; boundary="+form._boundary}}
        )
            .then((res) => {
                console.log("Success : ",res)
            })
            .catch(err => {
                console.log("Error : ",err)
            })
    })
    .catch((err) => {
        console.log("Error : ",err)
    })
}

readFromStream();

2.Attaching a Base64 pdf content to axios as a file.

const fs = require('fs')
const Buffer = require('buffer').Buffer
const axios = require('axios')
const FormData = require('form-data')

const readFromBase64 = () => {

    axios.get('your-api-for-getting-base64-string')
        .then((data) => {

// let the example string be 'SGksIHRoaXMgaXMgYSBzYW1wbGUgYmFzZTY0IHN0cmluZy4='
// Actual base64 string for a pdf file will be too large.

let base64Content = 'SGksIHRoaXMgaXMgYSBzYW1wbGUgYmFzZTY0IHN0cmluZy4='

            const buf = new Buffer.from(dbase64Content, 'base64')

            let form = new FormData()

            form.append('files', buf, { filename: 'testfile.pdf' });  

            axios.post(`your-api-for-post-call`, form, 
                {headers:{"Content-Type": "multipart/form-data; boundary="+form._boundary}}
            )
                .then((res) => {
                    console.log("Success : ",res)
                })
                .catch(err => {
                    console.log("Error : ", err)
                })
        })
        .catch((err) => {
            console.log("Error : ",err)
        })
}

readFromBase64()

The main point to note here is,

node doesn't have FormData by default. so need to use form-data package from npm
while attaching the buffer / stream to form data, filename should be added. Else it'll not consider proper MIME type for the file and may result in unwanted behaviour.
While making axios request, it's absolute necessary to mention Content-Type and boundary. Else, it'll throw error.

References :
*Send a File With Axios in Node.js

*Post file from local server
*Cover image - http://www.aisoftwarellc.com/blog/post/writing-custom-streams-in-nodejs---basics/2093

Hope this helps you. If you have any questions and suggestions, let me know in the comments.

DEV Community: K Ramakrishna Vaidya