DEV Community: Chandra Panta Chhetri

Docker, Postgres, Node, Typescript Setup

Chandra Panta Chhetri — Sun, 09 Jan 2022 13:46:57 +0000

When setting up the backend for my project I had many issues related to configuring & connecting to the DB running in a Docker container via Node & PgAdmin. And so, I wanted to explain how I fixed these issues in hopes that it can save you hours of frustrations.

We will be learning to:

Configure Typescript for Node.js
Run Node.js & Postgres in Docker containers
Use env variables in Docker Compose & Node.js
Connect to the DB running in a container via PgAdmin
Use Nodemon to automatically restart the server once the code changes

Prerequisite

Docker Desktop

Typescript & Nodemon

We will start by creating a basic Express server.

First, let's install the packages we will need:

//Dev Dependencies
npm i --save-dev typescript nodemon @types/pg @types/express dotenv

npm i pg express

Add the following scripts in package.json:

"scripts": {
    "start": "node ./dist/app.js",
    "dev": "nodemon -L -e ts --exec \"npm run build && npm start\"",
    "build": "tsc"
  }

build converts all our .ts files to .js and puts it in a dist folder (as configured below in tsconfig.json)
dev uses nodemon to watch for changes in any .ts file ('-e ts'). When there are changes, it will run the build & start scripts. Nodemon saves us from having to stop and start the server each time there is a change
- '-L' is required when using nodemon in containers
start starts up our server

To configure Typescript, create a tsconfig.json file at the root with the following:

{
    "compilerOptions": {  
      "target": "es6" /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', or 'ESNEXT'. */,
      "module": "commonjs" /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'. */,
      "outDir": "./dist" /* Redirect output structure to the directory. */,
      "strict": true /* Enable all strict type-checking options. */,
      "typeRoots": ["./node_modules/@types"] /* List of folders to include type definitions from. */,
      "esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */,
      "skipLibCheck": true /* Skip type checking of declaration files. */,
      "forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */
    }
}

Next, create an .env file at the root so that we use the same variables when configuring Docker Compose & the server. Also, we can hide the env variables used in Docker Compose as docker-compose.yml are commited to Github whereas the .env file is not.

For now, add a PORT variable to set the port the server will run at:

PORT=5000

Create a app.ts in a new src folder with the following content:

import express, { NextFunction, Request, Response } from "express";
import dotenv from "dotenv";

const app = express();
dotenv.config(); //Reads .env file and makes it accessible via process.env

app.get("/test", (req: Request, res: Response, next: NextFunction) => {
  res.send("hi");
});

app.listen(process.env.PORT, () => {
  console.log(`Server is running at ${process.env.PORT}`);
});

To verify everything is setup correctly thus far, start the server:

npm run dev

Now, make a GET request to localhost:5000/test. The response should be hi. Also, notice there should be a dist folder with all the converted .ts files.

Docker

Now, we will run the server & Postgres in a Docker container.

Before that, you might ask why use Docker at all?

Docker allows your app to run in isolated environments known as containers. Consequently, this solves the age-old problem of "the code works on my machine".

Also, it allows you to use all the tools you want without installing them locally but by using images.

Docker images can installed from Docker Hub or created using a Dockerfile.

Create a file named Dockerfile at the root:

# Installs Node.js image
FROM node:16.13.1-alpine3.14

# sets the working directory for any RUN, CMD, COPY command
# all files we put in the Docker container running the server will be in /usr/src/app (e.g. /usr/src/app/package.json)
WORKDIR /usr/src/app

# Copies package.json, package-lock.json, tsconfig.json, .env to the root of WORKDIR
COPY ["package.json", "package-lock.json", "tsconfig.json", ".env", "./"]

# Copies everything in the src directory to WORKDIR/src
COPY ./src ./src

# Installs all packages
RUN npm install

# Runs the dev npm script to build & start the server
CMD npm run dev

The Dockerfile will build our Express Server as an image, which we can then run in a container.

When creating applications that use multiple containers, it is best to use Docker Compose to configure them.

But before Docker Compose, let's add some more variables to the .env file as we will require them shortly.

DB_USER='postgres'
DB_HOST='db'
DB_NAME='db_name'
DB_PASSWORD='password'
DB_PORT=5432

DB_HOST corresponds to the name of the DB service below. This is because each Docker container has its own definition of localhost. You can think of db as the container's localhost.
DB_PORT is the default port Postgres uses
DB_PASSWORD & DB_USER are the default auth credentials Postgres uses

Create a docker-compose.yml file at the root:

version: '3.8'
services:
  api:
    container_name: api
    restart: always
    build: .
    ports:
      - ${PORT}:${PORT}
    depends_on:
      - db
    volumes:
    - .:/usr/src/app

  db:
    container_name: postgres
    image: postgres
    ports:
      - '5433:${DB_PORT}'
    volumes:
      - data:/data/db
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}

volumes: 
 data: {}

Note: The ${VARIABLE_NAME} syntax lets us use variables from the .env file. Docker Compose can automatically get variables from the root .env file.

For the api service, we are:

using the Dockerfile to build the container
exposing ${PORT} (which was 5000 from the .env file). When we expose a port, it allows us to access the server via localhost:${PORT}
only starting the container once the db service finishes starting up
mapping all the files in the project directory to WORKDIR of the container using volumes

For the db service, we are:

using the postgres image from Docker Hub
using volumes so that our DB data does not erase when we shut down the container
mapping port 5432 of the container to port 5433 of our localhost
using env variables from the .env file and passing it to the postgres image. The image requires at least the POSTGRES_PASSWORD as per the documentation on Docker Hub. We also included POSTGRES_DB as it specifies a different name for the default database that is created when the image is first started

Connecting To Postgres

To connect the server to Postgres container, add the following to app.ts:

import { Pool } from "pg";
const pool = new Pool({
  host: process.env.DB_HOST,
  user: process.env.DB_USER,
  database: process.env.DB_NAME,
  password: process.env.DB_PASSWORD,
  port: parseInt(process.env.DB_PORT || "5432")
});

const connectToDB = async () => {
  try {
    await pool.connect();
  } catch (err) {
    console.log(err);
  }
};
connectToDB();

Now, we can startup the server & DB by the following command:

docker-compose up

This will build & start the containers (api & db). Remember, first db will start then api as api depends on db.

Try making the same GET request we did earlier and you should get the same response.

Before we end the tutorial, you might be wondering, how do I view the DB and its contents? There are 2 ways:

You can add a new service to the docker-compose.yml file that uses the pgadmin4 image
If you have PgAdmin installed locally:
- Use localhost as the host & 5433 as the port when adding a new server. Why 5433 and not 5432 - the default port of Postgres? Earlier, we mapped port 5432 of the container to port 5433 of our localhost. But, why 5433? It could've been any port, just not 5432 because if you have Postgres already installed locally, it is already using port 5432. So, you cannot have the Postgres container also using the same port.

Conclusion

I hope my explanation was clear & helped you in some way. If you want the source code, you can find the full code here.

Infinite Scroll With Firebase, React, Intersection Observer & Redux Saga

Chandra Panta Chhetri — Thu, 07 Jan 2021 00:01:14 +0000

While working on a React project with Redux-Saga and Firebase, I wanted to add infinite scrolling to improve site performance and user experience. However, structuring the Firestore, Redux, Redux-Saga, and React code to maximize readability and maintainability was difficult.

End Result

We will be building a simple UI that displays 6 products initially and as the user scrolls to the end, we will load 6 more products. Building a simple UI will let us focus on the Redux, Firestore, and Redux-Saga logic.

Figure 1: Loading more products once user scrolls to the end

The code with all configurations can be found at https://github.com/Chandra-Panta-Chhetri/infinite-scroll-firebase-tutorial.

Prerequisite

Basic knowledge of Redux, Redux Saga, React
Basis understanding of Firestore
Basic understanding of generator functions as it will be used with Redux Saga

Redux

To setup the Redux portion, we will need the following dependencies:

Redux Store, Root Reducer & Root Saga

As with any React, Redux, and Redux-Saga project, the convention is to set up a root reducer, a root saga, and the Redux store.

In the root reducer, we will combine all the reducers, which in this case will only be a product reducer, and export it.

import productReducer from "./product/product.reducer";
import { combineReducers } from "redux";

export default combineReducers({
  product: productReducer
});

Similar to the root reducer, in the root saga we will combine all the sagas, which in this case will only be a product saga.

import { all, call } from "redux-saga/effects";
import productSagas from "./product/product.sagas";

export default function* rootSaga() {
  yield all([call(productSagas)]);
}

Now we need to connect the root saga and root reducer to the Redux store.

import createSagaMiddleware from "redux-saga";
import rootReducer from "./root.reducer";
import rootSaga from "./root.saga";
import { createStore, applyMiddleware } from "redux";

const sagaMiddleware = createSagaMiddleware();

const middlewares = [sagaMiddleware];

export const store = createStore(rootReducer, applyMiddleware(...middlewares));

sagaMiddleware.run(rootSaga);

To put it simply, the configuration above connects the root saga to the Redux store by passing the saga middleware to the applyMiddleware function and then calling the run method on the saga middleware.

If you want to understand the configurations in greater depth, refer to https://www.codementor.io/@rajjeet/step-by-step-how-to-add-redux-saga-to-a-react-redux-app-11xqieyj67.

When working with Redux, the convention is to define the action types, action creators, selectors, and a reducer so we can manage independent parts of the Redux store.

And so, we will create the action types, action creators, selectors, sagas, and a reducer to manage the product states in the Redux store.

Product Action Types

Let's start by defining the action types our product reducer and action creators will use. By defining constants, we will have consistent naming in the product reducer and action creators.

const PRODUCT_ACTION_TYPES = {
  START_INITIAL_PRODUCTS_FETCH: "START_INITIAL_PRODUCTS_FETCH",
  INITIAL_PRODUCTS_FETCH_FAIL: "INITIAL_PRODUCTS_FETCH_FAIL",
  INITIAL_PRODUCTS_FETCH_SUCCESS: "INITIAL_PRODUCTS_FETCH_SUCCESS",
  START_LOADING_MORE_PRODUCTS: "START_LOADING_MORE_PRODUCTS",
  LOADING_MORE_PRODUCTS_FAIL: "LOADING_MORE_PRODUCTS_FAIL",
  LOADING_MORE_PRODUCTS_SUCCESS: "LOADING_MORE_PRODUCTS_SUCCESS",
  NO_MORE_PRODUCTS_TO_LOAD: "NO_MORE_PRODUCTS_TO_LOAD"
};

export default PRODUCT_ACTION_TYPES;

If you are wondering why we are considering the initial product fetch and the subsequent product fetch as different action types, don't worry the reason will become quite clear when we write the sagas and Firestore queries.

Product Action Creators

Now that we have defined the action types, we will use them when creating the action creators we will dispatch to update the Redux store.

For each action type, we will create a function that returns an action. An action is an object of the form { type, payload }.

import PRODUCT_ACTION_TYPES from "./product.action.types";

export const startInitialProductsFetch = () => ({
  type: PRODUCT_ACTION_TYPES.START_INITIAL_PRODUCTS_FETCH
});

export const initialProductsFetchFail = (errorMsg) => ({
  type: PRODUCT_ACTION_TYPES.INITIAL_PRODUCTS_FETCH_FAIL,
  payload: errorMsg
});

export const initialProductsFetchSuccess = (products, lastVisibleDoc) => ({
  type: PRODUCT_ACTION_TYPES.INITIAL_PRODUCTS_FETCH_SUCCESS,
  payload: { products, lastVisibleDoc }
});

export const startLoadingMoreProducts = () => ({
  type: PRODUCT_ACTION_TYPES.START_LOADING_MORE_PRODUCTS
});

export const loadingMoreProductsFail = (errorMsg) => ({
  type: PRODUCT_ACTION_TYPES.LOADING_MORE_PRODUCTS_FAIL,
  payload: errorMsg
});

export const loadingMoreProductsSuccess = (newProducts, lastVisibleDoc) => ({
  type: PRODUCT_ACTION_TYPES.LOADING_MORE_PRODUCTS_SUCCESS,
  payload: { newProducts, lastVisibleDoc }
});

export const noMoreProductsToLoad = () => ({
  type: PRODUCT_ACTION_TYPES.NO_MORE_PRODUCTS_TO_LOAD
});

Product Reducer

The product reducer will manipulate the following states depending on the action types being dispatched.

const INITIAL_STATE = {
  products: [],
  isFetchingProducts: false,
  productsPerPage: 6,
  lastVisibleDoc: null,
  hasMoreToFetch: true
};

The purpose of each is as follows:

products
- Stores the product data fetched from Firestore
isFetchingProducts
- Indicates whether we are fetching products from Firestore
productsPerPage
- The maximum number of products we want to get on each request to Firestore
lastVisibleDoc
- Stores the last document snapshot from the most recent Firestore request
- When getting the next set of products from Firestore, we need to provide the last document snapshot. We will see an example when we write the Firestore queries later.
hasMoreToFetch
- Indicates whether there are more products to fetch from Firestore (Prevents making requests to Firestore if we have fetched all the products)

We can now define the skeleton of the reducer like so:

import PRODUCT_ACTION_TYPES from "./product.action.types";

const INITIAL_STATE = {
  products: [],
  isFetchingProducts: false,
  productsPerPage: 6,
  lastVisibleDoc: null,
  hasMoreToFetch: true
};

const productReducer = (prevState = INITIAL_STATE, action) => {
  switch (action.type) {
    default:
      return prevState;
  }
};

export default productReducer;

Using the action type constants, we can now add case statements so that we can manipulate the state when an action occurs.

import PRODUCT_ACTION_TYPES from "./product.action.types";

const INITIAL_STATE = {
  products: [],
  isFetchingProducts: false,
  productsPerPage: 6,
  lastVisibleDoc: null,
  hasMoreToFetch: true
};

const productReducer = (prevState = INITIAL_STATE, action) => {
  switch (action.type) {
    case PRODUCT_ACTION_TYPES.START_INITIAL_PRODUCTS_FETCH:
      return {
        ...prevState,
        isFetchingProducts: true,
        products: [],
        hasMoreToFetch: true,
        lastVisibleDoc: null
      };
    case PRODUCT_ACTION_TYPES.INITIAL_PRODUCTS_FETCH_FAIL:
    case PRODUCT_ACTION_TYPES.LOADING_MORE_PRODUCTS_FAIL:
    case PRODUCT_ACTION_TYPES.NO_MORE_PRODUCTS_TO_LOAD:
      return {
        ...prevState,
        isFetchingProducts: false,
        hasMoreToFetch: false
      };
    case PRODUCT_ACTION_TYPES.INITIAL_PRODUCTS_FETCH_SUCCESS:
      return {
        ...prevState,
        products: action.payload.products,
        lastVisibleDoc: action.payload.lastVisibleDoc,
        isFetchingProducts: false
      };
    case PRODUCT_ACTION_TYPES.START_LOADING_MORE_PRODUCTS:
      return {
        ...prevState,
        isFetchingProducts: true
      };
    case PRODUCT_ACTION_TYPES.LOADING_MORE_PRODUCTS_SUCCESS:
      return {
        ...prevState,
        isFetchingProducts: false,
        products: [...prevState.products, ...action.payload.newProducts],
        lastVisibleDoc: action.payload.lastVisibleDoc
      };
    default:
      return prevState;
  }
};

export default productReducer;

Now that we have implemented the product reducer, based on how the state is being manipulated, it should be more clear as to why we defined the action types we did.

Product Selectors

Selectors are functions that accept the entire Redux state as a parameter and return a part of the state.

export const selectProductsPerPage = (state) => state.product.productsPerPage;

export const selectLastVisibleDoc = (state) => state.product.lastVisibleDoc;

export const selectProducts = (state) => state.product.products;

export const selectIsFetchingProducts = (state) =>
  state.product.isFetchingProducts;

export const selectHasMoreProductsToFetch = (state) =>
  state.product.hasMoreToFetch;

For example, the selectIsFetchingProducts selector takes the Redux state and returns the isFetchingProducts state (the one we set up in the product reducer).

Product Sagas

Sagas can be thought of as event listeners as they watch the Redux store for any specified actions and call a specified callback when that action(s) occurs. In the callback, we can perform asynchronous code such as API requests and even dispatch additional actions.

Let's start by creating 2 sagas - one to watch for the latest "START_INITIAL_PRODUCTS_FETCH" action type and the other for the latest "START_LOADING_MORE_PRODUCTS" action type.

import PRODUCT_ACTION_TYPES from "./product.action.types";
import { takeLatest, put, call, all, select } from "redux-saga/effects";

function* watchProductsFetchStart() {
  yield takeLatest(
    PRODUCT_ACTION_TYPES.START_INITIAL_PRODUCTS_FETCH,
    fetchProducts
  );
}

function* watchLoadMoreProducts() {
  yield takeLatest(
    PRODUCT_ACTION_TYPES.START_LOADING_MORE_PRODUCTS,
    fetchMoreProducts
  );
}

We will define the fetchMoreProducts and fetchProducts functions soon.

To reduce the changes we need to make to the root saga, it is a good practice to create a main saga export instead of exporting each saga (i.e. watchProductsFetchStart and watchLoadMoreProducts).

import PRODUCT_ACTION_TYPES from "./product.action.types";
import { takeLatest, put, call, all, select } from "redux-saga/effects";

function* watchProductsFetchStart() {
  yield takeLatest(
    PRODUCT_ACTION_TYPES.START_INITIAL_PRODUCTS_FETCH,
    fetchProducts
  );
}

function* watchLoadMoreProducts() {
  yield takeLatest(
    PRODUCT_ACTION_TYPES.START_LOADING_MORE_PRODUCTS,
    fetchMoreProducts
  );
}

export default function* productSagas() {
  yield all([call(watchProductsFetchStart), call(watchLoadMoreProducts)]);
}

To create the fetchProducts function used above, we will import the action creators and selectors we created as we will need to access the Redux state and dispatch actions within fetchProducts.

import { takeLatest, put, call, all, select } from "redux-saga/effects";
import {
  initialProductsFetchFail,
  initialProductsFetchSuccess,
  noMoreProductsToLoad
} from "./product.actions";
import {
  getProducts
} from "../../firebase-utils/firebase.product_utils";
import {
  selectProductsPerPage
} from "./product.selectors";

function* fetchProducts() {
  try {
    const productsPerPage = yield select(selectProductsPerPage);
    const { products, lastVisibleDoc } = yield getProducts(productsPerPage);
    if (!products.length) {
      return yield put(noMoreProductsToLoad());
    }
    yield put(initialProductsFetchSuccess(products, lastVisibleDoc));
  } catch (err) {
    yield put(
      initialProductsFetchFail("There was a problem displaying the products.")
    );
  }
}

In the function above, we are getting the productsPerPage state using the selectProductsPerPage selector and passing it to getProducts. Although we have not implemented getProducts yet, it is evident that it takes the number of products we want to fetch initially and returns an object of the form { products, lastVisibleDoc }. If there are no products, we dispatch the noMoreProductsToLoad action creator, which then changes the hasMoreToFetch state to true. Otherwise, we dispatch the initialProductsFetchSuccess action creator which updates the lastVisibleDoc and products state.

Now, anytime an action with the type of "START_INITIAL_PRODUCTS_FETCH" is dispatched, the fetchProducts saga will run and update the Redux store accordingly.

The fetchMoreProducts function will be similar to fetchProducts except we will call the getMoreProducts function and pass it the lastVisibleDoc and productsPerPage state. The getMoreProducts will also be implemented later on.

import { takeLatest, put, call, all, select } from "redux-saga/effects";
import {
  initialProductsFetchFail,
  initialProductsFetchSuccess,
  loadingMoreProductsFail,
  loadingMoreProductsSuccess,
  noMoreProductsToLoad
} from "./product.actions";
import {
  getProducts,
  getMoreProducts
} from "../../firebase-utils/firebase.product_utils";
import {
  selectProductsPerPage,
  selectLastVisibleDoc
} from "./product.selectors";

function* fetchMoreProducts() {
  try {
    const productsPerPage = yield select(selectProductsPerPage);
    const lastDoc = yield select(selectLastVisibleDoc);
    const { products: newProducts, lastVisibleDoc } = yield getMoreProducts(
      lastDoc,
      productsPerPage
    );
    if (!newProducts.length) {
      return yield put(noMoreProductsToLoad());
    }
    yield put(loadingMoreProductsSuccess(newProducts, lastVisibleDoc));
  } catch (err) {
    yield put(
      loadingMoreProductsFail("There was a problem loading more products.")
    );
  }
}

For reference, here is the complete saga code:

import PRODUCT_ACTION_TYPES from "./product.action.types";
import { takeLatest, put, call, all, select } from "redux-saga/effects";
import {
  initialProductsFetchFail,
  initialProductsFetchSuccess,
  loadingMoreProductsFail,
  loadingMoreProductsSuccess,
  noMoreProductsToLoad
} from "./product.actions";
import {
  getProducts,
  getMoreProducts
} from "../../firebase-utils/firebase.product_utils";
import {
  selectProductsPerPage,
  selectLastVisibleDoc
} from "./product.selectors";

function* fetchProducts() {
  try {
    const productsPerPage = yield select(selectProductsPerPage);
    const { products, lastVisibleDoc } = yield getProducts(productsPerPage);
    if (!products.length) {
      return yield put(noMoreProductsToLoad());
    }
    yield put(initialProductsFetchSuccess(products, lastVisibleDoc));
  } catch (err) {
    yield put(
      initialProductsFetchFail("There was a problem displaying the products.")
    );
  }
}

function* fetchMoreProducts() {
  try {
    const productsPerPage = yield select(selectProductsPerPage);
    const lastDoc = yield select(selectLastVisibleDoc);
    const { products: newProducts, lastVisibleDoc } = yield getMoreProducts(
      lastDoc,
      productsPerPage
    );
    if (!newProducts.length) {
      return yield put(noMoreProductsToLoad());
    }
    yield put(loadingMoreProductsSuccess(newProducts, lastVisibleDoc));
  } catch (err) {
    yield put(
      loadingMoreProductsFail("There was a problem loading more products.")
    );
  }
}

function* watchProductsFetchStart() {
  yield takeLatest(
    PRODUCT_ACTION_TYPES.START_INITIAL_PRODUCTS_FETCH,
    fetchProducts
  );
}

function* watchLoadMoreProducts() {
  yield takeLatest(
    PRODUCT_ACTION_TYPES.START_LOADING_MORE_PRODUCTS,
    fetchMoreProducts
  );
}

export default function* productSagas() {
  yield all([call(watchProductsFetchStart), call(watchLoadMoreProducts)]);
}

Recap

Now that we are done with the Redux portion, anytime we dispatch the startInitialProductsFetch and the startLoadingMoreProducts action creators, the product sagas will call the getProducts and getMoreProducts functions and dispatch additional actions to update the product states we defined in the product reducer.

Firebase Paginated Queries

For this portion, we will need the following dependency:

Firebase

Before we can use Firestore, we need to configure Firebase like so:

import firebase from "firebase/app";
import "firebase/firestore";

const firebaseConfig = {
  apiKey: process.env.REACT_APP_FIREBASE_API_KEY,
  authDomain: process.env.REACT_APP_FIREBASE_AUTH_DOMAIN,
  databaseURL: process.env.REACT_APP_FIREBASE_DATABASE_URL,
  projectId: process.env.REACT_APP_FIREBASE_PROJECT_ID,
  storageBucket: process.env.REACT_APP_FIREBASE_STORAGE_BUCKET,
  messagingSenderId: process.env.REACT_APP_FIREBASE_MESSAGING_SENDER_ID,
  appId: process.env.REACT_APP_FIREBASE_APP_ID
};
firebase.initializeApp(firebaseConfig);

export const firestore = firebase.firestore();
export default firebase;

If you are confused about the configuration above, refer to https://dev.to/itnext/react-with-firebase-firestore-setup-4ch3.

We will now implement the getProducts and getMoreProducts function we used when we wrote the product sagas.

import { firestore } from "./firebase.config"; //We exported this earlier in the Firebase configuration
const productCollectionRef = firestore.collection("products");

export const getProducts = async (productsPerPage) => {
  const paginatedProductsQuery = productCollectionRef
    .orderBy("name", "asc")
    .limit(productsPerPage);
  const productsAndLastVisibleDoc = await excutePaginatedProductQuery(
    paginatedProductsQuery
  );
  return productsAndLastVisibleDoc;
};

As with any Firestore query, we first need a reference to a Firestore collection. Since we will be using the product collection ref in both getProducts and getMoreProducts, we should define it globally.

In the getProducts function, we are querying the product collection and ordering the documents by name in ascending order. Then we are selecting the first productsPerPage documents. Next, we call excutePaginatedProductQuery, which takes a paginated query, executes it, returns an object of the form: { products, lastVisibleDoc } and then we return this object from getProducts.

To improve code reusability, we are creating the excutePaginatedProductQuery function as the only difference between the getProducts and getMoreProducts function is the query we execute.

export const executePaginatedQuery = async (paginatedQuery) => {
  const querySnapshot = await paginatedQuery.get();
  const docSnapshots = querySnapshot.docs;
  const lastVisibleDoc = docSnapshots[docSnapshots.length - 1];
  return { lastVisibleDoc, docSnapshots };
};

export const excutePaginatedProductQuery = async (paginatedProductQuery) => {
  try {
    const {
      lastVisibleDoc,
      docSnapshots: productSnapshots
    } = await executePaginatedQuery(paginatedProductQuery);
    const products = productSnapshots.map((ps) => ({
      id: ps.id,
      ...ps.data()
    }));
    return { products, lastVisibleDoc };
  } catch (err) {
    return { products: [], lastVisibleDoc: null };
  }
};

The executePaginatedProductQuery function executes a query and returns the products and the last document snapshot from the query result.

Since we can abstract the process of executing a query, getting the document snapshots, and the last document snapshot, we have moved that logic to the executePaginatedQuery and called it within the executePaginatedProductQuery function.

"Why do we need the last document snapshot?"

Many databases have their own ways of skipping documents to get the next documents. In Firestore, we use the startAfter or startAt methods and pass a document snapshot to define the starting point for a query. We will see an example shortly.

So far, we have a function (getProducts) that queries the product collection and gets the first 6 products.

To get the next 6 products, we need to another function that uses the startAfter method.

export const getMoreProducts = async (lastVisibleDoc, productsPerPage) => {
  const nextProductsQuery = productCollectionRef
    .orderBy("name", "asc")
    .startAfter(lastVisibleDoc)
    .limit(productsPerPage);
  const productsAndLastVisibleDoc = await excutePaginatedProductQuery(
    nextProductsQuery
  );
  return productsAndLastVisibleDoc;
};

From above, it is clear that the getMoreProducts function is similar to the getProducts function except for the query. More specifically, the query uses the startAfter method which skips all the product documents before the lastVisibleDoc.

For reference, here is the complete code for this portion.

import { firestore } from "./firebase.config";
const productCollectionRef = firestore.collection("products");

export const executePaginatedQuery = async (paginatedQuery) => {
  const querySnapshot = await paginatedQuery.get();
  const docSnapshots = querySnapshot.docs;
  const lastVisibleDoc = docSnapshots[docSnapshots.length - 1];
  return { lastVisibleDoc, docSnapshots };
};

export const excutePaginatedProductQuery = async (paginatedProductQuery) => {
  try {
    const {
      lastVisibleDoc,
      docSnapshots: productSnapshots
    } = await executePaginatedQuery(paginatedProductQuery);
    const products = productSnapshots.map((ps) => ({
      id: ps.id,
      ...ps.data()
    }));
    return { products, lastVisibleDoc };
  } catch (err) {
    return { products: [], lastVisibleDoc: null };
  }
};

export const getProducts = async (productsPerPage) => {
  const paginatedProductsQuery = productCollectionRef
    .orderBy("price")
    .limit(productsPerPage);
  const productsAndLastVisibleDoc = await excutePaginatedProductQuery(
    paginatedProductsQuery
  );
  return productsAndLastVisibleDoc;
};

export const getMoreProducts = async (lastVisibleDoc, productsPerPage) => {
  const nextProductsQuery = productCollectionRef
    .orderBy("price")
    .startAfter(lastVisibleDoc)
    .limit(productsPerPage);
  const productsAndLastVisibleDoc = await excutePaginatedProductQuery(
    nextProductsQuery
  );
  return productsAndLastVisibleDoc;
};

Recap

Going back to why we considered the initial product fetch different from the subsequent product fetches, now that we have the getProducts and getMoreProducts function, the reason should be more clear. Put simply, when we make the initial request we cannot use the startAfter method as the last document snapshot is null. So, we need to make the initial product request, update the lastVisibleDoc state, and use that when fetching the next products.

usePaginationOnIntersection hook

The logic we have implemented so far will only work once the startInitialProductsFetch and startLoadingMoreProducts action creators are dispatched.

We can dispatch the startInitialProductsFetch action once a component mounts. But for the startLoadingMoreProducts action, we need to dispatch that each time the user has scrolled to the last product.

To do that, we can use the Intersection Observer. The Intersection Observer can run a callback once a specified DOM element appears on the screen.

In other words, we just need to observe the last product in the products state and dispatch the startLoadingMoreProducts action once it appears on the screen. Although we could put this logic in a component, this will reduce code reusability so instead we will create a hook.

The hook will have the following parameters:

fetchMore
- a function to call once an DOM element appears on screen
isFetchingMore
- Indicates whether more products are already being fetched
hasMoreToFetch
- Indicates whether there are more products to fetch
options
- When creating a new Intersection Observer instance, we can pass an options object. For example, we can set the threshold to 0.5, which will trigger the fetchMore function when the element is 50% visible.

import { useRef, useCallback } from "react";

const DEFAULT_OPTIONS = { threshold: 0.9 };

const usePaginationOnIntersection = (
  fetchMore,
  isFetchingMore,
  hasMoreToFetch,
  options = DEFAULT_OPTIONS
) => {
  const observer = useRef();
  const triggerPaginationOnIntersection = useCallback(
    (elementNode) => {
      if (isFetchingMore) return;
      //Removes the previously observed DOM node before observing another
      if (observer.current) {
        observer.current.disconnect();
      }
      if (!hasMoreToFetch) return;
      observer.current = new IntersectionObserver(([entry]) => {
        if (entry.isIntersecting) {
          fetchMore();
        }
      }, options);
      if (elementNode) {
        observer.current.observe(elementNode);
      }
    },
    [isFetchingMore, fetchMore, hasMoreToFetch]
  );

  return triggerPaginationOnIntersection;
};

export default usePaginationOnIntersection;

In the code above, we are using these hooks from React in the following way:

useRef
- To store a DOM reference to the element we are going to observe
useCallback
- To return a memoized function for performance reasons.

The triggerPaginationOnIntersection memoized function attaches a new Intersection Observer to the current property of the observer variable. Then it observes the DOM node passed to the function using the observe method (we can use this because current property is an Intersection Observer object). Doing this will trigger the fetchMore function whenever the elementNode appears on the screen.

Conclusion

Now the last thing remaining is to get the state from the Redux store so we can display the products and dispatch the actions to fetch products.

To get the state, we will use the selectors we created earlier.

import React, { useEffect } from "react";

import { connect } from "react-redux";
import {
  selectHasMoreProductsToFetch,
  selectIsFetchingProducts,
  selectProducts
} from "./redux/product/product.selectors";
import {
  startInitialProductsFetch
} from "./redux/product/product.actions";

function App({
  products,
  fetchProducts,
  fetchMoreProducts,
  hasMoreProductsToFetch,
  isFetchingProducts
}) {
  useEffect(() => {
    fetchProducts();
  }, [fetchProducts]);

  return (
    <section>
      <h1>Products</h1>
      <div>
        {(products || []).map((product, index) => (
          <div
            key={product.id}
          >
            <span>Name: {product.name}</span>
            <span>Price: ${product.price}</span>
          </div>
        ))}
        {isFetchingProducts && <p>Loading...</p>}
      </div>
    </section>
  );
}

const mapStateToProps = (state) => ({
  products: selectProducts(state),
  isFetchingProducts: selectIsFetchingProducts(state),
  hasMoreProductsToFetch: selectHasMoreProductsToFetch(state)
});

const mapDispatchToProps = (dispatch) => ({
  fetchProducts: () => dispatch(startInitialProductsFetch()),
  fetchMoreProducts: () => dispatch(startLoadingMoreProducts())
});

export default connect(mapStateToProps, mapDispatchToProps)(App);

In the component above, we are dispatching the startInitialProductsFetch action when the component mounts. Consequently, this will run the fetchProducts and query Firestore for the first 6 products.

To load more products once the user sees the last product, we can use the usePaginationOnIntersection hook we created.

If you remember correctly, the hook returns a memoized function that takes a DOM node as an argument. To pass a DOM node to the function, a shorthand we can use is to pass the function to the ref attribute if it is the last product in the products state (we only want to fetch more products once the user sees the last product).

import React, { useEffect } from "react";

import { connect } from "react-redux";
import {
  selectHasMoreProductsToFetch,
  selectIsFetchingProducts,
  selectProducts
} from "./redux/product/product.selectors";
import {
  startInitialProductsFetch,
  startLoadingMoreProducts
} from "./redux/product/product.actions";
import usePaginationOnIntersection from "./hooks/usePaginationOnIntersection.hook";

function App({
  products,
  fetchProducts,
  fetchMoreProducts,
  hasMoreProductsToFetch,
  isFetchingProducts
}) {
  const fetchMoreOnIntersection = usePaginationOnIntersection(
    fetchMoreProducts,
    isFetchingProducts,
    hasMoreProductsToFetch
  );

  useEffect(() => {
    fetchProducts();
  }, [fetchProducts]);

  return (
    <section>
      <h1>Products</h1>
      <div>
        {(products || []).map((product, index) => (
          <div
            key={product.id}
            ref={
              index + 1 === products.length
                ? fetchMoreOnIntersection
                : undefined
            }
          >
            <span>Name: {product.name}</span>
            <span>Price: ${product.price}</span>
          </div>
        ))}
        {isFetchingProducts && <p>Loading...</p>}
      </div>
    </section>
  );
}

const mapStateToProps = (state) => ({
  products: selectProducts(state),
  isFetchingProducts: selectIsFetchingProducts(state),
  hasMoreProductsToFetch: selectHasMoreProductsToFetch(state)
});

const mapDispatchToProps = (dispatch) => ({
  fetchProducts: () => dispatch(startInitialProductsFetch()),
  fetchMoreProducts: () => dispatch(startLoadingMoreProducts())
});

export default connect(mapStateToProps, mapDispatchToProps)(App);

Now anytime the user scrolls to the last product, the following events will happen if hasMoreToFetch is true:

startLoadingMoreProducts action will be dispatched
products state in Redux store will update
Component will re-render
A new Intersection Observer will be attached to last product and the previous observed element will be removed
Steps 1-4 will be repeated until hasMoreToFetch is false

Responsive React File Upload Component With Drag And Drop

Chandra Panta Chhetri — Sun, 03 Jan 2021 04:44:29 +0000

While working on a React project, I implemented a responsive file upload component that supports drag and drop without using any libraries. Most of the file upload components online used libraries such as react-dropzone to support drag and drop. So, I thought I'd share how I made the component and show a typical use case for it.

End result

Figure 1: File upload component used in a form (example use case)

Figure 2: Responsive file upload component

The features include:

drag and drop without using any libraries
displaying image preview for image files
displaying file size & name
removing files in the "To Upload" section
preventing user from uploading files bigger than a specified size
- Note: this should also be done on the backend for security reasons

Project Setup

Prerequisite: Node (for installing npm packages)

If you are familiar with building React applications, the easiest way to set up a new React project is by using create-react-app. So, run the following commands in a terminal/command-line:

npx create-react-app react-file-upload
cd react-file-upload

To ensure everything was set up properly after you run npm start, the following should appear once you visit localhost:3000 in a browser:

Before building the component, let's modify and remove some files to get rid of unnecessary code.

Change App.js to the following:

import React from 'react';

function App() {
  return (
    <div></div>
  );
}

export default App;

Change index.js to the following:

import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import App from './App';

ReactDOM.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
  document.getElementById('root')
);

Remove all files in the src folder except

App.js
index.js
index.css

File Upload Component

Installing Dependencies

The dependencies we will need are:

styled-components

For styling the component
- styled components allow for style encapsulation and creating dynamic styles via props

node-sass

For compiling Sass styles used in styled components (Optional, can use CSS)

To install them, run npm i styled-components node-sass.

Folder Structure

A good convention for structuring folders and files is to create a components folder that has a folder for each component. This makes it easier to find the logic and styles for each component.

Following this convention, create a components folder in the src folder and then a file-upload folder within the components folder.

Lastly, within the file-upload folder, create 2 new files.

file-upload.component.jsx
file-upload.styles.js

State

Since we are creating a functional component and need to use state, we will use the useState hook.

The useState hook returns a stateful value which is the same as the value passed as the first argument, and a function to update it.

For our purposes, we will need state to keep track of the uploaded files. So, in the file-upload.component.jsx file, add the following:

import React, { useState } from "react";

const FileUpload = () => {
  const [files, setFiles] = useState({});

  return (
   <div></div>
  )
}

export default FileUpload;

“Shouldn't we use an empty array instead of an empty object for the files state?”

Using an object will allow us to easily manipulate (add/remove) the files state and prevent files with the same name from being uploaded more than once. Here is an example of how the files state will look like:

{
 "file1.png": File,
 "file2.png": File
}

If we used an array it would require more work. For instance, to remove a file we would have to iterate through each file until we find the one to remove.

Note: File is a JS object. More info can be found at https://developer.mozilla.org/en-US/docs/Web/API/File.

useRef hook

If you look at Figure 1 above, you will notice the user can either drag and drop files or press the Upload Files button. By default, an file input tag will open the file explorer once it is clicked. However, we want to open it once the Upload Files button is clicked so we will require a DOM reference to the file input tag.

To create a DOM reference, we will use the useRef hook. The useRef hook returns a mutable ref object whose .current property refers to a DOM node (file input tag in this case).

Once we use the useRef hook, we must pass the returned value to the ref attribute of the file input tag, like so:

import React, { useState, useRef } from "react";

const FileUpload = (props) => {
  const fileInputField = useRef(null);
  const [files, setFiles] = useState({});

  return (
   <input type="file" ref={fileInputField} />
  )
}

export default FileUpload;

Props

The component will have the following props:

label
- Determines the label of the component (e.g. "Profile Image(s)" in Figure 1 above)
maxFileSizeInBytes
- For preventing files above the specified size from being uploaded
updateFilesCb
- A callback function used for sending the files state to the parent component

“Why do we need to send the files state to the parent component?”

Typically, the file upload component will be used in a form and when working with forms in React, the component stores the form data in the state. Thus, for the parent component to also store the uploaded files, we need the file upload component to send it.

“Why do we need use a callback function to send the files state to the parent component?”

Since React has unidirectional data flow, we cannot easily pass data from the child component (file upload component) to the parent component. As a workaround, we will pass a function declared in the parent component and the file upload component will call that function with the files state as an argument. This process of sending data from the child to the parent can be further explained at https://medium.com/@jasminegump/passing-data-between-a-parent-and-child-in-react-deea2ec8e654.

Using destructuring, we can now add the props like so:

import React, { useRef, useState } from "react";

const DEFAULT_MAX_FILE_SIZE_IN_BYTES = 500000;

const FileUpload = ({
  label,
  updateFilesCb,
  maxFileSizeInBytes = DEFAULT_MAX_FILE_SIZE_IN_BYTES,
  ...otherProps
}) => {
  const fileInputField = useRef(null);
  const [files, setFiles] = useState({});

  return (
   <input type="file" ref={fileInputField} />
  )
}

export default FileUpload;

“Why are we using the spread syntax when destructuring otherProps?”

When destructuring, we can assign all other values that were not explicitly destructured to a variable.

let props = { a: 1, b: 2, c: 3};
let {a, ...otherProps} = props;

//a = 1
//otherProps = {b: 2, c: 3};

In this case, for any props that we do not destructure, they will be assigned to the otherProps variable. We will see the use of this otherProps variable later.

HTML

For the icons shown in Figure 1, we will be using Font Awesome. To import it, add the following in the head tag in the public/index.html file:

<link
 rel="stylesheet"
 href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.13.0/css/all.min.css"
/>

From Figure 1, it is evident we can split the HTML for the component into 2 main parts.

Here is the component with the HTML for the first part:

import React, { useRef, useState } from "react";

const DEFAULT_MAX_FILE_SIZE_IN_BYTES = 500000;

const FileUpload = ({
  label,
  updateFilesCb,
  maxFileSizeInBytes = DEFAULT_MAX_FILE_SIZE_IN_BYTES,
  ...otherProps
}) => {
  const fileInputField = useRef(null);
  const [files, setFiles] = useState({});

  return (
      <section>
        <label>{label}</label>
        <p>Drag and drop your files anywhere or</p>
        <button type="button">
          <i className="fas fa-file-upload" />
          <span> Upload {otherProps.multiple ? "files" : "a file"}</span>
        </button>
        <input
          type="file"
          ref={fileInputField}
          title=""
          value=""
          {...otherProps}
        />
      </section>      
  );
}

export default FileUpload;

Earlier, we discussed that any props that we don't destructure will be assigned to the otherProps variable (i.e. any prop other than label, updateFilesCb, maxFileSizeInBytes). In the code above, we are taking that otherProps variable and passing it to the file input tag. This was done so that we can add attributes to the file input tag from the parent component via props.

“Why are we setting the title and value attribute to ""?”

Setting the title attribute to "" gets rid of the text that shows up by default when hovering over the input tag ("No file chosen").

Setting the value attribute to "" fixes an edge case where uploading a file right after removing it does not change the files state. Later, we will see that the files state only changes once the value of the input tag changes. This bug occurs because when we remove a file, the input tag's value does not change. Since state changes re-render HTML, setting the value attribute to "" resets the input tag's value on each files state change.

Before we write the HTML for the second part, keep in mind that React only allows for returning one parent element from a component. Thus, we will enclose both parts in a <></> tag.

Here is the component with the HTML for both parts:

import React, { useRef, useState } from "react";

const DEFAULT_MAX_FILE_SIZE_IN_BYTES = 500000;
const KILO_BYTES_PER_BYTE = 1000;

const convertBytesToKB = (bytes) => Math.round(bytes / KILO_BYTES_PER_BYTE);

const FileUpload = ({
  label,
  updateFilesCb,
  maxFileSizeInBytes = DEFAULT_MAX_FILE_SIZE_IN_BYTES,
  ...otherProps
}) => {
  const fileInputField = useRef(null);
  const [files, setFiles] = useState({});

  return (
    <>
      <section>
        <label>{label}</label>
        <p>Drag and drop your files anywhere or</p>
        <button type="button">
          <i className="fas fa-file-upload" />
          <span> Upload {otherProps.multiple ? "files" : "a file"}</span>
        </button>
        <input
          type="file"
          ref={fileInputField}
          title=""
          value=""
          {...otherProps}
        />
      </section>

      {/*second part starts here*/}
      <article>
        <span>To Upload</span>
        <section>
          {Object.keys(files).map((fileName, index) => {
            let file = files[fileName];
            let isImageFile = file.type.split("/")[0] === "image";
            return (
              <section key={fileName}>
                <div>
                  {isImageFile && (
                    <img
                      src={URL.createObjectURL(file)}
                      alt={`file preview ${index}`}
                    />
                  )}
                  <div isImageFile={isImageFile}>
                    <span>{file.name}</span>
                    <aside>
                      <span>{convertBytesToKB(file.size)} kb</span>
                      <i className="fas fa-trash-alt" />
                    </aside>
                  </div>
                </div>
              </section>
            );
          })}
        </section>
      </article>
    </>
  );
};

export default FileUpload;

In the second part of the HTML, we are iterating through each file in the files state and displaying the file name, size in KB, and an image preview if the file type is image/* (i.e. png, jpg...etc).

To display an image preview, we are using the URL.createObjectURL function. The createObjectURL function takes an object, which in this case is a File object, and returns a temporary URL for accessing the file. We can then set that URL to src attribute of an img tag.

Styling

We will now use the styled-components package we installed earlier.

Add the following in the file-upload.styles.js file:

import styled from "styled-components";

export const FileUploadContainer = styled.section`
  position: relative;
  margin: 25px 0 15px;
  border: 2px dotted lightgray;
  padding: 35px 20px;
  border-radius: 6px;
  display: flex;
  flex-direction: column;
  align-items: center;
  background-color: white;
`;

export const FormField = styled.input`
  font-size: 18px;
  display: block;
  width: 100%;
  border: none;
  text-transform: none;
  position: absolute;
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
  opacity: 0;

  &:focus {
    outline: none;
  }
`;

export const InputLabel = styled.label`
  top: -21px;
  font-size: 13px;
  color: black;
  left: 0;
  position: absolute;
`;

export const DragDropText = styled.p`
  font-weight: bold;
  letter-spacing: 2.2px;
  margin-top: 0;
  text-align: center;
`;

export const UploadFileBtn = styled.button`
  box-sizing: border-box;
  appearance: none;
  background-color: transparent;
  border: 2px solid #3498db;
  cursor: pointer;
  font-size: 1rem;
  line-height: 1;
  padding: 1.1em 2.8em;
  text-align: center;
  text-transform: uppercase;
  font-weight: 700;
  border-radius: 6px;
  color: #3498db;
  position: relative;
  overflow: hidden;
  z-index: 1;
  transition: color 250ms ease-in-out;
  font-family: "Open Sans", sans-serif;
  width: 45%;
  display: flex;
  align-items: center;
  padding-right: 0;
  justify-content: center;

  &:after {
    content: "";
    position: absolute;
    display: block;
    top: 0;
    left: 50%;
    transform: translateX(-50%);
    width: 0;
    height: 100%;
    background: #3498db;
    z-index: -1;
    transition: width 250ms ease-in-out;
  }

  i {
    font-size: 22px;
    margin-right: 5px;
    border-right: 2px solid;
    position: absolute;
    top: 0;
    bottom: 0;
    left: 0;
    right: 0;
    width: 20%;
    display: flex;
    flex-direction: column;
    justify-content: center;
  }

  @media only screen and (max-width: 500px) {
    width: 70%;
  }

  @media only screen and (max-width: 350px) {
    width: 100%;
  }

  &:hover {
    color: #fff;
    outline: 0;
    background: transparent;

    &:after {
      width: 110%;
    }
  }

  &:focus {
    outline: 0;
    background: transparent;
  }

  &:disabled {
    opacity: 0.4;
    filter: grayscale(100%);
    pointer-events: none;
  }
`;

export const FilePreviewContainer = styled.article`
  margin-bottom: 35px;

  span {
    font-size: 14px;
  }
`;

export const PreviewList = styled.section`
  display: flex;
  flex-wrap: wrap;
  margin-top: 10px;

  @media only screen and (max-width: 400px) {
    flex-direction: column;
  }
`;

export const FileMetaData = styled.div`
  display: ${(props) => (props.isImageFile ? "none" : "flex")};
  flex-direction: column;
  position: absolute;
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
  padding: 10px;
  border-radius: 6px;
  color: white;
  font-weight: bold;
  background-color: rgba(5, 5, 5, 0.55);

  aside {
    margin-top: auto;
    display: flex;
    justify-content: space-between;
  }
`;

export const RemoveFileIcon = styled.i`
  cursor: pointer;

  &:hover {
    transform: scale(1.3);
  }
`;

export const PreviewContainer = styled.section`
  padding: 0.25rem;
  width: 20%;
  height: 120px;
  border-radius: 6px;
  box-sizing: border-box;

  &:hover {
    opacity: 0.55;

    ${FileMetaData} {
      display: flex;
    }
  }

  & > div:first-of-type {
    height: 100%;
    position: relative;
  }

  @media only screen and (max-width: 750px) {
    width: 25%;
  }

  @media only screen and (max-width: 500px) {
    width: 50%;
  }

  @media only screen and (max-width: 400px) {
    width: 100%;
    padding: 0 0 0.4em;
  }
`;

export const ImagePreview = styled.img`
  border-radius: 6px;
  width: 100%;
  height: 100%;
`;

When using styled-components, we are creating components that render an HTML tag with some styles. For example, the ImagePreview is a component that renders an img tag with the styles inside the tagged template literal.

Since we are creating components, we can pass props to it and access it when writing the styles (e.g. FileMetaData in the example above).

We have now finished the styling and adding drag and drop.

“But wait, when did we add drag and drop?”

By default, the file input tag supports drag and drop. We simply just styled the input tag and made it absolutely positioned (refer to FormField above).

To use the styles we wrote, import all the styled components and replace the HTML in the file-upload.component.jsx file.

import React, { useRef, useState } from "react";
import {
  FileUploadContainer,
  FormField,
  DragDropText,
  UploadFileBtn,
  FilePreviewContainer,
  ImagePreview,
  PreviewContainer,
  PreviewList,
  FileMetaData,
  RemoveFileIcon,
  InputLabel
} from "./file-upload.styles";

const DEFAULT_MAX_FILE_SIZE_IN_BYTES = 500000;
const KILO_BYTES_PER_BYTE = 1000;

const convertBytesToKB = (bytes) =>
  Math.round(bytes / KILO_BYTES_PER_BYTE);

const FileUpload = ({
  label,
  updateFilesCb,
  maxFileSizeInBytes = DEFAULT_MAX_FILE_SIZE_IN_BYTES,
  ...otherProps
}) => {
  const fileInputField = useRef(null);
  const [files, setFiles] = useState({});

    return (
    <>
      <FileUploadContainer>
        <InputLabel>{label}</InputLabel>
        <DragDropText>Drag and drop your files anywhere or</DragDropText>
        <UploadFileBtn type="button">
          <i className="fas fa-file-upload" />
          <span> Upload {otherProps.multiple ? "files" : "a file"}</span>
        </UploadFileBtn>
        <FormField
          type="file"
          ref={fileInputField}
          title=""
          value=""
          {...otherProps}
        />
      </FileUploadContainer>
      <FilePreviewContainer>
        <span>To Upload</span>
        <PreviewList>
          {Object.keys(files).map((fileName, index) => {
            let file = files[fileName];
            let isImageFile = file.type.split("/")[0] === "image";
            return (
              <PreviewContainer key={fileName}>
                <div>
                  {isImageFile && (
                    <ImagePreview
                      src={URL.createObjectURL(file)}
                      alt={`file preview ${index}`}
                    />
                  )}
                  <FileMetaData isImageFile={isImageFile}>
                    <span>{file.name}</span>
                    <aside>
                      <span>{convertBytesToKB(file.size)} kb</span>
                      <RemoveFileIcon
                        className="fas fa-trash-alt"
                      />
                    </aside>
                  </FileMetaData>
                </div>
              </PreviewContainer>
            );
          })}
        </PreviewList>
      </FilePreviewContainer>
    </>
  );
}

export default FileUpload;

Functionality

We are almost finished with the file-upload component, we just need to add functions so that files state can be modified.

Earlier we created a DOM reference using the useRef hook. We will now use that to open the file explorer once the "Upload Files" button is clicked. To do this, add the following function within the component:

const handleUploadBtnClick = () => {
  fileInputField.current.click();
};

We also need to add an onClick attribute to the UploadFileBtn component to trigger the function above.

<UploadFileBtn type="button" onClick={handleUploadBtnClick}>

To process the files selected by the user once the "Upload Files" button is clicked, we need to add an onChange attribute to the FormField component.

<FormField
  type="file"
  ref={fileInputField}
  onChange={handleNewFileUpload}
  title=""
  value=""
  {...otherProps}
/>

Like with any DOM event (e.g. onClick), the function to handle the event will have access to the event object. So, the handleNewFileUpload function will have the event object as its first parameter.

 const handleNewFileUpload = (e) => {
    const { files: newFiles } = e.target;
    if (newFiles.length) {
      let updatedFiles = addNewFiles(newFiles);
      setFiles(updatedFiles);
      callUpdateFilesCb(updatedFiles);
    }
  };

In the function above, we access the files selected by the user from the e.target.files property then pass it to a function called addNewFiles. Then, we take the return value from addNewFiles and pass it to the setFiles to update the files state. Since any changes to the files state must be sent to the parent component, we need to call the callUpdateFilesCb function.

The addNewFiles function takes a FileList object (e.target.files above returns a FileList), iterates through it, and returns an object where the key is the file name and the value is the File object.

  const addNewFiles = (newFiles) => {
    for (let file of newFiles) {
      if (file.size <= maxFileSizeInBytes) {
        if (!otherProps.multiple) {
          return { file };
        }
        files[file.name] = file;
      }
    }
    return { ...files };
  };

“Why are checking if there is not a multiple property in otherProps?”

As explained earlier, we are using the otherProps variable to add attributes to the file input tag. So, if we don't pass a multiple prop to the file upload component, then the file input tag does not allow for selecting multiple files. Put simply, if there is a multiple prop, selected files will get added to the files state. Otherwise, selecting a new file will remove the previous files state and replace it with the newly selected file.

The callUpdateFilesCb function takes the value returned from addNewFiles, converts the files state to an array and calls the updateFilesCb function (from the props).

“Why do we pass updatedFiles to callUpdateFilesCb when we could just use the files state within the function?”

Since React state updates are asynchronous, there is no guarantee that when the callUpdateFilesCb gets called, the files state will have changed.

"Why do we have to convert the files state to an array?"

We don't! However, when uploading files in the files state to some third party service (e.g. Firebase Cloud Storage), it's easier to work with arrays.

const convertNestedObjectToArray = (nestedObj) =>
  Object.keys(nestedObj).map((key) => nestedObj[key]);

const callUpdateFilesCb = (files) => {
  const filesAsArray = convertNestedObjectToArray(files);
  updateFilesCb(filesAsArray);
};

To remove a file, we first need to add an onClick attribute to the RemoveFileIcon component.

<RemoveFileIcon
  className="fas fa-trash-alt"
  onClick={() => removeFile(fileName)}
/>

The removeFile function will take a file name, delete it from the files state, update the files state, and inform the parent component of the changes.

const removeFile = (fileName) => {
  delete files[fileName];
  setFiles({ ...files });
  callUpdateFilesCb({ ...files });
};

Here is the component with all the functions above:

import React, { useRef, useState } from "react";
import {
  FileUploadContainer,
  FormField,
  DragDropText,
  UploadFileBtn,
  FilePreviewContainer,
  ImagePreview,
  PreviewContainer,
  PreviewList,
  FileMetaData,
  RemoveFileIcon,
  InputLabel
} from "./file-upload.styles";

const KILO_BYTES_PER_BYTE = 1000;
const DEFAULT_MAX_FILE_SIZE_IN_BYTES = 500000;

const convertNestedObjectToArray = (nestedObj) =>
  Object.keys(nestedObj).map((key) => nestedObj[key]);

const convertBytesToKB = (bytes) => Math.round(bytes / KILO_BYTES_PER_BYTE);

const FileUpload = ({
  label,
  updateFilesCb,
  maxFileSizeInBytes = DEFAULT_MAX_FILE_SIZE_IN_BYTES,
  ...otherProps
}) => {
  const fileInputField = useRef(null);
  const [files, setFiles] = useState({});

  const handleUploadBtnClick = () => {
    fileInputField.current.click();
  };

  const addNewFiles = (newFiles) => {
    for (let file of newFiles) {
      if (file.size < maxFileSizeInBytes) {
        if (!otherProps.multiple) {
          return { file };
        }
        files[file.name] = file;
      }
    }
    return { ...files };
  };

  const callUpdateFilesCb = (files) => {
    const filesAsArray = convertNestedObjectToArray(files);
    updateFilesCb(filesAsArray);
  };

  const handleNewFileUpload = (e) => {
    const { files: newFiles } = e.target;
    if (newFiles.length) {
      let updatedFiles = addNewFiles(newFiles);
      setFiles(updatedFiles);
      callUpdateFilesCb(updatedFiles);
    }
  };

  const removeFile = (fileName) => {
    delete files[fileName];
    setFiles({ ...files });
    callUpdateFilesCb({ ...files });
  };

  return (
    <>
      <FileUploadContainer>
        <InputLabel>{label}</InputLabel>
        <DragDropText>Drag and drop your files anywhere or</DragDropText>
        <UploadFileBtn type="button" onClick={handleUploadBtnClick}>
          <i className="fas fa-file-upload" />
          <span> Upload {otherProps.multiple ? "files" : "a file"}</span>
        </UploadFileBtn>
        <FormField
          type="file"
          ref={fileInputField}
          onChange={handleNewFileUpload}
          title=""
          value=""
          {...otherProps}
        />
      </FileUploadContainer>
      <FilePreviewContainer>
        <span>To Upload</span>
        <PreviewList>
          {Object.keys(files).map((fileName, index) => {
            let file = files[fileName];
            let isImageFile = file.type.split("/")[0] === "image";
            return (
              <PreviewContainer key={fileName}>
                <div>
                  {isImageFile && (
                    <ImagePreview
                      src={URL.createObjectURL(file)}
                      alt={`file preview ${index}`}
                    />
                  )}
                  <FileMetaData isImageFile={isImageFile}>
                    <span>{file.name}</span>
                    <aside>
                      <span>{convertBytesToKB(file.size)} kb</span>
                      <RemoveFileIcon
                        className="fas fa-trash-alt"
                        onClick={() => removeFile(fileName)}
                      />
                    </aside>
                  </FileMetaData>
                </div>
              </PreviewContainer>
            );
          })}
        </PreviewList>
      </FilePreviewContainer>
    </>
  );
};

export default FileUpload;

Use Case

Let's now use the file upload component in App component to see it in action!

In the App.js file, we will create a simple form and add state to store the form data.

import React, { useState } from "react";

function App() {
  const [newUserInfo, setNewUserInfo] = useState({
    profileImages: []
  });

  const handleSubmit = (event) => {
    event.preventDefault();
    //logic to create a new user...
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <button type="submit">Create New User</button>
      </form>
    </div>
  );
}
export default App;

Now to add the file upload component.

import React, { useState } from "react";
import FileUpload from "./components/file-upload/file-upload.component";

function App() {
  const [newUserInfo, setNewUserInfo] = useState({
    profileImages: []
  });

  const handleSubmit = (event) => {
    event.preventDefault();
    //logic to create a new user...
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <FileUpload
          accept=".jpg,.png,.jpeg"
          label="Profile Image(s)"
          multiple
        />
        <button type="submit">Create New User</button>
      </form>
    </div>
  );
}

export default App;

Notice we have not added the updateFilesCb prop yet. Before we can do that, we need to create a function that updates only the profileImages property of the newUserInfo state.

const updateUploadedFiles = (files) =>
    setNewUserInfo({ ...newUserInfo, profileImages: files });

We will now pass this function as the updateFilesCb prop. So, any time the files state changes in the file upload component, it will be saved in the profileImages property of the newUserInfo state.

import React, { useState } from "react";
import FileUpload from "./components/file-upload/file-upload.component";

function App() {
  const [newUserInfo, setNewUserInfo] = useState({
    profileImages: []
  });

  const updateUploadedFiles = (files) =>
    setNewUserInfo({ ...newUserInfo, profileImages: files });

  const handleSubmit = (event) => {
    event.preventDefault();
    //logic to create new user...
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <FileUpload
          accept=".jpg,.png,.jpeg"
          label="Profile Image(s)"
          multiple
          updateFilesCb={updateUploadedFiles}
        />
        <button type="submit">Create New User</button>
      </form>
    </div>
  );
}

export default App;

“Why are we passing the accept and multiple prop to the file upload component?”

Since any additional props will get passed to the file input tag, the file input tag will have an accept and multiple attribute.

The multiple attribute allows a user to select multiple files in the file explorer.

The accept attribute prevents users from selecting file types different from the ones specified (i.e. jpg, png, jpeg in this case).

Now that we are finished, run npm start and visit localhost:3000. The following should appear:

For reference, the code can be found at
https://github.com/Chandra-Panta-Chhetri/react-file-upload-tutorial.

Sending Emails Securely Using Node.js, Nodemailer, SMTP, Gmail, and OAuth2

Chandra Panta Chhetri — Wed, 16 Dec 2020 23:05:54 +0000

Many solutions online regarding configuring Nodemailer to use your Gmail requires you to enable less secure app access. If that sounds too scary for you, then you have come to the right place! In this article, you will learn how to securely configure Nodemailer and Gmail.

Let's start by understanding what Nodemailer is.

Nodemailer is a module that makes sending emails from Node.js applications ridiculously easy.

The following are the main steps required to send emails:

Creating a transporter (object used to send emails) using either SMTP or some other transport mechanism
Setting up message options (who sends what to whom)
Sending the email by calling sendMail method on the transporter

Less Secure Configuration

Before we look at the secure solution for configuring Nodemailer and Gmail, let's look at the less secure solution.

Using the steps above as a reference, here is the corresponding code:



//Step 1: Creating the transporter
const transporter = nodemailer.createTransport({
    service: "Gmail",
    auth: {
          user: "******@gmail.com",
          pass: "gmail_password"
        }
});

//Step 2: Setting up message options
const messageOptions = {
  subject: "Test",
  text: "I am sending an email from nodemailer!",
  to: "put_email_of_the_recipient",
  from: "put_email_of_sender"
};

//Step 3: Sending email
transporter.sendMail(messageOptions);

Note: the solution above won't work until you enable less secure app access in Google account settings.

Now, let's look at the more secure solution.

Step 1: Creating a Google Project

Visit Google Developer Console to create a project. A project is needed so that we can create the necessary API credentials.

Once in the console, click the dropdown in the top left corner.

After the create project window loads, click New Project.

Enter in the project name and click create.

Step 2: Creating OAuth 2.0 API Credentials

To get the client secret and client id, we need to create OAuth credentials. A client id identifies our app to Google's OAuth servers so that we can securely send emails from Nodemailer.

Start by selecting credentials in the sidebar on the left. Once selected, the following screen should appear:

After clicking create credentials, a dropdown will appear. In the dropdown, select OAuth client ID.

Before proceeding, we need to configure the consent screen. The consent screen configuration is important when an application offers Google Sign In. Nevertheless, it must be completed so we can create a client id and secret.

Click configure consent screen.

Select external for the User Type and then click create.

After the multi-step form appears, fill out the required fields for each step.

Once on the last step, click back to dashboard.

Go back to the Create OAuth client ID screen (page with the configure consent screen button). If the consent screen has been configured successfully, an application type dropdown should appear. Select Web application and fill in the required field(s).

In the Authorized redirect URIs section, make sure to add https://developers.google.com/oauthplayground.

Now click create!

Copy the client ID and client secret shown on the screen and save it for later.

Step 3: OAuth 2.0 Playground

We also need a refresh token and access token which can be generated from the client id and secret.

Start by visiting https://developers.google.com/oauthplayground.
Once on the page, click the gear icon and check the Use your own OAuth credentials box. Then paste in the client id and secret from before.

On the left, under the Select & authorize APIs section, find Gmail API v1 and select https://mail.google.com/. Alternately, you can also type https://mail.google.com/ into the Input your own scopes field.

Now click Authorize APIs.

If the following pages appear, click allow so that Google OAuth 2.0 Playground has access to your Google account.

After being redirected back to the OAuth 2.0 Playground,
click the Exchange authorization code for tokens button under the Exchange authorization code for tokens section.

Once the refresh and access token is generated, copy the refresh token and save it for later.

Step 4: Writing Code

Now that we have the client id, client secret, and refresh token, we can now use them to send emails!

Start by making a new folder for the application and cd into the folder.



mkdir sendEmails
cd sendEmails

To initialize the app as a node project, run npm init.

Next, let's install the npm packages.



//Note: dotenv is a dev dependency
npm i nodemailer googleapis && npm i dotenv --save-dev

googleapis

library for using Google APIs
Will be used to dynamically generate access token

dotenv

library for using environment variables
Will be used to avoid having API keys in our code

Like with any NPM packages, we start by requiring the packages. So, create an index.js file and add the following:



const nodemailer = require("nodemailer");
const { google } = require("googleapis");
const OAuth2 = google.auth.OAuth2;

Environment Variables Setup

Typically when using sensitive info in code (e.g. API keys), the best practice is to use environment variables.

Create a .env file in the root directory of the project and add the following:



EMAIL=YOUR_GOOGLE_EMAIL_HERE
REFRESH_TOKEN=PASTE_REFRESH_TOKEN_HERE
CLIENT_SECRET=PASTE_CLIENT_SECRET_HERE
CLIENT_ID=PASTE_CLIENT_ID_HERE

Now, we need to require and call the config method before requiring all the packages:



require('dotenv').config();
const nodemailer = require("nodemailer");
const { google } = require("googleapis");
const OAuth2 = google.auth.OAuth2;

process.env now has the keys and values defined in the .env file. For example, we can access client id via process.env.CLIENT_ID

Creating a transporter

We first need to create an OAuth client with all of our info from before (client ID, client secret, and the OAuth Playground URL). The OAuth client will allow us to dynamically create an access token from a refresh token.

“But wait, why can't we just use the access token from the OAuth Playground? Or why are we creating the access token dynamically?”

Well, if you noticed earlier, there was a message indicating the access token would expire after 3582 seconds.

The following code creates the OAuth client and provides it with the refresh token:



const oauth2Client = new OAuth2(
    process.env.CLIENT_ID,
    process.env.CLIENT_SECRET,
    "https://developers.google.com/oauthplayground"
);

oauth2Client.setCredentials({
    refresh_token: process.env.REFRESH_TOKEN
});

Since getting the access token through the OAuth client is an asynchronous process, we need to wrap the above in an async function.



const createTransporter = async () => {
  const oauth2Client = new OAuth2(
    process.env.CLIENT_ID,
    process.env.CLIENT_SECRET,
    "https://developers.google.com/oauthplayground"
  );

  oauth2Client.setCredentials({
    refresh_token: process.env.REFRESH_TOKEN
  });
};

Now, we can get the access token by calling the getAccessToken method.



const accessToken = await new Promise((resolve, reject) => {
  oauth2Client.getAccessToken((err, token) => {
    if (err) {
      reject("Failed to create access token :(");
    }
    resolve(token);
  });
});

You might be wondering, why are we wrapping the getAccessToken method call in a promise? This is because getAccessToken requires a callback and does not support using async await. Thus, we can either wrap it in a promise or create the transporter inside the callback. I prefer the former as it is more readable.

Now for the main part, creating the transporter object itself. To create it, we pass some configurations to the createTransport method.



const transporter = nodemailer.createTransport({
  service: "gmail",
  auth: {
    type: "OAuth2",
    user: process.env.EMAIL,
    accessToken,
    clientId: process.env.CLIENT_ID,
    clientSecret: process.env.CLIENT_SECRET,
    refreshToken: process.env.REFRESH_TOKEN
  }
});

Note: If you receive an "unauthorized client", try adding the following to the JS object above.



tls: {
  rejectUnauthorized: false
}

After the transporter is created, the completed createTransporter function should look like this:



const createTransporter = async () => {
  const oauth2Client = new OAuth2(
    process.env.CLIENT_ID,
    process.env.CLIENT_SECRET,
    "https://developers.google.com/oauthplayground"
  );

  oauth2Client.setCredentials({
    refresh_token: process.env.REFRESH_TOKEN
  });

  const accessToken = await new Promise((resolve, reject) => {
    oauth2Client.getAccessToken((err, token) => {
      if (err) {
        reject();
      }
      resolve(token);
    });
  });

  const transporter = nodemailer.createTransport({
    service: "gmail",
    auth: {
      type: "OAuth2",
      user: process.env.EMAIL,
      accessToken,
      clientId: process.env.CLIENT_ID,
      clientSecret: process.env.CLIENT_SECRET,
      refreshToken: process.env.REFRESH_TOKEN
    }
  });

  return transporter;
};

Notice we are returning the transporter instead of writing the code to send an email. We will create another function for sending the email for the sake of code readability and separations of concerns.

Let's now create the sendEmail function. This function calls the createTransporter function and then the sendMail method that exists on the transporter.



//emailOptions - who sends what to whom
const sendEmail = async (emailOptions) => {
  let emailTransporter = await createTransporter();
  await emailTransporter.sendMail(emailOptions);
};

All that is left now is to send the email by calling the sendEmail function:



sendEmail({
  subject: "Test",
  text: "I am sending an email from nodemailer!",
  to: "put_email_of_the_recipient",
  from: process.env.EMAIL
});

The complete list of the email options can be found at https://nodemailer.com/message/.

Run node index.js from the terminal/command line and Voila! Here is the email we sent from the application!

For reference, here is the completed index.js file:



require("dotenv").config();
const nodemailer = require("nodemailer");
const { google } = require("googleapis");
const OAuth2 = google.auth.OAuth2;

const createTransporter = async () => {
  const oauth2Client = new OAuth2(
    process.env.CLIENT_ID,
    process.env.CLIENT_SECRET,
    "https://developers.google.com/oauthplayground"
  );

  oauth2Client.setCredentials({
    refresh_token: process.env.REFRESH_TOKEN
  });

  const accessToken = await new Promise((resolve, reject) => {
    oauth2Client.getAccessToken((err, token) => {
      if (err) {
        reject("Failed to create access token :(");
      }
      resolve(token);
    });
  });

  const transporter = nodemailer.createTransport({
    service: "gmail",
    auth: {
      type: "OAuth2",
      user: process.env.EMAIL,
      accessToken,
      clientId: process.env.CLIENT_ID,
      clientSecret: process.env.CLIENT_SECRET,
      refreshToken: process.env.REFRESH_TOKEN
    }
  });

  return transporter;
};

const sendEmail = async (emailOptions) => {
  let emailTransporter = await createTransporter();
  await emailTransporter.sendMail(emailOptions);
};

sendEmail({
  subject: "Test",
  text: "I am sending an email from nodemailer!",
  to: "put_email_of_the_recipient",
  from: process.env.EMAIL
});

Firebase Security Rules

Chandra Panta Chhetri — Sun, 06 Dec 2020 01:52:45 +0000

What is Firebase?

Firebase is a platform that provides easy integration of commonly used services for mobile and web applications. These services include

Firestore
Cloud Functions
Authentication
Cloud Storage
Realtime Database
Hosting
Firebase ML

How do security rules come into play?

Traditionally, access to a database is controlled via server-side code. However, Firebase services such as Firestore allow client-side code to access the database directly. This poses a security risk as client-side code can be viewed by anyone. This is where Firebase's concept of security rules come into play. Firebase security rules control access to specific resources. In other words, we can avoid the hassle of writing, maintaining, and deploying server-side code that controls access to resources. Security rules can only be written for Realtime Database, Firestore, and Cloud Storage.

Writing Security Rules

Security rules act as a middleware for when a service is used (e.g. reading a document in Firestore). When a request to access a service is denied due to security rules, the entire request fails. Rules can be written via Firebase console or an IDE that is using the Firebase CLI. Writing rules in the console provides the benefit of using Rules Playground before deploying the rules.

Firestore Security Rules

Firestore Security Rules consist of match statements and allow expressions. Match statements identify documents in your database and allow expressions control access to those documents.

The basic structure of a security rule:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /<some_path>/ {
      allow read: if <some_condition>;
      allow write: if <some_condition>;
    }
  }
}

In the example above, we are matching some path in the database and allowing read and write permissions based on a condition. In situations where we need finer access control, we can split read and write into more granular operations.

read can be broken into

get (applies to single document read requests)
list (applies to queries and collection read requests)

write can be broken into

create
update
delete

We can also define custom functions to improve readability and reusability.

Let's take a look at an example

rules_version = '2';  
service cloud.firestore {
  match /databases/{database}/documents {  
    function isAdmin() {
      let userDoc = get(/databases/$(database)/documents/users/$(request.auth.uid));
      return userDoc == null ? false : userDoc.data.isAdmin;
    }
    function isAuthenticated(){
        return request.auth != null;
    }
    match /users/{userId}{
        allow read, delete: if isAuthenticated() 
                            && request.auth.uid == resource.id;
        allow create: if request.resource.data.keys().hasAll(['name', 'email', 'isAdmin']) 
                      && request.resource.data.keys().hasOnly(['name', 'email', 'isAdmin'])
                      && request.resource.data.isAdmin == false;
        allow update: if isAuthenticated()
                      && request.auth.uid == resource.id
                      && request.resource.data.isAdmin == resource.data.isAdmin;
    }
 }
}

Based on the example above, we have written the following rules for the users collection:

reading user(s) and deleting a user is allowed if the user is authenticated & the user document being read/deleted belongs to the authenticated user
creating a user is allowed if the request body is of the form {name, email, isAdmin} && the isAdmin field is false (to prevent creating admin accounts from client-side)
updating a user is allowed if the user is authenticated, the user document being updated belongs to the authenticated user, and the isAdmin field remains unchanged

You might be wondering where variables and methods such as get(), request, resource are coming from. Well, any custom functions or match statements in the service cloud.firestore namespace has access to the following:

Properties

request (request context)
- Contains auth and incoming request information via auth & resource property
- request.resource is used to enforce write operations (create, update, delete) have specific structure and field validations (e.g. age >= 15)
- request.auth is used to provide access based on if the request is authenticated
- More info can be found at https://firebase.google.com/docs/reference/rules/rules.firestore.Request
resource
- resource being written to or read (i.e. document in a collection)
- Document data and id can be accessed via resource.data & resource.id
- Useful for enforcing certain fields cannot be changed. In the example above, we are preventing the isAdmin field from changing in update user requests by request.resource.data.isAdmin == resource.data.isAdmin.
- More info can be found at https://firebase.google.com/docs/reference/rules/rules.firestore.Resource

Methods

get(path) - path must be absolute path to a document
- Gets the contents of a document
- Useful when the modification of a document depends on another document
- e.g. The isAdmin() function in the example above gets the user document based on request.auth.uid then accesses the isAdmin field to check if the user is an Admin

The complete list of methods and properties provided in the service cloud.firestore namespace can be found at https://firebase.google.com/docs/reference/rules/rules.firestore

Firebase also provides functions for working with different property types (e.g. list, boolean, map...etc) when writing rules. In the example above, we are using hasOnly() and hasAll() to enforce a user document only has 'name', 'email', and 'isAdmin' field.

More info can be found at https://firebase.google.com/docs/reference/rules

Cloud Storage Security Rules

Security rules for Cloud Storage are similar to Firestore rules except instead of matching documents in a collection, we match directories or paths to files.

Let's take a look at an example

rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /product_images/{imageName}
    {
        allow read, delete: if request.auth != null;
        allow create: if request.auth != null 
                      && request.resource.contentType.matches('image/.*')
                      && request.resource.size < 100000;
    }
  }
}

Notice the similarities between Firestore rules and Cloud Storage rules. The main difference being service firebase.storage namespace provides us with different properties on the request variable.

Full list of the properties can be found at https://firebase.google.com/docs/storage/security/rules-conditions

Based on the example above, we have written the following rules for the product_images directory:

reading and deleting files is allowed if the user is authenticated
files can be added if the user is authenticated & the file is an image file that is smaller than 100 000 bytes

Realtime Database Security Rules

Firebase's Realtime Database stores data as a large JSON object. This means when writing security rules, we match keys in the object instead of documents in a collection (like in Firestore).

The basic structure of a security rule:

{
  "rules": {
    "parent_node": {
      "child_node": {
        ".read": <condition>,
        ".write": <condition>,
        ".validate": <condition>,
      }
    }
  }
}

Before writing rules, an important concept to keep in mind is that read & write rules cascade. In other words, if a rule grants read & write access at a particular path, then it grants access to all child nodes. Furthermore, shallower rules override deeper rules.

Let's take a look at an example

{
  "rules": {
     "foo": {
        // allows read to /foo/*
        ".read": "true",
        "bar": {
          /* bar can be read so this rule is ignored since read 
             was allowed already */
          ".read": false
        }
     }
  }
}

Firebase Realtime Database provides pre-defined variables similar to how service firebase.storage namespace provides global variables & methods.

The complete list of variables and methods can be found at https://firebase.google.com/docs/rules/rules-language#building_conditions

A more practical example:

{
  "rules": {
    "users": {
      "$uid": {
        ".read": "auth != null && auth.uid == $uid",
        ".write": "auth != null && auth.uid == $uid",
        ".validate": "newData.hasChildren(['name', 'age'])",
        "age": {
            ".validate": "newData.isNumber() &&
                          newData.val() > 0"
         }
      }
    }
  }
}

Based on the example above, we have written the following rules for the users node:

write & read access is allowed user node belongs to the authenticated user
a child node in the users node must have a 'name' & 'age' property
the 'age' property must be a number greater than 0

Conclusion

Although different services (Firestore, Cloud Storage, Realtime Database) have slightly different syntax when writing rules, at a high level we are doing the same thing. More specifically, we match a path to a resource and control access to that resource via read and write conditions.

Nevertheless, Firebase security rules serve the purpose of protecting data from malicious users. Thus, I hope this article helped you understand the main concepts when writing security rules! Now go protect your data!