DEV Community: Adarsh

A Brief Introduction To RxJS

Adarsh — Wed, 20 May 2020 06:02:36 +0000

ReactiveXJS or also known as RxJS is an asynchronous programming API that's built on the principles of Reactive Programming. It does so by borrowing concepts from functional programming and design patterns like the Observer, Publisher-Subscriber, Broadcast etc.

What is Reactive Programming?

Reactive Programming is a paradigm like the Object Oriented Programming that emphasizes on the fact that our code should react whenever there is a change in the input values. This paradigm is useful in handling huge volumes of asynchronous data such as streams, events and propagation of change that happens when an event is triggered.

You might be thinking "Reactive programming sounds a lot like what we already do in when we try to code asynchronous tasks". Yes it does but the code we write to manage the asynchronous calls can get quite messy and error prone as the application scales up. A piece of asynchronous code can quickly turn into a nightmare for developers when it's not handled properly.

Reactive Programming lays out practices and principles that allows developers to manage the asynchronous calls while making the code more readable, pure and less error prone. The reactive programming principle was introduced back in the 1960s and in 2014 the Reactive Manifesto was published which laid out the need and principles of Reactive Programming.

RxJS: An Introduction

RxJS is a JavaScript library that provides APIs to write reactive code in JavaScript.

Mentioned below are the terms that we will be using frequently,

Observable: An entity that our observer/subscriber monitors and reacts to. An example would be a data stream or sequence of events.

Subscriber: An entity that reacts to the values emitted from the oberservable.

Subjects: These are special class of observables that also is an subscriber and allows multiple subscribers to be subscribed to it. It's similar to the broadcast model.

Operators: Functions that allows to create, manage and transform observables. Some examples are map, filter, reduce etc.

Marble Diagrams

To allow us to better visualize the way data flows and changes through the RxJS API we will use "Marble Diagrams".

In the above diagram, right headed arrow represents time duration of the data stream which keeps increasing that is going from past to future. The circles/marbles represent the events that occur at a particular point in time. The vertical bar '|' represents the end of the stream and a successful completion while an 'X' signifies an error which occurred in the stream and an termination of execution.

The block represents the function which is being applied on the stream this function gets invoked whenever there is a new element emitted by the stream. The return type of this function is also a stream so we can chain multiple functions one after the other.

Observable

Observable are objects that allows an subscriber to be subscribed to it and emits values for the subscriber to act. In a reactive programming environment the observable is responsible for pushing changes and data to the subscribers.

There are generally two types of observables:

Hot: Starts emitting values as soon as it is created
Cold: Starts emitting values only after an subscriber is subscribed to it.

Let's take a look the following code

const { Observable } = require('rxjs');

const subscribe = (subscriber) => {
  subscriber.next('Hello');
  subscriber.next('World');
  subscriber.next(5);
  subscriber.next(true);
  subscriber.complete();
};

const observable = new Observable(subscribe);

const subscriber = {
  next: (value) => console.log(`Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

observable.subscribe(subscriber);
console.log('Subscribed');

The above code produces the following output:

Emitted: Hello
Emitted: World
Emitted: 5
Emitted: true
End of the stream
Subscribed

First we create an Observable by using the Observable constructor which takes a subscribe function as the parameter. This subscribe function itself has a parameter called subscriber. The subscriber comprises of 3 methods next, complete and error. The next method is used to emit the value to the subscriber, complete function is used to signal that we have reached the end and error function is used to process any error that is thrown by the observable.

If you notice we emit values of multiple data types. In the above instance the first two values we emit are strings, the third value is a number and the final value is a boolean. So an observable is not bound to any certain data type.

We then created a subscriber object which subscribes to the Observable instance to process the emitted values. The subscribe methods of an Observable instance takes an object as a parameter. The object again consists of three key, value pairs next, complete and error. The values for the keys are functions.

The functions subscriber.next() and subscriber.complete() in the observable instance merely executes the code we have written for the next and completed keys in our subscriber object.

Now let's simulate an error in the stream.

const { Observable } = require('rxjs');

const subscribe = (subscriber) => {
  subscriber.next('Hello');
  subscriber.next('World');
  subscriber.error(new Error('Some error in stream'));
  subscriber.next(5);
  subscriber.next(true);
};

const observable = new Observable(subscribe);

const subscriber = {
  next: (value) => console.log(`Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
  error: (err) => console.error(`${err}`),
};

observable.subscribe(subscriber);
console.log('Subscribed');

The above code will produce the following output:

Emitted: Hello
Emitted: World
Error: Some error in stream
Subscribed

Note that both 5 and true are missing from the output this is because once the subscriber encounters an error or complete signal it will stop processing the stream.

Whatever we wrote above is actually synchronous and not asynchronous. Let's modify it a little bit to make it asynchronous.

const { Observable } = require('rxjs');

const subscribe = (subscriber) => {
  setTimeout(() => subscriber.next('Hello'), 1000);
  setTimeout(() => subscriber.next('World'), 3000);
  setTimeout(() => subscriber.next(5), 5000);
  setTimeout(() => subscriber.next(true), 7000);
  setTimeout(() => subscriber.complete(), 9000);
};

const observable = new Observable(subscribe);

const subscriber = {
  next: (value) => console.log(`Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

observable.subscribe(subscriber);
console.log('Subscribed');

Now we will get the following output with an interval of 2 seconds before printing each line after the first line.

Subscribed
Emitted: Hello
Emitted: World
Emitted: 5
Emitted: true
End of the stream

If you notice the 'Subscribed' gets printed on to the console first because in our observable we emit values after a certain duration. Every time the subscriber.next() is called in the setTimeout it triggers the next function of our subscriber and prints the value finally we trigger subscriber.complete().

Last but not least we can create an observable in many ways, more details can be found here.

Congratulations, we have written our first code in RxJS. Let's go ahead the explore other features of the library.

RxJS Operators

Operators in RxJS are pure functions that transforms, process and manages the data that it receives from the observable. For example, map, filter, forEach etc. are some operators.

const { from } = require('rxjs');
const { map, take, filter } = require('rxjs/operators');

function* generateUserEverySecond() {
  previous = Date.now()
  while (true) {
    if (Date.now() - previous >= 1000) {
      previous = Date.now();
      yield {
        createdDate: Date.now(),
        updateDate: Date.now(),
        userId: Math.floor(Math.random() * 100000),
        numberOfLikes: Math.floor(Math.random() * 100),
      };
    }
  }
}

const observable = from(generateUserEverySecond())
  .pipe(
    map((x) => ({ user: x.userId, likes: x.numberOfLikes })),
    filter((x) => x.likes > 80),
    take(5)
  );

const subscriber = {
  next: (value) => console.log(`Emitted: ${JSON.stringify(value)}`),
  complete: () => console.log('End of the stream'),
};

observable.subscribe(subscriber);

Output:

Emitted: {"user":48219,"likes":93}
Emitted: {"user":7996,"likes":90}
Emitted: {"user":39907,"likes":82}
Emitted: {"user":53731,"likes":96}
Emitted: {"user":53499,"likes":84}
End of the stream

The from function is used to convert an Promise, Iterable or an Array into an Observable. The generateUserEverySecond generator will yield an object every second.

We have used the pipe function to apply the operators and each of the operators will be called whenever it encounters a new data. The pipe function kind of acts an pipeline and each operator can be thought of a gates in the pipeline that control how the data flows and changes across the pipeline.

We have applied three operators on our data map, filter and take function. The map functions transforms the input object into another object and produces it as the input to the next operator. The filter function picks only those objects which have likes greater than 80. Finally the take function tells the observable to signal complete after we obtain 5 objects as the output meaning our stream will end after the filter operator produces 5 objects.

Let's visualize our code with a marble diagram to see the data flow and changes.

Tip 1: Before writing a reactive code try to chart out a marble diagram so you will know what exactly you need to do at each step.

Tip 2: In-case you are stuck with wondering what operator to use. Use the operator decision tree tool in RxJS. It can be found here.

Subjects

Generally an observable allows only one subscriber to be subscribed to it at any given time. This doesn't bode well for certain use cases where we might need multiple subscribers to be subscribed to the same event for example, Broadcasting data.

Subjects are those special observables that can also act as subscribers meaning they can be used to subscribe to another Observable.

const { Subject } = require('rxjs');

const subject = new Subject();

const subscriberA = {
  next: (value) => console.log(`Subscriber A Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

const subscriberB = {
  next: (value) => console.log(`Subscriber B Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberA);
subject.subscribe(subscriberB);

subject.next('Foo');
subject.next('Bar')
subject.complete();

In the above code the subject acts as an Observable i.e. it only emits values. So the output is similar to how our code worked for synchronized Observable. However we have 2 subscribers A and B attached the subject so both gets invoked every time our subject emits an value.

Subscriber A Emitted: Foo
Subscriber B Emitted: Foo
Subscriber A Emitted: Bar
Subscriber B Emitted: Bar
End of the stream
End of the stream

Let's tweak the above code and see what happens.

const { Subject } = require('rxjs');

const subject = new Subject();

const subscriberA = {
  next: (value) => console.log(`Subscriber A Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

const subscriberB = {
  next: (value) => console.log(`Subscriber B Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

setTimeout(() => {
  subject.subscribe(subscriberA);
}, 2000);

setTimeout(() => {
  subject.subscribe(subscriberB);
}, 4000);

setTimeout(() => {
  subject.next('Foo');
}, 1000);
setTimeout(() => {
  subject.next('Bar')
}, 3000);
setTimeout(() => {
  subject.next('Baz')
}, 5000);

setTimeout(() => {
  subject.complete();
}, 7000);

Now the output becomes.

Subscriber A Emitted: Bar
Subscriber A Emitted: Baz
Subscriber B Emitted: Baz
End of the stream
End of the stream

Subscriber A caught every value starting from Bar while Subscriber B caught every value starting from Baz but none of them caught the value Foo that's because unlike general observables Subjects are Hot Observables so they don't wait for an subscriber to start emitting values.

Now let's take a look at an example where a subject acts as a subscriber.

const { Subject, Observable } = require('rxjs');

const observable = new Observable((subscriber) => {
  subscriber.next('Hello');
  subscriber.next('World');
  subscriber.next(5);
  subscriber.next(true);
  subscriber.complete();
});

const subject = new Subject();

const subscriberA = {
  next: (value) => console.log(`Subscriber A Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

const subscriberB = {
  next: (value) => console.log(`Subscriber B Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberA);
subject.subscribe(subscriberB);

observable.subscribe(subject);

The output will be

Subscriber A Emitted: Hello
Subscriber B Emitted: Hello
Subscriber A Emitted: World
Subscriber B Emitted: World
Subscriber A Emitted: 5
Subscriber B Emitted: 5
Subscriber A Emitted: true
Subscriber B Emitted: true
End of the stream
End of the stream

In the above example we have created an Observable, a Subject, two subscribers A and B. The the two subscribers A and B subscribe to subject while the subject itself subscribes to observable. So when the observable emits a value it triggers the subject which emits the same value to all the subscribers. This is a way of converting a Cold Observable to a Hot Observable.

Types of Subjects

There are 4 kinds of Subjects in RxJS:

PublishSubject or Subject: It emits only those values to the subscriber which are emitted after the subscriber subscribes to the subject. Whenever we import Subject from RxJS we are actually creating a PublishSubject.
ReplaySubject: It emits all the values that were emitted before the subscriber subscribed to the subject in the original order before emitting the latest value to all subscribers.
BehaviorSubject: When a subscriber subscribes to the subject it emits starting from the latest value that was emitted before the subscription happened.
AsyncSubject: This subject emits only the last value emitted from the source to all subscribers subscribed before the stream was completed.

The marble diagram and more in-depth explanation of these types can be found here.

Replay Subject

const { ReplaySubject } = require('rxjs');

const subject = new ReplaySubject();

const subscriberA = {
  next: (value) => console.log(`Subscriber A Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberA);

subject.next('Foo');
subject.next('Bar');

const subscriberB = {
  next: (value) => console.log(`Subscriber B Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberB);

subject.next('Baz');
subject.complete();

Output:

Subscriber A Emitted: Foo
Subscriber A Emitted: Bar
Subscriber B Emitted: Foo
Subscriber B Emitted: Bar
Subscriber A Emitted: Baz
Subscriber B Emitted: Baz
End of the stream
End of the stream

Since the values Foo and Bar were emitted before SubscriberB subscribed to the subject it emits the previous values to SubscriberB before emitting the next value Baz.

Behavior Subject

const { BehaviorSubject } = require('rxjs');

const subject = new BehaviorSubject();

const subscriberA = {
  next: (value) => console.log(`Subscriber A Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberA);

subject.next('Foo');
subject.next('Bar');

const subscriberB = {
  next: (value) => console.log(`Subscriber B Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberB);

subject.next('Baz');
subject.complete();

The output of the above code is

Subscriber A Emitted: undefined
Subscriber A Emitted: Foo
Subscriber A Emitted: Bar
Subscriber B Emitted: Bar
Subscriber A Emitted: Baz
Subscriber B Emitted: Baz
End of the stream
End of the stream

There are two things to notice in the output. The first line has emitted value as undefined this is because Behavior Subject by default will emit the latest emitted value before the subscriber subscribed to it. In our case the subject didn't emit any value before SubscriberA subscribed to the subject so it emitted undefined when subscriber A subscribed. Meanwhile the value Bar was the last emitted value before the subscriber B subscribed. So when the subscription of B happened the subject emitted the value Bar to B before proceeding to emit other values to all subscribers.

Async Subject

const { AsyncSubject } = require('rxjs');

const subject = new AsyncSubject();

const subscriberA = {
  next: (value) => console.log(`Subscriber A Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberA);

subject.next('Foo');
subject.next('Bar');

const subscriberB = {
  next: (value) => console.log(`Subscriber B Emitted: ${value}`),
  complete: () => console.log('End of the stream'),
};

subject.subscribe(subscriberB);

subject.next('Baz');
subject.complete();

Output:

Subscriber A Emitted: Baz
Subscriber B Emitted: Baz
End of the stream
End of the stream

Since AsyncSubject emits only the last value the subscribers got invoked for the value Baz and not other values regardless of the time of subscription.

Use Cases

We've gone over the basics of the library let's look at the use cases that can benefit using RxJS.

DOM Events

We can handle various DOM events such as change and click using RxJS to streamline the event handling process. Let's take a look at one such example of processing the input value.

Consider an input element with the id reactive-input in our DOM. We will write a reactive code that will convert the value to upper case every-time a user types in the input field.

const { fromEvent } = rxjs;
const { map } = rxjs.operators;

const eventObservable = fromEvent(document.getElementById('reactive-input'), 'input')
  .pipe(
      map((x) => x.target.value),
      map((x) => x.toUpperCase())
  )

eventObservable.subscribe(x => console.log(x));

In the above code we listen to the input event of reactive-input field and we convert the event to an Observable using fromEvent and we perform map two times. Firstly to extract the value that was typed and second to convert it to upper case. Finally we subscribe to the observable and print the value.

This is a simple use-case however we can orchestrate a whole lot of tasks for example fetching and processing an API output on the click of button. The most famous use case that can be solved easily with RxJS is the notorious Drag-n-Drop.

API Processing

There are a whole bunch of API related use case where using RxJS would result in a lot of benefit.

Response Processing

Most of the times we have to process the response of the API to use the data and often the processing logic can get complex but really can be aggregation of simple tasks. Let's take a look at one simple example.

const { from } = require('rxjs');
const{ filter, flatMap, distinct } = require('rxjs/operators');

const observable = from(fetch('<api>'))
  .pipe(
    flatMap(x => x),
    filter(x => x.tags.length > 3),
    flatMap(x => x.tags),
    distinct(),
  );

const subscriber = {
  next: (x) => { console.log(x); },
  error: (err) => { console.error(err); },
}

observable.subscribe(subscriber);

Our goal is to find the distinct tags for all objects which have more than 3 tags in the response of array of objects each of which contains tags. We first process the response array and split each element into individual elements using the first flatMap. Then we further filter the objects which have more than 3 tags using the filter we again use flatMap to flatten the lists and get the individual tags. Finally we apply distinct to get the distinct tags.

Getting the fastest API result

You would have come across the scenario to hit multiple APIs and process the result of the API to return the data first. We can achieve this using RxJS with just a slight modification from the previous code.

const { race } = require('rxjs');
const{ filter, flatMap, distinct } = require('rxjs/operators');

const observable = race(fetch('<api1>'), fetch('<api2>'))
  .pipe(
    flatMap(x => x),
    filter(x => x.tags.length > 3),
    flatMap(x => x.tags),
    distinct(),
  );

const subscriber = {
  next: (x) => { console.log(x); },
  error: (err) => { console.error(err); },
}

observable.subscribe(subscriber);

We use the race function to create a race between the different objects. The race emits the value of the first object to complete and ignores the result of rest of the objects.

API Retry

Often not we encounter a scenario where we would have to retry an API a certain number of times in-case of an error or because of some event and yes RxJS simplifies this as well for us.

const { from } = require('rxjs');
const{ filter, flatMap, distinct, retry } = require('rxjs/operators');


const observable = from(fetch('<api>'))
  .pipe(
    flatMap(x => x),
    filter(x => x.tags.length > 3),
    flatMap(x => x.tags),
    distinct(),
    retry(5),
  );

const subscriber = {
  next: (x) => { console.log(x); },
  error: (err) => { console.error(err); },
}

observable.subscribe(subscriber);

In the above example we retry five times before giving up and throwing the error using retry operator. So subscriber.error is called only after retrying five times.

Suppose we want to retry another API on failure we can do so using retryWhen operator as follows.

const { from } = require('rxjs');
const{ filter, flatMap, distinct, retryWhen } = require('rxjs/operators');


const observable = from(fetch('<api1>'))
  .pipe(
    flatMap(x => x),
    filter(x => x.tags.length > 3),
    flatMap(x => x.tags),
    distinct(),
    retryWhen((err) => {
      // Possibly can check the type of error before retrying.
      return from(fetch('<api2>'))
    }),
  );

const subscriber = {
  next: (x) => { console.log(x); },
  error: (err) => { console.error(err); },
}

observable.subscribe(subscriber);

Apart from the above mentioned use case we can also use RxJS with React or Vue to manage state and data, stream processing, data broadcasting, creating data streams etc.

Conclusion

RxJS is a very powerful library that provides easy, readable, maintainable solutions to many problems that we are facing currently with Promises and other asynchronous tasks in JavaScript. Many leading companies like Netflix, Airbnb, GitHub etc. all use ReactiveX libraries. Give it a try and play around with it who knows you might just discover a new and an easier solution to one of your use cases.

Blockchain using NodeJS and Socket.io

Adarsh — Wed, 31 Oct 2018 18:54:08 +0000

I was fascinated with the concept of Blockchain a decentralized trust based system to transfer and verify the data sent over a network. The concepts it was based on was quite simple so in order to understand the concepts involved I decided to create my own blockchain and soon the concepts turned into the story of easier said than done. So let me walkthrough what exactly a blockchain is and how to create one.

Blockchain 101

First off Blockchain is not Cryptocurrency. Blockchain is the technology behind Cryptocurrencies an analogy would be how Internet is the technology behind Emails. It is the process of maintaining a distributed digital ledger which contains a series of records that are linked together and cannot be corrupted. Well that's a fairly simple definition for a revolutionary technology. It has disrupted a lot fields like medicine, logistics, education and finance (majorly). Let's take a look at the components of a blockchain.

Components of Blockchain

Block: It is a collection of data usually capped at an upper limit either by the size or number of data units.
Chain: It is a collection of blocks that are linked together by an algorithm that uses the information of the previous block.
Node: A system in the blockchain that is used to verify the authenticity of block as well maintain the digital ledger that keeps record of the all the blocks in the chronological order.

And those are only components that make up the blockchain. What truly makes the blockchain secure are the processes that are involved.

Mining a block

Mining is process of generating a new block and adding it to the network and it involves two processes.

Proof of Work Generation:
As mentioned earlier blocks are linked using an algorithm that makes use of the information of the previous block to generate the next block. This algorithm is known as proof of work generally designed in a such a way that output is hard to generate but it's fairly easy to verify the output.
Proof of Work Verification:
When one of the nodes in the network successfully generates the block. The rest of nodes in the network must verify the authenticity of the proof and chain. So the nodes verify whether the proof is right and checks the authenticity of the chain. If everything is alright the block is mined and all other nodes update their ledger to include the newly mined block.

Event Flow of Blockchain

Let's look at how the events unfold when you add data to a blockchain

A request to exchange data is received from the sender to one of the nodes in the chain.
The node then broadcasts to other nodes about the incoming data to other nodes and adds it to the current transaction pool.
Once the limit of the block is reached (size or number of units). The nodes start mining the block.
The nodes compete against each other to find a proof of work solution. When one of the nodes succeeds in mining it broadcasts the solution.
The other nodes then verify the output and check whether is valid. Then they verify the blocks of the chain and add the newly mined block.

Creating Your Own Blockchain

So now we have brushed up with the fundamentals let's start creating our own blockchain. I have decided to use Socket.io to have a realtime communication system across the nodes. Let's proceed and create the models.

Models

Transaction Model:

class Transaction {
  constructor(sender, receiver, amount) {
    this.sender = sender;
    this.receiver = receiver;
    this.amount = amount;
    this.timestamp = Date.now();
  }

  /* Stringfying and Parser functions */ 
}

module.exports = Transaction;

This model pretty straight forward we have certain data as sender, receiver, amount and timestamp.

Block Model:

const crypto = require('crypto');

const Transaction = require('./transaction');

class Block {
  constructor(index, previousBlockHash, previousProof, transactions) {
    this.index = index;
    this.proof = previousProof;
    this.previousBlockHash = previousBlockHash;
    this.transactions = transactions;
    this.timestamp = Date.now();
  }

  hashValue() {
    const { index, proof, transactions, timestamp } = this;
    const blockString= `${index}-${proof}-${JSON.stringify(transactions)}-${timestamp}`;
    const hashFunction = crypto.createHash('sha256');
    hashFunction.update(blockString);
    return hashFunction.digest('hex');
  }

  setProof(proof) {
    this.proof = proof;
  }

  getProof() {
    return this.proof;
  }

  getIndex() {
    return this.index;
  }

  getPreviousBlockHash() {
    return this.previousBlockHash;
  }

  /* Stringify and Parsing functions */
}

module.exports = Block;

The important aspect of the Block is the hashValue() and previousBlockHash. The hashValue() is responsible for creating the hash value of the block. It generates a string expression of the block and that is sent to the NodeJS crypto module's createHash() function and hash is created using the specified sha256 algorithm. The generated hash is then stored the next block in previousBlockHash.

Chain Model:

const Block = require('./block');

const actions = require('../constants');

const { generateProof, isProofValid } = require('../utils/proof');

class Blockchain {
  constructor(blocks, io) {
    this.blocks = blocks || [new Block(0, 1, 0, [])];
    this.currentTransactions = [];
    this.nodes = [];
    this.io = io;
  }

  addNode(node) {
    this.nodes.push(node);
  }

  mineBlock(block) {
    this.blocks.push(block);
    console.log('Mined Successfully');
    this.io.emit(actions.END_MINING, this.toArray());
  }

  async newTransaction(transaction) {
    this.currentTransactions.push(transaction);
    if (this.currentTransactions.length === 2) {
      console.info('Starting mining block...');
      const previousBlock = this.lastBlock();
      process.env.BREAK = false;
      const block = new Block(previousBlock.getIndex() + 1, previousBlock.hashValue(), previousBlock.getProof(), this.currentTransactions);
      const { proof, dontMine } = await generateProof(previousBlock.getProof());
      block.setProof(proof);
      this.currentTransactions = [];
      if (dontMine !== 'true') {
        this.mineBlock(block);
      }
    }
  }

  lastBlock() {
    return this.blocks[this.blocks.length - 1];
  }

  getLength() {
    return this.blocks.length;
  }

  checkValidity() {
    const { blocks } = this;
    let previousBlock = blocks[0];
    for (let index = 1; index < blocks.length; index++) {
      const currentBlock = blocks[index];
      if (currentBlock.getPreviousBlockHash() !== previousBlock.hashValue()) {
        return false;
      }
      if (!isProofValid(previousBlock.getProof(), currentBlock.getProof())) {
        return false;
      }
      previousBlock = currentBlock;
    }
    return true;
  }

  /* Stringify and Parsing functions */
}

module.exports = Blockchain;

The chain has 2 major components currentTransactions and blocks. The currentTransactions contains a list of transactions that are yet to mined to a block. The blocks contains the list of all the blocks in the chain ordered according to time they were mined. The above chain also has the block size as 2 transactions.

In the constructor we are setting the blocks to initially have a single block with index as 0 previousBlockHash as 1 and proof as 0. This is also known as the initial block. But we have a list of blocks passed to the chain we just the blocks with value it received.

The addNode() functions is responsible for connecting the current node with other nodes of the blockchain network. The mineBlock() function adds the mined block to chain and signals other blocks to end the mining.

The most important methods are newTransaction() and checkValidity(). The newTransaction() method is called when the node receives a transaction request. We push the transaction to the currentTransactions pool. If the size of the currentTransaction pool is 2 we start to mine the block. We first get the latest block of the current chain. We create a block out of the latest block's hashValue, index and the currentTransactions pool. Then we generate the solution for the proof of work by passing the latest block's proof to generateProof() method (we will look into this implementation later). Once the solution is arrived we set the newly created block's proof. We then reset the currentTransaction pool and check whether this block can be mined using dontMine flag. If it can be mined we go ahead the mine the block.

The checkValidity() method checks the validity of the chain starting from the initial block. We get the currentBlock and previousBlock, we check whether the current block's previousHash to be same as previous block's hashValue. If they don't match we reject. We then check the validity of proof between the current and previous blocks. If they don't match either we reject the chain. We then check the currentBlock to be the previousBlock. We do this till the end of the chain and then return true if no discrepancies are found.

The above mentioned verification of the blocks is what makes the blockchain impossible to break and alter. If an attacker has to change the currentBlocks data he has to change the previousBlocks data as our hash calculation is based upon the data. If the data changes hash changes and therefore he has to do this till the initial block. Another secure aspect comes from the proof generation. If the attacker changes the tampers with the block the proof solution changes so again the attacker has to generate proof all the way from initial to the tampered block which can take a huge amount of time as the calculation of proof isn't very simple and easy.

Proof of Work Generation and Verification

const crypto = require('crypto');


const generateProof = (previousProof) => new Promise((resolve) => {
  setImmediate(async () => {
    let proof = Math.random() * 10000000001;
    const dontMine = process.env.BREAK;
    if (isProofValid(previousProof, proof) || dontMine === 'true') {
      resolve({ proof, dontMine });
    } else  {
      resolve(await generateProof(previousProof));
    }
  });
});

const isProofValid = (previousProof, currentProof) => {
  const difference = currentProof - previousProof;
  const proofString = `difference-${difference}`;
  const hashFunction = crypto.createHash('sha256');
  hashFunction.update(proofString);
  const hexString = hashFunction.digest('hex');
  if (hexString.includes('000000')) {
    return true;
  }
  return false;
};

exports.generateProof = generateProof;
exports.isProofValid = isProofValid;

This is the most time-consuming and the crucial part in any blockchain mining. Proof of Work is a solution for a problem that is harder to solve but easier to verify. An example would be generating two large prime numbers that when multiplied is divisible by 5. Finding the 2 large prime numbers is a daunting task we have to go through millions of combinations to land on one of the possible solutions. But verifying that whether product of the 2 large prime numbers is divisible by 5 or now is easy.

We have wrapper the generation algorithm inside a setImmediate function and wrapper it further under a Promise. Okay now you must be wondering why wrapping it under setImmediate. The reason is I have an environment variable that signals the end of the mining process. I set that variable to true if any other node in the network has finished mining a block. If I wrap the generation algorithm in a while loop it blocks the Event Loop and never checks the status of the environment variable until it has finished arriving at a solution. setImmediate allows it to bypass that as it waits until current process has finished executing before generating for the next time. This allows another module in my program to go and change the status of the environment variable. It also allows me to bypass the call stack limit when calling a recursive function.

Our proof of work problem is simple the hash value of the difference between current and previous proofs must contain six consecutive zeroes. We start with a random number and multiply it by a huge number. Then we verify whether proof satisfies the condition and also we verify whether end of mining has been set. If it satisfies we resolve the value else we try again. We repeat this process till we get a proof.

Server App

Great we have the model and generation setup all we need is a working server to orchestrate the action and interact with the blockchain.

const app = require('express')();
const bodyParser = require('body-parser');
const httpServer = require('http').Server(app);
const axios = require('axios');
const io = require('socket.io')(httpServer);
const client = require('socket.io-client');

const BlockChain = require('./models/chain');
const SocketActions  = require('./constants');

const socketListeners = require('./socketListeners');

const { PORT } = process.env;

const blockChain = new BlockChain(null, io);

app.use(bodyParser.json());

app.post('/nodes', (req, res) => {
  const { host, port } = req.body;
  const { callback } = req.query;
  const node = `http://${host}:${port}`;
  const socketNode = socketListeners(client(node), blockChain);
  blockChain.addNode(socketNode, blockChain);
  if (callback === 'true') {
    console.info(`Added node ${node} back`);
    res.json({ status: 'Added node Back' }).end();
  } else {
    axios.post(`${node}/nodes?callback=true`, {
      host: req.hostname,
      port: PORT,
    });
    console.info(`Added node ${node}`);
    res.json({ status: 'Added node' }).end();
  }
});

app.post('/transaction', (req, res) => {
  const { sender, receiver, amount } = req.body;
  io.emit(SocketActions.ADD_TRANSACTION, sender, receiver, amount);
  res.json({ message: 'transaction success' }).end();
});

app.get('/chain', (req, res) => {
  res.json(blockChain.toArray()).end();
});

io.on('connection', (socket) => {
  console.info(`Socket connected, ID: ${socket.id}`);
  socket.on('disconnect', () => {
    console.log(`Socket disconnected, ID: ${socket.id}`);
  });
});

blockChain.addNode(socketListeners(client(`http://localhost:${PORT}`), blockChain));

httpServer.listen(PORT, () => console.info(`Express server running on ${PORT}...`));

The server consists of an express and socket app that are bound to a http server running on a specific port. The /nodes endpoint allows us to connect to another node's socket app and sends the information for the other node to connect back. We also bind socket listeners to the socket connections created. The /transaction endpoint receives a transaction request and broadcasts the transaction information to other nodes. The /chain endpoint lists the details of the blockchain. There is also socket connection listener that actively logs the ID and monitors the connection status between nodes. Finally we make the server listen on a specific port.

Socket Listeners

The server app merely acts a facade to the socket listeners and only facilitates the connections between nodes. The socket listeners are responsible for triggering the events such as adding transactions to the blockchain, mining a new block and signalling successful mining status of a node to other nodes in the blockchain network.

const SocketActions = require('./constants');

const Transaction = require('./models/transaction');
const Blockchain = require('./models/chain');

const socketListeners = (socket, chain) => {
  socket.on(SocketActions.ADD_TRANSACTION, (sender, receiver, amount) => {
    const transaction = new Transaction(sender, receiver, amount);
    chain.newTransaction(transaction);
    console.info(`Added transaction: ${JSON.stringify(transaction.getDetails(), null, '\t')}`);
  });

  socket.on(SocketActions.END_MINING, (newChain) => {
    console.log('End Mining encountered');
    process.env.BREAK = true;
    const blockChain = new Blockchain();
    blockChain.parseChain(newChain);
    if (blockChain.checkValidity() && blockChain.getLength() >= chain.getLength()) {
      chain.blocks = blockChain.blocks;
    }
  });

  return socket;
};

module.exports = socketListeners;

The sockets listens to two events ADD_TRANSACTION and END_MINING events emitted by other nodes. The ADD_TRANSACTION listener actively listens to an incoming transaction event triggered by any node on the network. Adds it to the blockchain by calling the chain's newTransaction method.

The END_MINING event is triggered when one of the nodes successfully mine the block. It sets the BREAK flag to true which tells the other nodes on the network to stop mining and start verifying the solution. We parse the stringified chain to back to proper blockchain and call the checkValidity() method of the parsed chain. We also check whether the length of the parsed chain is greater than the current chain's length. If it is a success we go ahead and replace with the new chain else we just reject and hold on to our old chain.

And so we have the code of the chain set up let's run it and see the output.

Run It...

I used PM2 to spawn to instances of the app running on different ports. So once the two instances are up and running I trigger /nodes endpoint from one of the nodes to connect to another and the output is:

As the output states the two nodes have successfully established a websocket connection between them.

Then I trigger /transaction endpoint from one of the nodes. It then signals the other node about the incoming transaction and both of them add it to their transaction pool.

Then once again I trigger /transaction, since our block size is 2 the mining process starts on both the nodes. When one of the nodes successfully mines the block it signals the end of mining and starts the verification process and once the verification process is done the new chain is replaced across the whole network.

Then when I hit the /chain endpoint I receive the list of blocks in the chain.

And that's it we have created our own blockchain using NodeJS and Socket.io

Conclusion

What we have created is a simple blockchain. The whole purpose of this post is to summarize the fundamental processes that govern and power a blockchain by actually implementing them. There are concepts like forking of a blockchain that I haven't covered but do recommend to give it a read. There are few other process involved in a blockchain transaction that I haven't covered as a part of this post because there are other articles and posts that go in depth about the blockchain architecture. I have also committed my code to GitHub so you can clone the repository and play around by adding new processes ;).

Finally, when you want to learn about something try to learn the concepts and implement them on your own this will help you gain much deeper understanding and knowledge about the technology involved.

Java 11's Future

Adarsh — Fri, 28 Sep 2018 12:55:44 +0000

Hey guys, I am sure you all would've heard of the new JDK 11. Oracle has released it under the LTS banner. It has new features and improvements.

If you have heard of JDK 11 I am pretty sure you guys would've heard about the new licensing terms. In a nutshell it says "Hey, you can use JDK 11 for free during your development and testing but when you are deploying it to production you are supposed to pay Oracle".

Considering that Java is one of the integral part of the Open-Source community what are your feelings about Oracle changing the licensing terms?

Will OpenJDK which wasn't given much attention beat Oracle JDK to be the first choice of developers?

Demystifying Webpack

Adarsh — Tue, 18 Sep 2018 18:20:13 +0000

All of us would've definitely used webpack at one point of time. It is by far the most popular bundler due to the endless amount of loaders and customisability it brings to the bundling process. In one way webpack has influenced the growth of certain JavaScript ecosystems. But how often we have thought of opening that bundled file and understand what has happened while bundling. How does my app which contains hundreds of individual files works so beautifully and cohesively from that one single bundled file? Let's breakdown the concepts of webpack and understand what happens during the bundling process. I won't be going over the elements in webpack configuration as they are mentioned in detail in webpack documentation itself instead it will be on the core concepts of webpack.

What is a bundler?

Before we go any further let's understand what is a bundler. A bundler is a utility/program that takes a number of files and puts them together in such a way it doesn't change how the code works. This allows you write code in a modular fashion but serve them as a monolith file.

Why do we need a bundler?

Increasingly nowadays keeping maintainability and reusability in mind we write code in modules. This modular style works fine if the application is small. But as applications scale in complexity and size it becomes difficult to manage the increasing number of dependencies and code while running this modularized code. For Example consider you are creating an HTML/JavaScript application that consists of 50 JS modules. Now from your HTML you cannot afford to have 50 script tags to use them in the page. This is where bundler kicks in, it bundles all those 50 files together and gives one file which you can use from your HTML with a single script tag.

Demystifying webpack

Okay enough of the basics let's dive into webpack now.

Consider the three files

// A.js

const B = require('./B');

B.printValue();

// B.js

const C = require('./C.js')

const printValue = () => {
  console.log(`The value of C.text is ${C.text}`);
};

module.exports = {
  printValue,
};

// C.js

module.exports = {
  text: 'Hello World!!!',
};

And I defined A.js as my entry point for webpack and the output to be a single bundled file. When you run webpack build these two things happen.

Form the dependency graph
Resolve the dependency graph and Tree-Shaking

Form the dependency graph

The first thing webpack will do is analyze the modules that are present and form a dependency graph. A dependency graph is a directed graph that says how each module is connected to another module. It's quite popular among package managers such as npm, maven, snap etc. It starts off the from the entry point A.js and our graph initially looks like this with just one node.

Then webpack comes to know that B.js is being required by A.js and so it goes and creates a link from A to B in the graph.

Now analysing B.js it figures out that it needs C.js as well. So again in the graph it creates a link from B to C.

Now hypothetically if A.js requires another file called D.js which in turn requires C.js the graph becomes

See it's relatively simple stuff. Now in C.js webpack realises that it doesn't have any more modules as dependencies and so outputs the complete dependency graph.

Resolving the modules

Okay now, webpack has the graph and the modules. It must put all of them into one file and so it takes one node at a time from the graph starting from the root node A.js. It copies over the content of A.js to the output files, marks the node as resolved and then goes to the children of A.js. Suppose if the module which was already resolved earlier appears again it just skips it. Likewise it keeps adding content of the modules to the output file till it has finished traversing the dependency graph.

Tree-Shaking

Tree-Shaking is the process of removing dead code from the output. While webpack is creating the graph it also marks whether the module is used or not. If it's not used anywhere it removes the module as it's effectively a dead code. A point to note webpack does this in production mode only.

Let's take a look at the bundled code of the above three files.

/******/ (function(modules) { 
// webpackBootstrap 
/******/    // Load entry module and return exports
/******/    return __webpack_require__(__webpack_require__.s = 0);
/******/ })
/************************************************************************/
/******/ ([
/* 0 */
/***/ (function(module, exports, __webpack_require__) {

// A.js

const B = __webpack_require__(1);

B.printValue();

/***/ }),
/* 1 */
/***/ (function(module, exports, __webpack_require__) {

// B.js

const C = __webpack_require__(2)

const printValue = () => {
  console.log(`The value of C.text is ${C.text}`);
};

module.exports = {
  printValue,
};

/***/ }),
/* 2 */
/***/ (function(module, exports) {

// C.js

module.exports = {
  text: 'Hello World!!!',
};

/***/ })
/******/ ]);

You can immediately recognise that it's an IIFE. The functions takes in a list of modules and executes the commands runs the code of each module. We can see that the first module is our entry file A.js second is B.js and the third is C.js. And we can see each of those modules are modified as functions that can be executed.

The module parameter is the replacement for default node module object. exports is the replacement for exports object and __webpack_require__ is the replacement for require used in our programs. The // webpackBootstrap contains the implementation of the function which is quite long. Let's just see the implementation of __webpack_require__

function __webpack_require__(moduleId) {
/******/
/******/        // Check if module is in cache
/******/        if(installedModules[moduleId]) {
/******/            return installedModules[moduleId].exports;
/******/        }
/******/        // Create a new module (and put it into the cache)
/******/        var module = installedModules[moduleId] = {
/******/            i: moduleId,
/******/            l: false,
/******/            exports: {}
/******/        };
/******/
/******/        // Execute the module function
/******/        modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
/******/
/******/        // Flag the module as loaded
/******/        module.l = true;
/******/
/******/        // Return the exports of the module
/******/        return module.exports;
/******/    }

The code is quite simple to understand it takes in a moduleId and checks whether that module is present in installedModules cache. If it's not present it creates an entry in the cache. The next line modules[moduleId].call(module.exports, module, module.exports, __webpack_require__); actually executes the module function in the modules array which we passed earlier to the parent function. Comparing that to the fn.call() syntax we can deduce that module is the object created earlier, exports and this scope is the exports object of the created module object, and __webpack_require__ is the function itself. It then sets the module as loaded in the cache and returns the exports of the module.

That's all folks this is how webpack works on a fundamental level. There are still a lot more powerful things webpack does such as minimising the initial load by ordering modules in a particular way which I highly encourage you guys to go and explore.

It's always better to understand how a piece of utility works before we begin to use it. This helps writing better optimised code keeping in mind the inner workings and constraints of the utility we are using.

Linux distro you are using for development?

Adarsh — Fri, 20 Jul 2018 06:22:54 +0000

Hi all I just wanted to know the different linux distros you are using for development and why you chose that.

React - Asynchronous Component Rendering Wrapper

Adarsh — Wed, 09 May 2018 16:16:23 +0000

Most of the time our front-end applications interact with a wide range of services and APIs for populating and displaying the necessary data. We usually display loading screens for the same and we make the user wait a certain amount of time before we actually allow them to use the page. But sometimes most of the necessary content for the user is available but the user is made to wait for the unnecessary data on the page to load. This is very bad when it comes to user experience perspective.

Consider this scenario, You are opening a blog link. The text loads much faster but then the page doesn't allow you to navigate until the pictures and side links are loaded. Instead the page can allow you to navigate while the pictures and other things load simultaneously.

One of the ways of tackling this issue in react is to use an asynchronous wrapper for rendering the component. Let's take two components HeadingComponent and ParagraphComponent.

const HeadingComponent = props => <h1>{props.data}</h1>;

const ParagaphComponent = props => <p>{props.data}</p>;

We will now create the AsyncComponent that acts a wrapper for the HeadingComponent and ParagraphComponent which displays data from two different APIs.

class AsyncComponent extends React.PureComponent {
  constructor(props) {
    super(props);
    this.state = {
      resolvedError: false,
      resolvedSuccess: false,
      data: '',
      error: '',
    };
    this.renderChildren = this.renderChildren.bind(this);
  }

  componentDidMount() {
    this.props.promise()
      .then(data => this.setState({ resolvedSuccess: true, data }))
      .catch(error => this.setState({ resolvedError: true, error }));
  }

  renderChildren() {
    return React.Children.map(this.props.children, child => (
      React.cloneElement(child, {
        data: this.state.data,
      })
    ))
  }

  render() {
    if (this.state.resolvedError) {
      return <h1>Error Encountered</h1>;
    } else if (this.state.resolvedSuccess) {
      return <div>{ this.renderChildren() }</div>;
    } else {
      return <h1>Loading...</h1>;
    }
  }
}

The AsyncComponent takes a prop called promise that it calls from componentDidMount. If it resolves successfully it stores the data in the state and error in case of a rejection. Then in the render the method we render

Error component incase of error
The child nodes if resolves successfully
Loading component otherwise

Sometimes the child components needs the response data. React doesn't allow us to get the component directly from the child elements so we use React's inbuilt functions like React.Children.map and React.cloneElement. We traverse the children of the component and we clone each child element by adding a prop data which has the actual response from the API so that the response is accessible to children as well.

Final piece of code that puts all of the above together

const HeadingAPI = () => new Promise((resolve, reject) => {
  setTimeout(() => resolve('Heading'), 5000);
});

const ParagraphAPI = () => new Promise((resolve, reject) => {
  setTimeout(() => resolve('Paragraph data'), 2000);
});

const App = () => (
  <div>
    <AsyncComponent promise={HeadingAPI}>
      <HeadingComponent />
    </AsyncComponent>
    <AsyncComponent promise={ParagraphAPI}>
      <ParagaphComponent />
    </AsyncComponent>
  </div>
);

Here's a Codepen running the scenario with both the promises resolving successfully.

Here's a Codepen running the scenario when one of the promises is rejected.

As you can see the failure of one API doesn't affect the rendering of the other component and the user can continue navigating the webpage regardless. This greatly improves the user experience and also reduces the amount of redundant code created by API calls across components.

You can still improve the wrapper by giving custom loader and error components to make it look more fancy.

Capture error and data in async-await without try-catch

Adarsh — Mon, 30 Apr 2018 14:21:09 +0000

One of the things that took the Javascript community by storm was the introduction of async-await. It was simple and looked a lot better than the then-catch of Promises and also more readable and debuggable than the callback hell. But one thing that bothered me was the usage of try-catch. At first I thought it isn't a problem but as fate would have it I was working on chained API calls and the problem cropped up where each API call has specific error message that had to be printed. I realized soon that I was creating a try-catch hell.

Let's consider this Promise that resolves or rejects after 2 seconds based on a parameter rejectPromise

// api.js

const fetchData = async (duration, rejectPromise) => (
  new Promise((resolve, reject) => {
    setTimeout(() => {
      if (rejectPromise) {
        reject({
          error: 'Error Encountered',
          status: 'error'
        })
      }
      resolve({
        version: 1,
        hello: 'world',
      });
    }, duration);
  })
);

module.exports = {
  fetchData,
};

So my typical usage of async-await is gonna be like this.

const { fetchData } = require('./api');

const callApi = async () => {
  try {
    const value = await fetchData(2000, false);
    console.info(value);
  } catch (error) {
    console.error(error);
  }
}

callApi();

/* 
 OUTPUT: 
 { version: 1, hello: 'world' } (rejectPromise=false)

 { error: 'Error Encountered', status: 'error' } (rejectPromise=true)

 */

As you can see when the rejectPromise parameter is false the await resolves to { version: 1, hello: 'world' } and when it's true it rejects the promise and catch is called and the error is { error: 'Error Encountered', status: 'error' }.

That's the typical implementation of async-await. Now we will leverage the promise functions then-catch to make the process simpler. Let's write a wrapper that does this.

// wrapper.js

const wrapper = promise => (
  promise
    .then(data => ({ data, error: null }))
    .catch(error => ({ error, data: null }))
);

module.exports = wrapper;

We can see that the wrapper takes a promise as an input and returns the resolved/rejected values through then-catch. So let's go and modify the original code we wrote in try-catch to utilize the wrapper.

const { fetchData } = require('./api');
const wrapper = require('./wrapper');

const callApi = async () => {
  const { error, data } = await wrapper(fetchData(2000, false));
  if (!error) {
    console.info(data);
    return;
  }
  console.error(error);
}

callApi();

/* 
 OUTPUT: 
 { version: 1, hello: 'world' } (rejectPromise=false)

 { error: 'Error Encountered', status: 'error' } (rejectPromise=true)

 */

Voila the same output but the this way makes it better to understand the code.

Virtual DOM - The backbone of React

Adarsh — Wed, 18 Apr 2018 15:51:29 +0000

We sing praises to React's performance when it comes to handling page updates, data binding etc. But there are lot's of things that happens behind the scenes for achieving this performance. One of the influencing factors is the Virtual DOM.

Wait now, There was this concept of DOM and now you are telling me there is this whole new thing called Virtual DOM. What is going on?. Before we go any further let's understand what is actually DOM.

What is DOM?

In front-end web-development Document Object Model or DOM as it's commonly known it's a tree representation of a html document where each node of the tree is the html element and the parent-child relationships of the html elements are also maintained in the tree. This DOM structure provides us with a way to traverse the DOM nodes and make changes. Vanilla Javascript provides an API to manipulate the DOM this was further abstracted and simplified by jQuery.

Okay so, you're saying Vanilla Javascript provides an API and also jQuery provides a simplified API if these are there what is the need for React and other front-end frameworks. To answer that question as well the factors that gave rise to Virtual DOM we must understand what the DOM lacks.

Drawbacks of DOM

Yeah DOM is great for simple, static sites and sites with very few UI updates. But then when people started to move towards more UI, Mobile, Responsive driven approach to create websites they started to see the drawbacks of DOM.

1. Cost of DOM mutations

Updating the DOM is a much slower operation when compared to other Javascript operations. Whenever you want to update or add a DOM it has to find the DOM node you want to update or find the position the new node has to be inserted in. This is not a problem in small websites where the number of DOM nodes are less. But with large scale sites with huge amount of DOM nodes and with cost of updating it becomes a performance concern. Also when there is a DOM update browser repaints the DOM nodes making it even more time consuming.

2. Inefficient Updates

DOM is highly inefficient when it comes to figuring out what exactly needs to be updated. Sometime it might update more than what's necessary. For Example, Let's consider a scenario. You have an <ul> with a list of <li> items which generated from an array of values. Now when one of the values in the array changes the whole list gets re-rendered which is unnecessary as only one item got updated.

Typically these steps take place whenever a single DOM mutation call occurs:

Traverse the DOM till the position where the node must be inserted or updated.
Update the DOM node or add it.
Recalculate positions and CSS.
Traverse again and repaint the node on the page.

Those 2 points coupled with the rise of Single Page Applications (SPAs) which typically has huge amount of DOM nodes, DOM mutations and listeners listening to the site for changes led for people to come up with frameworks to bypass these performance concerns.

The Virtual DOM

Firstly, let me say this React did not invent Virtual DOM they just made it better in terms of performance. Virtual DOM in it's core sense is just a Javascript object which contains a list of nodes that have to be rendered in the real DOM and updating the values in this Javascript object is much faster compared to updating the actual DOM tree.

Basically, if DOM is the actual building then Virtual DOM is the blueprint of the building. The core idea is it's always easy and fast to modify the blueprint than the real building.

The 2 ways of implementing a Virtual DOM are:

Dirty Check: Which involves periodically checking the components for changes.
Observable: Which involves listening to changes via listeners to find out the components that has to be updated.

As guessed React uses the latter way of implementation and this is one of the major reasons for react having a one-way data binding and it's amazing performance.

Wow, All of this is nice. But how is it implemented in React?

Synchronizing Virtual and Real DOM

The process of updating the DOM nodes based on component updates is usually done by the ReactDOM library. The following usually happens in this process:

Diffing/Reconciliation
Batch Updating the real DOM

Diffing/Reconciliation

When an update like setState() occurs a new Virtual DOM tree is created from scratch (Don't worry it's fast because it's just a Javascript object). The diffing algorithm compares the old and the new Virtual DOM tree to find the dirty components. Once it finds all the dirty components. It then identifies the minimum number of steps to update the real DOM. This diffing algorithm works on certain assumptions in order to make them fast as React uses a heuristic approach for things which I won't be discussing here but if you want to know you can visit the Reconciliation documentation of React.

Batch Updating The Real DOM

Once the steps to update are identified the ReactDOM puts all of those in a single call in the Event Loop. ReactDOM orders these steps in such a manner it only calls the repaint of DOM as the last step. So once all the steps are executed the Event Loop then calls the DOM repaint thus during the entire process only once the DOM gets repainted leading to a faster performance. When an update in the component occurs while ReactDOM is updating the real DOM it will wait for the real DOM update to finish.

And that's how the React's Virtual DOM leads to an amazing performance which has become a trademark of React. So while creating React applications please keep these concepts of Virtual DOM in the mind in order to utilize the performance boost given by React to it's full potential.

GraphQL or REST? What should I use?

Adarsh — Thu, 12 Apr 2018 09:43:54 +0000

This is a topic that's been debated within the dev community for sometime now and many people have different opinions on this. So which one should I use? The new kid on the block or the old person with lots of experience?. Before we proceed let's understand REST and GraphQL.

What is REST?

REpresentational State Transfer or REST is an architectural style that conforms to specific guidelines, constraints for implementation of Web APIs. It was proposed by Roy Fielding in his Ph.D thesis. It encourages separation of client and server with stateless mode of information exchange. Keep in mind not all APIs are REST but all RESTful services are APIs.

What is GraphQL?

GraphQL is a query language for Web APIs. It was created by Facebook in 2012 and open-sourced in 2015. It's neither an architectural pattern nor a web service. It acts an intermediary that helps in querying the data received from various datasources. These datasources can be databases or web services.

REST has become the dominant standard for Web API over the years. Since it used standard HTTP methods (GET, POST, PUT, DELETE etc.) it gained momentum and popularity as the web applications in the internet increased. Moreover it's language and platform agnostic which made it a better choice in creating web services. Because every data is treated as a resource to be sent over when a URL is called it can be called using even web browsers or using cURL requests.

Cons of REST

Although REST was extremely successful it had it's shortcomings which became very apparent as RESTful services grew in size and complexity.

1. Multiple Endpoints (Multiple Round Trips)

In RESTful services an URL just denotes a single resource. So when there is an need for accessing multiple resources you need to call multiple endpoints and therefore leading to multiple round trips for getting all the necessary data.

For example, let's consider a blogging application. You have a blog post and each post has comments. A typical REST endpoints for this will look like

GET /posts/<postId> - To fetch a particular post
GET /posts/<postId>/comments - To fetch all comments related to a post
GET /posts/<postId>/comments/<commentId> - To fetch a particular comment of a particular post

You can see that the length of the endpoints getting increased. As the relations between entities increases so does the amount endpoint URLs. As the application grows it becomes cumbersome to maintain these APIs.

2. Overfetching/Underfetching Data

There are times when you interface with an API that you get unnecessary data along with the relevant ones and sometimes you don't receive enough data so you end up making multiple trips. This is a common problem in RESTful services. There are situations where you may need only 2-3 values but you get around 20-25 values as the response. This just leads to a transferring large amounts of unused data there by increasing the response time. In the latter case you may need more information than what you get from a single endpoint so therefore it's necessary to make multiple round trips. This also leads to increase in the overall time taken for the client to have all the required data.

3. API Versioning

API versioning is a method followed to avoid breaking the client application with the changes in response format. When there is an change in the API response format a new version is created. This is done so that production client applications can run as expected and giving some breathing time to developers to migrate to the new API version.

But this versioning becomes a problem as when a new version is released it means new endpoints. Maintenance and consuming of API becomes tough and also often leads to duplicated code.

4. Weak Typing

Not all the data we receive from RESTful service are strongly typed i.e. they are not properly given a particular data. This becomes a problem while documenting APIs as we have to specify what kind of data the client can expect by calling the endpoint.

5. Client Kept In The Dark

The client isn't aware of the response structure until it receives it. So the client is kept in the dark regarding what kind of response it can expect. This might often lead to some errors and data not handled properly thus bringing the reliability of the consuming the API low.

Pros of GraphQL

GraphQL was invented by Facebook primarily to overcome the shortcomings of REST.

1. One Request To Get Them All

A GraphQL service exposes only one endpoints through which the client can pass the necessary query to retrieve the data. With the same example that was considered previously let's look at the GraphQL query

{
    findPost(id: <postId>) {
        id
        title
        content
        author
        comments {
            id
            comment
            commentedBy
        }
    }
}

As you see we get all the necessary data in just a single request. So when you want a new field you just add it to the query and it will start appearing in the response.

2. Strongly Typed

GraphQL is dominated by schema which are strongly typed. These types can either be primitive or derived. The strong typing system allows the API to be self documented thus making the client aware on what response he will get when querying a particular query.

3. Client Is The Driver

GraphQL provides a declarative syntax that allows clients to specify which fields they need exactly. This eliminates the possibility of overfetching and underfetching of data as it's client who states his data requirements to the GraphQL server based on the schema.

4. API Evolution

Since everything in GraphQL is schema driven extending it won't be a problem as the new field won't affect the existing fields and also GraphQL introduces @deprecated annotation to deprecate fields. This eliminates the need for versioning the API.

5. Transport Layer Agnostic

This is a very nice advantage of GraphQL. The API server can exchange information over any protocol HTTP, HTTPS, WebSockets, TCP, UDP etc. This is because GraphQL is least bothered about how the information is exchanged between the client and server.

Cons of GraphQL

Wow, GraphQL is great. But it's a universally known truth everything in this world has a catch and GraphQL can't escape this truth.

1. Non-Existent Caching

GraphQL doesn't have support for browser and mobile caching unlike RESTful service which uses native HTTP caching mechanisms. This leads to a developmental effort in order to achieve caching. Although tools like Relay give some support for caching they are not as mature as the caching mechanisms used by RESTful services.

2. Monitoring and Error Reporting

RESTful services leverage the HTTP status codes for different errors that can be encountered. This makes the monitoring the APIs very easy and effortless for the developer. But with GraphQL services always return 200 OK response. A typical GraphQL error message looks like this with status code 200 OK.

{
    errors: [
        { 
            message: 'Some error occurred'
        }
    ]
}

This is makes it very difficult to handle the error scenarios and the makes the monitoring process cumbersome.

3. Exposed Schema and Resource Attacks

Unlike RESTful services GraphQL services mandates that the client has to know about the data schema to query. If you are exposing your API to a third party you are basically exposing your internal data structure. A great care has to be taken care so that the client doesn't create expensive join queries that can potentially lead to Denial of Service (DoS) attacks on your server.

4. Security - Authentication and Authorization

There is still a confusion going on among the GraphQL community on how to handle the security part of the GraphQL server. There is still not a native solution to integrate authentication and authorization. It is usually abstracted to the business logic layer to authorize user but do we really have to parse and validate the query for an unauthenticated user is still a question in the realm of GraphQL.

5. N + 1 Query Problem

In RESTful services it's very easy to log the SQL query performed and further optimize it. But in the case of GraphQL where the resolving nature is dynamic it's very tough to get the exact query and optimize it further. Sometimes the field resolvers might lead to N + 1 query problems and expensive join operations. But Facebook is developing tools like DataLoader to solve this exact problem. So maybe in the future this won't be a disadvantage.

6. Young Ecosystem

GraphQL is pretty much like a baby in this API ecosystem which means there will be issues and breaking changes every now and then so we have to be very diligent while using any libraries and modules associated with GraphQL.

Conclusion

Let me start by saying that GraphQL is merely a tool and REST is an architectural pattern. It would be very wrong to say GraphQL can replace REST. But in this age of Microservices where we segregate and create APIs to an atomic level we can leverage both there isn't one silver bullet.

GraphQL servers keep performance as the foremost priority while RESTful services keep reliability.

A GraphQL endpoint can be exposed via existing RESTful service as an endpoint such as /graphql which can act as a gateway to run GraphQL query while also maintaining REST endpoints for certain use cases.

There are certain use-cases where using GraphQL would be better and there are certain use-cases where REST will certainly win. So before saying which can be better just analyze the requirements and data involved to come to a conclusion on which can be used.