DEV Community: Michiel Mulders

Exchange-Based Trading vs. CFD Trading - Which Is Best for YOU?

Michiel Mulders — Mon, 05 Apr 2021 16:24:08 +0000

Most people start trading cryptocurrencies through crypto exchanges, which is the most obvious option.

However, there’s another option that allows traders to speculate on price movements without actually owning or holding the underlying cryptocurrency. We can achieve this type of trading using contracts for difference (CFDs), an instrument that’s very popular in the world of stock trading.

But how does a CFD work, and why would you use it? This article explores crypto CFDs and compares them against exchange-based trading.

What is a Contract For Difference (CFD)?

First, let’s dive a bit deeper into the ‘What’ of CFDs. Trading contracts for difference is a way of speculating on financial markets that don’t require the buying and selling of any underlying assets. When you trade a CFD, you agree to exchange the difference in the price of an asset from the point at which the contract is opened to when it is closed. You can create such contracts via CFD brokers.

The main benefit of this approach is that you can speculate on price movements in either direction. For instance, you expect the price of Bitcoin to enter a bear market for the upcoming months. It’s a great entry point for opening a short contract with a CFD broker. More on this strategy later.

So, what are the pros and cons of cryptocurrency CFD trading?

Cryptocurrency CFD Trading vs Exchange-based Trading

Here’s a list of arguments that favor CFD trading or exchange-based crypto trading.

1. CFD: Cheaper option for leveraged trading

CFDs offer a cheaper alternative for leveraged trading. Many crypto traders avoid opening leveraged positions due to the high fees exchanges charge. It’s certainly not a good idea to open a long-term leveraged position via a crypto exchange.

Some exchanges charge up to 0.02% to open a position and up to 0.02% (per 4 hours) in rollover fees to keep it open. This means you’ll pay six times per day a fee of 0.02% on your total position. In other words, fees accumulate rapidly and discourage traders from opening long-term leveraged positions.

2. Exchange: A wider variety of crypto pairs

While CFD trading might be cheaper for opening leveraged or short positions, you are often limited in the variety of cryptocurrencies offered. Some crypto exchanges offer more than 300 cryptocurrency pairs for you to choose from. For instance, popular crypto exchange Binance offers its users 1132 crypto pairs.

(Screenshot: CoinMarketCap.com)

However, a CFD broker like easyMarkets typically offers the most popular cryptocurrencies such as Bitcoin, Ripple, Ethereum, Litecoin, Bitcoin Cash, and Stellar. And most CFD traders prefer to trade cryptocurrencies with a lot of volume like the cryptos as mentioned earlier. As you would have guessed, Bitcoin trading is most popular for CFD trading.

(Screenshot: easyMarkets.com Bitcoin trading)

3. CFD: Great for hedging risks

Traders want to find new ways to protect their trades. Hedging is one such example of protecting a trade against volatility or external circumstances that can affect a cryptocurrency’s price.

Most often, traders use hedging to offset the risk of their position by opening a trade in the other direction when the market is unclear or they are concerned about the stability of the market.

For instance, you want to hold Bitcoin long-term, but you expect the Bitcoin market to turn into a short-term bear market. To offset the risk of this bear market, you can open a short position via CFDs. In other words, if the market falls, you can cover some of the loss to your initial position with the gain from this short position.

The most common types of risk include but are not limited to:

Crypto hacks
Smart contract failure
High market volatility
Regulatory risks
Entering a bear or bull market

4. Exchange: You own the cryptocurrency

When using an exchange to trade cryptocurrency, you can choose between an exchange that uses hot wallets or one that allows you to generate new wallets to which you own the private and public keys.

When you want to truly own your crypto, you should pick an exchange that provides you with the private and public keys of your crypto wallet.

When trading CFDs, you don’t own any crypto. A CFD is a derivative instrument that is merely used for speculation. If you want to truly own Bitcoin or any other cryptocurrency, your best choice is to pick an exchange that gives you access to your crypto wallet. You can also buy cryptocurrencies and send them to a non-custodial wallet you own.

5. CFD: Better risk management tools

By now, most exchanges and CFD brokers offer risk management instruments or tools such as setting a Take-profit or a Stop-loss order.

However, CFD brokers like easyMarkets offer users other protection mechanisms. For instance, they protect users against negative balances, which means you can’t lose more money than you’ve deposited. It’s a promise many crypto exchanges can’t always guarantee. When trading, you don’t want to worry about the possibility of losing more money than you initially planned as the worst-case scenario.

Besides that, they also offer a guaranteed stop-loss to ensure your position is closed at the pre-selected price. It’s an important risk management tool some exchanges can’t offer due to slippage during high volatility.

In other words, CFD brokers understand the need for reliable risk management tools for traders. It’s also one of the most important elements of trading.

What’s best: Crypto exchanges or CFD brokers?

We can’t conclude on a definitive winner just looking at the above arguments. Both approaches have their own set of advantages. Yet, we can conclude the following.

If you are a long-term crypto investor, you can use CFDs to hedge your position against any type of risk. Moreover, it’s much cheaper to use a CFD broker than to open a leveraged trade than to use a crypto exchange to hedge your risk.

On the other hand, if you are a trader who likes to frequently open short positions, CFD trading might be a cheaper option for you. If you would like to explore CFD trading, easyMarkets is a great place to get started. They offer traders many built-in risk management solutions to make trading a more pleasant experience.

What are React Server Components and will you need to use them in the future?

Michiel Mulders — Wed, 03 Feb 2021 10:26:31 +0000

Around Christmas, the React team introduced React Server Components. It’s a complete surprise for the React ecosystem as React has always focused on the client-side with a small focus on server-side rendering. Server-side rendering can, for instance, be accomplished using Next.js.

However, with React Server Components (RSC), it’s now possible to write server-rendered code using React only. Note that RSC is not a final feature as of now. Currently, React Server Components are undergoing beta-testing.

In this article, you’ll learn the following concepts about React Server Components:

What is a React Server Component?
Why do we need React Server Components? What problems do they solve?
What does a React Server Component look like, and what are the benefits?
What’s the difference between React Server Components and Server Side Rendering (SSR)?
Is there any future for React Server Components?

What is a React Server Component?

Let’s make this clear first: React always renders React Server Components on the server. It allows frontend developers to create components that span both server and client.

Why is this useful? This property allows developers to quickly fetch data from the backend. Therefore, you don't have to send an expensive request from your client to your backend to fetch data. As you render your components on the server that also hosts your backend, you can make a local call to your API to fetch data. In other words, locally fetched data is super fast (cheap).

To summarize, React Server Components allow you to access the speed of fetching data on the server-side while maintaining its rich interactivity of client-side apps. That’s a fantastic improvement for the React ecosystem to build faster applications.

Why do we need React Server Components?

UI development needs to offer the following properties:

It should offer a good user experience
It should be cheap to maintain
It should offer fast performance

However, these properties are at odds with each other. To illustrate this problem with the current state of React, let’s imagine a Spotify artist page that has the following components.

function ArtistPage({ artistID }) {

  return (
    <ArtistDetails artistId={artistId}>
      <TopTracks artistId={artistId} />
      <Discography artistId={artistId} />
    </ArtistDetails>
  );
}

Now, let’s fetch all data at the top of the component and pass each component’s required data. This approach’s benefit is that it’s performant as we only send one API call to retrieve all data we need.

We end up with the below code.

function ArtistPage({ artistID }) {
  const artistData = fetchAllArtistData();

  return (
    <ArtistDetails 
      details={artistData.details}
      artistId={artistId}>
      <TopTracks 
        topTracks={artistData.topTracks}
        artistId={artistId} />
      <Discography 
        discography={artistData.discography}
        artistId={artistId} />
    </ArtistDetails>
  );
}

The API response becomes very coupled to the component. We expect the API call to return artist details, top tracks, and discography. If we want to change the component’s behavior in the future, we might need to change the behavior of the API endpoint as well.

Further, if we remove a component inside the <ArtistDetail/> component, we might forget to remove this data from the API call. Therefore, this approach affects code maintenance. But don't worry, there’s nothing fundamentally wrong with this approach. It’s how React works.

What if we allow each component to fetch its data to decouple the API response from the component? Our code might look like this.

function ArtistDetails({ artistId, children }) {
  const artistData = fetchDetails(artistId)'
  // ...
}

function TopTracks({ artistId }) {
  const topTracks = fetchTopTracks(artistId)'
  // ...
}

function Discography({ artistId }) {
  const discography = fetchDiscography(artistId)'
  // ...
}

While this code is much easier to maintain, it creates performance issues. If you remember our initial component, we’ll get a cascade of API calls that wait for each other to finish.

In other words, the <ArtistDetails/> component fetches data, then <TopTracks/> fetches its data, and finally, the <Discography/> component starts fetching the required data. They also call this a network waterfall.

Our previous example used a single API call to fetch all data to increase performance.

function ArtistPage({ artistID }) {

  return (
    <ArtistDetails artistId={artistId}>
      <TopTracks artistId={artistId} />
      <Discography artistId={artistId} />
    </ArtistDetails>
  );
}

So, can we come up with a solution that addresses all three properties in the React ecosystem: good UX, cheap maintenance, and fast performance? The potential answer here is React Server Components (RSC) by moving our components to the server.

What does a React Server Component look like?

It’s time to explore a React Server Component. This component is an example out of the React demo for React Server Components. We have a list of notes that we can expand when we click it.

You can distinguish a server-side component from a client-side component by appending the file name with .server.js. The file below is named NoteList.server.js. You can apply the opposite for a client-side file: myFile.client.js.

// NoteList.server.js
import {fetch} from 'react-fetch';

import SidebarNote from './SidebarNote';

export default function NoteList({searchText}) {
  const notes = fetch('http://localhost:4000/notes').json();

  return notes.length > 0 ? (
    <ul className="notes-list">
      {notes.map((note) => (
        <li key={note.id}>
          <SidebarNote note={note} />
        </li>
      ))}
    </ul>
  ) : (
    <div className="notes-empty">
      {searchText
        ? `Couldn't find any notes titled "${searchText}".`
        : 'No notes created yet!'}{' '}
    </div>
  );
}

First of all, we can immediately fetch notes data from the server using http://localhost:4000/notes. This ticks off the fast performance property. On top of that, we can use interactive logic on our server, such as return notes.length > 0 ? <code> : <code>.

Next, let’s look at an example where we have an import that we only use once to format dates data-fns. Often, front-end developers have to evaluate if they want to include a dependency that they only use once or twice in the application. It will increase the final bundle size. Developers often ask the question: is it worth it?

import {format, isToday} from 'date-fns';

export default function SidebarNote({note}) {
  const updatedAt = new Date(note.updated_at);
  const lastUpdatedAt = isToday(updatedAt)
    ? format(updatedAt, 'h:mm bb')
    : format(updatedAt, 'M/d/yy');

  return (
    <div className="sidebar-note-list-item">
      <header className="sidebar-note-header">
        <strong>{note.title}</strong>
        <small>{lastUpdatedAt}</small>
      </header>
      <button classname="sidebar-note-open" />
    </div>
  );
}

With React Server Components, we don’t have to worry about this evaluation. React won’t include a dependency that you use only in a React Server Component in the bundle size. Therefore, the user won’t have to download this dependency. The RSC will use this dependency on the server to render the required output and send this output to the client. Cool, right?

What’s the difference between React Server Components and Server Side Rendering (SSR)?

Server-side rendering focuses on reducing the initial page load. When using SSR, you send HTML to the client. The client then loads all of the React JavaScript you need to make your web page interactive. In other words, you have a traditional React app after the initial page load when React injects all JavaScript.

The most significant difference is that your components are still client-side components. All required dependencies are still downloaded. The only difference when using SSR is that your initial page load contains only HTML to improve performance.

Is there any future for React Server Components?

There’s a massive potential for React Server Components as they address many tradeoffs React developers experience. Some of the most important benefits include:

Fast data fetching locally on a server
Server-side dynamic logic
Ability to include any dependency for RSCs without having to worry about increasing the cost to load a web page

On the other hand, many React users might get confused at first by this domain switch. React has shown a strong focus on the client-side in the past. React Server Components suddenly span both server and client. This change requires a significant shift in the way we think about React components.

For that reason, I expect React developers to embrace this approach but also to see many people struggle to grasp this concept and make correct decisions about server/client-side components.

If you want to play around with the demo code for React Server Components, you can find this code on GitHub.

Fetch vs Axios: Which Is the Best Library for Making HTTP Requests?

Michiel Mulders — Mon, 01 Feb 2021 23:21:14 +0000

In our previous post “How to make HTTP requests with Axios,” we’ve explained how to use Axios for sending basic CRUD requests. One of the main benefits of using Axios is the automatic JSON stringification when sending requests. A feature not supported by the native Fetch API.

Our post concluded that Axios is a lightweight library that offers a lot of helpful functionality when dealing with HTTP requests. For example, Axios provides an easy way to track upload progress via its request config. Moreover, Axios allows you to define interceptors to automate specific tasks for requests easily.

However, let’s also take a look at the Fetch API. You can perform the same tasks with the Fetch API. On top of that, the Fetch API is already available in all modern browsers.

This article aims to provide you with an overview of both tools so you can make a better-informed decision when picking a tool to send HTTP requests.

We’ll compare:

CRUD operations
Automatic JSON parsing
Interceptors
Error handling
Request timeout functioning

Comparison: CRUD operations and automatic JSON parsing

Let’s first take a look at sending basic requests. Let’s retrieve some data from a publicly available API at typicode.com.

First, the Axios object exposes different methods to retrieve, send, update, or delete a resource from an API.

axios.get(url[, config])
axios.post(url[, data[, config]])
axios.put(url[, data[, config]])
axios.patch(url[, data[, config]])
axios.delete(url[, config])

For a GET request, the function accepts an API URL and an optional configuration object.

const axios = require('axios')

const getRequest = async () => {
    try {
        const response = await axios.get('https://jsonplaceholder.typicode.com/todos/1');
        console.log(response.data);
    } catch (err) {
        console.error(err);
    }
}

getRequest()

We can use the config object to send additional parameters or headers. You can find all possible request configuration parameters in the Axios documentation.

const response = await axios.get('https://jsonplaceholder.typicode.com/todos', {
    params: {
        ID: 1
    },
    headers: {'X-Custom-Header': 'foobar'},
});

Now, let’s take a look at the fetch() version. The fetch API is the same as the Axios API. Note that you can pass any method to the fetch() function via the options object.

fetch(url, [options])

As you can see, it accepts an API URL and an options object. We have to pass the method with the options object. Further, we can also pass headers, parameters, or a body with this object.

const options = {
  method: 'GET',
  headers: {
    'Accept': 'application/json',
    'Content-Type': 'application/json;charset=UTF-8'
  }
}

fetch('https://jsonplaceholder.typicode.com/todos/1', options)
  .then(response => response.json())
  .then(console.log)

Note how we have to manually parse the resolve response object to JSON before we can log the returned data. Axios automatically takes care of parsing the response to JSON.

Comparison: Interceptors

Axios provides interceptors for both requests and responses. In other words, you can modify a request before sending it to an API or modify a response object that’s been returned. It’s a powerful concept that allows the automation of specific tasks.

You can use request interceptors to automatically add authorization headers. Furthermore, response interceptors are useful for modifying the response object to fit a different format. It’s even possible to intercept error responses and send them to a monitoring tool.

Here’s an example of an Axios interceptor used in our previous tutorial.The axios.interceptors.request.use() method allows you to define a request interceptor.

axios.interceptors.request.use(config => {
    config.name = 'my-axios-app'
    console.log(`Sending ${config.method} request to: ${config.url}`);

    return config;
}, error => {
    return Promise.reject(error);
});

// send GET request
axios.get('https://jsonplaceholder.typicode.com/todos/1')

How can we do this with fetch()? By default, fetch() doesn’t support a way to define request interceptors. Luckily, we can fall back to overwriting the function so we can add custom logic to the fetch method.

fetch = (originalFetch => {
    return (...arguments) => {
        console.log('Adding headers')

        arguments[1].headers = {
            'Accept': 'application/json'
        }

      return originalFetch.apply(this, arguments);
    };
})(fetch);

fetch('https://jsonplaceholder.typicode.com/todos/1', { method: 'GET' })
    .then(response => response.json())
    .then(data => {
      console.log(data) 
    });

As you can see, overwriting the original fetch() function affects the readability of your code. Personally, I prefer using Axios’ request and response interceptors. Yet, there’s an NPM package that allows you to register custom interceptors to the fetch() method in a more readable manner. The package is called fetch-intercept. Here's an example that solves the problem.

const fetchIntercept = require('fetch-intercept')

const unregister = fetchIntercept.register({
    request: function (url, config) {
        // Modify the url or config here
        console.log(`Sending ${config.method} request to: ${url}`);
        return [url, config];
    },

    response: function (response) {
        // Modify the reponse object
        console.log('Response received')
        return response;
    }
});

// Call fetch to see your interceptors in action.
fetch('https://jsonplaceholder.typicode.com/todos/1');

// Unregister your interceptor
unregister();

Next, let’s learn how both tools take care of error handling.

Comparison: Error handling

It’s essential to understand the difference between Axios and fetch() in error handling. When Axios receives a 404 or 400 error, it will catch the error using the catch() block. This catch() block allows you to handle the error.

Instead, the Fetch API prefers completing the request successfully via the then() method but includes a 400 status. We can also find this in the fetch() documentation.

The Promise returned from fetch() won’t reject for a HTTP error status even if the response is an HTTP 404 or 500. Instead, it will resolve normally (with ok status set to false), and it will only reject on network failure or if anything prevented the request from completing.

You can try it out yourself using the below code.

const url = 'https://jsonplaceholder.typicode.com/toodoos/1'
axios.get(url)
    .then(response => console.log('good'))
    .catch(error => console.error('error')) // enters catch method -> prints 404

fetch(url, { method: 'GET' })
    .catch(error => console.log('BAD', error))
    .then(response => console.log('GOOD', response.status, response.statusText)); // enters then method -> prints 404 status

In the end, this is a personal preference. Some industry experts argue that the request didn’t fail, so it finished successfully. This is independent from whether the server could find the resource or not.

However, it’s a significant difference to know when working with one of both libraries.

Comparison: How to timeout a request?

Having the ability to define a timeout for a request prevents your application from hanging requests. These hanging requests can slow down your application or cause a memory leak, and therefore, are pretty dangerous.

As indicated by Roberto Vitillo, “Modern applications don’t crash; they hang. One of the main reasons for it is the assumption that the network is reliable. It isn’t.”

So, how do fetch() and Axios handle request timeouts?

First, Axios doesn’t set a default timeout. Therefore, it relies on the browser's request timeout. In Chrome, a network request times out at 300 seconds. That’s way too long. Let’s change that using the config object. The example below aborts the request after a 1000 milliseconds waiting time.

const response = await axios.get('https://jsonplaceholder.typicode.com/todos/1', {
    timeout: 1000
});

For fetch(), there was originally no way to set a timeout. Luckily, we can make use of the experimental Abort API that supports request timeouts. Here’s an example of how to abort a request after 1000ms with fetch().

const controller = new AbortController();
const fetchPromise = fetch('https://jsonplaceholder.typicode.com/todos/1', { signal: controller.signal });  

// Abort request using setTimeout
setTimeout(() => controller.abort(), 1000); 

fetchPromise.then(response => {
    // Finished request
    console.log(response)
})

Note how we use the AbortController and pass the abort signal via our config object to the fetch() method. Once the setTimeout() timer runs out, it will call the abort() method on the controller object. This abort call will send the abort signal to our fetch() call.

Again, fetch() opts for a more lengthy workaround to implement a timeout for requests.

Conclusion

Axios provides a more comfortable to use API in comparison with fetch(). The most significant disadvantage of using fetch() is the verbose workaround to set a timeout for your request. It’s a much-needed functionality to prevent hanging requests.

On the other hand, remember that Axios sets the default timeout to zero. Therefore, don’t forget to set a timeout for each request. Alternatively, you can use a request interceptor to automatically set the request timeout.

Furthermore, for Node.js especifically, automatic JSON parsing is a useful feature to keep your code clean. Again, you can use an interceptor for the Fetch API to parse a response object automatically to JSON.

Yet, as seen in this guide, we have to overwrite the fetch() method to add custom logic. I think it adds complexity to your code. For that reason, Axios is a clear winner to send HTTP requests based on its simplicity and ease-of-use.

Why Use Redux - Reasons With Clear Examples

Michiel Mulders — Wed, 20 Jan 2021 22:00:10 +0000

Redux is a pattern and library for managing and updating application state, using events called "actions". In other words, the Redux pattern provides state management for JavaScript apps. You can use the Redux library with any frontend framework, such as React, Angular, or even Vue. In the end, you can implement the Redux pattern in any vanilla JS application.

This article covers what Redux is, why you need it, how it works, the benefits, and when not to use Redux. To make Redux clearer, we’ll update you with code examples to make the concept easier to grasp. Let’s get started!

What is Redux?

As mentioned in the introduction, Redux is a pattern that facilitates state management. It allows you to maintain a predictable state container for your JavaScript apps. This is important for consumer-facing applications where the interface changes based on user input.

On top of that, Redux prevents race conditions where two components simultaneously try to update the state. It accomplishes this task by defining actions that get dispatched to reducers.

Each action contains a type (also seen as an identifier) and a payload. Next, a reducer accepts the action and changes the state based on the received action type and payload.

Reducers are pure functions, which means they are predictable. A pure function returns the same output for the same input. You can use reducers to generate a new application state.

Lastly, to notify our interface that the application state has changed, we can subscribe to data changes. Whenever the application state changes, we update the UI.

It’s a simple, yet elegant solution to facilitate predictable state management for small and large applications. Luckily, most popular frameworks offer support for Redux. Here's a quick overview:

React -> react-redux: You can add Redux to your React application by installing the react-redux dependency.
Angular -> @ngrx/store or @angular-redux/store: Both options work well to implement Redux into your Angular application. According to npmtrends.com, @angular-redux/store is the most popular library in terms of weekly downloads.
Vue -> vuejs-redux: This dependency offers a light-weight implementation of Redux for Vue with only 45 lines of code and no dependencies. It provides the same API as the react-redux dependency.

Next, why should you use Redux?

Why use Redux?

Instead of directly exploring examples, let’s learn more about the problem Redux solves.

As with many applications, they start small. Imagine a pyramid structure of seven components where each component as two child components. Each component manages its state. However, situations occur where we have to share a state with a child component or a child component wants to modify the parent component’s state.

Do you see the problem? While our application grows to a higher number of components, maintaining data consistency becomes a hairy challenge. It’s not an easy task to manage each component’s state while sharing it with many other components. You’ll likely experience data inconsistency bugs, a fearsome nightmare for frontend developers.

Image source: Codecentric blog

As shown in the image, Redux takes away the responsibility from individual components to manage a state. Instead, we create a single store that handles our state management. On top of that, all communication regarding reading, updating, or creating data happens via the store. It prevents data inconsistency bugs from appearing. Moreover, components can listen to state changes to update the UI and avoid these data inconsistency bugs.

Lastly, you can install the Redux DevTools that give you insights into your application’s current state to simplify debugging or testing your application. It’s a great incentive to get started with Redux.

Next, let’s explore Redux with code examples.

Redux with code examples - How does it work?

Let’s recap the Redux cycle before we take a look at our code example. This is how the Redux cycle looks like:

Users interact with the interface and triggers an action
Action with/without payload is sent to a reducer using the dispatcher
Reducer checks if it handles the action and produces a new state based on the action and its payload
State changes are notified via subscription methods
UI renders again based on state changes received via the subscription method

Now, let’s explore how this works using code. We’ll create a simple webpage that allows you to increase or decrease a counter in the state using plus and minus buttons. We’ll use a single index.html document that contains a script tag with all the necessary code.

You can find the completed code via CodeSandbox.io.

Step 1: Exploring index.html

First, let’s create an index.html document with the following HTML setup. This will render the current counter value and buttons to increase or decrease the counter.

<!DOCTYPE html>
<html>
  <head>
    <title>Redux basic example</title>
    <script src="https://unpkg.com/redux@latest/dist/redux.min.js"></script>
  </head>
  <body>
    <div>
      <p>
        Counter: <span id="count">0</span>
        <button id="increment">+</button>
        <button id="decrement">-</button>
      </p>
    </div>
    <script>
    </script>
  </body>
</html>

Next, let's take a look at how we can define actions.

Step 2: Define Redux actions

Let’s define actions we want to dispatch to the reducer when the user clicks the increase or decrease button.

We can listen for the click event and dispatch a new action to the Redux store, which contains the reducer.

Make sure to dispatch an object that contains the type property. This property contains the name of the action. As a best practice, use the format <reducer-name>/<action>. This makes it easier to identify actions as multiple components can send the same action. In our example, we will name the reducer counter. Therefore, we get the following actions:

counter/increment
counter/decrement

<script>
      document
        .getElementById("increment")
        .addEventListener("click", function () {
          store.dispatch({ type: "counter/increment" });
        });

      document
        .getElementById("decrement")
        .addEventListener("click", function () {
          store.dispatch({ type: "counter/decrement" });
        });
</script>

Next, let’s define the reducer.

Step 3: Define a reducer

As we can dispatch multiple actions to the reducer, we’ll use a switch statement to handle the different actions.

First, we define the initial state for the application by setting the count equal to zero. Next, we define a counterReducer function that accepts the current state and the dispatched action.

Two scenarios are possible here:

Reducer receives an counter/increment action to increase the counter
Reducer receives an counter/decrement action to decrease the counter

Note that we use the reducer function as an argument for the createStore function to define a new Redux store for our application.

<script>
      // Define an initial state for the app
      const initialState = {
        count: 0
      };

      // Create a "reducer" function that determines what the new state
      // should be when something happens in the app
      function counterReducer(state = initialState, action) {
        switch (action.type) {
          case "counter/increment":
            return { ...state, count: state.count + 1 };
          case "counter/decrement":
            return { ...state, count: state.count - 1 };
          default:
            // If the reducer doesn't care about this action type,
            // return the existing state unchanged
            return state;
        }
      }
</script>

Tip: Don’t forget to add a default clause to your switch statement that returns the current state. You may have multiple reducers for your application. When you dispatch an action, the action is sent to all reducers and not only the intended one. Therefore, you want all reducers to return the current state to avoid errors and only the intended reducer to return the updated state.

Step 4: Subscribing to state changes

In this step, we want to subscribe to state changes to update the UI when the state has changed.

We define a render method that retrieves the current state and renders the count property. Further, we pass this render function as an argument to the store.subscribe method to update the UI automatically when the state changes. This method exposed by our newly created store will call the render function when the state has changed.

<script>
      // Our "user interface" is some text in a single HTML element
      const countEl = document.getElementById("count");

      // Whenever the store state changes, update the UI by
      // reading the latest store state and showing new data
      function render() {
        const state = store.getState();
        countEl.innerHTML = state.count.toString();
      }

      // Update the UI with the initial data
      render();

      // And subscribe to redraw whenever the data changes in the future
      store.subscribe(render);
</script>

Make sure to take a look at the completed code. You can use the CodeSandbox to play with the code yourself or fork the example.

Completed code below:

<!DOCTYPE html>
<html>
  <head>
    <title>Redux basic example</title>
    <script src="https://unpkg.com/redux@latest/dist/redux.min.js"></script>
  </head>
  <body>
    <div>
      <p>
        Counter: <span id="count">0</span>
        <button id="increment">+</button>
        <button id="decrement">-</button>
      </p>
    </div>
    <script>
      const initialState = {
        count: 0
      };

      function counterReducer(state = initialState, action) {
        switch (action.type) {
          case "counter/increment":
            return { ...state, count: state.count + 1 };
          case "counter/decrement":
            return { ...state, count: state.count - 1 };
          default:
            return state;
        }
      }

      const store = Redux.createStore(counterReducer);

      const countEl = document.getElementById("count");

      function render() {
        const state = store.getState();
        countEl.innerHTML = state.count.toString();
      }

      render();

      store.subscribe(render);

      document
        .getElementById("increment")
        .addEventListener("click", function () {
          store.dispatch({ type: "counter/increment" });
        });

      document
        .getElementById("decrement")
        .addEventListener("click", function () {
          store.dispatch({ type: "counter/decrement" });
        });
    </script>
  </body>
</html>

That's it!

What are the benefits of using Redux?

There are many benefits to using Redux. The most prominent benefit is improved state management for your application. Yet, there are many other benefits.

Easy debugging and testing. You can use the Redux DevTools or log the state to understand better what’s happening in your application.
Reducer functions can be tested quickly. As reducer functions are pure functions, they produce the same output for the same input. Therefore, testing pure functions become a simple task.
Hook monitoring tools. You can hook monitoring tools to your application’s state to monitor the state in real-time. This improves visibility for your application and allows you to track different metrics.
Predictable outcome. Every action produces a predictable outcome. Your state store acts as a single source of truth. Therefore, you can avoid data inconsistency bugs and don’t have to worry about data synchronization issues between components.

When not to choose Redux?

For beginners, it’s an obvious choice to opt for Redux. Yet, you don’t always need Redux to manage the state of your application.

Applications that consist of mostly simple UI changes most often don’t require a complicated pattern like Redux. Sometimes, old-fashioned state sharing between different components works as well and improves the maintainability of your code.

Also, you can avoid using Redux if your data comes from a single data source per view. In other words, if you don’t require data from multiple sources, there’s no need to introduce Redux. Why? You won’t run into data inconsistency problems when accessing a single data source per view.

Therefore, make sure to check if you need Redux before introducing its complexity. Although it’s a reasonably efficient pattern that promotes pure functions, it might be an overhead for simple applications that involve only a couple of UI changes. On top of that, don't forget that Redux is an in-memory state store. In other words, if your application crashes, you lose your entire application state. This means that you have to use a caching solution to create a backup of your application state, which again creates extra overhead.

If you want to learn more about Redux, check out the Redux FAQ section, which contains tons of interesting questions you might have about Redux. Also, check out this amazing analogy for Redux by Hitesh Choudhary.

Instagram Automation Bot for JavaScript: How to use InstAuto?

Michiel Mulders — Thu, 19 Nov 2020 14:27:45 +0000

For many Instagram users, using an Instagram automation bot has proven to grow their account and audience.

An Instagram bot provides a simple way to find new accounts to follow, which are likely interested in your post's content.

However, we must use an Instagram bot correctly. There's no point in targeting people who like content about cars while posting content about flowers.

On top of that, Instagram implements stringent guidelines for bot automation. Although we don't know the exact limitations, a study by SocialPros.co showed the following average interaction statistics for users in 2020:

13 follows per hour, or 100-150 follows per day
300 to 400 likes per day (of followed accounts)
2 to 5 comments per hour, or 20-30 comments per day

In this article, we'll learn how to set up the JavaScript Instagram automation bot InstAuto and how to automate within Instagram's terms of service without receiving a shadowban.

Why use an Instagram bot?

Instagram bots help us with repetitive tasks, such as liking, commenting, and (un)following. These are the core actions of the Instagram platform and consume a lot of time when done manually.

Therefore, this presents us with the opportunity to use an Instagram automation bot to automate the above tasks.

On top of that, an Instagram bot offers us much more flexibility.

Let's say we follow five new accounts, and we want to unfollow them after three days. It's quite a daunting task to manually verify if this person has followed you and then unfollow him.

For this reason, we use an Instagram bot as it allows for much more complex scenarios. Therefore, an Instagram automation bot often keeps track of likes, follows, and comments using a database. A database allows the bot to quickly lookup accounts that we followed three days ago, which we want to unfollow.

Let's take a look at how we can use InstAuto to configure our first Instagram automation bot.

Requirements

First, let's discuss the requirements for this tutorial.

Node.js version 8 or newer
New project setup using npm init with an index.js file in the root
Install required dependencies: npm install --save dotenv puppeteer instauto

Done? Let's continue!

Step 1: Creating Your First Instagram Automation Bot

It's time to modify the index.js file. We first want to load the required dependencies. We'll use dotenv to hide secrets, such as our Instagram username and password, where we use puppeteer for controlling a Chrome or Chromium browser instance to automate all aspects of our Instagram tasks.

Add the following code.

require('dotenv').config()
const puppeteer = require('puppeteer')
const Instauto = require('instauto')

Next, let's prepare the options object to configure our Instagram bot. Here, we can configure the limits for our Instagram bot. To be safe, we only allow 20 new follows per hour and limit the total number of followers per day to 150. Also, we set a maximum number of likes per day equal to 50.

Note that you toggle a dry run on or off. For testing purposes, enable the dryRun property.

const options = {
  cookiesPath: './cookies.json',

  username: process.env.INSTA_USERNAME,
  password: process.env.INSTA_PASSWORD,

  // Global limit that prevents follow or unfollows (total) to exceed this number over a sliding window of one hour:
  maxFollowsPerHour: 20,
  // Global limit that prevents follow or unfollows (total) to exceed this number over a sliding window of one day:
  maxFollowsPerDay: 150,
  // (NOTE setting the above parameters too high will cause temp ban/throttle)

  maxLikesPerDay: 50,


  dontUnfollowUntilTimeElapsed: 3 * 24 * 60 * 60 * 1000,

  // Usernames that we should not touch, e.g. your friends and actual followings
  excludeUsers: [],

  // If true, will not do any actions (defaults to true)
  dryRun: true,
};

Next, we need to add our Instagram username and password. As you can see, we import those secrets using process.env (dotenv via npm).

Therefore, let's create a .env file in the root of your project. Make sure to add both INSTA_USERNAME and INSTA_PASSWORD variables.

Here's an example of the .env file. Make sure to replace the values!

INSTA_USERNAME=myusername
INSTA_PASSWORD=mypassword

All good? Let's add the bot execution code. This snippet follows the code for the options object.

Here, we launch an instance of Puppeteer to enable automatization. Next, we also create a database that keeps track of followed and unfollowed users, and liked photos.

Then, we pass the Puppeteer browser instance, database, and options object to the InstAuto dependency to create our bot.

It's possible to pass one or multiple users for which we want to follow their followers. At the moment, we've added only one Instagram username dedik_armawann.

This is the current setup for the bot:

Follow all users (within the predefined limits) for user dedik_armawann
Skip all private accounts
Like 1-2 user images
When the bot has completed following all accounts, it will sleep for 10 minutes and start the unfollow process for accounts that haven't followed us back. Remember that we've set the dontUnfollowUntilTimeElapsed to three days.
Unfollow all users that didn't follow us back after 60 days (clean up phase)

(async () => {
  let browser;

  try {
    browser = await puppeteer.launch({ headless: false });

    // Create a database where state will be loaded/saved to
    const instautoDb = await Instauto.JSONDB({
      // Will store a list of all users that have been followed before, to prevent future re-following.
      followedDbPath: './followed.json',
      // Will store all unfollowed users here
      unfollowedDbPath: './unfollowed.json',
      // Will store all likes here
      likedPhotosDbPath: './liked-photos.json',
    });

    const instauto = await Instauto(instautoDb, browser, options);

    // List of usernames that we should follow the followers of, can be celebrities etc.
    const usersToFollowFollowersOf = ['dedik_armawann'];

    // Now go through each of these and follow a certain amount of their followers
    await instauto.followUsersFollowers({ usersToFollowFollowersOf, skipPrivate: true, enableLikeImages: true });

    await instauto.sleep(10 * 60 * 1000);

    // This is used to unfollow people - auto-followed AND manually followed -
    // who are not following us back, after some time has passed
    // (config parameter dontUnfollowUntilTimeElapsed)
    await instauto.unfollowNonMutualFollowers();

    await instauto.sleep(10 * 60 * 1000);

    // Unfollow auto-followed users (regardless of whether they are following us)
    // after a certain amount of days
    await instauto.unfollowOldFollowed({ ageInDays: 60 });

    console.log('Done running');

    await instauto.sleep(30000);
  } catch (err) {
    console.error(err);
  } finally {
    console.log('Closing browser');
    if (browser) await browser.close();
  }
})();

Good, let's try our bot!

Step 3: Starting a Bot Dry Run

Now, let's start an Instagram bot dry run. We can execute the index.js file from our terminal.

node index.js

If we've passed the correct Instagram username and password, the bot will open a new Chromium browser and log in to your Instagram account.

Have some patients while the bot is running. The bot will eventually look up the usernames we've passed to follow and start the follow and liking process.

We can always track the progress in the terminal as the bot prints everything it does. On top of that, you can also open the following files to track the progress:

followed.json
unfollowed.json
liked-photos.json

Here's an example of how the bot works:

Does your bot work? Yes! Next, let's customize our bot.

Step 4: Instagram Bot Customization

In this step, we want to discuss different parameters to tailor your bot. Let's discuss four essential parameters.

followUserRatioMin: Don't follow users that have a follower to following ratio of less than the defined value.
followUserRatioMax: Don't follow users that have a follower to following ratio higher than the defined value. Often, spam accounts have a very high ratio. To avoid those accounts, you can set this property.
followUserMaxFollowers: Don't follow users who have more followers than the defined value. This property is useful to avoid popular accounts as they often don't follow back.
followUserMaxFollowing: Similar property to not follow users who have more people following them than the defined value.

Of course, you can always exclude certain users from your bot using the excludeUsers array property.

Conclusion

In this article, we've learned how to set up an Instagram automation bot, operate a bot within the Instagram terms of service, and customize the InstAuto bot.

Tip: Don't forget to set the dryRun property to false when using the bot in production.

You can find more information about the bot on GitHub and more information about running the bot on a server using a tool such as pm2.

Creating Custom Blockchain Transactions with the SDK — Introducing Lisk Bills

Michiel Mulders — Thu, 22 Aug 2019 14:28:03 +0000

The Lisk Bills webinar is now live on our YouTube channel. Subscribe for more educational developer content.

Lisk’s Alpha SDK Phase officially began in late July with the release of SDK 2.1.0. We decided what better way to showcase the potential of custom transactions than to create our own proof-of-concept (PoC) blockchain application. To explore the possibilities of custom transactions at their best, we decided to build an invoicing application and through this register two new custom transactions on our blockchain.

Introduction to Custom Transactions

Lisk SDK allows you to define your own custom transaction types where you can implement the required logic for your blockchain use-case. The custom transaction types are an extension to the default set of transactions that is already part of the Lisk Protocol. You can read more about the predefined types here.

The beginning of the Alpha SDK phase of our roadmap allows you to create your own proof-of-concept blockchain applications aligned with our architecture. This phase of our roadmap also allows us to get feedback on how the development experience can be improved via discussion on our community channels.

Custom Transactions to Grow the Lisk Ecosystem

Custom transactions offer great business value to the Lisk Ecosystem as they allow for a lot of creativity. We believe custom transactions are the “creative spark” for the Lisk Ecosystem to see a whole bunch of innovative projects being created. We already see community members coming up with their own solutions ranging from hardware-utilizing rental bike tracking system, Lisk.Bike, to the usage of our modular JavaScript library for an innovative take on a classic strategy game, Lisk Tic-Tac-Toe. Now is your time to get creative!

Benefits of Custom Transactions

Every account object has the ability to store data in its asset field. Custom transactions make clever use of this. The use of the asset field allows for any type of stringified data to be passed to the transaction. This allows for greater flexibility and creativity when defining custom logic.

Besides that, every custom transaction can access and modify all account-related data and only read transaction-related data from the database. This allows for more advanced interactions between data and even between different custom transactions. For example, our PoC used the data from the Invoice transaction to verify the validity of the Payment transaction.

You can also create a token in the asset field with some basic transfer and verification logic. In the end, this is just another way of smart contract logic.

Let’s continue with exploring the technicals of our Lisk Bills PoC.

Lisk Bills — Blockchain-Based Invoicing

As we like the Keep it Simple and Stupid (KISS) approach, we have built a minimal frontend with React that uses the Lisk Alpha SDK to interact directly with your blockchain application. The PoC includes two actors, the client and the freelancer.

Imagine Alice (Freelancer) and Bob (Client). Bob is looking for a new logo for his website and decides to consult a freelancer. While looking for a good designer, he comes across Alice who offers some spectacular designs in her portfolio. Bob is so excited he decides to immediately employ Alice’s skillset.

A few days go by, and Alice returns the promised logo together with an invoice. However, Bob is a big fan of blockchain technology as it helps to ease the settlement process. It often happens parties disagree about the agreed price, product, or even shipping terms. Bob, therefore, believes blockchain can help with recording all this information right from the beginning, so no disputes can occur and human error can be eliminated. The blockchain should act as proof for the invoice.

For the above reason, Bob asks Alice to create the invoice via a Lisk custom transaction.

In order to do so, Alice first has to log in with her passphrase to the Lisk Bills application.

Custom Transaction 1: Invoice

Now Alice has been logged in, she can create an invoice. To create the custom invoice transaction, Alice has to input the following details:

Client holds Bob’s Lisk address or business name.
RequestedAmount holds the amount Bob is due to Alice.
Description to describe the delivered design service.

The following data is stored in the asset field of the transaction. As this is a normal BaseTransaction, we can simply specify Bob’s Lisk address as the recipient for the transaction.

Before we dive into the technicals, make sure to open or clone the lisk-sdk-examples repository. The code for both custom transactions can be found in invoice/transactions/invoice_transaction.js and invoice/transactions/payment_transaction.js.

Technicals

First of all, let’s take a look at the class definition. The InvoiceTransaction extends the BaseTransaction which means it’s inheriting its properties. As the name suggests, BaseTransaction is the most basic interface for creating new transaction types. Other transaction types also exist in the system, later we will show an example of extending the TransferTransaction type.

When extending the BaseTransaction we can provide extra business logic for the following methods: Prepare, Validate Asset, Apply Asset and Undo Asset. You can find out more about these methods in our documentation.

Also, pay attention to the static getter function for retrieving the type of the transaction. As an example, we have chosen 13 to be the type number for this transaction. Besides that, you can set the fee you want users to pay for using this transaction type. For now, we have set this to 1 LSK (10 to the 8th beddows).

class InvoiceTransaction extends BaseTransaction {
  static get TYPE () {
    return 13;
  }

  static get FEE () {
    return `${10 ** 8}`;
  }

  ...
}

Prepare

The prepare function is responsible for loading the required data used inside the applyAsset() and undoAsset() function. Here, we try to load the account data for the sender as we want to add data to his asset field in the applyAsset() function. This data will be loaded from the StateStore object that provides access to data in the database.

We can cache the sender account like this by passing an array with filters.

await store.account.cache([
    {
        address: this.senderId,
    },
]);

However, we actually don’t have to manually cache the data. We can simply call the parent method for the prepare function in the abstract BaseTransaction class which will by default cache the sender account to deduct the fee in the apply step.

async prepare(store) {
    await super.prepare(store);
}

Validate Asset

Before a transaction reaches the apply step it gets validated. Check the transaction’s asset correctness from the schema perspective (no access to StateStore here). You can invalidate the transaction by pushing an error into the result array.

validateAsset() {
    const errors = [];
    if (!this.asset.client || typeof this.asset.client !== 'string') {
        errors.push(
            new TransactionError(
                'Invalid "asset.client" defined on transaction',
                this.id,
                '.asset.client',
                this.asset.client,
                'A string value',
            )
        );
    }
    if (!this.asset.requestedAmount || typeof this.asset.requestedAmount !== 'string') {
        errors.push(
            new TransactionError(
                'Invalid "asset.requestedAmount" defined on transaction',
                this.id,
                '.asset.requestedAmount',
                this.asset.requestedAmount,
                'A string value',
            )
        );
    }
    if (!this.asset.description || typeof this.asset.description !== 'string') {
        errors.push(
            new TransactionError(
                'Invalid "asset.description" defined on transaction',
                this.id,
                '.asset.description',
                this.asset.description,
                'A string value',
            )
        );
    }
    return errors;
}

Apply Asset

As you can see, we finally use the loaded account which we put in the store during the prepare step. Next, we update the invoice count and record the ID of the invoice in an array with sent invoices. We will use this data in our frontend to display all invoices.

applyAsset(store) {
    const sender = store.account.get(this.senderId);

    // Save invoice count and IDs
    sender.asset.invoiceCount = sender.asset.invoiceCount === undefined ? 1 : sender.asset.invoiceCount++;
    sender.asset.invoicesSent = sender.asset.invoicesSent === undefined ? [this.id] : [...sender.asset.invoicesSent, this.id];
    store.account.set(sender.address, sender);
    return [];
}

Undo Asset

Do not underestimate the importance of the undoAsset() function. The Undo function allows us to rollback to a previous blockchain state. Therefore, we should exactly tell our blockchain application how it should roll back changes.

The Undo function is of most importance for the fork recovery mechanism. In case a fork occurs on a chain with tip B and we want to roll back to a common height in order to re-apply blocks up to the tip of chain A, we need the Undo function to do the actual rollback to this common height.

For the invoice proof of concept, the code reduces the invoiceCount and removed the invoice ID from the invoicesSent array.

undoAsset(store) {
    const sender = store.account.get(this.senderId);

    // Rollback invoice count and IDs
    sender.asset.invoiceCount = sender.asset.invoiceCount === 1 ? undefined : sender.asset.invoiceCount--;
    sender.asset.invoicesSent = sender.asset.invoicesSent.length === 1 
        ? undefined 
        : sender.asset.invoicesSent.splice(
            sender.asset.invoicesSent.indexOf(this.id),
            1,
        );
    );
    store.account.set(sender.address, sender);
    return [];
}

Ok, we have explored the functions for the invoice transaction. Let’s move to the payment transaction.

Custom Transaction 2: Payment

Now Bob has received the Invoice Transaction in his wallet, he decides to pay for the invoice. In order to fulfill the transaction, we would normally send a TransferTransaction which is natively supported by Lisk SDK.

However, doing so would make this a very boring tutorial. Therefore, Bob decides to use another custom transaction to showcase Lisk’s possibilities. This custom Payment Transaction holds logic to verify the transferred amount is at least equal to the RequestedAmount. Also, the transaction requires Bob to specify the ID of the invoice he wants to fulfill.

If the transferred amount is too low or the invoice ID just doesn’t exist, the transaction fails. Bob keeps his side of the agreement and sends the requested amount to Alice’s invoice ID. Bob even adds a tip for Alice’s great work.

This is how the UI implementation looks like for paying an invoice with our Lisk Bills application.

Technicals

Again, let’s take a look at the class definition. The PaymentTransaction extends the TransferTransaction which means it’s inheriting its properties like a different fee and transfer-related verification checks. Also, pay attention to the static getter function for retrieving the type of the transaction. As we can not have identical transaction types, the PaymentTransaction has received type 14.

class PaymentTransaction extends TransferTransaction {

    static get TYPE () {
        return 14;
    }

    ...
}

Also, note we don’t define a static getter function for FEE. We didn’t implement it here as we don’t want to overwrite the FEE defined in the TransferTransaction. In short, we want to use the 0.1 fee defined in TransferTransaction.

Prepare

The prepare function is responsible for loading the required data into the store to be used inside the applyAsset() and undoAsset() function. For the PaymentTransaction, we are loading the transaction that holds the invoice using the ID sent with this.asset.data.

async prepare(store) {
    await super.prepare(store);
    await store.transaction.cache([
        {
            id: this.asset.data,
        },
    ]);
}

Validate Asset

As you might have noticed, we didn’t implement the validateAsset() function for the payment transaction. The only check we have to perform is validating if the sent number of tokens is at least equal to the requested number of tokens.

In order to validate this, we need access to the StateStore as we need to cache the invoice transaction. Because we can only perform static checks in the validateAsset() function that don’t use the StateStore, this check is moved to the apply step.

Apply Asset

The applyAsset() function first tries to find the corresponding invoice transaction. If this transaction exists, we further check for the number of tokens transferred is at least equal to the requested amount in the invoice. If this check succeeds, the transaction gets applied.

applyAsset(store) {
    const errors = super.applyAsset(store);

    const transaction = store.transaction.find(
        transaction => transaction.id === this.asset.data
    ); // Find related invoice in transactions for invoiceID

    if (transaction) {
        if (this.amount.lt(transaction.asset.requestedAmount)) {
            errors.push(
                new TransactionError(
                    'Paid amount is lower than amount stated on invoice',
                    this.id,
                    '.amount',
                    transaction.requestedAmount,
                    'Expected amount to be equal or greated than `requestedAmount`',
                )
            );
        }
    } else {
        errors.push(
            new TransactionError(
                'Invoice does not exist for ID',
                this.id,
                '.asset.invoiceID',
                this.asset.data,
                'Existing invoiceID registered as invoice transaction',
            )
        );
    }

    return errors;
}

Undo Asset

No rollback logic is required for the undo step of the payment transaction. We do not modify any data in the store with the set method, so no need for defining undo steps to revert this data change.

However, do not forget to call super.undoAsset(store) as the Undo step will make sure the fee Alice has paid gets returned to her account’s balance.

How to Register Custom Transactions?

Ok, we have prepared both of our custom transactions. Bob and Alice are very happy to use both transactions in order to finalize their deal. However, we don’t know yet how to register these new transactions on our blockchain application.

The invoice/index.js file holds the startup code for running your custom blockchain and also registers both transactions. It’s as simple as that!

const { Application, genesisBlockDevnet, configDevnet } = require('lisk-sdk');
const { InvoiceTransaction, PaymentTransaction } = require('./transactions/index');

const app = new Application(genesisBlockDevnet, configDevnet);

app.registerTransaction(InvoiceTransaction);
app.registerTransaction(PaymentTransaction);

app
    .run()
    .then(() => app.logger.info('App started...'))
    .catch(error => {
        console.error('Faced error in application', error);
        process.exit(1);
    });

Ok, we are all done! At last, let’s take a brief look at the considerations regarding the use of custom transactions.

Considerations of Using Custom Transactions

Currently, we expect users to run their own blockchain instance that registers their freshly created custom transaction.

It took us a few weeks to build this prototype. We have deliberately kept it simple to act as a learning resource, and as inspiration for the community. It is not production-ready.

Conclusion

Lisk aims to allow for creativity within the blockchain industry by providing the ability to process data with custom business logic. This concept is very similar to smart contracts as they also hold custom business logic. We are pleased to introduce you to Lisk Bills as the first example of what is possible with our SDK.

We hope this freedom will spark a whole bunch of new innovative blockchain applications build on top of Lisk using the newly released Lisk Alpha SDK. Currently, we do not plan to support custom transactions on the Lisk mainnet but they are intended to be used inside your own blockchain application.

Lisk is on a mission to enable you to create decentralized, efficient, and transparent blockchain applications. Join us:

Why we stopped using 'npm start' for running our blockchain core's child processes

Michiel Mulders — Thu, 23 May 2019 15:52:18 +0000

You shouldn't start applications through npm when you have child processes natively supported by Node.js. In this article, we will provide a list of best practices for Node.js applications with a code snippet that outlines the core problem and shows you how to reproduce the issue in 3 steps. In short, we stopped using npm start to run our blockchain's core and instead opted for using the native node command.

Introduction to npm and its most well-known command 'npm start'.

Npm is the go-to node package manager when you are working on a JavaScript project. It allows you to install other people's code packages into your own project so you don't have to code everything you need from scratch. Npm also became famous because of its industry-wide usage of script commands that can be entered in the shell to start your application. Of course, the most well-known command is npm start that acts as a wrapper for node app.js.

Our challenge: npm runs app.js files as a child process ofnpm.

However, what many don't know is that while using npm start to trigger node app.js, npm is actually running your app.js file as a child process of npm which manages this. In 99% of the cases, you shouldn't care about this, but things can get tricky when working with child processes in your own project. Can you feel the inception happening here? #child-process-inception

If you want to know more about Lisk first, check out this short explainer clip and our documentation!

To give you a better understanding of how this is relevant to our "npm vs node" problem, let's talk about how we are running Lisk Core. For those who don't know what Lisk Core is, essentially, it is a program that implements the Lisk Protocol which includes consensus, block creation, transaction handling, peer communication, etc. Every machine must set it up to run a node that allows for participation in the network.

Intro to PM2, a production process manager for Node.js apps.

In our case, we use PM2 to restart the application upon failure. PM2 is a production process manager for Node.js applications with a built-in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks.

A few weeks ago, we decided to provide the ability to run the http_api module as a child process to improve the overall efficiency of the Lisk Core application while using the same allocated resources.

Rationale behind the decision to run http_api module as a child process.

The idea behind this decision was mainly funded by the fact that functionally isolated components can form the basis of a multi-process application, in order to utilize the potential of multiple hardware cores of the physical processor if available. Also, to design each component in a resilient way to tackle brittleness of the multi-processing. This means that a failure of one component will have minimal impact on other components and that components can recover individually. More information about child processes can be found in our proposal to introduce a new flexible, resilient and modular architecture for Lisk Core.

We were not able to gracefully exit Lisk Core with npm.

While implementing child processes for the http_api module, Lightcurve Backend Developer Lucas Silvestre discovered that Lisk Core was not exiting gracefully while running the http_api module as a child process using PM2. This resulted in a tricky situation where the http_api kept on running in the background whenever the main process (Lisk Core) crashed.

Whenever this happens, PM2 will attempt to recover the Lisk Core process. However, this would spawn a new http_api process which was not possible as the port was already in use because of the cleanup process not being called. The resulted in PM2 not being able to restore the application which is a big issue when running a blockchain node that is part of the network. In this case, the user has to manually restart the blockchain node which we absolutely want to avoid.

Running Lisk Core with node command

This issue made us aware of the difference between npm and node and made us reconsider the way we were running Lisk Core. Previously, we just accepted the npm start industry standard as the go-to way of running an application.

Later, we found the best practices provided by the docker-node GitHub repository dedicated to Dockerizing Node.js applications. Here, a clear warning message can be found about the usage of npm inside of a Dockerfile or any other higher-level application management tool like PM2.

"When creating an image, you can bypass the package.json's start command and bake it directly into the image itself. First off this reduces the number of processes running inside of your container. Secondly, it causes exit signals such as SIGTERM and SIGINT to be received by the Node.js process instead of npm swallowing them."

Whenever we tried to exit Lisk Core or the application crashed, a SIGINT signal is sent to the application. In Node.js, you can listen for this signal and execute a cleanup function in order to gracefully exit the application. In our case, we are removing various listeners and pass the SIGINT signal to the child process in order to exit this one gracefully as well.

As stated by docker-node, npm swallows this signal and does not trigger our listeners for the SIGINT signal causing the application to not being able to clean up gracefully. That's also the reason why the http_api module kept running inside of PM2.

Nick Parson, an expert when it comes to running Node applications with PM2 also mentions the fact that it is important to gracefully shut down your application in order to maximize robustness and enable fast startup (no downtime) when using PM2.

Termination signals: what are SIGKILL, SIGTERM, and SIGINT?

We have to dive quite deep to find out what these signals are about. These signals are part of a collection of signals to tell a process to terminate, actually many more exist, and can be found in the documentation provided by gnu.org under section 24.2.2 Termination Signals.

SIGKILL: "The SIGKILL signal is used to cause immediate program termination. It cannot be handled or ignored, and is therefore always fatal. It is also not possible to block this signal."
SIGTERM: "The SIGTERM signal is a generic signal used to cause program termination. Unlike SIGKILL, this signal can be blocked, handled, and ignored. It is the normal way to politely ask a program to terminate." Interesting to know that the shell command kill generates SIGTERM by default.
SIGINT: "The SIGINT ('program interrupt') signal is sent when the user types the INTR character (normally C-c)." Developers will probably be more familiar with the CTRL/CMD+C command to interrupt a running process in the shell.

Moving Docker and PM2 to Node.

This made us decide to get rid of npm start and replacing it by the node command. The start command was being used in both the Dockerfile as the PM2 run file.

The following image shows a snippet of the typical ENTRYPOINT for Docker. Previously, this would contain ENTRYPOINT ["npm", "start"]. This file can be found now in our new Lisk Core repositor which is extracted from the Lisk-SDK Monorepo.

Lisk-SDK Dockerfile.Also, the same applies to the pm2-lisk.json file which contains the PM2 configuration for starting Lisk Core. The script property now contains the relative path to the index file.

Learn how to reproduce the issue in 3 steps.

We can find a cool snippet created by GitHub user EvanTahler addressing the above-mentioned issue. Let's reproduce this!

Step 1. Create package.json and app.js

To emulate this issue, you need to create two files (package.json and app.js) in the same directory. Make sure you have Node.js version 10.x or higher installed on your machine to run the snippet with the node command. As we don't need any code dependencies, we don't have to install anything else.

Package.json

{
  "name": "test",
  "scripts": {
    "start": "node ./app.js"
  }
}

App.js

process.on('SIGINT',  function(){ console.log("SIGINT");  shutDown() });
process.on('SIGTERM', function(){ console.log("SIGTERM"); shutDown() });

var string = ".";

var shutDown = function(){
  console.log("off-ing...");
  string = "x";

  setTimeout(function(){
    console.log("bye!");
    process.exit();
  }, 1000 * 5);
}

setInterval(function(){
  console.log(string);
}, 500)

Snippet clarification - The snippet will print a dot every 0.5 seconds and listens for the SIGINT and SIGTERM signals. Once one of the two termination signals is received, we will delay the shutdown by 5 seconds (5 * 1000ms) and print out "bye!".

Before running this snippet, I want to show you how a killed process is indicated in your terminal when hitting CTRL/CMD+C. You can notice it by the ^C characters.

Shows Lisk Core running for exactly 17 minutes after getting killed with the SIGINT signal.

Step 2. Run the snippet with node.

Now we know how the SIGINT is represented in our terminal, let's start the snippet with node app.js. Let it run for 5 seconds, and hit CTRL/CMD+C. You will see that the kill signal is properly handled by Node and waits for 5 more seconds before shutting down.

Step 3. Run the snippet with npm start

However, when we run the snippet with npm start, you will notice two kill signals being received. As we now know, the start command will run node app.js as a child process. So, when receiving ^C, it will try to exit the npm process and pass this termination signal to the child which causes the problem that the main process exits but the child is still active for 5 more seconds.

As explained before, this will give all sorts of problems when you try to listen for termination signals while running applications with npm start, especially when operating child processes.

Interested in learning how to set up and run your own Lisk node? More information can be found in the Lisk Core documentation on the website. You can choose between the binary setup which is the default (and most simple) installation technique. Other options include running Lisk Core with Docker to support other platforms or for more advanced users, it is possible to build from Lisk Core.

Because of this "child process inception", the http_api module could not gracefully exit and kept on running. The only way to stop this process is by using a shell command that kills all Node processes: sudo killall node (or target the specific process ID to be killed). Luckily, this could be easily resolved by using node to start the application.

Best Practices for Handling Node.js Applications

Felix Geisendörfer, an early contributor of Node.js, makes it very clear how to handle crashed applications:

Source: Node.js Best Practices SlideShare

What does the above teach us? Avoid spinning up your application through npm start but use node instead. Also, if something goes wrong, exit the process gracefully and accept it. Felix recommends using higher level tools like PM2 to deal with recovering and restarting the application.

We learned from this that you not always should take standards for granted. It is sometimes better to keep things simple and run it with a simple node command.

To conclude what we did at Lisk, we decided to solve the issue by changing the npm start command to node src/index in both the PM2 run configuration and Dockerfile. Now, upon receiving a SIGINT signal, the node process receives this directly and can communicate the SIGINT signal to its child processes so every process can be exited gracefully.

Therefore, PM2 can easily restart the application without any downtime. Running our application via this setup allows us to deploy a more stable application which is utterly important for creating a stable blockchain network.

Lisk empowers individuals to create a more decentralized, efficient and transparent global economy. We welcome you to join us in our mission: