DEV Community: Sumit Kumar Singh

List of most used Key-Value stores and databases

Sumit Kumar Singh — Sun, 05 Mar 2023 16:29:03 +0000

Originally Published At: themythicalengineer.com

⚡ List of most used Key-Value stores and databases

redis

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.

Created Date: 2009-03-22

All Languages: C, Tcl, Ruby, Python, Shell, Makefile, C++, Smarty

etcd

Distributed reliable key-value store for the most critical data of a distributed system

Created Date: 2013-07-07

All Languages: Go, Shell, Jsonnet, Procfile, Makefile, Python, Dockerfile

leveldb

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

Created Date: 2014-08-28

All Languages: C++, C, CMake

rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.

Created Date: 2012-11-30

All Languages: C++, Java, C, Python, Perl, Shell, Makefile, CMake, PowerShell, Assembly, Dockerfile

dragonfly

A modern replacement for Redis and Memcached

Created Date: 2021-12-11

All Languages: C++, C, Python, CMake, Shell, Smarty, Dockerfile, Makefile, Go

arangodb

🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

Created Date: 2011-10-26

All Languages: C++, JavaScript, NASL, CMake, C, Python, SCSS, EJS, CSS, TypeScript, Shell, Yacc, Pascal, NSIS, HTML, LLVM, Perl, PowerShell, Dockerfile

tikv

Distributed transactional key-value database, originally created to complement TiDB

Created Date: 2015-12-31

All Languages: Rust, Makefile, Shell, Python, C, Dockerfile

memcached

memcached development tree

Created Date: 2009-04-25

All Languages: C, Perl, C++, M4, Python, Lua, DTrace, Shell, Makefile, Roff, Starlark

foundationdb

FoundationDB - the open source, distributed, transactional key-value store

Created Date: 2017-12-14

All Languages: C++, C, Python, Java, CMake, Go, C#, HTML, Ruby, Shell, Assembly, JavaScript, CSS, Dockerfile, Rich Text Format

immudb

immudb - immutable database based on zero trust, SQL and Key-Value, tamperproof, data change history

Created Date: 2019-11-07

All Languages: Go, Shell, Yacc, Makefile, Dockerfile, Roff, Python, Smarty, HTML, JavaScript

KeyDB

A Multithreaded Fork of Redis

Created Date: 2019-02-01

All Languages: C++, C, Tcl, Shell, Ruby, Makefile, Roff, Python, Assembly, Dockerfile, Smarty

hazelcast

Open-source distributed computation and storage platform. Real-time Stream Processing Unconference. Save Your Spot https://hazelcast.com/lp/unconference/

Created Date: 2012-03-21

All Languages: Java, FreeMarker, Shell, Python, Batchfile, C, Kotlin

ignite

Apache Ignite

Created Date: 2015-02-19

All Languages: Java, C#, C++, Python, Shell, CMake, Batchfile, Jinja, FreeMarker, HTML, Dockerfile, PHP, PowerShell, C, JavaScript, Makefile

ledisdb

A high performance NoSQL Database Server powered by Go

Created Date: 2014-04-30

All Languages: Go, Python, Shell, Dockerfile, C++, Makefile, C

riak

Riak is a decentralized datastore from Basho Technologies.

Created Date: 2010-04-16

All Languages: Shell, Erlang, Makefile, Emacs Lisp

skytable

Skytable is a fast, secure and reliable realtime NoSQL database with keyspaces, tables, data types, authn/authz, snapshots and more to build powerful apps

Created Date: 2020-06-30

All Languages: Rust, Perl, Shell, C, PowerShell, Dockerfile, Makefile

aerospike-server

Aerospike Database Server – flash-optimized, in-memory, nosql database

Created Date: 2018-03-08

All Languages: C, C++, Makefile, Shell, CSS, Python, Roff, HTML, DTrace

anna

A low-latency, cloud-native KVS

Created Date: 2019-07-26

All Languages: C++, Python, CMake, Shell, Dockerfile

splinterdb

High Performance Embedded Key-Value Store

Created Date: 2021-07-13

All Languages: C, Shell, Python, Makefile, C++, Dockerfile

rondb

This is RonDB, a distribution of NDB Cluster developed and used by Hopsworks AB. It also contains development branches of RonDB.

Created Date: 2020-12-21

All Languages: C++, C, Java, NASL, Makefile, CMake, Objective-C, Perl, HTML, Shell, Yacc, JavaScript, Python, Pascal, Go, PHP, Roff, Starlark, Assembly, SWIG, DIGITAL Command Language, M4, CSS, Lex, Pawn, Awk, RPC, SourcePawn, LLVM, POV-Ray SDL, SAS, Module Management System, sed, Batchfile, CoffeeScript, Visual Basic 6.0, PLSQL

hibari

Hibari is a production-ready, distributed, ordered key-value, big data store. Hibari uses chain replication for strong consistency, high-availability, and durability. Hibari has excellent performance especially for read and large value operations.

Created Date: 2010-12-28

All Languages: Erlang, Makefile, Shell, Batchfile

speedb

A library that provides an embedded, persistent key-value store.

Created Date: 2022-02-22

All Languages: C++, Java, C, Python, Shell, Makefile, CMake, PowerShell, Assembly, Dockerfile

Converting Markdown to Epub or Mobi

Sumit Kumar Singh — Mon, 26 Apr 2021 03:09:55 +0000

In this blog post, I will show you the process of combining and converting a list of markdown files into kindle supported ebook (.mobi) format.

In this example I would be converting official documentation of Docker website into an ebook for offline use.

I will use Pandoc to combine and convert the markdown (.md) files into .epub format. Then I will use Calibre to convert it to .mobi format.

You can follow the steps and modify the linux commands according to your needs.

If you don't have a linux machine, you can use Deepnote platform to run linux commands in jupyter notebook.

Note: If you're running this in Deepnote, you need to append "!" as prefix to each command, so that deepnote can identify it as bash command. If running in linux machine, remove "!" from all commands written below.

Here is a small video tutorial of the working script.
Video Link: https://www.youtube.com/watch?v=L4hGDPLMPcw

Steps to be followed:

# Download Pandoc executable
!wget https://github.com/jgm/pandoc/releases/download/2.11.3.2/pandoc-2.11.3.2-1-amd64.deb

# Install pandoc
!sudo dpkg -i pandoc-2.11.3.2-1-amd64.deb

# Clone Docker CLI documentation github repository
!git clone https://github.com/docker/cli.git

# Verify the list of markdown files and contents of files
!cd cli/docs/reference/commandline && ls -la

# Generate ebook from markdown files, *.md picks up all the files with .md as extension
!cd cli/docs/reference/commandline && pandoc -o docker_cli.epub --metadata title="Docker CLI Docs" *.md

# Move generated file to required location
!mv cli/docs/reference/commandline/docker_cli.epub ~/work/docker_cli.epub

# Install required dependencies for Calibre
!apt update -y
!apt install libgl1-mesa-glx -y

# Download and install calibre command line utility
!wget -q -O- https://download.calibre-ebook.com/linux-installer.sh | sudo sh /dev/stdin

# Convert ebook to kindle supported .mobi format
!ebook-convert docker_cli.epub docker_cli.mobi

Now you can transfer generated .mobi file to your kindle device.

There is also a browser extension called EpubPress, which can convert any webpage into a kindle book in a single click. You can use it convert any webpage or blogs to read on Kindle.

You can follow the discussion thread on Hackernews

Originally published at https://themythicalengineer.com on Apr 21, 2021.

MongoDB Auto Expire Documents

Sumit Kumar Singh — Sun, 20 Sep 2020 11:07:28 +0000

When running MongoDB at scale, you might need to remove some data when your collection starts overflowing and your database memory reaches the max limit.

Few types of data that need to be cleaned periodically are:

Logs
User sessions
Notifications
Advertisements for limited time offers and discounts etc.

There are certain ways to tackle this issue.

Trigger deletion via application events
Creating cron scripts for deleting old records based on timestamp
Using a capped collection which limits the number of records that you can store in it.
Create Time to Live indexes on the collection and let database do it automatically.

In this post, I'll cover implementation using Time to Live indexes in detail. I'll also discuss best practices and technical gotchas you should care about while implementing this technique.

What is a Time to Live Index?

It's nicely explained in the official documentation. I'll quote the few important lines here.

👉 TTL indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time or at a specific clock time.

👉 A background thread in mongod reads the values in the index and removes expired documents from the collection.

How do I implement it?

Let's say you want to get the record deleted after 1 day of its creation.

#create a new record
> db.notifications.insert({"createdAt": new Date(), "text": "Test Notification", "user_id": 1234})
{
    "_id" : ObjectId("5f5ce5f166fc42c137815c9b"),
    "createdAt" : ISODate("2020-09-12T15:14:57.806Z"),
    "text" : "Test Notification",
    "user_id" : 1234
}

To achieve this using TTL indexes you can do this:

> db.notifications.createIndex({"createdAt":1}, {expireAfterSeconds:86400})
{
    "createdCollectionAutomatically" : false,
    "numIndexesBefore" : 1,
    "numIndexesAfter" : 2,
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1599923870, 2),
        "signature" : {
            "hash" : BinData(0,"A5uE8V6MQf04wNlrmIxaWuRVnQo="),
            "keyId" : NumberLong("6848960729458933762")
        }
    },
    "operationTime" : Timestamp(1599923870, 2)
}

You can verify if the index is created properly by running getIndexes command

> db.notifications.getIndexes()
[
    {
        "v" : 2,
        "key" : {
            "_id" : 1
        },
        "name" : "_id_",
        "ns" : "themythicalengineer.notifications"
    },
    {
        "v" : 2,
        "key" : {
            "createdAt" : 1
        },
        "name" : "createdAt_1",
        "ns" : "themythicalengineer.notifications",
        "expireAfterSeconds" : 86400
    }
]

You can see that an index with expireAfterSeconds parameter is created.
_id field is always indexed by default.

Let's take a look at important points that should be taken care of, for successful implementation in a production environment.

The Seven Commandments of Implementation:

TTL indexes are a single-field indexes. You cannot create a compound index with expireAfterSeconds option. Creating TTL index using two or more fields will throw an error.

    > db.notifications.createIndex({"createdAt":1,"text":1}, {expireAfterSeconds:86400})
    {
        "operationTime" : Timestamp(1599925286, 1),
        "ok" : 0,
        "errmsg" : "TTL indexes are single-field indexes, compound indexes do not support TTL. Index spec: { key: { createdAt: 1.0, text: 1.0 }, name: \"createdAt_1_text_1\", expireAfterSeconds: 86400.0 }",
        "code" : 67,
        "codeName" : "CannotCreateIndex",
        "$clusterTime" : {
            "clusterTime" : Timestamp(1599925286, 1),
            "signature" : {
                "hash" : BinData(0,"mCmm0gKiYCSNIpYEggT3LloGKdw="),
                "keyId" : NumberLong("6848960729458933762")
            }
        }
    }

The _id field does not support TTL indexes. Indexed field should contain value of ISODate (YYYY-MM-DD HH:MM.SS.millis) format or an Array which contains dates. You will be able to create a ttl index on field which doesn't have a ISODate format, but it will not expire.

    # creating index on _id field
    > db.notifications.createIndex({"_id":1,}, {expireAfterSeconds:86400})
    {
        "operationTime" : Timestamp(1599925396, 1),
        "ok" : 0,
        "errmsg" : "The field 'expireAfterSeconds' is not valid for an _id index specification. Specification: { ns: \"5f1118e130ef1b0c1b6271f2_themythicalengineer.notifications\", v: 2, key: { _id: 1.0 }, name: \"_id_1\", expireAfterSeconds: 86400.0 }",
        "code" : 197,
        "codeName" : "InvalidIndexSpecificationOption",
        "$clusterTime" : {
            "clusterTime" : Timestamp(1599925396, 1),
            "signature" : {
                "hash" : BinData(0,"bBhDRiEDafDblhodBTb8NZXVRs0="),
                "keyId" : NumberLong("6848960729458933762")
            }
        }
    }

    # able to create index but it won't expire, as text field does not hold a value of ISODate format
    > db.notifications.createIndex({"text":1,}, {expireAfterSeconds:86400})
    {
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1,
        "$clusterTime" : {
            "clusterTime" : Timestamp(1599925696, 2),
            "signature" : {
                "hash" : BinData(0,"yOPV7m5w8mRKQvAIMcJJ36HYvOw="),
                "keyId" : NumberLong("6848960729458933762")
            }
        },
        "operationTime" : Timestamp(1599925696, 2)
    }

You should not create a TTL index on a capped collection. mongod background process cannot remove documents from a capped collection and will throw an error.

👉 The biggest drawback of capped collections is that you cannot anticipate how large they really need to be. If your application scales up and you receive very large number of documents, you will start to lose important data.
You cannot use createIndex() to change the value of expireAfterSeconds of an existing index. You need to use collMod database command to modify the expireAfterSeconds value. Or you can just drop the index and recreate it.

    > db.runCommand({collMod:'notifications', index:{name:'createdAt_1',expireAfterSeconds:60}})
    {
        "expireAfterSeconds_old" : 86400,
        "expireAfterSeconds_new" : 60,
        "ok" : 1,
        "$clusterTime" : {
            "clusterTime" : Timestamp(1599927916, 1),
            "signature" : {
                "hash" : BinData(0,"+XZKfn8QVM2wZxHIhLlfXSf+nik="),
                "keyId" : NumberLong("6848960729458933762")
            }
        },
        "operationTime" : Timestamp(1599927916, 1)
    }

On replica set members, the TTL background thread only deletes documents when a member is in state primary. The TTL background thread is idle when a member is in state secondary. Secondary members replicate deletion operations from the primary.
The background task that removes expired documents runs every 60 seconds. The TTL index does not guarantee that expired data will be deleted immediately. Duration of removal operation depends on workload hitting your mongod. Expired data may exist beyond the 60 second period between runs of the TTL monitor.
If collection is large, it will take a long time to create an index. Better purge data first, then create index on smaller collection, or create TTL index when creating the collection.

Best Practices?

There are two ways to implement TTL indexes.

Delete after a certain amount of time.
Delete at specific clock time.

If you want to keep your system flexible, choose the second option.

So what can be the issue in first option?

Let's say you intially decided to keep the records for a month before deletion, you'll create a ttl index with expireAfterSeconds = 30*86400.

db.collection('notifications').createIndex({"createdAt": 1}, {epireAfterSeconds: 2592000})

But, your company policy changes or the scale of your application increases and you want the records to be deleted in 15 days now. You might have to either drop and recreate the index or run collMod command to modify the ttl value.

Building indexes during time periods where the target collection is under heavy write load can result in reduced write performance and longer index builds.

If the indexes are being built in background, it can affect the read/write efficiency. When you want to modify your index, you might need to have a maintenance period allocated to complete the index build operation. This can lead to some downtime in your application.

Choosing the second option can let you do the modifications without any downtime. You can easily configure the expiry time using environment variables.

db.collection('notifications').createIndex({"expireAt": 1}, {epireAfterSeconds: 0})

As you can see that expireAfterSeconds is set to 0 here, which means record will expire at the value set to expireAt field by the application.

You can use modify your logic anytime to set the value of expireAt field without any downtime.

Please feel free to ask any question in the comment section below.

Originally published @ https://themythicalengineer.com/mongodb-auto-expire-documents-ttl.html

Getting Started with Express

Sumit Kumar Singh — Thu, 13 Aug 2020 05:41:44 +0000

In the previous blog post I explained basic routing with inbuilt http module in Node.js.

In this blog, I will cover the basics of express module. Express JS is a 3rd party web framework for Node.js which provides small, fast and robust tooling for HTTP servers.

Let’s get right into the code.

Install express

Since this module is not available by default in Node.js. We have to install it from npm registry. This is where developers publish their node modules for the world to use.

Open up your terminal in the demo-project directory and type

$ npm install express --save

This creates a node_modules folder in your root directory and also creates a package-lock.json file. This file is a representation of dependency tree that is modified by npm commands. This command installs the express module and the dependencies that are needed by express module inside the node_modules directory.

Install nodemon

This module watches your project directory files and restarts your server if any file is modified. It will help you to test your changes without manually stopping and restarting your server.

$ npm install nodemon --save-dev

As this module is required only for development and testing purposes, we'll install it as a dev dependency. If you have deployed your code on server using environment variable NODE_ENV=production, these dependencies will not get installed.

Instead of using node to execute index.js file, we'll use nodemon. If you type the command below in your terminal you'll be able to see that it executes the index.js file and it also waits for any modifications in the directory.

$ ./node_modules/.bin/nodemon index.js
[nodemon] 2.0.4
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
[nodemon] clean exit - waiting for changes before restart

You can also add scripts to your package.json file to execute commands from locally installed modules.

Let's create a start script in package.json so that we don't have to type the complete command again. Your package.json should look like this after all the steps we've done so far.

{
  "name": "demo-project",
  "version": "1.0.0",
  "description": "Getting started with Backend using Node.js",
  "main": "index.js",
  "scripts": {
    "start": "./node_modules/.bin/nodemon index.js"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.1"
  },
  "devDependencies": {
    "nodemon": "^2.0.4"
  }
}

Now you can use scripts inside your package.json using npm. Type this in your terminal and your command will executed.

$ npm start

Creating a HTTP server

const express = require('express');

const app = express();

// all accepts any HTTP method
app.all('*', function (req, res) {
    // express handles basic headers and mime-types automatically
    res.send("<h1>Demo page</h1>");
})

app.listen(3000, function () {
    console.log("Listening on port 3000. Go to http://localhost:3000");
});

If we use http module, we need to set status codes, headers, write the data and end the response, while in express we just need one send() statement.

If you visit your browser you will see that express added extra header X-Powered-By and if you refresh it multiple times status code will be 304 Not Modified which means express is also handling caching mechanism.

Let's add more methods and routes

const express = require('express');

const app = express();

// Application-level middleware to log request method and path
app.use(function(req, res, next) { 
    console.log(req.method, req.path); 
    next();
});

app.get('/', function (req, res) {    
    res.send("<h1>Demo page Get</h1>");
});
app.post('/', function (req, res) {    
    res.send("<h1>Demo page Post</h1>");
});
app.put('/', function (req, res) {    
    res.send("<h1>Demo page Put</h1>");
});
app.delete('/', function (req, res) {    
    res.send("<h1>Demo page Delete</h1>");
});

app.listen(3000, function () {
    console.log("Listening on port 3000. Go to http://localhost:3000");
});

Browser's default request method is GET, so we can use curl command to test other methods.

$ curl -X METHOD http://localhost:3000

here METHOD can be replaced by GET, PUT, POST, DELETE and various other HTTP methods. You can see the request method and path getting logged when you hit localhost via curl.

app.use() function is used to define middlewares in express.

Middlewares are functions that have access to the request object (req), the response object (res), and the next middleware function in the application’s request-response cycle. Middlewares are used to implement authentication, error handling, logging etc.

Serving static files

Let's create a directory named static in root directory and an index.html file inside that folder. Also download an image so that you can test if you can serve images in response.

<html>
<head></head>
<body>
    <h1>Demo page Get</h1>
    <img src="./demo.jpeg">
</body>
</html>

Your directory structure should look like this.

demo-project
│----index.js
│----package.json
│----package-lock.json
│----node_modules/
│----static
    │----index.html
    │----demo.jpeg

Now modify index.js to add middleware to serve static files from directory named static. In app.get() callback function send the html file as response.

const express = require('express');
const path = require('path')

const app = express();

app.use(function (req, res, next) {
    console.log(req.method, req.path);
    next();
});

app.use(express.static(path.join(__dirname, 'static')))

app.get('/demo', function (req, res) {    
    res.sendFile(path.join(__dirname, 'static/index.html'))
});

app.listen(3000, function () {
    console.log("Listening on port 3000. Go to http://localhost:3000");
});

Refresh the page in browser and you will be able to see 3 requests logged in the console.

GET /
GET /demo.jpeg
GET /favicon.ico

Please feel free to ask any question in the comment section below.

Getting Started with Node.js Backend Development

Sumit Kumar Singh — Sun, 02 Aug 2020 13:30:42 +0000

The Internet, or the World Wide Web has taken over the world and changed our livestyles. We are spending on average 6 - 8 hours online each day. Sites like Youtube, Wikipedia, Netflix etc. have evolved a lot and provide uniterrupted service 24x7 to the whole world. Software Engineers are working hard to create and maintain highly reliable systems like these. These engineers can be classified into two broad categories, Front-End & Back-End.

Front-End engineers focus on User Interfaces, User Experience, Interactions, Animations and look and feel of the website which a user sees on the browser. Back-End engineers focus on providing functionality, building APIs, maintaining databases, scaling of infrastructure, improving performance etc.

In this blog post. I'll explain how to get started with back-end development with Node.js. This will be a very basic introduction.

Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine. A JavaScript engine is a program or an interpreter which executes JavaScript code. It allows javascript code to run on the server side.

Let's dive right into the code.

Disclaimer: It's assumed that you have some sort of programming experience before. Even if you don't, let's hope you leave with an understanding of how backend technologies work.

Installing Node.js

You can follow the instructions on official Node.js website.
There are numerous articles on getting Node.js installed on specific platforms. You can look up on google if you face any issues.

Open your terminal and verify if node and npm are installed.

$ node -v
v12.18.1

$ npm -v
6.14.5

Code Editor / IDE

I'll be using Visual Studio Code for this demo. Open up your code editor and create a demo-project directory. Open integrated terminal in VS Code and change the currect directory to demo-project. Once inside the target directory follow the below steps.

Initiate Project

$ npm init

You can keep pressing enter to choose the default configuration parameters. I've just changed the description option in this configuration.

package name: (demo-project) 
version: (1.0.0) 
description: Getting started with Backend using Node.js
entry point: (index.js) 
test command: 
git repository: 
keywords: 
author: 
license: (ISC)

It will write a new package.json file in demo-project directory, which holds the information about dependencies, config, repository, keywords, licence, author, scripts etc.

About to write to ~/Desktop/dev/demo-project/package.json:

{
  "name": "demo-project",
  "version": "1.0.0",
  "description": "Getting started with Backend using Node.js",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

Is this OK? (yes) yes

Now we'll get started with creating a HTTP server and serve some data to the client(browser).

Before getting into that let's clear up some basics.

HTTP (Hyper Text Transfer Protocol) is a client-server protocol which allows the fetching of resources, such as HTML documents. It is an application layer protocol that is sent over TCP.

A Protocol is a set of rules that we use for specific purposes. In the current scenario, it is about communication between front-end and back-end components. To make communication possible, we need both the parties to agree on the format of the data that is being exchanged.

Let's start with writing code for creating a server.

Create an index.js file in the root (demo-project) directory.

// http is an inbuilt module in Node.js
const http = require('http');

// createServer method takes a callback function as argument
// the callback function takes two arguments req and re
const server = http.createServer(function (req, res) {
    res.statusCode = 200; // 200 = OK
    res.setHeader('Content-Type', 'text/html');
    res.write("<h1>Demo page</h1>");
    res.end();
});

// server is listening to incoming requests on port 3000 on localhost
server.listen(3000, function () {
    console.log("Listening on port http://localhost:3000");
});

Now open up your browser and go to http://localhost:3000.

I have opened the developer console, where we can inspect the request and the response in detail.

Note: You can use shortcut Option ⌥ + ⌘ + J (on macOS), or Shift + CTRL + J (on Windows/Linux) to open developer console in Chrome

Click on Network tab in developer console panel and click on the first request localhost. You will be able to see the output as below.

In Request Headers section we can see that browser has made a GET request to path / on host localhost:3000. It has added some other default headers to the request.

In Response Headers section in Status Line, we have status code as 200 and human readable meaning of the status OK. We have explicitly specified Content-Type: text/html, so browser has interpreted and rendered it as HTML. HTTP supports various MIME types to be served. Connection: keep-alive means that it's a persistent connection.

HTTP version 1.1 specifies a persistent connection by default in which the server leaves the connection open for more requests after sending a response. The server can close the connection at the request of a client or if a time-out has been reached.

Now try to change the URL in the browser's address bar.

http://localhost:3000/hello

http://localhost:3000/hello123

As you might have noticed on other websites that response on the web page changes as the url changes, but it's not happening in our case, since we haven't defined any conditional response yet.

This is called routing. Routing refers to determining how an application responds to a client request to a particular endpoint (URI) and a specific HTTP request method.

Let's try to write some conditions in our code. Make these changes to the code and restart the server again by using command node index.js.

const http = require('http');

const server = http.createServer(function (req, res) {
    console.log(`${req.method} request received at ${req.url}`);
    if (req.url === '/html') {
        res.setHeader('Content-Type', 'text/html');
        res.statusCode = 200; // 200 = OK
        res.write("<h1>Demo page</h1>");
        res.end();
    } else if (req.url === '/plain') {
        res.setHeader('Content-Type', 'text/plain');
        res.statusCode = 200; // 200 = OK
        res.write("<h1>Demo page</h1>");
        res.end();
    } else if (req.url === '/json') {
        res.setHeader('Content-Type', 'application/json');
        res.statusCode = 200; // 200 = OK
        res.write(JSON.stringify({"firstName": "Harry", "lastName": "Potter"}));
        res.end();
    } else {
        res.setHeader('Content-Type', 'text/html');
        res.statusCode = 400; // 400 = Bad request
        res.write("<h1>Sorry, this page is not available</h1>");
        res.end();
    }
});

server.listen(3000, function () {
    console.log("Listening on port http://localhost:3000");
});

After running this code, try to visit these urls in your browser

http://localhost:3000/html

http://localhost:3000/plain

http://localhost:3000/json

http://localhost:3000/hello

Now you can see that, output is changing as the relative path is changing. If you give anything other than /html, /plain and /json, it will show "Sorry, this page is not available".

You can see that <h1>Demo Page</h1> is not rendered as html when we explicitly specified Content-Type: text/plain.

json is the most used format for building backend APIs with node.js.

But this is not how the production code is written for the backend APIs. Libraries such as Express are widely used, which allows us to write robust and secure code.

We've just scratched the surface of backend development. There's much more to learn, which I will cover in upcoming articles. Stay tuned for more <3.

Originally published at https://themythicalengineer.com on July 31, 2020.