DEV Community: Miftahul Arifin

I Built Dexplorer: Disposable Cosmos-based Blockchain Explorer

Miftahul Arifin — Sun, 30 Apr 2023 06:46:29 +0000

Recently, I built an exciting project called Dexplorer. As a developer working on a Cosmos-based blockchain on my local machine, I found it quite challenging to test a local chain because we had to query through CLI all the time. So, I came up with an idea: what if we built a temporary explorer that works for all Cosmos-based chains? That's when I started developing Dexplorer, a disposable light explorer specifically designed to connect to any Cosmos SDK chain using only WebSocket RPC.

Dexplorer is only a frontend application, which means there is no cache or pre-processing. However, it pulls data from RPC as needed, making it an excellent tool for developers working on Cosmos-based chains who want to explore blockchain data through a user interface.

Currently, the app has several features, including:

The ability to connect to any Cosmos-based RPC
A dashboard to easily monitor chain activity
The ability to subscribe to the latest blocks and transactions
A search function that allows you to quickly find blocks, transactions, and accounts
A list of active validators
A list of proposals
Blockchain parameters

With Dexplorer, you can quickly and easily monitor the activity of Cosmos-based chains, making it an invaluable tool for developers. And, in the future, I plan to connect Dexplorer with a wallet, enabling users to process transactions directly through the app.

If you're interested in checking out Dexplorer for yourself, head to https://dexplorer.space. And if you want to dive into the code, you can find it on Github at https://github.com/arifintahu/dexplorer.

How to Build API with Go and QuestDB

Miftahul Arifin — Fri, 09 Sep 2022 10:21:47 +0000

QuestDB is a relational column-oriented database designed for time series and event data. It uses SQL with extensions for time series to assist with real-time analytics.

If you are not familiar enough with QuestDB, here is demo link to get in touch.

In this tutorial, we will build simple API and implement QuestDB as timeseries database in our project. Then, we will use Gin Framework for handling HTTP routes.

Before we begin, I’ll assume that you:

Have Go installed on your machine
Understand the basics of Go language
Have a general understanding of RESTful API

Running QuestDB

Firstly, we need to run QuestDB in our local. There are several methods to install it, you can find here. But we will use Docker and the latest questdb Docker image for convenience.

To start QuestDB via Docker, run the following:

docker run -p 9000:9000 \
  -p 9009:9009 \
  -p 8812:8812 \
  questdb/questdb

Alternatively, macOS users can use Homebrew:

brew install questdb
brew services start questdb

After starting QuestDB, the web console is available on port 9000, so navigating to localhost:9000 should show the UI which looks like the following:

Alright, QuestDB is ready. Now let's begin to the next step.

Building a REST API in Go using Gin and Gorm

Let’s start by initializing a new Go module to manage our project’s dependencies.

go mod init

Now let's install required dependencies

go get -u github.com/joho/godotenv
go get -u gorm.io/gorm
go get -u github.com/gin-gonic/gin

After installation is complete the folder should contain go.mod and go.sum. Both of the files contain information of the packages that we have installed.

For reference, I published the entire source code in my github. Feel free to clone it.

git clone https://github.com/arifintahu/go-api-questdb

Setting up database

Let's start by creating our database connection and models.

// models/tracker.go

package models

import (
    "time"
)

type Tracker struct {
    Timestamp time.Time     `gorm:"type:timestamp" json:"timestamp"`
    VehicleId int           `gorm:"type:int" json:"vehicleId"`
    Latitude  float64       `gorm:"type:double" json:"latitude"`
    Longitude float64       `gorm:"type:double" json:"longitude"`
}

We have tracker models that will record every position of vehicles. Each tracker should have a timestamp, a VehicleId with type of integer, a Latitude and a Longitude with type of float. We should consider if our types are available in QuestDB types or not as stated here.

Next, we will create setup function to connect to our database. We can interact with a QuestDB database by connecting to various network endpoints such as Web Console, InfluxDB Line Protocol, PostgreSQL Wire Protocol, HTTP REST API.

We will use PostgreSQL Wire Protocol by connecting to port 8812 because we can use gorm as ORM in golang. Before that we need to install gorm driver postgres because we will connect QuestDB using Postgres driver.

go get -u gorm.io/driver/postgres

Then we will write function for database connection.

// models/setup.go

package models

import (
    "fmt"

    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

var DB *gorm.DB

type DBConfig struct {
    Host string
    User string
    Password string
    Name string
    Port string
}

func (dbConfig *DBConfig) ConnectDatabase() error {
    dsn := fmt.Sprintf(
        "host=%s user=%s password=%s dbname=%s port=%s",
        dbConfig.Host,
        dbConfig.User,
        dbConfig.Password,
        dbConfig.Name,
        dbConfig.Port,
    )

    database, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})

    if err != nil {
        return err
    }

    database.AutoMigrate(&Tracker{})

    DB = database

    return nil
}

In setup.go, we also define auto migration for tracker model. Therefore, we don't need to create table in our database first.

Writing controllers

Next, we will build simple controllers where we can create and find trackers.

// controllers/trackers.go

package controllers

import (
    "go-api-questdb/models"
    "net/http"
    "time"

    "github.com/gin-gonic/gin"
)

type CreateTrackerInput struct {
    VehicleId int    `json:"vehicleId"`
    Latitude  float64 `json:"latitude"`
    Longitude float64 `json:"longitude"`
}

func CreateTracker(c *gin.Context) {
    var input CreateTrackerInput
    if err:= c.ShouldBindJSON(&input); err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"data": err.Error()})
        return
    }

    tracker := models.Tracker{
        Timestamp: time.Now().UTC(),
        VehicleId: input.VehicleId,
        Latitude: input.Latitude,
        Longitude: input.Longitude,
    }

    models.DB.Create(&tracker)

    c.JSON(http.StatusOK, gin.H{"data": tracker})
}

func GetTrackers(c *gin.Context) {
    var trackers []models.Tracker
    models.DB.Find(&trackers)

    c.JSON(http.StatusOK, gin.H{"data": trackers})
}

In trackers controller, we have CreateTrackerInput to validate request body in CreateTracker handler, then we just call our DB instance to execute row creation. We also have GetTrackers handler to fetch all rows.

RESTful routes

We almost there!

The last thing we need to do is creating route handler and application entry point.

// main.go

package main

import (
    "go-api-questdb/controllers"
    "go-api-questdb/models"
    "os"

    "github.com/gin-gonic/gin"
    _ "github.com/joho/godotenv/autoload"
)

func main() {
    r := gin.Default()

    dbConfig := models.DBConfig{
        Host: os.Getenv("DB_HOST"),
        User: os.Getenv("DB_USER"),
        Password: os.Getenv("DB_PASSWORD"),
        Name: os.Getenv("DB_NAME"),
        Port: os.Getenv("DB_PORT"),
    }
    err := dbConfig.ConnectDatabase()

    if err != nil {
        panic(err)
    }

    r.POST("/trackers", controllers.CreateTracker)
    r.GET("/trackers", controllers.GetTrackers)

    r.Run("localhost:3000")
}

In main.go, we have dbConfig for initializing our database connection and we load our database credentials in .env file. Therefore, we need to add .env file in our project.

We will use default user and password of QuestDB as stated here

// .env

DB_HOST=localhost
DB_USER=admin
DB_PASSWORD=quest
DB_NAME=qdb
DB_PORT=8812

Alright, let's run out API

go run main.go

Great, our app is successfully running in localhost:3000 and successfully migrating new table.

Let's test it out by sending POST request to localhost:3000/trackers

curl --request POST 'localhost:3000/trackers' --header 'Content-Type: application/json' --data-raw '{ "vehicleId": 1, "latitude": -7.626923, "longitude": 111.5213978 }'

Then we got

{
    "data": {
        "timestamp": "2022-09-09T09:56:01.8970862Z",
        "vehicleId": 1,
        "latitude": -7.626923,
        "longitude": 111.5213978
    }
}

Let's test again by sending GET request to localhost:3000/trackers and we got

{
    "data": [
        {
            "timestamp": "2022-09-09T09:56:01.8970862Z",
            "vehicleId": 1,
            "latitude": -7.626923,
            "longitude": 111.5213978
        }
    ]
}

Yeay we have successfully built API with Go and QuestDB 🌟 🌟 🌟

API Project Template V2.0.0

Miftahul Arifin — Mon, 05 Sep 2022 10:05:52 +0000

Several months ago, I posted API Project Template, Written in Typescript about boilerplate for building API project in Typescript.

Now, I made some changes especially in code style consistency, unit test coverage, swagger docs, and types.

What's new?

Improve coverage unit test on repositories and services using Jest

Refactor project structure as below

├── .husky/                     # Pre-commit config for lint staged
├── docs/                       # Swagger API docs
├── src/                        # All application source
├──── @types/                   # Type definition for modules
|
├──── api/
├────── controllers/            # Define all controllers
├────── middlewares/            # Define all middlewares
├────── models/                 # Define all sequelize models
├────── repositories/           # Define all repositories
├────── routes/
├──────── v1/                   # Define all v1 routes
├────── services/               # Define all services
├────── types/                  # Define all input types
|
├──── config/
├────── appConfig.ts            # Define app configuration
|
├──── constants/                # Define all constants
├──── database/                 # Define database connection and sync tables
├──── utils/                    # Define reusable libs
├──── server.ts                 # Create express config
├──── index.ts                  # ENTRYPOINT - Start server
|
└── ...

Improve code style using class in repositories, services, and controllers
Add express validator as middleware to validate API request
and moreover...

See details of API Project Template v2.0.0 in this repo

Writing Well-Structured Unit Test in TypeScript

Miftahul Arifin — Tue, 09 Aug 2022 23:29:00 +0000

The purpose of this post is to discover the implementation of writing unit test using Jest, a JavaScript testing Framework, in Sequelize and TypeScript project.

Setup Project

Let's create a new brand project using NPM and Git Versioning.



mkdir my-project
cd /my-project
git init
npm init

Then we will install some dependencies, we will use babel for running Jest using TypeScript



npm install --save sequelize pg pg-hstore
npm install --save-dev typescript ts-node jest babel-jest @types/sequelize @types/jest @babel/preset-typescript @babel/preset-env @babel/core

As we use TypeScript, we need to create tsconfig.json to indicate how transcript TypeScript files from src to dist folders.



//tsconfig.json
{
    "compilerOptions": {
        "module": "commonjs",
        "moduleResolution": "node",
        "target": "es2017",
        "rootDir": "./src",
        "outDir": "./dist",
        "esModuleInterop": false,
        "strict": true,
        "baseUrl": ".",
        "typeRoots": ["node_modules/@types"]
    },
    "include": ["src/**/*"],
    "exclude": ["node_modules", "**/*.test.ts"]
}

Then, we need to add babel.config.js in project folder, so we can run the unit test directly.



//babel.config.js
module.exports = {
    presets: [
        ['@babel/preset-env', {targets: {node: 'current'}}],
        '@babel/preset-typescript',
    ],
};

Okay, now let's start writing the code.

Write Code

We will follow a design pattern with a model, a repository, a database lib, and a service. It will be as simple as possible, so we could write simple unit test with full coverage. The project structure will be like this



my-project/
├──src/
|   ├──bookModel.ts
|   ├──bookRepo.test.ts
|   ├──bookRepo.ts
|   ├──bookService.test.ts
|   ├──bookService.ts
|   └──database.ts
├──babel.config.js
├──package.json
└──tsconfig.json

Firstly, we need to create database.ts, it is a database connection lib in Sequelize.



//database.ts
import { Sequelize } from 'sequelize';

export const db: Sequelize = new Sequelize(
    <string>process.env.DB_NAME,
    <string>process.env.DB_USER,
    <string>process.env.DB_PASSWORD,
    {
        host: <string>process.env.DB_HOST,
        dialect: 'postgres',
        logging: console.log
    }
);

Now, let's define the model. Models are the essence of Sequelize. A model is an abstraction that represents a table in your database. In Sequelize, it is a class that extends Model. We will create one model using Sequelize extending Class Model representing Book Model.



//bookModel.ts
import { db } from './database';
import { Model, DataTypes, Sequelize } from 'sequelize';

export default class Book extends Model {}
Book.init(
    {
        id: {
            primaryKey: true,
            type: DataTypes.BIGINT,
            autoIncrement: true
        },
        title: {
            type: DataTypes.STRING,
            allowNull: false
        },
        author: {
            type: DataTypes.STRING,
            allowNull: false
        },
        page: {
            type: DataTypes.INTEGER,
            allowNull: false,
            defaultValue: 0
        },
        publisher: {
            type: DataTypes.STRING
        },
        quantity: {
            type: DataTypes.INTEGER,
            allowNull: false,
            defaultValue: 0
        },
        created_at: {
            type: DataTypes.DATE,
            defaultValue: Sequelize.fn('now'),
            allowNull: false
        },
        updated_at: {
            type: DataTypes.DATE,
            defaultValue: Sequelize.fn('now'),
            allowNull: false
        }
    },
    {
        modelName: 'books',
        freezeTableName: true,
        createdAt: false,
        updatedAt: false,
        sequelize: db
    }
);

Cool, next we will create a repository layer. It is a strategy for abstracting data access. It provides several methods for interacting with the model.



//bookRepo.ts
import Book from './bookModel';

class BookRepo {
    getBookDetail(bookID: number): Promise<Book | null> {
        return Book.findOne({
            where: {
                id: bookID
            }
        });
    }

    removeBook(bookID: number): Promise<number> {
        return Book.destroy({
            where: {
                id: bookID
            }
        });
    }
}

export default new BookRepo();

Then we will create a service layer. It consists of the business logic of the application and may use the repository to implement certain logic involving the database.
It is better to have separate repository layer and service layer. Having separate layers make the code more modular and decouple database from business logic.



//bookService.ts
import BookRepo from './bookRepo';
import Book from './bookModel';

class BookService {
    getBookDetail(bookId: number): Promise<Book | null> {
        return BookRepo.getBookDetail(bookId);
    }

    async removeBook(bookId: number): Promise<number> {
        const book = await BookRepo.getBookDetail(bookId);
        if (!book) {
            throw new Error('Book is not found');
        }
        return BookRepo.removeBook(bookId);
    }
}

export default new BookService();

Alright, we have done with the business logic. We will not write the controller and router because we want to focus on how to write the unit test.

Write Unit Test

Now we will write the unit test for repository and service layer. We will use AAA (Arrange-Act-Assert) pattern for writing the unit test.
The AAA pattern suggests that we should divide our test method into three sections: arrange, act and assert. Each one of them only responsible for the part in which they are named after. Following this pattern does make the code quite well structured and easy to understand.

Let's write the unit test. We will mock the method from bookModel to isolate and focus on the code being tested and not on the behavior or state of external dependencies. Then we will assert the unit test in some cases such as should be equal, should have been called number times, and should have been called with some parameters.



//bookRepo.test.ts
import BookRepo from './bookRepo';
import Book from './bookModel';

describe('BookRepo', () => {
    beforeEach(() =>{
        jest.resetAllMocks();
    });

    describe('BookRepo.__getBookDetail', () => {
        it('should return book detail', async () => {
            //arrange
            const bookID = 1;
            const mockResponse = {
                id: 1,
                title: 'ABC',
                author: 'John Doe',
                page: 1
            }

            Book.findOne = jest.fn().mockResolvedValue(mockResponse);

            //act
            const result = await BookRepo.getBookDetail(bookID);

            //assert
            expect(result).toEqual(mockResponse);
            expect(Book.findOne).toHaveBeenCalledTimes(1);
            expect(Book.findOne).toBeCalledWith({
                where: {
                    id: bookID
                }
            });
        });
    });

    describe('BookRepo.__removeBook', () => {
        it('should return true remove book', async () => {
            //arrange
            const bookID = 1;
            const mockResponse = true;

            Book.destroy = jest.fn().mockResolvedValue(mockResponse);

            //act
            const result = await BookRepo.removeBook(bookID);

            //assert
            expect(result).toEqual(mockResponse);
            expect(Book.destroy).toHaveBeenCalledTimes(1);
            expect(Book.destroy).toBeCalledWith({
                where: {
                    id: bookID
                }
            });
        });
    });
});

Then, we will write unit test for service layer. Same as repository layer, we will mock repository layer in service layer test to isolate and focus on the code being tested.



//bookService.test.ts
import BookService from './bookService';
import BookRepo from './bookRepo';

describe('BookService', () => {
    beforeEach(() =>{
        jest.resetAllMocks();
    });

    describe('BookService.__getBookDetail', () => {
        it('should return book detail', async () => {
            //arrange
            const bookID = 1;
            const mockResponse = {
                id: 1,
                title: 'ABC',
                author: 'John Doe',
                page: 1
            };

            BookRepo.getBookDetail = jest.fn().mockResolvedValue(mockResponse);

            //act
            const result = await BookService.getBookDetail(bookID);

            //assert
            expect(result).toEqual(mockResponse);
            expect(BookRepo.getBookDetail).toHaveBeenCalledTimes(1);
            expect(BookRepo.getBookDetail).toBeCalledWith(bookID);
        });
    });

    describe('BookService.__removeBook', () => {
        it('should return true remove book', async () => {
            //arrange
            const bookID = 2;
            const mockBookDetail = {
                id: 2,
                title: 'ABC',
                author: 'John Doe',
                page: 1
            };
            const mockResponse = true;

            BookRepo.getBookDetail = jest.fn().mockResolvedValue(mockBookDetail);
            BookRepo.removeBook = jest.fn().mockResolvedValue(mockResponse);

            //act
            const result = await BookService.removeBook(bookID);

            //assert
            expect(result).toEqual(mockResponse);

            // assert BookRepo.getBookDetail
            expect(BookRepo.getBookDetail).toHaveBeenCalledTimes(1);
            expect(BookRepo.getBookDetail).toBeCalledWith(bookID);

            //assert BookRepo.removeBook
            expect(BookRepo.removeBook).toHaveBeenCalledTimes(1);
            expect(BookRepo.removeBook).toBeCalledWith(bookID);
        });

        it('should throw error book is not found', () => {
            //arrange
            const bookID = 2;
            const mockBookDetail = null;
            const errorMessage = 'Book is not found';

            BookRepo.getBookDetail = jest.fn().mockResolvedValue(mockBookDetail);

            //act
            const result = BookService.removeBook(bookID);

            //assert
            expect(result).rejects.toThrowError(errorMessage);
            expect(BookRepo.getBookDetail).toHaveBeenCalledTimes(1);
            expect(BookRepo.getBookDetail).toBeCalledWith(bookID);
        });
    });
});

Alright, we have done writing the unit test.
Before running the test, we will add script test in our package.json as follows:



//package.json
...
"scripts": {
    "build": "tsc",
    "build-watch": "tsc -w",
    "test": "jest --coverage ./src"
},
...

Cool, finally we can run the test with this command in our terminal:



npm test

After running, we will get this result telling our unit test is success and fully coverage 🎉

Beautiful! ✨

How to Set Up Free SSL for NodeJS App in AWS EC2

Miftahul Arifin — Wed, 27 Oct 2021 01:04:32 +0000

Recently, I figured out how to set up a Free SSL certificate NodeJS App in AWS EC2. Then, I would like to share what I've done so far.

Firstly we need some prerequisites below:

AWS EC2 Instance with NodeJS installed
Domain purchased from any provider

Step 1. EC2 Setup

Create Instance

I've used a t2.micro Linux instance, choose your desired instance and click on Review and Launch

Security Group

Setup inbound security group settings to allow incoming traffic on http port 80, https port 443, and 22 for SSH as well.

Elastic IP

Allocate static IP address for your instance

Go to EC2 Dashboard > Network & Security > Elastic IPs
Click on Allocate Elastic IP Address
Select the newly generated static IP. Click on the dropdown Actions > Associate Elastic IP Address > Select Instance > Associate

SSH to your instance

Go to EC2 Dashboard > Instances > Select Instance ID and click on connect in the right section

Step 2. Set Up IP Tables

We need to set up IP Tables because NodeJS Express Server cannot access port 80/443. We'll set up server to listen on port 8443 for HTTPS (8000 for HTTP) and redirect traffic to it.



# Lookup IP routing tables
sudo iptables -t nat -L

# Add HTTP port 80 and 443 traffic redirect rule
sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8000
sudo iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 8443

Step 3. Generate Private Key, CA Bundle and SSL Certificate

We'll use SSL For Free for generating key and SSL certificate for free.

Firstly we need to create account in SSL For Free
Click on Certificates > New Certificate
Enter your domain or subdomain and click Next Step
Select 90-Day Certificate for free SSL and Next Step
Finalize order and make sure select free
Then, you need to verify your domain, I choose verification using DNS (CNAME) because it's easier
Finally, it'll take some time to verify our domain
If the order has been issued, then download certificate

Step 4. Domain Routing

Next we'll route our instance to our domain provider.

Go to your DNS Management dashboard
Create new A Record with Elastic IP Address as a value, and fill host with the same address which we have registered in SSL For Free
Then, wait for some time to activate the configuration

Step 5. Certificate Activation

Finally, we'll create a simple server to apply our SSL certificate.

Firstly, SSH to your EC2 instance
Upload and extract certificate zip file from SSL For Free to our project folder
Create index.js file and write the following code

We'll create simple server as below:



const https = require('https');
const fs = require('fs');
const https_options = {
 ca: fs.readFileSync("ca_bundle.crt"),
 key: fs.readFileSync("private.key"),
 cert: fs.readFileSync("certificate.crt")
};
https.createServer(https_options, function (req, res) {
 res.writeHead(200);
 res.end("Welcome to Node.js HTTPS Server");
}).listen(8443)

Run node index.js
Now open your browser and go to your domain
If everything was set up correctly you'll see green https in your browser address bar.

Folder Structure



Node Project
│   index.js
│   private.key        // Zip file from SSL For Free
│   ca_bundle.crt      // Zip file from SSL For Free
│   certificate.crt    // Zip file from SSL For Free

API Project Template, Written in Typescript

Miftahul Arifin — Sun, 10 Oct 2021 23:59:52 +0000

Building project with standardized structure could save much our time. We could focus on business process without considering too much on project structure. On the other hand, a good project structure should be clean, nice refactored, and easy to maintain.

Here's why:

Your time should be focused on creating something amazing. A project that solves a problem and helps others.
You shouldn't be doing the same tasks over and over like structuring project
You should implement dry principles to the rest of your life

This time, I created complete project template for API written in Typescript which is available in this github. Of course, no one template will serve all projects since your needs may be different.

Built With

This project structure is built using

Features

Everything is modular and unit testable
Typescript everything
Express API with dependency injected routes, controllers, repositories, middleware, and services
Centralized configuration loading and validation
API Documentation using swagger

Folder Structure

├── .husky/                     # Pre-commit config for lint staged
├── src/                        # All application source
├──── @types/                   # Type definition
|
├──── api/                       
├────── controllers/            # Define all controllers
├────── helpers/                # Define all helpers
├────── lib/                    
├──────── logger.ts             # Define logger
├────── middlewares/
├──────── authorization.ts      # Define authorization using JWT
├──────── morgan.ts             # Define morgan   
├────── models/                 # Define all schema models
├────── repositories/           # Define all repositories
├────── routes/                 
├──────── v1/                   # Define all v1 routes
├────── services/               # Define all services
├────── validations/            # Define all controller validations
|
├──── config/
├────── swagger/                # Define swagger configuration
├────── database.ts             # Define postgres database connection
├────── express.ts              # Define express configuration
|
├──── constants/                # Define all constants                   
├──── server.ts                 # Create express config
├──── index.ts                  # ENTRYPOINT - Start server
|
├── test/                       # End-to-end test
|
└── Lots of random build-related files

See full project here. Let me know if you have any suggestions😄.

DEV Community: Miftahul Arifin

I Built Dexplorer: Disposable Cosmos-based Blockchain Explorer

How to Build API with Go and QuestDB

Running QuestDB

Building a REST API in Go using Gin and Gorm

Setting up database

Writing controllers

RESTful routes

API Project Template V2.0.0

What's new?

Writing Well-Structured Unit Test in TypeScript

Setup Project

Write Code

Write Unit Test

Links:

How to Set Up Free SSL for NodeJS App in AWS EC2

Step 1. EC2 Setup

Create Instance

Security Group

Elastic IP

SSH to your instance

Step 2. Set Up IP Tables

Step 3. Generate Private Key, CA Bundle and SSL Certificate

Step 4. Domain Routing

Step 5. Certificate Activation

Folder Structure

API Project Template, Written in Typescript

Built With

Features

Folder Structure