DEV Community: Ankit Kumar

Go - Project Structure and Guidelines

Ankit Kumar — Tue, 07 Sep 2021 09:06:42 +0000

Assuming you read my post you should have the starting point for a minimal go web service. For your first project it is easier to keep all your code in one folder, in the base of your project, but at some point you want to restructure things.

Packages
All go code is organised into packages and its simply consists of several .go files. A package is the entry point to access Go code.

Best Practice to use multiple files
a. Feel free to separate you code into as many files as possible.
b. Aim to ensure it is easy to navigate your way around
c. Loosely couple sections of the service or application
Keep types close
It is often a good practice to keep the core types grouped at the top of a file.
Organise by responsibility
In other languages we organise types by models or types but in go we organise code by their functional responsibilities.
Start to Use Godoc
Godoc extracts and generates documentation for Go programs. It runs as a web server and presents the documentation as a web page.
Dont't write your business logic to main.go

Naming Convention

Package names should be lowercase. Don't use snake_case or camelCase.
Avoid overly use terms like util, common, script etc
Use singular form
Rename should follow the same rules


 (
    gourl "net/url"
    myurl "myApp/url"
)

*Repository Structures examples: *

```fooApp/


  circle.yml
  Dockerfile
  cmd/
    foosrv/
      main.go
    foocli/
      main.go
  pkg/
    fs/
      fs.go
      fs_test.go
      mock.go
      mock_test.go
    merge/
      merge.go
      merge_test.go
    api/
      api.go
      api_test.go


```

As you noticed, there is func_test.go file in the same directory. In Go, you save unit tests inside separate files with a filename ending with _test.go. Go provides go test command out of the box which executes these files and runs tests.

Common terms we use in package structure

/cmd
This folder contains the main application entry point files for the project, with the directory name matching the name for the binary.
/pkg
This folder contains code which is OK for other services to consume, this may include API clients, or utility functions which may be handy for other projects but don’t justify their own project.
/internal
This package holds the private library code used in your service, it is specific to the function of the service and not shared with other services.
go.mod
The go. mod file defines the module's module path, which is also the import path used for the root directory, and its dependency requirements, which are the other modules needed for a successful build.
go.sum contains all the dependency check sums, and is managed by the go tools. The checksum present in go.sum file is used to validate the checksum of each of direct and indirect dependency to confirm that none of them has been modified.

e.g.

```


├── LICENSE
├── README.md
├── config.go
├── go.mod
├── go.sum
├── clientlib
│   ├── lib.go
│   └── lib_test.go
├── cmd
│   ├── modlib-client
│   │   └── main.go
│   └── modlib-server
│       └── main.go
├── internal
│   └── auth
│       ├── auth.go
│       └── auth_test.go
└── serverlib
    └── lib.go


```

Thanks!!

Distributed System Configuration Management

Ankit Kumar — Fri, 30 Jul 2021 19:49:59 +0000

This post has based on how can we configure our system so that it shouldn't be restarted to load new configurations. A distributed system's services use configuration to operate.

Let's say we have a distributed system that has multiple services running that process the message stream from different sources based on their configuration.

What is Zookeeper?

ZooKeeper is a distributed coordination service that also helps to manage a large set of hosts. Since managing and coordinating a service especially in a distributed environment is a complicated process, so ZooKeeper solves this problem due to its simple architecture as well as API.

At its best, ZooKeeper allows developers to focus on core application logic without worrying about the distributed nature of the application.

What problem does Zookeeper Solve?

Services needs a restart to load the new configuration or need a new service that syncs the existing configuration
For need to make API Calls to update configuration in each service.

Coordination is of utmost importance in distributed applications. In an application involving multiple services, the components of the system need to work together and coordinate to achieve a result.

That's where Apache Zookeeper is useful. Distributed systems like Apache Hadoop, Apache Kafka, Apache Hive and many more are using zookeeper. All these distributed systems are using a zookeeper as a coordinator between all nodes and store all shared config, state and metadata.

Architecture of Zookeeper:

ZooKeeper follows a simple client-server model where clients are nodes (i.e., machines) that make use of the service, and servers are nodes that provide the service.

Znode - Znodes are the fundamental abstraction provided by Zookeeper to represent a node in a tree-like structure.
A znode can store data in the form of a byte array and it can have child nodes.
Each znode also has an additional data structure called Stat which contains transaction id which created or modified the znode, transaction id which created or modified its children, timestamp, version for the data change, version for the child node change, ACL change, and the owner of the znode.

So that's how zookeeper tree looks like:

Watchers - While reading a particular znode, clients can set watchers. Moreover, for any of the znode (on which client registers) changes, watchers send a notification to the registered client.

Type of Nodes:
Ephemeral Nodes - These Znodes exists as long as the session that created the Znode is active.

Sequence Nodes - When creating this type of node, Zookeeper will add a unique sequence number in the name.

Some of the Key Znode features:

Some of the common Zookeeper Recipes:

Any system that needs a centralised reliable service to manage its configuration across the environment.
Leader election
Naming Service
Queuing the messages
Managing the Notification System

Several Hadoop projects are already using ZooKeeper to coordinate the cluster and provide highly available distributed services.
Apache HBase, which uses ZooKeeper to track the master, the region servers, and the status of data distributed throughout the cluster.

For Zookeeper Installation and References:
Zookeeper

Data Modelling / Database Design

Ankit Kumar — Fri, 09 Jul 2021 20:32:38 +0000

Database design is structuring data and organised relationships in a database. It goes beyond how the database further works.
I was learning about cool stuff dbdigram.io for designing a database so hitting a random try for Bank Database design.

Requirements

Create and manage accounts { owner, branch, balance details...}
Record all transactions history from accounts -> to accounts etc.
Perform money transfer between 2 accounts consistently within a transactions
Maintaining a Loan Records

Database Design

The basics of database design are identifying what data needs to get stored, grouping such data into tables or collections and adding relationships between related data points. The whole database design process involves

Using Entity-Relationship(ER) diagrams to visualize the objects that get stored and their relationships.

Let's see how this can be designed on diagrams using dbdigram.io and then converted to SQL and PostgreSQL.

Exciting right?

How to use DB diagram
The tool is simple: you write code, it renders the ER diagram 👌. You can then export to PDF, PNG or generate SQL code with it ⭐️.

It uses DBML (database markup language) to define and document database schemas.

customer_id is the primary key(pk keyword) and auto-incremented
created_at enclose timezone information as well
deleted_at for maintaining the deletion record

bigserial in PostgreSQL is basically a big autoincrementing integers (8-byte/64-bit)
customer_id int [ref: > C.customer_id], customer_id is foreign key user here

Hence the Final ER - Diagram Result:

Let's write/generate Mysql Code for this schema now

CREATE DATABASE bank;
USE bank;

CREATE TABLE customers (
customer_id int PRIMARY KEY AUTO_INCREMENT,
first_name VARCHAR,
last_name VARCHAR,
city VARCHAR,
mobile_no VARCHAR,
pancard_no VARCHAR,
dob VARCHAR,
created_at timestampz DEFAULT "now()",
deleted_at timestampz
);

CREATE TABLE branchs (
branch_id int PRIMARY KEY AUTO_INCREMENT,
branch_name VARCHAR,
branch_location VARCHAR,
created_at timestampz DEFAULT "now()",
deleted_at timestampz
);

CREATE TABLE accounts (
account_id bigserial PRIMARY KEY,
customer_id int,
balance bigint,
account_status VARCHAR,
account_type VARCHAR,
currency VARCHAR,
created_at timestampz DEFAULT "now()",
deleted_at timestampz
);

CREATE TABLE transactions (
transaction_id bigserial PRIMARY KEY,
transaction_type VARCHAR,
from_account_id bigint,
to_account_id bigint,
date_issued date,
amount bigint,
transaction_medium VARCHAR,
created_at timestampz DEFAULT "now()",
deleted_at timestampz
);

CREATE TABLE loans (
loan_id bigserial PRIMARY KEY,
customer_id int,
branch_id int,
loan_amount bigint,
date_issued date,
created_at timestampz DEFAULT "now()",
deleted_at timestampz
);

ALTER TABLE accounts ADD FOREIGN KEY (customer_id) REFERENCES customers (customer_id);

ALTER TABLE transactions ADD FOREIGN KEY (from_account_id) REFERENCES accounts (account_id);

ALTER TABLE transactions ADD FOREIGN KEY (to_account_id) REFERENCES accounts (account_id);

ALTER TABLE loans ADD FOREIGN KEY (customer_id) REFERENCES customers (customer_id);

ALTER TABLE loans ADD FOREIGN KEY (branch_id) REFERENCES branchs (branch_id);

PostgreSQL Code

CREATE TABLE "customers" (
"customer_id" SERIAL PRIMARY KEY,
"first_name" VARCHAR,
"last_name" VARCHAR,
"city" VARCHAR,
"mobile_no" VARCHAR,
"pancard_no" VARCHAR,
"dob" VARCHAR,
"created_at" timestampz DEFAULT 'now()',
"deleted_at" timestampz
);

CREATE TABLE "branchs" (
"branch_id" SERIAL PRIMARY KEY,
"branch_name" VARCHAR,
"branch_location" VARCHAR,
"created_at" timestampz DEFAULT 'now()',
"deleted_at" timestampz
);

CREATE TABLE "accounts" (
"account_id" bigserial PRIMARY KEY,
"customer_id" int,
"balance" bigint,
"account_status" VARCHAR,
"account_type" VARCHAR,
"currency" VARCHAR,
"created_at" timestampz DEFAULT 'now()',
"deleted_at" timestampz
);

CREATE TABLE "transactions" (
"transaction_id" bigserial PRIMARY KEY,
"transaction_type" VARCHAR,
"from_account_id" bigint,
"to_account_id" bigint,
"date_issued" date,
"amount" bigint,
"transaction_medium" VARCHAR,
"created_at" timestampz DEFAULT 'now()',
"deleted_at" timestampz
);

CREATE TABLE "loans" (
"loan_id" bigserial PRIMARY KEY,
"customer_id" int,
"branch_id" int,
"loan_amount" bigint,
"date_issued" date,
"created_at" timestampz DEFAULT 'now()',
"deleted_at" timestampz
);

ALTER TABLE "accounts" ADD FOREIGN KEY ("customer_id") REFERENCES "customers" ("customer_id");

ALTER TABLE "transactions" ADD FOREIGN KEY ("from_account_id") REFERENCES "accounts" ("account_id");

ALTER TABLE "transactions" ADD FOREIGN KEY ("to_account_id") REFERENCES "accounts" ("account_id");

ALTER TABLE "loans" ADD FOREIGN KEY ("customer_id") REFERENCES "customers" ("customer_id");

ALTER TABLE "loans" ADD FOREIGN KEY ("branch_id") REFERENCES "branchs" ("branch_id");

References:
https://github.com/holistics/dbml

Get the code here
https://github.com/jinxankit/Bank-DataBase-Design

THANK YOU!!!!!

How to upgrade/install Ubuntu 20.04 LTS (Focal Fossa)

Ankit Kumar — Thu, 08 Jul 2021 17:28:11 +0000

How to upgrade/install Ubuntu 20.04 LTS (Focal Fossa)
The latest version of ubuntu 20.04 LTS released on 23rd April 2020. This article will show you how to upgrade into the new version and also how to install ubuntu from scratch.
Installation requirement for Ubuntu 20.04 LTS is
2 GHz dual-core processor
4 GB RAM
25 GB of hard drive space (or USB stick, memory card or external drive but see Live CD for an alternative approach)
VGA capable of 1024x768 screen resolution
Either a CD/DVD drive or a USB port for the installer media
Internet access is always helpful

So let's start:-
Part 1: Upgrading ubuntu 18.04 to ubuntu 20.04 through the command line
If you're using old ubuntu version like ubuntu 18.04 LTS then you can directly upgrade into it through the command line. Make sure to backup your important files into secondary storage for better precautions.
step 1: Upgrade all install packages on ubuntu using
sudo apt update && sudo apt upgrade
step 2: Now you've to make sure to install update-manager-core package by using
sudo apt install update-manager-core
step 3: The final command for checking & upgrading into the new LTS version using
sudo do-release-upgrade
This will search for a newer version and If it finds a newer version then it will require around 1.5 GB data and ask you to install new packages. After choosing yes, your procedure will start accordingly.
If you're showing this:
Note: Checking for a new Ubuntu release
There is no development version of an LTS available.
To upgrade to the latest non-LTS development release
set Prompt=normal in /etc/update-manager/release-upgrades.
that means you might not see the upgrade immediately, as it takes time from the older version. But if you want to upgrade forcefully, you can do it by using
sudo do-release-upgrade -d
"-d" refers to get the latest support version forcefully.
Part 2: Fresh installation from a live USB/CD.
step 1: Download the iso files of latest ubuntu release from their official site https://releases.ubuntu.com/20.04/
step 2: Create a bootable disk. You can use rufus,( https://rufus.ie/ ) easy to use and faster too.
Press START after choosing ISO filestep 3: Prepare your PC to boot from USB.
Every bios screen is different so for better help. You can check this page https://support.endlessm.com/hc/en-us/articles/210527103-How-do-I-start-boot-my-computer-from-a-USB-device-or-DVD-with-Endless-OS-
for better help.
step 4: Starting the installation.
Loading screen for ubuntu 20.04step 5: In the opening screen you've to choose between try ubuntu and install ubuntu.
step 6: Choose your preferred language and keyboard layout.
step 7: Get connected to any wireless, if any:
step 8: Install third party software and updates by choosing them and click on continue further.
step 9: Go for something else if you need dual booting.
step 10: Let's allocate, swap memory = 2 * Ram Size for better performance of RAM and Create / partition of remaining unallocated size, Select the free space and then Click on the "+" symbol to create a new partition.
e.g: / - 8 GB (for root space)
SWAP - 2 GB (for RAM Performance)
/home (for home directory)
I've allocated like this :
step 12: Proceed to continue for further steps.
step 13. Select a region and input your account info like user and password details and at last, ubuntu is started to install. It'll take a few minutes to install and then restart your system.
Your ubuntu installation is ready. Enjoy Ubuntu.

References:
Ubuntu 20.04 LTS (Focal Fossa)
Ubuntu is distributed on three types of images described below. The desktop image allows you to try Ubuntu without…releases.ubuntu.com
The Beginner's Guide To Installing Ubuntu Linux 18.04 LTS
Grab a USB stick and start your Linux journey!medium.com
How to install Ubuntu 18.04 alongside Windows 10
After so many efforts I've figured out what went wrong while installing. Now at last I've installed Ubuntu 18.04…askubuntu.com

Author:
Ankit Kumar
Thank you for reading this far. If you enjoyed this post, please like,share and comment. For more contacts and query you can reach me at ankit.2official@gmail.com. Follow me on Dev.to if you're interested in more in-depth and informative write-ups like these in the future!

So Monolith or Microservice?

Ankit Kumar — Thu, 08 Jul 2021 17:05:11 +0000

MONOLITHIC ARCHITECTURE

A monolithic architecture refers to the fact that your entire application is one large project or codebase. Let’s say you were building an online store. This single codebase must be responsible for every concern of the application from login, to account management, to the product listings, to the shopping cart, payment processing etc.

It sounds immediately critical but there's advantage of monolithic architecture as well.
1. Monoliths are also generally easier to test, as all of their components are in one place for end-to-end access.
2. They are also easy to deploy and scale horizontally as a single package that you can place on multiple server instances.
3. Initial development of monolithic architecture is fast because it is built with the mindset that all parts should be interconnected and dependent on each other.
4. Monoliths don't require API for communication between components, enabling higher performance.

There are also several drawbacks of it:
1. When you get to an enterprise-level application, a single codebase becomes large and hard to handle.
2. Every change in any feature requires a redeployment of the entire application.
3. Making updates, making additions is time-consuming and need to be carefully coordinated.
4. It is extremely problematic to apply new technology in a monolithic application because then the entire application has to be rewritten.

MICROSERVICE ARCHITECTURE

A microservice architecture aims to break down a large application into several more manageable chunks. Using the example of an online store above, you could have a microservice, or even a micro application, dedicated to login. Another dedicated to your product catalogue. Yet a third to handle payments and/or shipping. Essentially, each concern of the application is broken into a separate project or service.

But there's a double-edged sword, just like the monolith.
There are many advantages to a microservice approach as well though.

Each problem can be handled and managed independently.
It makes the codebases more easily managed and the project can be broken up into more easily managed chunks and handled by specialized teams.
New technologies can be adopted more easily, as you can update the services one by one, rather than all at once.
Changes and enhancements to the application don’t require the entire application to be deployed.

The major disadvantage of Microservice is

The initial deployment and development of a microservices architecture requires a high number of resources and can take time. Since there needs to be a seamless integration of the varying individual components.
Due to the complexity of this architecture, testing can prove to be difficult, especially in terms of the interactions between services.

So Monolith or Microservice?
Choosing a monolithic architecture: Small teams, simple application, fast deployment, No microservice expertise, quick release.

Choosing a microservice architecture: Complex and scalable application, long term plans, larger data scale, have a team with different language expertise.