DEV Community: Vishnu Chilamakuru

Using Geospatial Queries in MySQL

Vishnu Chilamakuru — Thu, 02 Feb 2023 11:24:34 +0000

Introduction

Geospatial data, or data that represents a location on the Earth's surface, is becoming increasingly prevalent in today's data-driven world. To effectively work with geospatial data, databases need to have the ability to perform geospatial queries, which allow users to search, filter and analyze this data based on location. In this blog post, we will focus on implementing geospatial queries in MySQL, one of the most popular open-source relational databases.

1.Enabling Geospatial Support in MySQL

MySQL does not have built-in geospatial support, but it can be enabled through the installation of a plugin called "MySQL Spatial Extensions." This plugin adds a set of geospatial data types and functions that allow for the creation and querying of geospatial data. To install the plugin, you can run the following command:

INSTALL PLUGIN mysql_no_login SONAME 'libgeometry.so';

2.Creating a Geospatial Table in MySQL

To create a geospatial table in MySQL, you will need to specify the geospatial data type for each column that will contain geospatial data. Some common geospatial data types in MySQL include POINT, LINESTRING, and POLYGON.

Here is an example of how you can create a table to store restaurant locations:

CREATE TABLE restaurants (
  id INT PRIMARY KEY AUTO_INCREMENT,
  name VARCHAR(255),
  location POINT
);

3.Indexing Geospatial Data in MySQL

To improve query performance, it is recommended to create a spatial index on the geospatial column. This can be done by using the following syntax:

ALTER TABLE restaurants ADD SPATIAL INDEX (location);

4.Performing Geospatial Queries in MySQL

Once the geospatial data has been indexed, it can be queried using geospatial functions. Some common geospatial functions in MySQL include ST_Distance, ST_Within, and ST_Contains.

Here is an example of how you can find all restaurants within a certain radius of a location:

SELECT * FROM restaurants WHERE ST_DWithin(location, POINT(longitude, latitude), radius);

5.Other Databases Supporting Geospatial Queries

MySQL is not the only database that supports geospatial queries. Other popular databases that support geospatial queries include:

PostgreSQL with PostGIS extension
Microsoft SQL Server with spatial data types and functions
Oracle with Spatial and Graph option
MongoDB with native support for geospatial data and indexing

Conclusion

Geospatial queries allow databases to effectively work with geospatial data, enabling users to perform location-based analysis and search operations. Implementing geospatial queries in MySQL requires enabling geospatial support through the installation of a plugin and creating a geospatial table with the proper data types and indexes. Whether you are working with real estate data, tracking delivery trucks, or mapping out locations, geospatial queries provide a powerful tool for analyzing and understanding location-based data.

Mastering Database Indexing: A Comprehensive Guide

Vishnu Chilamakuru — Wed, 25 Jan 2023 09:25:46 +0000

Introduction

Database indexing is a fundamental concept for optimizing the performance of database queries. It enables faster data retrieval by providing a quick way to look up specific rows of data in a table. In this blog post, we will take a deep dive into the world of database indexing, including the different types of indexes, the benefits of indexing, and best practices for indexing your database.

What is an Index?

An index is a data structure that improves the performance of database queries by providing quick access to specific rows of data in a table. It works by creating a copy of the data in a specific order, making it easy to find specific rows of data. Indexes can be thought of as an extra layer of organization on top of the data stored in a table.

Why Use Indexes?

Without an index, a database must scan the entire table to find the specific rows of data requested in a query. This can be slow when the table has a large number of rows. Indexes improve the performance of queries by allowing the database to quickly find the specific rows of data requested in a query without having to scan the entire table. This can lead to significant performance improvements, especially for large tables or tables with a large number of rows.

Types of Indexes

Several types of indexes can be used in a database. The most common types of indexes are:

B-tree index: This is the most common type of index and is used in most relational databases. B-tree indexes are efficient for both small and large tables and are well-suited for equality and range queries.

Hash index: This type of index is used primarily for equality queries on a small number of columns. Hash indexes are faster than B-tree indexes for equality queries but are not efficient for range queries.

Bitmap index: This type of index is used to efficiently find specific rows of data in a table with a large number of columns. Bitmap indexes are efficient for both equality and range queries but are not well-suited for large tables.

Clustered index: This type of index determines the physical order of the data in a table. Each table can have only one clustered index.

Creating Indexes

To create an index in a database, you must specify the name of the index, the name of the table that the index will be created on, and the columns that will be included in the index. The syntax for creating an index will vary depending on the type of database you are using. Here is an example of how to create an index in a MySQL database:

CREATE INDEX index_name ON table_name (column1, column2);

It is important to note that indexing can also hurt performance if done excessively or inappropriately, so it's crucial to carefully select the columns that need to be indexed and avoid indexing columns that are rarely used in queries.

Maintenance

Indexes require maintenance to keep them up-to-date and efficient. When a table is modified (rows are inserted, updated, or deleted), the corresponding index must be updated as well. This is known as index maintenance. If the index is not maintained properly, it can lead to poor performance.

Conclusion

In this blog post, we've discussed the basics of database indexing and why it's an important aspect of database management. We've also shown how to implement an index and the importance of proper index selection. By understanding and implementing proper indexing, you can greatly improve the performance of your database queries and ensure efficient data retrieval.

Git Show Command Cheat Sheet

Vishnu Chilamakuru — Tue, 24 Jan 2023 18:35:07 +0000

A Beginner's Guide for the 'git show' command

Git is a powerful version control system that allows developers to track and manage changes to their code. One of the most useful Git commands is git show, which allows you to view the details of a specific commit. In this post, we'll explore the git show command and provide a cheat sheet of some of the most commonly used options.

1. Basic Usage

The basic usage of the git show command is to display the details of a specific commit. You can use the following syntax to view the details of a commit:

git show <commit-hash>

This command will display the commit message, the author, the date, and the changes made in the commit.

2. Showing Changes

One of the most useful options of the git show command is the ability to view the changes made in a specific commit. You can use the following syntax to view the changes made in a commit:

git show <commit-hash> --stat

This command will display a summary of the changes made in the commit, including the number of files changed and the number of lines added or removed.

3. Showing Diffs

Another useful option of the git show command is the ability to view the actual diffs (differences) made in a specific commit. You can use the following syntax to view the diffs made in a commit:

git show <commit-hash> --patch

This command will display the actual diffs made in the commit, including the lines added and removed.

4. Showing File Changes

You can also use the git show command to view the changes made to a specific file in a specific commit. You can use the following syntax to view the changes made to a file:

git show <commit-hash> <file-path>

This command will display the changes made to the specified file in the specified commit.

5. Showing Branch

you can also use git show command to see the latest commit on a branch, you can use the following syntax

git show <branch-name>

Conclusion:

The git show command is a powerful Git command that allows you to view the details of a specific commit. It's a useful tool for understanding the changes made to your code and for debugging. The basic usage, showing changes, showing diffs, showing file changes, and showing branch are some of the most commonly used options of the git show command. By mastering these options, you can quickly and easily view the details of specific commits in your Git repository.

4 Service Discovery Patterns for your Microservice

Vishnu Chilamakuru — Sat, 31 Jul 2021 06:33:41 +0000

Let's say you’re writing some code that invokes a service that has a REST API. In order to make a request, your code needs to know the network location (IP address and port) of a service instance. In a traditional application, the network locations of service instances are usually static.

For example, your code could read the network locations from a configuration file that’s occasionally updated. But in a modern, cloud-based microservices application, it’s usually not that simple. It is much more dynamic.

Service instances have dynamically assigned network locations. Moreover, the set of service instances changes dynamically because of autoscaling, failures, and upgrades.Consequently, your client code must use service discovery.

What is Service Discovery?

From the above image you can see, you can’t statically configure a client with the IP addresses of the services. Instead, an application must use a dynamic service discovery mechanism.

Service discovery is conceptually quite simple: its key component is a service registry, which is a database of the network locations of an application’s service instances.

The service discovery mechanism updates the service registry when service instances start and stop. When a client invokes a service, the service discovery mechanism queries the service registry to obtain a list of available service instances and routes the request to one of them.

In this article, I will mention about 4 service discovery patterns that you should consider for your microservices.

1. Self Registration Pattern
2. Client Side Discovery Pattern
3. Server Side Discovery Pattern
4. 3rd Party Registration Pattern

The services and their clients interact directly with the service registry:

1. Self Registration Pattern

The first pattern is the Self Registration pattern. A service instance invokes the service registry’s registration API to register its network location. It may also supply a health check URL.

The health check URL is an API end-point that the service registry invokes periodically to verify that the service instance is healthy and available to handle requests.

A service registry may require a service instance to periodically invoke a “heartbeat” API in order to prevent its registration from expiring.

A service instance registers itself with the service registry. Find out more details here.

2. Client Side Discovery Pattern

The second pattern is the Client-side discovery pattern. When a service client wants to invoke a service, it queries the service registry to obtain a list of the service’s instances.

To improve performance, a client might cache the service instances. The service client then uses a load-balancing algorithm, such as a round-robin or random, to select a ser-
vice instance. It then makes a request to a select service instance.

Find out more details here

The deployment infrastructure handles service discovery:

3. Server Side Discovery Pattern

In this pattern, Instead of a client querying the service registry, it makes a request to a DNS name, which resolves to a request router that queries the service registry and load-balances requests.

A client makes a request to a router, which is responsible for service discovery. Find out more details here.

4. 3rd Party Registration Pattern

In this pattern, Instead of a service registering itself with the service registry, a third party called the registrar, which is typically part of the deployment platform, handles the registration.

Service instances are automatically registered with the service registry by a third party.
Find out more details here.

Advantages of platform-provided service discovery:

The key benefit of platform-provided service discovery is that all aspects of service discovery are entirely handled by the deployment platform. Neither the services nor the
clients contain any service discovery code. Consequently, the service discovery mechanism is readily available to all services and clients regardless of which language or framework they’re written in.

Disadvantages of platform-provided service discovery:

One drawback of platform-provided service discovery is that it only supports the discovery of services that have been deployed using the platform. For example, Kubernetes-based discovery only works for services running on Kubernetes.

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

Kubernetes Cheatsheet

Vishnu Chilamakuru — Thu, 22 Jul 2021 09:29:29 +0000

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of scaling and failover for your application, provides deployment patterns, canary deployments, and more. In this blog post, I will mention Kubernetes commands which we need for most of the use-cases.

I will list down kubectl commands in the sections below as a quick reference to work with Kubernetes.

List Resources
Create Resources
Update Resources
Display the State of Resources
Delete Resources
Execute Command
Print Container Logs
Modify Kubeconfig Files

List Resources

Get list of all namespaces

kubectl get namespaces

Get list of all pods

kubectl get pods

Get list of all pods with detailed information like IP, Node Name etc...

kubectl get pods -o wide

Get list of all pods running on a particular node server

kubectl get pods --field-selector=spec.nodeName=[server-name]

Get list of all replication controllers and services

kubectl get replicationcontroller,services

Get list of daemonsets

kubectl get daemonset

Create Resources

Create a new namespace

kubectl create namespace [namespace-name]

Create a new namespace from JSON or YAML file.

kubectl create –f [filename]

Update Resources

To apply or update a resource use the kubectl apply command.

Create a new service with the definition contained in [service-config].yaml

kubectl apply -f [service-config].yaml

Create a new replication controller with the definition contained in [controller-config].yaml

kubectl apply -f [controller-config].yaml

Create the objects defined in any .yaml, .yml, or .json file in a directory

kubectl apply -f [yaml-file/directory-name]

Edit a service config

kubectl edit svc/[service-name]

Above command opens the file in your default editor. To choose another editor, specify it in front of the command:

KUBE_EDITOR=”[editor-name]” kubectl edit svc/[service-name]

Display the State of Resources

Get Details about Particular Node

kubectl describe nodes [node-name]

Get Details about a Particular pod

kubectl describe pods [pod-name]

Get Details about a Particular pod whose name and type are listed in pod.json

kubectl describe –f pod.json

Get Details about a Particular pod managed by a specific replication controller

kubectl describe pods [replication-controller-name]

Get Details about all pods

kubectl describe pods

Delete Resources

Delete a pod using the name and type mentioned in pod.yaml

kubectl delete -f pod.yaml

Delete all pods and services with a specific label

kubectl delete pods,services -l [label-key]=[label-value]

Delete all pods

kubectl delete pods --all

Execute Command

Get output from a command run on the first container in a pod

kubectl exec [pod-name] -- [command]

Get output from a command run on a specific container in a pod

kubectl exec [pod-name] -c [container-name] -- [command]

Run /bin/bash from a specific pod. The received output comes from the first container

kubectl exec -ti [pod-name] -- /bin/bash

Print Container Logs

Print Logs from Pod

kubectl logs [pod-name]

Stream Logs from Pod

kubectl logs -f [pod-name]

Tail Logs from Pod (Print last 200 logs from pod)

kubectl logs --tail=200 [pod-name]

Modify Kubeconfig Files

kubectl config command lets you view and modify kubeconfig files.

Get Current Context

kubectl config current-context

Set cluster entry in kubeconfig

kubectl config set-cluster [cluster-name] --server=[server-name]

Unset an entry in kubeconfig

kubectl config unset [property-name]

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

6 Observable Patterns to consider for your Microservice

Vishnu Chilamakuru — Fri, 16 Jul 2021 09:35:17 +0000

When any Microservice is deployed to production, you most probably want
to know what the application is doing like

No of requests per second
Memory Utilization of application instances
CPU utilization of application instances
Disk Utilization of application instances etc...

You also need to be alerted if there’s a problem, such as a failed service instance or a disk filling up, ideally before it impacts a user. If there’s a problem, you need to be able to troubleshoot and identify the root cause.

Many aspects of managing an application in production are outside the scope of the developer, such as monitoring hardware availability and utilization. These are clearly the responsibility of operations. But there are several patterns that you, as a service developer must implement to make your service easier to manage and troubleshoot. Below are 6 key Observable patterns that you need to consider for your Microservice

Health Check API
Log Aggregation
Distributed Tracing
Exception Tracking
Application Metrics
Audit Logging

Health Check API

Expose an endpoint that returns the health of the service.

A service instance needs to be able to tell the deployment infrastructure whether or not it’s able to handle requests. A good solution is for a service to implement a health check endpoint.

For example, implement a GET /health endpoint, which returns 200 if and only if the service is healthy, and 503 otherwise.

The deployment infrastructure periodically invokes this endpoint to determine the health of the service instance and takes the appropriate action if it’s unhealthy.

Log Aggregation

Log service activity and write logs into a centralized logging server, which provides searching and alerting.

Logs are a valuable troubleshooting tool. If you want to know what’s wrong with your application, a good place to start is the log files. But using logs in a microservice architecture is challenging.

Most of the time, the log entries you need are scattered across the log files of the API gateway and several services. The solution is to use log aggregation.

The log aggregation pipeline sends the logs of all of the service instances to a centralized logging server. Once the logs are stored by the logging server, you can view, search, and analyze them. You can also configure alerts that are triggered when certain messages appear in the logs.

Distributed Tracing

Assign each external request a unique ID and trace requests as they flow between services.

A good way to get insight into what your application is doing is to use distributed tracing. Distributed tracing is analogous to a performance profiler in a monolithic application. It records information (Ex: start time and end time) about the tree of service calls that are made when handling a request.

Exception Tracking

Report exceptions to an exception tracking service, which de-duplicates exceptions, alerts developers and tracks the resolution of each
exception.

A service should rarely log an exception, and when it does, it’s important that you identify the root cause. The exception might be a symptom of a failure or a programming bug. The traditional way to view exceptions is to look in the logs. You might even configure the logging server to alert you if an exception appears in the log file. A better approach is to use an exception tracking service.

Application Metrics

Services maintain metrics, such as counters and gauges, and expose them to a metrics server

A key part of the production environment is monitoring and alerting. So it's important to have a monitoring system that gathers metrics, which provide critical information
about

Health of an application, from every part of the technology stack.
Metrics range from infrastructure-level metrics, such as CPU, memory, and disk utilization, to application-level metrics, such as service request latency and a number of requests executed, etc...

Ex: Newrelic, Datadog, etc...

Audit Logging

Log user actions

The purpose of audit logging is to record each user’s actions. An audit log is typically used to help customer support, ensure compliance, and detect suspicious behavior.Each audit log entry records the identity of the user, the action they performed, and the business objects.

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

Interprocess Communication in Microservices

Vishnu Chilamakuru — Mon, 12 Jul 2021 09:56:00 +0000

There are lots of different technologies available to choose from that can be used to communicate between your microservices. In this blog post, I will explain a few important ways that can be used to communicate between microservices.

The communication between microservices can be one of the following two ways.

Synchronous Communication
Asynchronous Communication

Synchronous Communication

Synchronous Communication is real-time communication between two services. The caller service will wait until it gets the response from the invoked service within the configured timeout threshold.

Services can use synchronous request/response-based HTTP communication mechanisms, such as

REST
gRPC

REST

REST is an acronym for REpresentational State Transfer. It is an architectural style for distributed hypermedia systems and was first presented by Roy Fielding in 2000.

REST also does have it's own 6 guiding constraints which must be satisfied if an interface needs to be referred to as RESTful.

1. Uniform interface - A resource in the system should have only one logical URI, and that should provide a way to fetch related or additional data.
*2. Client-server * - This constraint essentially means that client application and server application MUST be able to evolve separately without any dependency on each other. A client should know only resource URIs, and that’s all.
3. Stateless - Make all client-server interactions stateless. The server will not store anything about the latest HTTP request the client made. It will treat every request as new. No session, no history.
4. Cacheable - In REST, caching shall be applied to resources when applicable, and then these resources MUST declare themselves cacheable. Caching can be implemented on the server or client-side
5. Layered system - REST allows you to use a layered system architecture where you deploy the APIs on server A, and store data on server B and authenticate requests in Server C, for example. A client cannot ordinarily tell whether it is connected directly to the end server or an intermediary along the way.
6. Code on demand (optional) - Most of the time, you will be sending the static representations of resources in the form of XML or JSON. But when you need to, you are free to return executable code. e.g., clients may call your API to get a UI widget rendering code. It is permitted.

Find out more details about REST constraints here.

gRPC

gRPC is a modern, open-source, high-performance remote procedure call (RPC) framework that can run anywhere. gRPC enables client and server applications to communicate transparently and simplifies the building of connected systems. It was initially developed at Google in 2015 as the next generation of the RPC infrastructure Stubby.

In gRPC, a client application can directly call a method on a server application on a different machine as if it were a local object, making it easier for you to create distributed applications and services.

On the server-side, the server implements this interface and runs a gRPC server to handle client calls. On the client-side, the client has a stub (referred to as just a client in some languages) that provides the same methods as the server.

By default, gRPC uses Protocol Buffers, Google’s mature open source mechanism for serializing structured data (although it can be used with other data formats such as JSON).

Find out more details about gRPC here.

Asynchronous Communication

Asynchronous Communication is when you send a message without expecting an immediate response from the invoked service. The caller service will just send the request and invoked service will queue the request and process it.

Services can use asynchronous, message-based communication mechanisms such as

Message Queues
Pub/Sub

Message Queues

A message queue is an architecture that provides asynchronous communication, allowing microservices to interact with each other without coupling. These messages are then stored (in memory or persisted) in a queue and processed by another microservice (called the Consumer).

Below are the few popular Messaging Queues that can be used for your application.

Pub/Sub

Pub/Sub enables you to create systems of event producers and consumers, called publishers and subscribers. Publishers communicate with subscribers asynchronously by broadcasting events, rather than by synchronous calls.

Publishers send events to the Pub/Sub service, without regard to how or when these events will be processed. Pub/Sub then delivers events to all services that need to react to them.

Compared to synchronous communication through REST or RPCs, where publishers must wait for subscribers to receive the data, such asynchronous integration increases the flexibility and robustness of the system overall.

Below are the few popular pub/sub services that can be used for your application.

References:

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

5 Open-Source Search Engines For your Website

Vishnu Chilamakuru — Mon, 05 Jul 2021 08:34:10 +0000

A search engine is a software program that helps people find the information they are looking for online using search queries containing keywords or phrases.

Search engines are able to return results quickly even with millions of records by indexing every data record they find. In this blog post, I will list 5 popular open-source search engines which can be used to build search functionality in your website.

Apache Lucene
Apache Solr
Elasticsearch
MeiliSearch
Typesense

Apache Lucene

Apache Lucene is a free and open-source search engine software library, originally written completely in Java. It is supported by the Apache Software Foundation and is released under the Apache Software License.
It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.

Below are some of the key features of Apache Lucene.

Data Indexing
Search
Spellchecking
Keyword Highlighting
Advanced analysis/tokenization capabilities.

Apache Solr

Apache Solr is the popular, blazing-fast, open-source enterprise search platform built on Apache Lucene. Solr is a standalone search server with a REST-like API. You can put documents in it (called "indexing") via JSON, XML, CSV, or binary over HTTP. You query it via HTTP GET and receive JSON, XML, CSV, or binary results.

Below are some of the key features of Solr.

*Advanced Full-Text Search Capabilities *: Solr enables powerful matching capabilities including phrases, wildcards, joins, grouping, and much more across any data type.
Optimized for High Volume Traffic: Solr is proven at extremely large scales the world over
Comprehensive Administration Interfaces: Solr provides a responsive administrative user interface to make it easy to control your Solr instances.
Easy Monitoring: Solr publishes loads of metric data via JMX which helps to get more insights about your Solr instances.

Elasticsearch

Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. It is built based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.

Elasticsearch provides key features like Advanced Full-Text Search Capabilities like Data indexing, Search capabilities including phrases, wildcards, auto suggestions, filters & facets, etc... Elasticsearch can also be used for other use-cases like

Logs - Storing logs via ELK (Elasticsearch, Logstash, Kibana)
Metrics - Monitor and Visualise your system metrics
APM - Get insights into your application performance
App Search - Search across your documents, geodata, and more.

MeiliSearch

MeiliSearch is an open-source, blazingly fast and hyper-relevant search engine that will improve your search experience. It provides an extensive toolset for customization. It works out-of-the-box with a preset that easily answers the needs of most applications. Communication is done with a RESTful API because most developers are already familiar with its norms.

Below are the few key features of MeiliSearch.

*Synonyms *: Ability to create synonyms for a better search experience.
Highlight: With highlight, users understand their search results and act upon them.
Custom Relevancy: It gives you the possibility to add new sorting rules. You can order results by date, likes, whatever suits your dataset.
Filters: Improve your search query by adding custom filters.
Faceting: Empower users to drill down on search results and find what they need faster.

Typesense

Typesense is a fast, typo-tolerant search engine for building delightful search experiences. It claims that it is an Easier-to-Use ElasticSearch Alternative & an Open Source Algolia Alternative.

Below are few key features of Typesense

Typo Tolerance: Handles typographical errors elegantly, out-of-the-box.
Simple and Delightful: Simple to set up, integrate with, operate, and scale. -⚡** Blazing Fast**: Built-in C++. Meticulously architected from the ground-up for low-latency (<50ms) instant searches.
Tunable Ranking: Easy to tailor your search results to perfection. Sorting: Sort results based on a particular field at query time (helpful for features like "Sort by Price").
Faceting & Filtering: Drill down and refine results.
Grouping & Distinct: Group similar results together to show more variety.
Federated Search: Search across multiple collections (indices) in a single HTTP request.
Synonyms: Define words as equivalents of each other, so searching for a word will also return results for the synonyms defined.
Geo search - Search and sort by results around a geographic location

Other Popular Enterprise Search Engines

Below are a few other popular Enterprise Search Engines that are not free

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

Docker Cheatsheet

Vishnu Chilamakuru — Sat, 26 Jun 2021 08:49:17 +0000

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. In this post, I will mention docker commands which we need or most of the use-cases.

Lifecycle Commands
Starting and Stopping Containers
Docker Image Commands
Docker Container And Image Information
Network Commands

Lifecycle Commands

Create a container (without starting it):

docker create [IMAGE]

Rename an existing container

docker rename [CONTAINER_NAME] [NEW_CONTAINER_NAME]

Run a command in a new container

docker run [IMAGE] [COMMAND]

Remove container after it exits

docker run --rm [IMAGE]

Start a container and keep it running

docker run -td [IMAGE]

Start a container and creates an interactive bash shell in the container

docker run -it [IMAGE]

Create, Start, and run a command inside the container and remove the container after executing command.

docker run -it-rm [IMAGE]

Execute command inside already running container.

docker exec -it [container]

Delete a container (if it is not running)

docker rm [CONTAINER]

Update the configuration of the container

docker update [CONTAINER]

Starting and Stopping Containers

Start Container

docker start [CONTAINER]

Stop running Container

docker stop [CONTAINER]

Stop running Container and start it again

docker restart [CONTAINER]

Pause processes in a running container

docker pause [CONTAINER]

Unpause processes in a running container

docker unpause [CONTAINER]

Block a container until others stop

docker wait [CONTAINER]

Kill a container by sending a SIGKILL to a running container

docker kill [CONTAINER]

Attach local standard input, output, and error streams to a running container

docker attach [CONTAINER]

Docker Image Commands

Create an image from a Dockerfile

docker build [URL/FILE]

Create an image from a Dockerfile with Tags

docker build -t <tag> [URL/FILE]

Pull an image from a registry

docker pull [IMAGE]

Push an image to a registry

docker push [IMAGE]

Create an image from a tarball

docker import [URL/FILE]

Create an image from a container

docker commit [CONTAINER] [NEW_IMAGE_NAME]

Remove an image

docker rmi [IMAGE]

Load an image from a tar archive or stdin

docker load [TAR_FILE/STDIN_FILE]

Save an image to a tar archive

docker save [IMAGE] > [TAR_FILE]

Docker Container And Image Information

List running containers

docker ps

Lists both running containers and ones that have stopped

docker ps -a

List the logs from a running container

docker logs [CONTAINER]

List low-level information on Docker objects

docker inspect [OBJECT_NAME/ID]

List real-time events from a container

docker events [CONTAINER]

Show port mapping for a container

docker port [CONTAINER]

Show running processes in a container

docker top [CONTAINER]

Show live resource usage statistics of container

docker stats [CONTAINER]

Show changes to files (or directories) on a filesystem

docker diff [CONTAINER]

List all images that are locally stored with the docker engine

docker [image] ls

Show the history of an image

docker history [IMAGE]

Network Commands

List networks

docker network ls

Remove one or more networks

docker network rm [NETWORK]

Show information on one or more networks

docker network inspect [NETWORK]

Connects a container to a network

docker network connect [NETWORK] [CONTAINER]

Disconnect a container from a network

docker network disconnect [NETWORK] [CONTAINER]

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

10 Websites To Find Your Next Programming Course

Vishnu Chilamakuru — Fri, 25 Jun 2021 08:36:55 +0000

In this blog, I will list the Top 10 Popular Websites where you can find Programming-related courses covering multiple topics including Frontend & Backend Development.

1. Freecodecamp

Freecodecamp provides multiple courses starting from Python, Responsive Web Design, Javascript Algorithms & Data Structures, React, Machine Learning, Data Analysis, etc...

They also offer tutorials/courses on how to build real-world applications in multiple programming languages like Ruby on Rails, Python, Node Js, etc... in their YouTube Channel and its pretty popular one.

2. Coursera

Coursera is a popular website where you can find courses from popular Ivy League colleges. It provides courses in multiple subjects ranging from Programming Languages, Data Engineering, Machine Learning, Mobile Development, Cloud Computing, etc...

It offers Paid certifications and Free courses as well. If you want to just learn the contents of the course irrespective of certification you can just Audit the course.

3. Udemy

Udemy is the most popular website where most of the developers and people from the tech community refer to. Udemy is pretty much like a marketplace of courses where Instructors can signup and create courses. If you are an expert in any domain you can create a course on the same and upload it in Udemy.

Udemy has many great courses for almost all Programming Languages, Databases, Mobile development, Frontend, and Backend technologies.

Most of the Udemy Courses are paid courses. It offers great discounts on selected courses most of the time. So make sure u check out offers for your course before you purchase it.

4. LinkedIn Learning

LinkedIn Learning offers video courses often taught by domain and industry experts in business, leadership, creative, and software technologies. It is one of the subsidiaries of LinkedIn. Most of the courses in LinkedIn Learning fall into 3 categories.

Business
Creative
Technology

It offers paid courses and the first month of your LinkedIn learning is free.

5. Pluralsight

Pluralsight offers a variety of video training courses for software developers, IT administrators, and creative professionals. It provides multiple products like

Paths - Where courses are curated by Experts based on difficulty level to learn the skill.
Course Discussions - Where you can engage and discuss with the community and experts to discuss course-related topics.

It offers paid courses and the first 10 days of free subscription on signup.

6. Udacity

Udacity offers Nanodegree programs that are built in partnership with the tech companies and taught by industry leaders like

Google
Amazon
IBM
Microsoft
Intel etc...

It provides courses in subjects like Programming, Databases, Machine Learning, Artificial Intelligence, Robotics, Self-Driving Cars, etc...It also provides financial aid to the learners based on their merit and performance in the Nanodegree Program.

7. FutureLearn

FutureLearn is a UK-based digital educational platform founded in 2012. It is a Massive Open Online Course learning platform and it has almost 250 UK + International partners offering courses in multiple subjects ranging from Business, Creative Arts, Healthcare, IT & Computer Science, Law, Literature, etc...

It offers a flexible monthly subscription plan and a 7-day free trial as well.

8. Datacamp

Datacamp provides specialization courses related to

Data Engineering
Machine Learning
Artificial Intelligence

It provides Courses, Career and Skill Tracks on Data courses with R, Python, etc...If you are someone who is looking for Data Engineering related courses, this is the best place to start. It offers Personal and Business plans with different pricing levels.

9. Educative

Educative provides technology courses where you can learn without the hassle of environment setup or videos. It is trusted by the learners working in tech giants like Microsoft, Netflix, Facebook, Google, etc... (as mentioned on the website).

Learners can buy a monthly/annual subscription plan or can purchase the individual course.

10. Treehouse

Treehouse or is an online technology school that offers beginner to advanced courses in

Web Design
Web Development
Mobile Development
Game Development. Its courses are aimed at beginners looking to learn coding skills for a career in the tech industry.

It offers a 7-day free trial as well after which you need to subscribe for monthly plans.

Hope you like the article.

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

6 JSON Tools to Improve Your Productivity

Vishnu Chilamakuru — Fri, 18 Jun 2021 16:02:16 +0000

JSON is a data format with a number of data types such as strings, booleans, lists, numbers, objects, etc... It is one of the popular, easiest, and lightweight and formats used for interaction between services. In this blog post, I will list down 10 JSON tools that help to improve your parse, format, and visualize JSON in a better and easier way.

1. JSON Formatter

JsonFormatter helps to

Format/ Beautify JSON
Validate JSON
Minify JSON
Convert JSON to XML, YAML, CSV

2. JSONLint

JSONLint is a validator and reformatter for JSON. We can copy and paste, or we can also directly type or input a URL in the editor and let JSONLint validate our messy JSON code.

3. JSONView Chrome Plugin

JSONView is a Chrome Plugin that offers features of JSON support, Syntax highlighting, Collapsible trees, with indent guides, Clickable URLs, Toggle between raw and parsed JSON.

4. JSON to POJO

JSON2CSHARP is another cool online tool that helps to convert JSON to C# or Java Classes. This tool pretty much saves time for you in generating classes to map and parse JSON fields.

5. JSON to POJO In any Programming Language

Quicktype is another great online tool that helps to convert JSON to classes/structs in any one of the following programming languages. It supports

Go
Java
Python
Ruby
Swift
Objective-C and many more...

6. JSON Pretty Printer

JSON Pretty Printer utility prints JSON data in a legible, indented format. It provides the most benefit to people who are used to the command line. This tool is pretty useful for developers building or consuming JSON-based APIs.

curl http://api.plos.org/search\?q\=title:%22Drosophila%22%20and%20body:%22RNA%22\&fl\=id\&start\=1\&rows\=3 | json_pp

Response Looks like

{
   "response" : {
      "numFound" : 1400,
      "docs" : [
         {
            "id" : "10.1371/journal.pone.0188133"
         },
         {
            "id" : "10.1371/journal.pbio.1000320"
         },
         {
            "id" : "10.1371/journal.pbio.0000060"
         }
      ],
      "start" : 1
   }
}

Hope you like the article.

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.

Learn Redis with Class Central Study Group, Earn a Free Certificate

Vishnu Chilamakuru — Sun, 06 Jun 2021 17:09:17 +0000

I work at Class Central as a Backend Engineer. Recently, we started free open Study groups where you can join and learn together with learners around the world with weekly live sessions.

A month ago, Class Central started the first public study group to take together the course Mountains 101 from the University of Alberta. It’s been a great experience so far to meet other people in the study group and to know about their experiences.

Why Study Groups?

Before we started the public study group, we used to have our internal study groups where we use to pick one course every month, do it together and discuss the same every week with our learnings and experiences.

I personally completed the following three courses via our internal study groups.

Normally, I used to signup mostly for Technology related courses like programming, databases, frameworks, etc... Because of our internal study group, I got motivated to take these non-technical courses and the main reason for this is the social activity, discussions, and learning from others' experiences every week.

Another advantage of study groups I found is because of peers in the group, I feel more motivated to learn and complete the course on schedule :)

Before, I signed up for few courses to complete them on my own. But, most of the time I used to drop out of the course after a couple of weeks.

Upcoming Study Groups

Class Central announced three upcoming study groups in three different areas of study (tech, business, humanitarian/psychology) to promote learning and learners around the globe. The invite is open to everyone from all backgrounds to join our study groups.

All three study groups start on June 16th. Full schedule will be available in the coming days here: (https://www.classcentral.com/study-groups)

If you are interested in joining one of the following study groups, please sign up here.

Course	Basic Info	Free Certificate
A Life of Happiness and Fulfillment from Indian School of Business (ISB)	6 weeks long, 28 hours of material	yes
Introduction to Redis Data Structures from Redis University	6 weeks long	yes
Excel Fundamentals for Data Analysis from Macquarie University	5 weeks long, 15 hours of material	no

You can refer to the following link to know about Class Central's upcoming study groups details and signup for the same.

https://www.classcentral.com/study-groups

Hope to see you soon in the study group :)

Thank you for reading

Hope you find these resources useful. If you like what you read and want to see more about system design, microservices, and other technology-related stuff... You can follow me on

Twitter - Follow @vishnuchi
Subscribe to my weekly newsletter here.