DEV Community: Sachin Jha

What is a Service Mesh? -the breakout area chat between an Account manager and a Solutions Engineer!

Sachin Jha — Thu, 30 Apr 2020 18:56:01 +0000

❓ "Who is this new kid in town people are talking about, Service Mesh?" -- John asked in a very confused state.

🤔 "What’s with service mesh? You look, anxious man, what happened?" -- VJ asked.

John is an “Account Manager” and VJ is a “Solutions Engineer”!

John: Things are changing fast, first it used to be those simple applications and I used to tell them to use a number of virtual machines. Now, it gets difficult when the engineers talk in client architecture review calls!”

VJ: ok 😄

John: What was that, this is serious man?

VJ: ok-ok, I’ll explain what is a service-mesh- but I need to start from the time of those Simple applications; as you have mentioned.

John: Go ahead, I’m listening!

VJ: Ok, I’ll draw something for you!

source

Monolithic: 

- The software built using a monolithic approach is self-contained; 
  its components are interconnected and interdependent. 

- If developers want to make any changes or updates to a monolithic 
  system, they need to build and deploy the entire stack at once. 

- It’s the same thing with scalability: the entire system, not just 
  the modules in it, is scaled together. 

- With the monolithic architecture, it can be difficult to adopt a 
  new technology stack, and in case you want to use a new platform or 
  framework, you’ll have to rewrite the entire solution.

Microservices:

- The microservice software architecture allows a system to be divided 
  into a number of smaller, individual, and independent services. 

- Each service is flexible, robust, composable, and complete. 
  They run as autonomous processes and communicate with one another 
  through APIs. 

- Each microservice can be implemented in a different programming 
  language on a different platform. Almost any infrastructure can 
  run in a container which holds services encapsulated for operation. 

- Since these containers can be operated in parallel, the existing 
  infrastructure is easier to maintain.

VJ: Although, when you have independent services or units there are a couple of things that you need to take care of.

Challenges with the microservices architecture:

……
Service discovery
Load balancing
Fault tolerance
Distributed tracing
Telmetrics
Security (mTLS, policies, patches)
Independently releasable
Service contracts
……

John: Yeah, I know that. Ok, not all of it 😄 but I know, and isn’t the container-based implementations like Kubernetes solves them?

VJ: Oh, that’s impressive man 😉

Yes, that's correct. Kubernetes is a very capable platform that has been well proven in production deployments of container applications. It provides a rich networking layer that brings together service discovery, load balancing, health checks, and access control in order to support complex distributed applications.

These capabilities are more than enough for simple applications and for well‑understood, legacy applications that have been containerized. They allow you to deploy applications with confidence, scale them as needed, route around unexpected failures and implement simple access control.

John: Ok, that makes sense. So, if I’m running a simple application or even a complex application, Kubernetes is more than enough to handle things.

VJ: Partially yes! So only when the applications become more complex individually, it puts an overall burden on the application developer and operations team to accommodate it!

[Security, monitoring, and traffic management]

Your code should just focus on business logic, and not the infrastructure part!

Let discuss a few of the challenges:

Service discovery:

Say, now you have moved your Monolithic into Microservices. Each microservice would be running on a different virtual machine (VM) with its own application and named with an IP address and a port number.

How do you think service discovery will happen? You can’t hardcode the values, as in a cloud environment that’s not a permanent state. A simple machine restart or VM failure might make that not so optimum solution.

Hence, you need something like a service registry to take care of this problem. In simple terms, a service registry would be the boss, and every service when it comes up has to register itself to the registry.

In the following example, say if the Driver Management service wants to talk to Passenger Management service it’s gonna send a “lookup” request to the “service registry”.

Problems:
1. Service registry (Server): is a standalone application now, 
and needs your attention to manage it- potentially across availability zones (AZs).

2. Single point of failure. 

3. You will have to add libraries to all the services and write code for them.

Canary/Rolling deploys:

Rolling deploys:

You want to roll out a new version update of your service. In this case, you will make sure you have two identical production environments working in parallel, with different versions.

At the right time here, you want to switch traffic from v1.0 to v.1.1. If there is an issue after v1.1 becomes live, traffic can be routed back to v1.0. All the time you are making sure that services are up and behind the scenes, you are doing an update.

Canary deploys:

It works on very similar principles, and goes as- let’s say we have updated a feature and want to test it only for 5% of the clients.

Jason Skowronski explains:

With the canary deployment, you deploy a new application code in a small part of the production infrastructure. Once the application is signed off for release, only a few users are routed to it. This minimizes any impact.

With no errors reported the new version can gradually roll out to the rest of the infrastructure.

Security:

How do you ensure services are talking on HTTPS? How do you make sure certificates rotation: renewing/replacing after 30 days is taken care.
Network policies/whitelisting
Transparent patching

VJ:

So, to rephrase it, each challenge described above puts a burden on the application developer and operations team to accommodate it!

Individually, the burdens are light because the solutions are well understood, but the weight accumulates.

Eventually, organizations running large‑scale, complex applications might reach a tipping point where enhancing the application service-by-service becomes too difficult to scale.

John: That makes complete sense! So, can I call Service Mesh as:

Do it for me and reduce the overhead on my application developers?

VJ: You are fast man! 🚀

In a nutshell, the answer is `yes`.

At a high level, a service mesh ensures communication between containerized application infrastructures. It provides features such as traffic routing, load balancing, service discovery, encryption, authentication, and authorization.

Brian “Redbeard” Harrington, principal product manager at Red Hat, describes service mesh as a mash-up of several better-known technologies:

A service mesh is a set of software components that act as the “glue” for a set of independent applications. The goal of the mesh is to guarantee secure communications between each application and be able to redirect traffic in the event of failures.

Often the features of a service mesh look like a mash-up between a load balancer, a web application firewall, and an API gateway.

source

VJ: Let’s say initially we had services (Service A and Service B) directly talking to each other. Now, a program called Sidecar is installed with each service, and they sit where the services sit- can be a container or a VM.

Service A will now talk to sidecar and will act as a proxy or hiding factor for service A. It's the sidecar's responsibility now to talk to other service’s sidecars.

All these sidecars will be managed by a program called the Control Tower. It is responsible for sending all those instructions about service discovery, networking policies, load balancing, etc.

Hence, control tower and sidecar implementation takes care of all those heavy lifting and offloads all the burden from your “application code”- and that's the logic behind Service Mesh.

Let's look at some actual implementation of Service Mesh. The available projects in the market are:

Istio, Linkerd and Consul

This is how an actual very close architecture of Istio- looks like.

John: Until now it was fine, although that architecture! 😕

VJ: (Interrupting)

Na, don't get confused- the logic is the same.

Envoy: is a project that's actually the Sidecar we talked about.

Control Plane: is just divided into 3 different services:

Istio-Manager ||  Mixer || Istio-Auth

You can read about the full architecture, and what each of the components does here!

Istio’s main features are:

-- Automatic load balancing for HTTP, gRPC, and TCP traffic.

-- Fine-grained control of traffic behavior with rich routing rules.

-- Traffic encryption, service-to-service authentication and strong 
identity assertions.

-- Fleet-wide policy enforcement.

-- In-depth telemetry and reporting.

John: That was a lot of information man, however, I’m glad that I've got something here 😄.

In a nutshell again:

"Your code should just focus on business logic, and not the infrastructure part!"

"Eventually, organizations running large‑scale, complex applications might reach a tipping point where enhancing the application service-by-service becomes too difficult to scale, that's where a service mesh can help you!"

VJ: Correct, you got it, man!

Now, have your coffee and let me go- I have an architecture review call with one of our potential clients. 😉

John: Ok, talk to their non-tech folks like this and they would love you!

VJ: 😊 I’ll see. Although, you look happy now! 😄

Note: This was just me story-telling, and representing this in a form that is easier to understand. This is in no way a full implementation or covers how to use service mesh.

I will continue to share insights in a phased-out manner.

📓 You can read this on my personal blog as well: https://sachcode.com/tech/tea-time-chat-service-mesh

[More Topics]-https://sachcode.com/

References and Motivation:

https://istio.io/docs/concepts/what-is-istio/

https://istio.io/docs/ops/deployment/architecture/

Save your time, have these advanced Linux commands in your cheatsheet!

Sachin Jha — Mon, 27 Apr 2020 19:50:51 +0000

🐚 I describe Shell with words omnipresence or has ubiquity. Quick filtering or command chaining is like magic and gives us more powers, with Pipe ("|") and CLI utilities like sed, awk, grep, etc.

📓 Hence, if you are related to Linux and Server troubleshooting in any way, and that involves your day to day work. Here is some quick reference to command-line know-hows:

Note: This is Part 1, just a quick reference guide that I've had in my cheatsheet for server troubleshooting. The list I have is exhaustive, so I will keep things simple and continue to share insights in a phased-out manner.

---Super-powers with:
ps | SSH | Removing files | Grep | Find | Sudo | SSL
---

# SSL

With curl's insecure option we can filter the output and get only the Server certificate information:

~ $ curl --insecure -v https://www.sachcode.com 2>&1 | \
> awk 'BEGIN { cert=0 } /^\* Server certificate:/ { cert=1 } \
>  /^\*/ { if (cert) print }'

* Server certificate:
*   subject: CN=*.sachcode.com
*   start date: Mar 22 04:34:05 2020 GMT
*   expire date: Jun 20 04:34:05 2020 GMT
*   common name: *.sachcode.com
*   issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US
* Connection #0 to host www.sachcode.com left intact

# ps(processes status)

Sort by the highest CPU utilization in ascending order:

~ $ ps -aux --sort -pcpu | less

Sort by the highest Memory utilization in ascending order:

~ $ ps -aux --sort -pmem | less

Print all processes running as Root:

~ $ ps -U root -u root

Show process tree of all PIDs:

~ $ ps auxwf

Show all process info and hierarchy:

~ $ ps -efH

Find top running processes by highest memory and CPU usage:

~ $ ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head

~ $ ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head

# SSH

Generate generic ssh key pair:

~ $ ssh-keygen -q -t rsa -f ~/.ssh/<name> -N '' -C <name>

# Removing files

Remove files over 30 days old:

~ $ find . -mtime +30 | xargs rm -rf

Remove files older than 7 day starting with 'backup':

~ $ find . -type f -name "backup*" -mtime +7 -exec rm {} \;

# Grep

Look through all files in current dir for word “foo”:

~ $ grep -R "foo” .

View last ten lines of output:

~ $ grep -i -C 10 "invalid view source” /var/log/info.log

Display line number of message:

~ $ grep -n “pattern” <file>

# Find

Exclude directories in find:

~ $ find /tmp -not \( -path /tmp/dir -prune \) -type p -o -type b

# Sudo

To check sudo access for a user!

Method 1:

~ $ sudo -l -U sachcode 
User sachcode may run the following commands on host:
    (ALL) ALL

Method 2: Another way to find out if a user has sudo access is by checking if the said user is member of the sudo group.

~ $ groups sachcode
sachcode : sachcode wheel docker

Method 3: list all sudo users of your system

~ $ genet group sudo

Read more about me: https://sachcode.com/
Reference on my website: https://sachcode.com/tech/linux-troubleshooting-cheatsheet/

Python 3 for System Administrators: The Plot!

Sachin Jha — Sun, 26 Apr 2020 19:25:27 +0000

📓 Shell or Python? Should I replace my Shell scripts with Python?

It’s debatable!

That’s what Nick, who has been a mentor for Ajay, added.

It certainly depends, whether you work in Sysadmin, Cloud operations, DevOps, or SRE. But that love for Shell would always be there. We all started with it and it’s the glue!

Ajay: But isn’t Shell scripts slow? In addition, it lacks data structure and gets highly complex with long scripts.

Nick: Maybe, but that’s not the point 😄 There are day-to-day tasks that can be performed in both, say checking application ports, starting/stopping the server, checking the resources consumption/load, backing up your database and exporting it to object storage.

🐍 Python is clean, it feels like writing pseudocode. In other words, it’s easier to write and maintain, provides better error handling, can perform OOP (object-oriented programming), and the best part is argument parsing and modules.

🐚 Shell is something that we all have started with, hence I describe it with words “omnipresence” or “has ubiquity”. Quick filtering or command chaining with Pipe ("|") and CLI utilities like sed, awk, grep, etc.

Ajay: Awesome, let me give it a shot; it would certainly help to add new skills to my knowledge bank, at the very least.

Hence, Ajay- who has been a sysadmin all the while writing scripts in shell, just wants to try Python and see how he can make use of both to add more efficiency to his day-to-day work.

So, this is Ajay’s review on Linux academy’s course for Python which he found out while browsing the internet and it simply said: “Learn or start only with modules and structure that would be useful for you as Sysadmin and not the whole Python at once!”

Ajay can be any person who has spent a good time with shell, and not specially sysadmin. So this is for anyone who just wants to add an element to fun to work and has been wanting to learn Python for a long time!

The entire story/review can be found with the following link, it was a bit long to add here!

Review: https://sachcode.com/tech/linux-academy-review-system-administrators-python/
Acloudguru: https://acloudguru.com/

Note: This is in no way a connection or recommendation from me to try only Linux Academy; I just loved the course and this was an honest review.

It contains the use-case on who this course is for, my experience on the operations side, and some key takeaways. Each section’s outcome(s), and the overall links to codes, as well as entire Python projects-repos on GitHub that came out as learning.

Some of the fun exercise(s) and key outcomes- snip!

Project:
We have many database servers that we manage, and we want to create a 
single tool that we can use to easily back up the databases to either 
AWS S3 or locally. We would like to be able to:

> Specify the database URL to backup.
> Specify a “driver” (local or s3)
> Specify the backup “destination”. This will be a file path for local 
and a bucket name for s3.
> Depending on the “driver”, create a local backup of the database or 
upload the backup to an S3 bucket.

https://github.com/linuxacademy/content-python-for-sys-admins/tree/master/pgbackup

Usage:
..............
Pass in a full database URL, the storage driver, and destination.
S3 Example w/ bucket name:
$ pgbackup postgres://bob@example.com:5432/db_one --driver s3 backups

Local Example w/ local path:
$ pgbackup postgres://bob@example.com:5432/db_one --driver local \
/var/local/db_one/backups/dump.sql

🖊️ https://sachcode.com/