DEV Community: Sandip Das

Basic AI & ML Concepts for MLOps Engineers

Sandip Das — Fri, 31 Jan 2025 15:14:06 +0000

🚀 Basic AI & ML Concepts for MLOps Engineers

Many engineers misunderstand AI & ML concepts before jumping into MLOps. Let’s clear up those fundamentals RIGHT NOW!

🤖 What is AI?

Artificial Intelligence (AI) simulates human intelligence in machines to perform tasks like learning, reasoning, and problem-solving.

🔍 What is ML?

Machine Learning (ML) is a subset of AI that enables systems to learn from data and make predictions or decisions without explicit programming.

📊 What is an ML Model?

An ML Model is a mathematical representation trained on data using an algorithm to recognize patterns, make predictions, or decisions without explicit programming.

🔥 ML Model Training Methods:

Supervised Learning - Learns from labeled data (e.g., regression, classification).
Unsupervised Learning - Identifies patterns in unlabeled data (e.g., clustering, dimensionality reduction).
Reinforcement Learning - Trains agents to make sequential decisions by maximizing rewards.
Semi-Supervised Learning - Mixes labeled and unlabeled data to improve accuracy.
Deep Learning (DL) - Uses multi-layered neural networks for complex feature learning.
Online Learning - Continuously updates the model with new data.
Transfer Learning - Adapts knowledge from one task to another.
Ensemble Learning - Combines multiple models to enhance accuracy.

🏗 Foundation Models (FMs)

Foundation Models (FMs) are large-scale AI models trained on massive datasets, making them adaptable to multiple tasks like NLP, image generation, and coding.

Key Characteristics:

✅ Pretrained on massive datasets (text, images, code, videos).

✅ General-purpose capabilities (e.g., GPT-4, Stable Diffusion, Code Llama).

✅ Fine-tuned for custom use cases (e.g., a healthcare chatbot trained on medical literature).

✅ Scalability & API access (AWS, Azure, Google Cloud).

🔥 Famous ML Models

DeepSeek R1 - High-performance AI for reasoning & language tasks.
Sonet - Efficient LLM for resource-constrained environments.
Meta's LLaMA - Open-weight AI for research & deployment.
OpenAI's GPT - Powers ChatGPT & generative AI apps.
Google's Gemini - Multimodal AI for text, images, & reasoning.
BERT - Google's NLP model for search ranking & text classification.
Claude (Anthropic) - AI model optimized for safety & accuracy.

🚀 Hugging Face: The Open-Source AI Hub

Hugging Face is an open-source AI platform providing pretrained AI models, datasets, and developer tools for NLP, computer vision, and beyond.

Key Features:

✅ Hosts thousands of open-source AI models.

✅ Provides the Transformers library for NLP.

✅ Supports fine-tuning & deployment via API.

✅ Enables AI research & collaboration.

🧠 LLMs: Large Language Models

Large Language Models (LLMs) are deep learning models trained on vast text datasets to understand and generate human-like text.

How LLMs Work?

🔹 Training on massive datasets (books, websites, articles).

🔹 Tokenization (breaking text into smaller units).

🔹 Self-Attention Mechanism (understanding context in sentences).

🔹 Billions of parameters (e.g., GPT-3 has 175B parameters).

LLM Limitations:

❌ Hallucinations - May generate incorrect information.

❌ Bias - Reflects biases in training data.

❌ Computational cost - Requires massive power.

❌ Context limitations - Limited memory in long conversations.

🎨 Generative AI: Content Creation with AI

Generative AI can create text, images, code, music, and videos based on learned data patterns.

How It Works?

Pre-trained on massive datasets.
Uses transformer-based architectures (GPT, Stable Diffusion).
Prompt-based generation (input text → AI generates content).
Fine-tuning for specific domains (e.g., DevOps automation, cybersecurity).

Key Generative AI Models:

LLMs - GPT, LLaMA, Falcon (for text & code generation).
Image Generators - DALL·E, MidJourney, Stable Diffusion.
Audio & Music - OpenAI's Jukebox, Google's MusicLM.
Video - RunwayML, Sora.

🔍 RAG: Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) enhances LLM responses by retrieving external data before generating answers.

How RAG Works?

User Query → Model receives a question.
Retrieval Step → Searches external sources (DBs, APIs).
Augmentation Step → Retrieved data is fed into the LLM.
Generation Step → Model generates an improved response.

💡 This technique improves accuracy, context, and reduces hallucinations.

☁️ Amazon Bedrock: GenAI on AWS

Amazon Bedrock is a fully managed AWS service for building scalable Generative AI applications using Foundation Models (FMs).

Why Use Amazon Bedrock?

✅ Access to multiple FMs (Claude, LLaMA, Cohere, Stability AI).

✅ Fine-tuning & RAG support (improve accuracy with enterprise data).

✅ Seamless AWS integration (S3, Lambda, SageMaker, DynamoDB, RDS).

🏁 Next Steps: Data Extraction, Validation & Preparation for MLOps

This guide covered AI/ML fundamentals for MLOps Engineers. Next, we’ll dive into Data Extraction, Validation & Preparation for MLOps! 🚀

👉 Follow Sandip Das for more updates!

Complete Cloud Migration Guide

Sandip Das — Thu, 26 Aug 2021 10:07:23 +0000

One simple question that everyone reading this article will definitely have is:

"What is Cloud Migration"?

So, just to clarify, Cloud migration is when a company moves some / all of its on-premises or data center or other cloud resources & capabilities such as data, applications, or any other IT resources/processes/workload into the target cloud.

In more simple words: Although Cloud migration usually refers to moving all tech resources from old/legacy on-prem/data center to the Cloud, Cloud migration also can be moving from one cloud to another Cloud e.g. From On-prem to AWS, from Google Cloud to AWS Cloud or AWS Cloud to Azure

Compare it like moving a smaller office workspace to a bigger office workspace (on-prem to the cloud) or maybe challenging office locations! (One cloud to another cloud)

Before jumping in-depth into Cloud migration, let's understand few key concepts:

Cloud: Cloud in technology space means, Cloud Computing! and Cloud computing is the delivery of computing services—including servers, storage, databases, networking, software, analytics, and intelligence—over the Internet to offer faster innovation, flexible resources, and economies of scale. We typically have to pay only for cloud services you use, helping lower our operating costs, run our infrastructure more efficiently, and scale as business needs change.

Data: In computing, Data is information that has been translated into a form that is efficient for movement or processing.

SLA: A service-level agreement (SLA) defines the level of service you expect from a vendor, laying out the metrics by which service is measured, as well as remedies or penalties should agreed-on service levels not be achieved. It is a critical component of any technology vendor contract.

Legacy Infra: In computing, hardware or software is considered "legacy" if it is outdated but still in use. Legacy products and processes are usually not as efficient or secure as more up-to-date solutions. Businesses stuck running legacy systems are in danger of falling behind their competitors; they also face an increased risk of data breaches.

Why do Companies/Businesses need to move to Cloud?

Running from an on-premise or single data center, have three biggest limitations i.e.

1) Scalability: Running from an on-premise or single data center has limited resources hence limited scaling capability, moving to the cloud gives virtually limitless resource options, so limitless scalability opportunity & as per need!

2) Single Point of Failure: Running from an on-premise or data center create another big issue, i.e. it creates a single point of failure, in cases of natural disaster, un-wanted power supply cut or maybe a strike, the reason could be anything and operations/work processing can get stopped, but with Cloud Computing, it completely eliminates single point of failure if configured properly and there is no such chance and often Cloud Providers give SLA as well for it's services.

3) Cost: The cost of maintaining infra on-premise, operations, maintenance is huge, Cloud gives the "Pay as You Go" model which gives flexibility to only for using what is needed and stops using when not needed, in that way saving cost as well as reservations models which can bring up to 70% or more discount.

While talking about Cloud and migration, we also need to know about Types of Cloud Computing, Cloud Service Models, and Cloud Deployment Models

Types of Cloud Computing

Public Cloud: This is a publicly accessible framework where one can store data or use it as a virtual machine. This can be done either by programming or autonomously. Here individual does not have to invest time and effort in buying physical servers but can get started in no time. Public clouds are available to use on pay per approach basis. e.g. Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Alibaba Cloud, IBM Cloud, and Oracle Cloud.
Private Cloud: If one needs to have a cloud exclusively for the organization then a private cloud is the best option. Along with the flexibility it provides one can opt for a data center on the premises for security and compliance needs. A dedicated professional is required to manage the private cloud framework. e.g. Hewlett Packard Enterprise (HPE), Dell, IBM, Oracle, and some familiar names from the public cloud provider space, including AWS, Google, and Microsoft.
Hybrid Cloud: Hybrid is a combination of public and private clouds. For certain business needs who can benefit from the combination are the ones who use a hybrid cloud.

Cloud Service Models

IaaS (Infrastructure as a Service): IaaS contains the basic building blocks for cloud IT. It typically provides access to networking features, computers (virtual or on dedicated hardware), and data storage space. IaaS gives you the highest level of flexibility and management control over your IT resources. It is most similar to the existing IT resources with which many IT departments and developers are familiar.
PaaS (Platform as a service): PaaS removes the need for you to manage underlying infrastructure (usually hardware and operating systems), and allows you to focus on the deployment and management of your applications. This helps you be more efficient as you don’t need to worry about resource procurement, capacity planning, software maintenance, patching, or any of the other undifferentiated heavy lifting involved in running your application.
SaaS (Software as a Service): SaaS provides you with a complete product that is run and managed by the service provider. In most cases, people referring to SaaS are referring to end-user applications (such as web-based email). With a SaaS offering, you don’t have to think about how the service is maintained or how the underlying infrastructure is managed. You only need to think about how you will use that particular software.

Cloud Deployment Models

Cloud: A cloud-based application is fully deployed in the cloud and all parts of the application run in the cloud. Applications in the cloud have either been created in the cloud or have been migrated from an existing infrastructure to take advantage of the benefits of cloud computing. Cloud-based applications can be built on low-level infrastructure pieces or can use higher-level services that provide abstraction from the management, architecting, and scaling requirements of core infrastructure.
On-premises: Deploying resources on-premises, using virtualization and resource management tools, is sometimes called “private cloud”. On-premises deployment does not provide many of the benefits of cloud computing but is sometimes sought for its ability to provide dedicated resources. In most cases, this deployment model is the same as legacy IT infrastructure while using application management and virtualization technologies to try and increase resource utilization.
Hybrid: A hybrid deployment is a way to connect infrastructure and applications between cloud-based resources and existing resources that are not located in the cloud. The most common method of hybrid deployment is between the cloud and existing on-premises infrastructure to extend, and grow, an organization's infrastructure into the cloud while connecting cloud resources to internal system

Now, that we have covered key concepts of cloud and pre-requisite knowledge, let's learn about the step by step migration flow:

Step 1: Cloud Assessment:

The very first thing we must do is the below assessments and document everything!

Asses Cost:

We have to do below cost assessment and estimate the total cost of migration to make sure it doesn't exceed the assigned budget

The cost of the team what working on the migration project
The cost of creating infrastructure
The cost of Cloud Service Provider (CSP)
The Cost of Data Transfer
Post-migration development & maintenance cost

Asses Architecture:

We have to below Architecture assessment before starting actual cloud migration

Create a list of existing infrastructure resources such as servers RAM, CPU, storage, firewall, Databases, etc any and all IT Infra Resources.
Create a list of expected resources that will need to be created in the remote Cloud Services Provider's platform, based on the existing list.
Find out the way to save Infrastructure costs and what possible options are available and select the best possible options as per business budget and needs.
Do a GAP analysis, to make a list of the things that need to be done before moving applications and data to the Cloud

Asses Security

Before moving to the cloud we have to consider below security assessments

List existing security policies, frameworks, business mandatory policies which need special considerations.
List how to achieve the above-existing infra requirements in the target cloud or how to improve it even further using available Cloud Service provider given tools
List all the relevant Security options such as how to use them, are they incur the extra cost or are free of cost, how effective it is, etc

Step 2: Create Proof of Concept (POC):

Before starting actual full-scale migration, there has to be a proof of concepts with at least to see if we can move core services and/or core data or not, otherwise if start full-scale migration and find out that some or most important services are incompatible or unable to migrate due to any reason, that might cause havoc loss to the company/project, so it's best practice to create a PoC before processing further! consider the below points while creating the Pilot project:

After above three assessment it will be clear with which cloud provider the PoC need to be made i.e. with best possible less cost + more performance service + infrastructure + Best Security options
Plan to move core service and core data + service first rather than making it feature-rich, try to make a smaller app/data migration only, just to verify what you are expecting to work, are working at all for your app/service or not
Utilizing Cloud service provider provided tools to make this POC faster
Not just depend on the Cloud Service Provided Tools, but also have a look at the best possible open-source options

Step 3: Data Migration:

After PoC succeeds and is passed by top management, it's time to start the actual migration process and first, we will move the data:

Leverage Storage Options:

Select best possible storage option based on requirement + cost/budget + performance
While selecting the storage option, check for tools provided by the cloud service provider for the data migration
Check SLA, most of the service providers give SLA (Service Level Agreement) for their storage options, which might be a selection factor for mid/large/enterprise businesses.

Migrate:

After selecting the storage option, it's time to migrate actual data, and consider the below points:

Estimate the time it's going to take to complete the Data Migration, let the Operations team and management know, and get concept/approval
Make Schema / Format ready (if needed) in the remote cloud storage
Check if data pre-processing is needed, if so, do make processing is done before starting the migration process, e.g. for DB storage export data to CSV or JSON, and for an object at such as pic and doc file to zipping before moving, etc
Find out when application traffic is lowest, and notify customers that you are going to move data in that time, make an application in maintenance mode so that it doesn't create new data, then start the migration and keep the service unavailable during the migration time
After the data migration is done, don't just start using the new storage, instead use live data migration tools, which keep the old data storage and new data storage in sync until completely move to the new data storage option, and only move to the new data storage option till fully satisfied with the performance, security and stability of new storage, for this either use cloud service provided tool, or open sources data migration tools or just create your own custom solution!
If the data is too huge that it might not be wise to do a live migration, instead it will be better to use physical data move using cloud service provided paid data migration service, where they will come to you and copy all the data into some large drive and then take it back to their data center where they do copy data from the drive, it's faster and then you can do live migration for any new data.

Step 4: Application Migration:

There are 6 R’s of cloud migration define cloud migration strategies, which include — Rehost, Refactor, Revise, Rebuild, Replace, and Retiring.

Rehost - Rehosting can be thought of as "the same thing, but on cloud servers". Companies that choose this strategy will select an IaaS (Infrastructure-as-a-Service) provider and recreate their application architecture on that infrastructure. in short: moving applications to the cloud as-is.
Refactor - Companies that choose to refactor will reuse already existing code and frameworks, but run their applications on a PaaS (Platform-as-a-Service) provider's platform – instead of on IaaS, as in rehosting. in short: changing the app’s code to better support the cloud environment
Revise - This strategy involves partially rewriting or expanding the code base, then deploying it by either rehosting or refactoring. in short: moving applications to the cloud without major changes but taking advantage of the benefits of the cloud environment.
Rebuild - To "rebuild" means rewriting and re-architecting the application from the ground up on a PaaS provider's platform. This can be a labor-intensive process, but it also enables developers to take advantage of modern features from PaaS vendors. in short: rewrite the application from scratch.
Replace - Businesses can also opt to discard their old applications altogether and switch to already-built SaaS (Software-as-a-Service) applications from third-party vendors. in short: retire the application and replace it with a new cloud-native application
Retiring — After looking over everything in your environment, you’re likely to discover there are things you can remove without consequences, doing it can save money, improves security, and gives your people one less tool they have to learn to use. in short: Remove unnecessary resources

List of Available Cloud Migration Tools & Service

Leading Cloud service providers such as AWS, Azure, GCP gives us excellent migration tools as follows:

AWS cloud migration tools:

AWS Provides below Cloud Migration tools:

AWS Migration Hub — This hub lets you track the progress of migrations across AWS solutions, helping you pick the right tools, track metrics, and more.
AWS Application Delivery Service — Plan for your migration by letting AWS review your on-prem data setup. Collected data is encrypted and accessible from the Migration Hub.
AWS Server Migration Service — This service makes it easy and quick to move workloads to AWS, particularly when dealing with large-scale server migrations.
AWS Database Migration Service — Easily and securely move your databases to AWS. The source database remains functional throughout the migration, minimizing downtime.
CloudEndure Migration — This automated lift-and-shift solution is free for 90 days.
Migrating data and files: AWS Snowball, AWS Snowball Edge, AWS Snowmobile, AWS DataSync, and AWS Transfer for SFTP. Read more here

Microsoft Azure cloud migration tools

Azure provides below migrations tools/services

Azure Migrate — Microsoft’s built-in migration service serves as a central hub for tools, progress tracking, insights, and guidance to plan and successfully migrate to the cloud. Most other tools covered below are integrated into this central dashboard.
Azure Migrate: Server Assessment and Server Migration — These tools let you assess and migrate servers to Azure, including physical servers and VMware, Hyper-V, public cloud, and other VMs.
Data Migration Assistant — DMA helps locate compatibility issues that could derail your migration. It points out unsupported features, new features, and helps you plot a proper path for database migration.
Azure Database Migration Service — Migrate on-prem databases to Azure VMs.
Azure Data Box — Move large amounts of offline data to the Azure cloud.
Movere — This SaaS platform was acquired by Microsoft in 2019. It’s a discovery solution that increases business intelligence to see and control environments across.

GCP cloud migration tools

GCP Provides below migrations tools/services:

Transfer Service — Execute large-scale data transfers from online and on-premises sources to Google Cloud Storage.
Transfer Appliance — For offline bulk data migration, Transfer Appliance lets you securely capture, ship, and upload data using 100TB or 480TB models.
Migrate for Anthos — Migrate and modernize existing workloads to containers.
Migrate for Compute Engine — Get enterprise applications running in Google Cloud while data migrates in the background. Validate, run, and migrate applications without reworking them.
BigQuery Data Transfer Service — Let your analytics team set the foundation for a BigQuery data warehouse and schedule and automate data transfers from your SaaS applications.

Step 5: Leverage Cloud:

Finally, after migration over, remember migration is not just moving data and application to the cloud, don't forget your primary goal, which was to leverage full power, do below to harness it:

Scale:

So set auto-scaling policies properly, don't overestimate and don't underestimate, set auto-scaling policies based on application need.
Over-provisioning costs more and under-provisioning costs clients, which can impact Companie's Goodwill, so either is not an option, do experiment scaling as much as possible with Cloud Service provided tools and as well as open-source tools, practice chaos engineering!
Look at Multi-Cloud, using containerization or function-based application running approach, we can separate codes(business logic) from Environment (OS/other dependencies), Cloud gives us awesome multi-cloud features, where we can leverage multi-cloud so that in case any outage in any cloud, there is another backup Cloud service provider ready to compensate that.

Automate:

Cloud gives us wonderful automation options such as:

We can automate infrastructure creation/provision, maintenance, upgrade/downgrade, security, kind of everything via Infrastructure as code tools/services such as Terraform, CloudFormations etc, we should and must utilizing this awesome feature
Using Automation, migration from one cloud to another is also easy (in case don't use the property tools/service, instead use opensource options)
Using Automation tools, also help estimate better in cost calculation and lets us know what resources going to be created before creating it!
Even using automated CI/CD process, from source code to deployment all can happen automation or using less manual international, making release process much faster and smooth

Test

The test will get even easier:

If we maintain infrastructure as code, we also can add automated test cases that will make cloud resource testing much easier
If we maintain the automated CI/CD Process in the cloud, we can do easily add test cases in source code and do run the test cases after build successes and after testing, if passed, then moving to staging/production
If during the test it failed, using a different Cloud feature we can notify the concerned team about the failure and as soon the fixes and done and codes pushed, the testing can happen automatically again

Step 6: Optimizations :

After all the above in place, it's time to do the below:

Monitor:

Monitor cost, do create budget and notification, as it's very easy to thing can go out of budget and hard to manage, so have to make sure it's always under control and keep it under control.
Monitor Security, do make sure its least privileged approach is followed, monitor each and every access using cloud service proved tools, set alert for possible breaches, make sure strict policies are in place and set an alert if anyone tries to deviate anything, should trigger alert and automation there to rectify it.
Monitor performance, do monitor performance of each and every resources, check if any improvements if not, and do as needed
Monitor usage pattern, create separate usage profiles, like many hours running, when the load is high or less, how much usage happening, over the month, quarter, year etc, it will use in the next stage

Redesign:

After motioning and tethering enough data :

Improve based on fact, based on monitoring data, find out what resources needs improvements/upgrade/downgrade, etc
Look for Cost Saving opportunities, by profiling data and requirements, we can predict the next usage pattern and by leveraging this knowledge we can save cost by either by doing reservations or purchasing savings plans provided by Cloud Service proves, for this we need how much to reserve and monitoring gives us that data.
Improve security, by looking monitoring data & analyzing the data, find out anomalies, possible breaches, and find opportunities to improve it
Remove unnecessary resources, based on monitoring remove the resources which is unnecessary and not getting used at all

Migrations Challenges:

Duration: Although with help of modern tools and services migration takes less time, but in the case of legacy systems it takes a lot of time.

Complexity: Mapping existing systems and requirements to Cloud is no easy task and often it's very easy to get things more complicated, need very talented/experienced resource to handle the migration

Huge Bills: If not monitored and managed properly, cloud costs can go out of the hands

Make Management Understand: Let's agree, Cloud is complicated for non-teach / non-trained persons, management often doesn't know why and what, it's tough to make them understand what is needed and why is needed and its impact on laymen understandable way.

Human Resource: Probably the most important is this, Cloud migration need experienced knowledge hence experienced professional needed for do asses existing infra, plan the architecture of the cloud and not to mention actually implementing the infra/automate it and do the migration. For this we need, Cloud Architect, DevOps Engineers, Operations Engineer, etc, and since Cloud professionals are in demand, finding the right person is challenging.

Conclusion:

Even though at a glance it looks like Cloud migration is moving resources to Cloud is just moving on-prem to Cloud, but it requires a lot of considerations before actually starting migrations and post migrations manage it.

Hope the information I have tried to provide will be beneficial for many others and if you really loved this article, do like this article, share it with others, and comment on what is yours thought about this article.

About the Author:

Sandip Das works as a Sr. Cloud Solutions Architect & DevOps Engineer for multiple tech product companies/start-ups, have AWS DevOps Engineer Professional certification, also holding the title of "AWS Container Hero",

He is always in "keep on learning" mode, enjoys sharing knowledge with others, and currently holds 5 AWS Certifications. Sandip finds blogging as a great way to share knowledge: he writes articles on Linkedin about Cloud, DevOps, Programming, and more. He also creates video tutorials on his YouTube channel.

Microservices CI/CD With Terraform, AWS ECS & AWS CI/CD Tools

Sandip Das — Sun, 15 Aug 2021 11:45:58 +0000

Learn how to configure AWS CodeBuild & CodePipeline for any applications using Terraform

By the end of tutorial you will learn:

What exactly are Microservices?
What is Continuous Integration?
What is Continuous Delivery & Deployment?
What is Infrastructure as Code & Terraform?
AWS Tools for CICD & Microservices
Microservices CI/CD Demo with AWS + Terraform

What is Microservices?

Microservices are an architectural and organizational approach to software development where software is composed of small independent services that communicate over well-defined APIs. These services are owned by small, self-contained teams.

Microservices architectures make applications easier to scale and faster to develop, enabling innovation and accelerating time-to-market for new features.

What is Terraform?

Terraform is an infrastructure as code (IaC) tool that allows you to build, change, and version infrastructure safely and efficiently. This includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc. Terraform can manage both existing service providers and custom in-house solutions.

What are the AWS CI/CD Tools?

AWS CodeCommit: Managed Git Repository Services (Source)

AWS CodeBuild: Container-based Serverless Build tool/Service (Build)

AWS CodeDeploy: Deployment service

AWS CodePipeline: Workflow management e.g. Source ➡️ Build ➡️ Deploy

▬▬▬▬▬▬ Useful Links 🔗 ▬▬▬▬▬▬
Terraform Course For Beginners: ►https://bit.ly/379r5dg
Demo Project GitHub Repo: ►https://bit.ly/3scmsZs
Terraform AWS CodeBuild Documentation: ►https://bit.ly/3jLyDss
Terraform AWS CodePipeline Documentation: ►https://bit.ly/3m1ZweC
Terraform CodePipeline Providers Reference: ►https://amzn.to/3yLrAq0

▬▬▬▬▬▬ Session Ebook 🚀 ▬▬▬▬▬▬
Microservices CI/CD With Terraform Ebook: ► https://bit.ly/3xJd2WB

▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
My Site ► https://bit.ly/3rz82SQ
INSTAGRAM ► https://bit.ly/3iFWfy8
TWITTER ► https://bit.ly/3i6lkDs
LINKEDIN ► https://bit.ly/3rAeCsm

Hashicorp Terraform Associate Certification Study Course. Pass the Exam in 2 hours!

Sandip Das — Tue, 27 Jul 2021 16:10:40 +0000

This is a complete end-to-end course to help you pass the Hashicorp Terraform Certification.

Learn:

What is Infrastructure as Code (IaC)
What is Infrastructure as Code
IaC and its benefits
What is Terraform?
Terraform Features
Terraform Basic Commands
Terraform Main Commands In-depth explanation (Terraform get, init, validate, plan, apply, state, workspaces, destroy, import, taint, graph)
Terraform Coding (Structure, resources, data sources, variable, providers, functions, modules, provisioners, Lifecycle Policy, Remote State backend)
Terraform Cloud & Enterprise

Terraform is our tool of choice to manage the entire lifecycle of infrastructure using infrastructure as code. That means declaring infrastructure components in configuration files that are then used by Terraform to provision, adjust and tear down infrastructure in various cloud providers

In this tutorial, you will learn in-depth about what is Terraform, the important commands we use daily, how we can code with Terraform, and how to work with teams while coding infra as code with Terraform.

Course Outline ⏰

0:00 - Intro
0:33 - What is Infrastructure as Code (IaC)?
01:48 - IaC benefits
04:40 - What is Terraform?
05:33 - Terraform Features
7:28 - Terraform CLI overview
7:54 - Terraform Installation Instructions
8:26 - Terraform Init
10:26 - Terraform Validate
12:10 - Terraform Plan
14:49 - Terraform Apply
19:42 - Terraform State
22:56 - Terraform Workspaces
26:54 - Terraform Destroy
28:12 - Terraform Import
29:12 - Terraform Taint
31:22 - Terraform Coding Intro
32:38 - Terraform Resource Referencing
35:12 - Terraform Example Code Explanation
42:40 - Supply AWS Credentials to Terraform
43:54 - Terraform Data Sources
45:10 - Terraform Variables Overview
45:56 - Input Variable In-depth explanation
48:12 - Different Ways to Pass Input variables in Terraform
49:56 - Input validation
52:46 - Input Variable Order
52:46 - Output Values
55:12 - Local Values
56:14 - Terraform Providers
58:32 - Terraform Functions
1:00:00 - Terraform Modules
1:01:54 - How to Use Terraform Modules
1:04:45 - Terraform Provisioners
1:07:28 - Lifecycle Management In Terraform
1:09:18 - Remote State Backends in Terraform (Very Important)
1:15:00 - Terraform Cloud
1:22:11 - Terraform Enterprise
1:23:34 - Closing Notes

Useful Links 🔗

Course Ebook 🚀

Download Terraform Full Course Ebook

Connect with me 👋

Containerization Best Practices

Sandip Das — Mon, 11 Jan 2021 04:27:38 +0000

Container technology now getting used by Developers, DevOps Professions, Data Scientists, and other tech professionals, especially it helped developers to develop and deploy applications a lot faster than before, it brings us application portability and a lot of other features, that we will through one by one and we will learn which best practices we can follow to optimize it further!

Ok, but for those the word container/containerization still an alien word, below is a brief about containerization:

What is Containerization?
I like this definition from Citrix:

Containerization is defined as a form of operating system virtualization, through which applications are run in isolated user spaces called containers, all using the same shared operating system (OS). A container is essentially a fully packaged and portable computing environment that already has all the necessary packages to run the application inside.

In a more simple way: Containerization is the process of packaging an application along with its required libraries, frameworks, and configuration files together so that it can be run in various computing environments efficiently

In one line: Containerization is the encapsulation of an application and its required environment.

The below diagram Explain it well 👍

Popular Container Engines:

Docker:
The first and still most popular container technology, Docker's open-source containerization engine works with most of the commercial/enterprise products, as well as many open-source tools.

CRI-O:
CRI-O, a lightweight alternative to using Docker, allows you to run containers directly from Kubernetes, without any unnecessary code or tooling.

Rktlet:
The aforementioned rkt, redesigned and retooled to use the CRI as rktlet, now has a set of supported tools and community to rival Docker.

runC:
runC, a lightweight universal container runtime, is a command-line tool for spawning and running containers according to the Open Container Initiative (OCI) specification.

Containerd:
A project of the Cloud Native Computing Foundation, containerd was an early container format. More recently the developers of containerd built a CRI plugin that lets Kubernetes run containerd in the same way it runs rktlet or CRI-O.

Microsoft Containers:
Positioned as an alternative to Linux, Microsoft Containers can support Windows containers under very specific circumstances. They generally run in a true virtual machine and not in a cluster manager like Kubernetes.

Kata Containers:
Kata Containers is an open-source container runtime, building lightweight virtual machines that seamlessly plug into the containers ecosystem.

But Again Why Use Containers?

Consistent Environment

Containers give developers the ability to create predictable environments that are isolated from other applications. Containers can also include software dependencies needed by the application, such as specific versions of programming language runtimes and other software libraries. From the developer’s perspective, all this is guaranteed to be consistent no matter where the application is ultimately deployed. All this translates to productivity: developers and IT Ops teams spend less time debugging and diagnosing differences in environments, and more time shipping new functionality for users. And it means fewer bugs since developers can now make assumptions in dev and test environments they can be sure will hold true in production.

Run Anywhere

Containers are able to run virtually anywhere, greatly easing development and deployment: on Linux, Windows, and Mac operating systems; on virtual machines or bare metal; on a developer’s machine or in data centers on-premises; and of course, in the public cloud. The widespread popularity of the Docker image format for containers further helps with portability. Wherever you want to run your software, you can use containers.

Isolation

Containers virtualize CPU, memory, storage, and network resources at the OS-level, providing developers with a sandboxed view of the OS logically isolated from other applications.

From Code to Applications

Containers allow you to package your application and its dependencies together into one succinct manifest that can be version controlled, allowing for easy replication of your application across developers on your team and machines in your cluster.

Less overhead

Containers require fewer system resources than traditional or hardware virtual machine environments because they don’t include operating system images.

Greater efficiency

Containers allow applications to be more rapidly deployed, patched, or scaled.

Containerization Best Practices:

Choose Container Engine Properly:

Based on the project and use cases, select the right container engine for your application, in case not sure about what would be the right container engine for your application, then good to go with the most popular one i.e. Docker but don't just select it because someone telling it, do your own research and then select the right one for your project.

Keep images as small as possible:

To Keep images smaller:

-> Use multi-stage builds (e.g. docker multi-stage build)

-> Use smaller base image

-> While building image make sure only include packages what is necessary, should remove what's not needed and always tag the images.

Store Persistent Data Outside Container:

Avoid storing application data in your container’s writable layer. This increases the size of your container and is less efficient from an I/O perspective than using volumes or bind mounts, instead of store data in external data storage e.g volumes in docker

Don't store secrets in simple text:

Never store secrets / sensitive files which have contents like DB username/passwords, or any other sensitive credentials in plain text, instead use native secret management tools available in container engine or use efficient external plugins.

Dev, Test, Staging, Production:

Make sure to separately and dynamically manage environments in containerization, use environments variable accordingly, can use a script to automate this and optimize further.

Utilize CI/CD Tools for Testing & Deployment:

When you check in a change to source control or create a pull request, use any CI/CD pipeline to automatically build, tag images and test it and deploy automatically.

Pay attention to Container Security:

Security matters and considering the current situation it matters a lot! , so make sure to sign images, scan any external images before using them, search for vulnerability in images/packages/dependencies, and fix the root cause and of course audit those regularly. Try to use trusted/secure images

Single App per Container:

Of course you can run multiple, but it's a good practice to run a single app per container. Because a container is designed to have the same lifecycle as the app it hosts, each of your containers should contain only one app. When a container starts, so should the app, and when the app stops, so should the container.

Never Run Container as Root:

Out of security concerns, never run any container as root as it will get root access to the host system, any security flow in any installed packages/dependencies will affect the original host system.

Don't Run Many Processed in Single Container:

Running many processes inside a single process will affect the performance of the actual application, so instead make multiple containers as services that can interact with each other.

Monitor Your Containers:

It's best practice to monitor your containers, check how it's performing, collect CPU, RAM, I/O usage and other resource data periodically and automatically, also collect and analyze the logs time to time. It will be very helpful at the time of debugging or reporting purposes.

There are many tools available e.g. container insight by AWS, newrelic, App Dynamics, App Optics, Prometheus, cAdvisor, Dynatrace, DataDog, and many more.

Thanks for reading this article 🙏

Did I miss any point? let me know in the comments!

About the Author:

Sandip Das works as a Sr. Cloud Solutions Architect & DevOps Engineer for multiple tech product companies/start-ups, also holding the title of "AWS Container Hero".

He is always in "keep on learning" mode, enjoys sharing knowledge with others, and currently holds 4 AWS Certifications. Sandip finds blogging as a great way to share knowledge: he writes articles on Linkedin about Cloud, DevOps, Programming, and more. He also creates video tutorials on his YouTube channel.

DevOps Roadmap

Sandip Das — Wed, 15 Jul 2020 16:20:07 +0000

There has been a constant request on giving a roadmap on becoming a DevOps Engineer and of course the required skills that you need to master to become a DevOps Engineer.
In this article, I am going to explain all that in greater detail. But looks at this first 👇

Ok! Ok! I know it's a bit too much, don't get overwhelmed, let get started from basics.

What is DevOps?

DevOps is a set of practices that combines software development and IT operations. It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. DevOps is complementary with Agile software development; several DevOps aspects came from Agile methodology. - Wikipedia

I like this definition from AWS too:

DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.

From the definition and above image you are already getting the idea which might be DevOps.

In simple terms, it's anything that can make below process faster & smoother :

Coding -> Building -> Testing -> Packaging -> Releasing -> Configuring -> Monitoring -> Repeat & Improve <-

We talked about what is DevOps, now let's talk about DevSecOps

I like Sumologic's explanation on DevSecOps 👇

What is DevSecOps:

"DevSecOps is the philosophy of integrating security practices within the DevOps process. DevSecOps involves creating a ‘Security as Code’ culture with ongoing, flexible collaboration between release engineers and security teams. The DevSecOps movement, like DevOps itself, is focused on creating new solutions for complex software development processes within an agile framework.

DevSecOps is a natural and necessary response to the bottleneck effect of older security models on the modern continuous delivery pipeline. The goal is to bridge traditional gaps between IT and security while ensuring fast, safe delivery of code. Silo thinking is replaced by increased communication and shared responsibility for security tasks during all phases of the delivery process."

It's simply means make sure security as part of your DevOps process and make it automated.

Before even going further it's important to understand the daily tasks of DevOps Engineers, below are some examples :

Design, build, test and deploy scalable, secure, distributed systems from development through production
Manage the code repository(such as Git, SVN, BitBucket, etc.) including code merging and integrating, branching and maintenance and remote repository management
Manage, configure and maintain infrastructure system
Design the database architecture and database objects and synchronize the various environments
Design implement and support DevOps Continuous Integration and Continuous Delivery pipelines
Research and implement new technologies and practices
Document processes, systems, and workflows
Creation and enhancement of dynamic monitoring and alerting solutions using industry-leading services
Continuously analyze tasks that are performed manually and can be replaced by code
Creation and enhancement of Continuous Deployment automation built on Docker and Kubernetes.
You might be feeling that it's all responsibility of the developer and operational personals combined, and it's really it is, that's exactly why it's:

Dev(elopment) + Op(eration)s = DevOps

Ok, enough with the definitions, now let's talk about recommended roadmap:

1) Learn More on DevOps Culture & Practices:

DevOps is everyone's responsibility, it's not a hard written rule or such, instead, it's when all team comes together to achieve the ultimate goal i.e. optimize both the productivity of developers and the reliability of operations, make a quality product, release features quickly and fix bugs faster, introduce more automation and follow best DevOps practices such as:

Continuous Integration:

Continuous integration is a software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run. The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

Continuous Delivery:

Continuous delivery is a software development practice where code changes are automatically built, tested, and prepared for a release to production. It expands upon continuous integration by deploying all code changes to a testing environment and/or a production environment after the build stage. When continuous delivery is implemented properly, developers will always have a deployment-ready build artifact that has passed through a standardized test process.

Microservices:

The microservices architecture is a design approach to build a single application as a set of small services. Each service runs in its own process and communicates with other services through a well-defined interface using a lightweight mechanism, typically an HTTP-based application programming interface (API). Microservices are built around business capabilities; each service is scoped to a single purpose. You can use different frameworks or programming languages to write microservices and deploy them independently, as a single service, or as a group of services.

Infrastructure as Code:

Infrastructure as code is a practice in which infrastructure is provisioned and managed using code and software development techniques, such as version control and continuous integration. The cloud’s API-driven model enables developers and system administrators to interact with infrastructure programmatically, and at scale, instead of needing to manually set up and configure resources. Thus, engineers can interface with infrastructure using code-based tools and treat infrastructure in a manner similar to how they treat application code. Because they are defined by code, infrastructure and servers can quickly be deployed using standardized patterns, updated with the latest patches and versions, or duplicated in repeatable ways.

Monitoring and Logging:

Organizations monitor metrics and logs to see how application and infrastructure performance impacts the experience of their product’s end user. By capturing, categorizing, and then analyzing data and logs generated by applications and infrastructure, organizations understand how changes or updates impact users, shedding insights into the root causes of problems or unexpected changes. Active monitoring becomes increasingly important as services must be available 24/7 and as application and infrastructure update frequency increases. Creating alerts or performing real-time analysis of this data also helps organizations more proactively monitor their services.

Communication and Collaboration :

Increased communication and collaboration in an organization is one of the key cultural aspects of DevOps. The use of DevOps tooling and automation of the software delivery process establishes collaboration by physically bringing together the workflows and responsibilities of development and operations. Building on top of that, these teams set strong cultural norms around information sharing and facilitating communication through the use of chat applications, issue or project tracking systems, and wikis. This helps speed up communication across developers, operations, and even other teams like marketing or sales, allowing all parts of the organization to align more closely with goals and projects.

2) Learn at least one back-end Programming language:

Yes, you have to learn at least one programming language so that you can program automation scripts and if you could able to learn multiple demanding languages, it's even better but not all 😉

My Personal recommendations are: Go, Python, JavaScript (Node.js)

Good Courses:

Go:

Paid: https://www.udemy.com/course/go-the-complete-developers-guide/

Free: https://www.youtube.com/watch?v=YS4e4q9oBaU

Python:

Paid: https://www.udemy.com/course/python-the-complete-python-developer-course/

Free:

https://www.youtube.com/watch?v=_uQrJ0TkZlc

https://www.youtube.com/watch?v=rfscVS0vtbw

https://www.youtube.com/watch?v=WGJJIrtnfpk

Node.js

Paid: https://www.udemy.com/course/the-complete-nodejs-developer-course-2/

Free:

https://www.youtube.com/watch?v=JnvKXcSI7yk

https://www.youtube.com/watch?v=RLtyhwFtXQA

3) Understand Different OS Concepts:

Essential OS concepts necessary for a DevOps engineer, since as a DevOps engineer ultimately will be working with different OS (Mostly Linux) during development and/or deployment.

DevOps engineer often has to do system-level OS optimizations to increase performance and/or do other optimizations as per application needs, knowledge of Process Management, Threads and Concurrency, Sockets, I/O Management, Virtualization, Memory storage, and File systems would be very much useful in this process.

Good Courses:

Paid:

https://www.udemy.com/course/operating-system-concepts/

Free:

https://www.youtube.com/watch?v=Snrh580U3tI

https://www.youtube.com/watch?v=mXw9ruZaxzQ

4) Learn about managing servers:

As a DevOps engineer, we have to manage all kind of servers, and interesting enough it's not just 1 or 2 but a fleet of servers (sometimes even 100+), mostly consist of Linux instances (and rarely other OS ) and that's why the previous step i.e. basic understanding different OS concepts are needed, and after that deepen your knowledge in Linux Systems, that will make your daily life easier 😉

As a DevOps Engineer, We should have the knowledge of dynamically scaled up/down the servers, without rewriting the configuration files and make necessary recommendation to developers so that the application itself comply with scalability approach.

Also, you will definitely have to handle web servers at some point in time if not all the time 😁, so having knowledge and practical experience of Reverse Proxy servers such as Nginx and/or Apache is also necessary as well.

Good Courses:

Paid:

https://www.udemy.com/course/linux-administration-bootcamp/

https://www.udemy.com/course/nginx-fundamentals/ (nginx)

Free:

https://www.youtube.com/watch?v=wsh64rjnRas

https://www.youtube.com/watch?v=v_1zB2WNN14

https://www.youtube.com/watch?v=lWjZSgXu5VU (nginx)

5) Networking, Security, and Protocols

Knowledge of networking, security, and basic protocols are absolutely necessary, as a DevOps engineer we have to set-up VPC and security groups/firewalls and utilize various protocols such as TCP/IP (Transfer Control Protocol/Internet Protocol), HTTP (Hypertext Transfer Protocol), SSL (Secure Sockets Layer), SSH (Secure Shell), FTP (File Transfer Protocol), SMTP (Simple Mail Transfer Protocol) and more.

In particular to networking concepts, necessary concepts such as DNS record management, routing, firewalls and ports, basic utilities like ping, ssh, netstat, ncr and ip, load balancing, and TLS encryption are also necessary.

Good Courses:

Paid:

https://www.udemy.com/course/introduction-to-computer-networks/

https://www.udemy.com/course/network-security-course/

Free:

https://www.youtube.com/watch?v=QKfk7YFILws

https://www.youtube.com/watch?v=U_P23SqJaDc

6) Learn Infrastructure as Code:

"Infrastructure as code is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools."

I also like this definition from AWS and Microsoft:

AWS:

"Infrastructure as code is a practice in which infrastructure is provisioned and managed using code and software development techniques, such as version control and continuous integration. The cloud’s API-driven model enables developers and system administrators to interact with infrastructure programmatically, and at scale, instead of needing to manually set up and configure resources. Thus, engineers can interface with infrastructure using code-based tools and treat infrastructure in a manner similar to how they treat application code. Because they are defined by code, infrastructure and servers can quickly be deployed using standardized patterns, updated with the latest patches and versions, or duplicated in repeatable ways."

Microsoft:

"Infrastructure as Code (IaC) is the management of infrastructure (networks, virtual machines, load balancers, and connection topology) in a descriptive model, using the same versioning as DevOps team uses for source code. Like the principle that the same source code generates the same binary, an IaC model generates the same environment every time it is applied. IaC is a key DevOps practice and is used in conjunction with continuous delivery.

Infrastructure as Code evolved to solve the problem of environment drift in the release pipeline. Without IaC, teams must maintain the settings of individual deployment environments. Over time, each environment becomes a snowflake, that is, a unique configuration that cannot be reproduced automatically. Inconsistency among environments leads to issues during deployments. With snowflakes, administration and maintenance of infrastructure involves manual processes which were hard to track and contributed to errors."

All In Simple Words:

Infrastructure as code (IaC) means to manage your IT infrastructure using configuration files.

As a DevOps engineer, we should know about containers (Docker), Container Orchestration (Kubernetes, Docker Swarm), Configuration management tools like Ansible, Chef, Salt, and Puppet, Infrastructure Provisionings tools like Terraform and Cloud formation.

Good Courses:

Paid:

https://www.udemy.com/course/docker-mastery/ (Docker)

https://www.udemy.com/course/learn-kubernetes/ (Kubernetes )

https://www.udemy.com/course/learn-ansible/ (Ansible)

https://www.udemy.com/course/chef-fundamentals-a-recipe-for-automating-infrastructure/ (Chef)

https://www.udemy.com/course/learning-salt/ (Salt)

https://www.udemy.com/course/learn-puppet/ (Puppet)

https://www.udemy.com/course/learning-path-automation-with-ansible-puppet-and-salt/ (Ansible, Puppet and salt)

https://www.udemy.com/course/learn-devops-infrastructure-automation-with-terraform/ (Terraform)

https://www.udemy.com/course/aws-cloudformation-master-class/ (Cloud Formation)

Free:

https://www.youtube.com/watch?v=RSIstPUiEjY (Docker)

https://www.youtube.com/watch?v=Mi3Lx7yk3Hg (Kubernetes)

https://www.youtube.com/watch?v=EcnqJbxBcM0 (Ansible)

https://www.youtube.com/watch?v=LTIjUJEehDA (Chef)

https://www.youtube.com/watch?v=XovHdJZp1X4&list=PLgGQIE0cGrkiDRmJ3YWBKlI2KuYfFvfzV (Salt)

https://www.youtube.com/watch?v=kHD4KQKKP5Y (Puppet)

https://www.youtube.com/watch?v=0-z1G0BFZSU (Terraform)

https://www.youtube.com/watch?v=6R44BADNJA8 (Cloud Formation)

7) Learn Some CI/CD Tools:

Continuous Integration (CI):
Continuous Integration (CI) is a development practice where developers integrate code into a shared repository frequently, preferably several times a day. Each integration can then be verified by an automated build and automated tests. While automated testing is not strictly part of CI it is typically implied.

Continuous Delivery(CD):

Continuous Delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time and, when releasing the software, doing so manually. It aims at building, testing, and releasing software with greater speed and frequency.

As a DevOps Engineer, widely used tool you should be aware of, learn, and utilize:

Jenkins , GitLab CI, CircleCI , Microsoft VSTS, CodeShip, Bamboo, GitHub Actions

Good Courses:

Paid:

https://www.udemy.com/course/jenkins-from-zero-to-hero/ (Jenkins)

Free:

https://www.youtube.com/watch?v=FX322RVNGj4 (Jenkins)

8. Learn to monitor software and infrastructure:

It's very important for a DevOps Engineer to collect feedback and implement the changes quickly, for that you should have knowledge of monitoring tools like: Nagios, Prometheus, Grafana

Good courses:

Paid:

https://www.udemy.com/course/setting-up-nagios-4/ (Nagios)

https://www.udemy.com/course/monitoring-and-alerting-with-prometheus/ (Prometheus)

https://www.udemy.com/course/grafana-graphite-and-statsd-visualize-metrics/ (Grafana)

Free:

https://www.youtube.com/watch?v=knCO6wzCW3w (Negios)

https://www.youtube.com/watch?v=9GMWvFcQjYI (Prometheus)

https://www.youtube.com/watch?v=sTP7yzXmdFk&list=PLyJqGMYm0vnO9osZ-EBV6iu2l10muE2A- (Grafana)

9) Cloud Providers:

I always say:

There is NO DevOps Without Cloud

It's actually self-explanatory but still, let me simplify a bit, for that let me ask you a question?

Would you think about doing DevOps for your local machine?

The answer is straight cut no, isn't it!! yes, we don't bother DevOps for local development environments but we do care when we are thinking about hosting applications in the Cloud, managing multiple applications & servers, and monitoring both application as well as Infrastructure for staging and production environments.

Currently, there are multiple cloud service provider out there out which 3 are holing top market share, they are AWS, Azure, and GCP, let me tell you what tools for each cloud provider you should learn:

AWS: The Cloud Leader
CI & CD: AWS CodePipeline, AWS CodeBuild, AWS CodeDeploy, AWS CodeStar

Microservices: Amazon Elastic Container Service (ECS), AWS Lambda,

Infrastructure as Code: Cloud Formation, AWS OpsWorks

Configuration Management: AWS System Manager

Policy as Code: AWS Config

Monitoring and Logging: Amazon CloudWatch, X-Ray AWS CloudTrail

Platform as a Service: AWS Elastic Beanstalk

Version Control: AWS CodeCommit

Microsoft Azure:
CI & CD: Azure Pipelines, Azure Test Plans, Azure Artifacts

Version Control: Azure Repos

GCP:
CI/CD: Cloud Build, Tekton, Artifact Registry, Deployment Manager

Good Courses:

Paid:

https://www.udemy.com/course/aws-certified-developer-associate/ (AWS)

https://www.udemy.com/course/aws-certified-developer-associate-dva-c01/ (AWS)

https://www.udemy.com/course/aws-certified-devops-engineer-professional-hands-on/ (AWS)

https://www.udemy.com/course/azure-devops-for-beginners/ (Azure)

https://www.udemy.com/course/azure-devops-ci-cd-pipelines/ (Azure)

https://www.udemy.com/course/microsoft-azure-devops/ (Azure)

https://www.udemy.com/course/google-certified-architect-developer-engineer-data-devops/ (GCP)

Free:

https://www.youtube.com/watch?v=RrKRN9zRBWs (AWS)

https://www.youtube.com/watch?v=PxGGeNEdb3E (AWS)

https://www.youtube.com/watch?v=k1RI5locZE4 (AWS)

https://www.youtube.com/watch?v=tDuruX7XSac (Azure)

https://www.youtube.com/watch?v=QmvAYDc4UUw (Azure)

https://www.youtube.com/watch?v=RfbXgK-T0dM (GCP)

Summary:

I would say it's a long and never-ending journey, keep learning, keep improving, don't try to learn everything at once, instead of one at a time and related topic together to make the learning process a bit faster. Don't just learn, do hands-on, that's what important and will help you get quality jobs afterward.

I wish you good luck on your DevOps journey 😊

About the Author:

Sandip Das works as a Sr. Cloud Solutions Architect & DevOps Engineer for multiple tech product companies/start-ups, also holding the title of "AWS Container Hero".

Scams That Software Developers Need To Be Aware of

Sandip Das — Thu, 20 Jun 2019 09:30:51 +0000

Scam 1 - Technical Help Scam:
In this scam some fake and very intelligent profit seeking person will say that someone is in very big technical trouble and need your help, after that you reply with your charges, those persons will agree, then those persons will give a huge technical issue which normally takes days to solve but will ask to solve within few hours, now if you can't do the work within few hours then even though they have promised per hour charges , they will not pay you anything, they will block your number and stop all the communication, and if even if you solve the technical issue, these persons will take money from the main client and will not pay you a single rupee and block all kind of communication.

Scam 1 Solution: Only trust if this opportunity comes from a person you personally know from long time or before you have worked in any other projects with same person and got the money, or use escrow account like - ask the person to open that as an opportunity in any well-known freelancing site (but they will also charge a certain percentage, which hated by everyone the most), if any moment you feel this is a scam then halt immediately, that will save your time and effort.

Scam 2 - US / UK /EU / International Work Visa Scam:
This is "The Games Of Greed", the more greedy you are the more the chances that you are gonna fall for it.

There are people, who use people's greed to make money, yes, they have no other skills than to con greedy people, they will say there is an opportunity available for Software Engineers with more than $100000 USD per annum , and you want money right ? but to work there you will need visa as well, oh , they are very good visa consultant just pay $5000 (even more sometime), they will arrange everything .

You arrange the amount by any means (even selling own bike, own savings money, ask parents for the amount etc) to get the visa, and pay to that fake Visa consultant and that's it, they will block you in LinkedIn, in phone and every where possible. You lose your $5000.

Scam 2 Solutions: Apply for proper international jobs or acquire international clients, if you are skilled, and a valuable asset to team and company , they will fund your visa, your relocation charges, and of course you will get salary too, you need to work hard and smart way, there is no shortcut.

Scam 3- Training Requirement Scam:
If you are experienced software developer then you might be getting training requirements from many unknown person.

Your time is precious , you work day / night , you get very low time for your own, even that time if you agree to give commercial training to a company, and expect the payment will come in time.

But the twist, days after days passed but you don't get the money in account, you call the company but no one picking your call, you whatsapp but not seen (probably blocked), that's not a good experience, but common these days, have seen and talked with many other professionals who faced it and shared their experience with me, even faced same issue, that's why stopped giving training to training specific companies, because they are scammers.

Scam 3 Solution: Don't respond to or agree to these training companies, instead if a normal company directly contact you then you can take advance and you think they are seems genuine then only agree else don't.

Scam 4: Angry Client Scam:
In your career you will at least once will face this kind of scam.

Normally all of clients make contracts that's most favourable to them, of course they want win situation even it means you lose, until you are a big deal they will impose all the conditions possible, you are their target if you are a mid level developer and not senior developers as they probably already know about this.

Now comes, the main part, you need money, to have money, you need client, you are desperate to get the client, you agree to have any contract as you just need the money, here comes the angry client scam, these are the people who will come to you as client and will give you normal work, and you are super happy then they will give tremendous work pressure that you can't possible handle, and if it's a 1 month work they will want you to do that within a week or less, and if you can't they will blame you, threat you by saying that since you didn't able to deliver the work within the mentioned dates they lost more than USD $ 10,000 or more and now you will have to pay that amount by any means possible, they will start deliver threat by using lawyers and try to pressurize you to pay the money. Sometime their motto is that, they will pay the first month, then you will work for next 1 month, but they will have some financial problem so they will pay next month all together, and when the next month come, they will repeat their works again, then when it's start crossing limit to you and you complain they will give some impossible job to create this situation.

Scam 4 Solution: If you are a very rich, then pay the amount so that you get rid of issue. If you are intelligent, you will stay strong, ask for all the dues if they didn't pay yet, threat them back that you will reverse case against them for mentally pressuring you or not paying the due money. No matter what stay strong and don't pay them single rupee, they will bark and then will stop barking within few weeks. This is their strategy. What you should do is, if you understand or doubt this can happen, step out of the client as soon as possible. If legal email they send, ask them to come to India and have legal flight, they will not bother you anymore. It's an experience that will stay with you and make you strong if it ever happened to you and happen in future.

Scam 5 - International Opportunity Scam:
This scam are two type,

A) Work At Low Rate:

IN this method they will ask you to work in very low rate, so that within 1 year they will arrange a job in US / UK etc and you will get huge money once such happen, and you work for 1 year and you remind them their words they will say your work is not that impressive and they don't want to work with you anymore.

B) Pay $2500 or more or less To get International Job Scam:

Now this is a straight forward scam, they will ask you to pay $2500 or more or less than that and will guarantee a international job opportunity, and if you are greedy, lazy, rich and pay for it, then say bye bye to the money and bye bye to the person to whom you have paid it, as both not going to come back to you.

Scam 5 Solution: Just don't fall far it, all this scammers uses the same pattern, same words, easily identifiable. Never lower the rate that you are currently getting and aim for higher as you get more experienced. Never pay anyone to get job, instead invest that money in self development, learn new skills, join new courses , be a better version of yourself, upskill, reskill and never fall for scams.

Scam 6- Type "Interested" , "Yes", "I am In", "Email", "Phone number" scam:
This is quite normal in linkedin.

This is just plain and simple marketing skills they are applying, they are trying to fool you, they either want your email id or phone number for scamming you or marketing purposes, no proper recruiter will want you to type "interested" or mobile number instead of sending your resume. And do you really think a person can read or view your profile if there is more than 100's of comments , it's not possible, generally they want more profile views or want that post to go viral that's it.

Scam 6 Solution: Don't fall for it, think logically, use your engineer brain and if you are looking for job then apply from proper medium, or reach out to the recruiter or entrepreniour directly.

Scam 7 - Free Internship Scam:
The target's are mostly freshers who are not from IIT, NIT or such top tier college.

The first phase or say most challenging phase in life of a software developer is to get first job and if you are mediocre then that's a hell for you.

Here comes the evils, the already broke people who seeking the innocency of these freshers and utilizing them as workforce without paying anything, they will even try to make fun of your knowledge in order to break you and indirectly force you to do job for free.

Pattern as follow:
"Oh, you don't have much knowledge, learn work here for 6 months, if you learn well then we will start paying salary"

If you stay there for the whatever agreed period, you may or may not get salary, if they don't want to give you salary and not willing to continue, they will say that you learn nothing and they don't want you in team anymore. Even if they start you paying, that will be very less so you will leave anyway, in this process you just lost that many months.

Scam 7 Solution: Never ever join this kind of greedy, evil peoples, they are just trying to use you and after that they will throw you away, instead join product development companies if you like their idea, be it start-ups or established companies, see no one will hire you if you lack skills, and they will hire you if you have skills and potential. Learn new demanding skills, ready yourselves, work hard (if you are mediocre then work even harder ), work smart, try to give interviews as much as possible, soon you will be get selected for paid job and not fake free jobs.

Scam 8 - Income Guarantee Schemes Scam:
This one is on latest trend. Target is rich and middle class candidates.

Pattern: Simple yet not so simple pattern. These scammers knows marketing tricks very well, they have the perfect convincing power unless you are the clever one. They will say they can guarantee your income at least more than Rs. 10 Lakh per year but you will have to pay around just Rs. 1 -2 Lakh to join their course, and how good people they are , they will refund the money back if you don't able to gain at least Rs. 10 lakh per year.

Wait!! here's the twist, their goodness comes with few conditions:

1) Your attendance should be more than 96% in a online course 🤣

2) Your marks at the end of course should be more than 95% 😅

and few more bunch of conditions that you can never will be able to achieve and say goodbye to refund and say hi to regret.

Wow!! talk about some unique way to scam, and more interestingly their posts goes viral just like anything (Greed wins over common sense)

Scam 8 Solution: Use common sense and brain a little bit, use the money for good online technical courses, learn more on new technologies, make yourself market ready, and try interview in companies to get job, don't assume that they will come and invite you to job unless you are a big deal, so become a big deal instead by improving yourself, that will go very long.

Some good online training sites are below:

A Cloud Guru

Linux Academy

Linkedin Learning

Udemy

Conclusion:

Don't take action without giving detailed thought.

Normally scammers pressurize you to take action quickly so that you can't get enough time to think. Reach out to more senior persons to get their suggestions or friends to take action and never fall for such scams.

Thanks for reading this post till the end!