DEV Community: Anuj Dube

Learning AWS Lambda, API Gateway, and S3 by Building Your Own QR Code Generation API

Anuj Dube — Tue, 31 Oct 2023 17:39:55 +0000

Step 1: Creating S3 Bucket
Step 2: Creating a Lambda Function
Step 3: Attaching API gateway to lambda function
Step 4: Integrating the Created API into a Website

In this tutorial, we will develop a custom QR code generator API and then integrate it into a basic website.

We'll build the API using Node.js and deploy it on AWS Lambda. The API endpoints will be established through API Gateway, and the generated QR codes will be stored in Amazon S3. The only requirement is having an AWS account. If you're within the free tier, following this tutorial should incur no costs, as we'll stay within the free tier limits. However, it's important to delete all AWS resources once you're done practicing to prevent any unexpected charges.

AWS Lambda is a serverless compute service provided by AWS (Amazon Web Services) that allows developers to run code in response to events without having to manage servers or runtime environments. Lambda Supports lots of Runtime environments like Node.js, Python, Go, Ruby, Java, .NET etc. It supports custom runtime environments also. You only pay for the compute time you consume - there is no charge when your code is not running.

Under Free Tier AWS Gives 1,000,000 free requests per month which is 3.2 million seconds of free compute time per month for 12 months.

Step 1: Creating S3 Bucket

We will be creating an S3 bucket. and making content inside it publically accessible so we can access our generated QR codes.

Click on Create Bucket

Now Enter the Bucket name it must be unique. Keep all options as default. and then click on the "create bucket" button at the bottom.

Now After Your bucket is created click on the "Permissions" tab. Click on the Edit button on "Block public access".

uncheck all options and click on "Save Changes". You will be asked to confirm. type confirm.

Now Move down and click on Edit button in "Bucket policy".

Update Bucket Policy like below. Keep in mind to add Your Bucket ARN. You can take it just from above.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "<Your ARN>/*"
    }
  ]
}

Now Click on "Save Changes"

Now Our S3 Bucket is ready. All files put inside this bucket now will be publically accessible.

Step 2: Creating a Lambda Function

Search lambda in the search bar then click on the first result as shown in the image.

Now Click on Create Function

Then Select the First Option "Author from scratch" and then Enter the Function name. Also, Runtime should be selected as Node.js. Keep Everything default. and Just click on the "Create Function" button.

You Will come across a window like below.

You can explore this window to learn more.

Go to link https://github.com/Anujsd/lambda-qr-code

And Clone this Repo in your PC.

Open that in your Favourite IDE I will be using vs code.

Open index.mjs go to line number 9 and update the bucket name with your bucket name we just created.

Now Run the Command "npm install" to install node dependencies

Now We are going to create a ZIP of this whole "lambda-qr-code" folder.

For Windows open it in File Explorer select all, right-click and click on compress to zip.

name zip file "lambda-qr-code".

For the Sake of this tutorial, we will be uploading code using zip.

This ZIP file contains all code and node modules required for the lambda function. As we are using custom packages we need to upload node modules in zip format. otherwise, we could have directly written code on the web interface.

Now to upload this ZIP go to lambda in AWS in the "code" section and click on "upload from".

select option ".zip file" and upload that zip file.

After Uploading the File You will see an interface like below.

The code won't be visible in the web interface because the file is too large; however, it's quite straightforward, and you can view it below.

import AWS from 'aws-sdk';
import QRCode from 'qrcode';

// Configure AWS SDK
AWS.config.update({ region: 'us-east-1' });
const s3 = new AWS.S3();

// Define S3 bucket and key
const bucketName = '<s3 bucket name>';
const key = `qr-codes/${Date.now()}.png`;

export const handler = async (event) => {
  console.log(JSON.stringify(event));

  // Process POST request
  try {
    let requestBody;
    if (typeof event.body === 'string') {
      requestBody = JSON.parse(event.body);
    } else {
      requestBody = event.body; // Assuming it's already parsed
    }

    const { url } = requestBody;

    if (!url) {
      return {
        statusCode: 400,
        body: JSON.stringify({ message: 'URL is required', body: requestBody }),
      };
    }

    // Generate QR code
    const qrCodeBuffer = await QRCode.toBuffer(url);

    // Upload QR code to S3
    await s3
      .putObject({
        Bucket: bucketName,
        Key: key,
        Body: qrCodeBuffer,
        ContentType: 'image/png',
      })
      .promise();

    // Construct the public URL of the QR code
    const publicUrl = `https://${bucketName}.s3.amazonaws.com/${key}`;

    // Return response with CORS headers
    return {
      statusCode: 200,
      body: JSON.stringify({ qrCodeUrl: publicUrl }),
    };
  } catch (error) {
    console.error('Error generating QR code:', error);
    return {
      statusCode: 500,
      body: JSON.stringify({ message: 'Internal Server Error' }),
    };
  }
};

Now We will be giving our lambda function permission to add generated QR in s3.

for that go to the "Configuration" Tab then "Permissions" and click on the "Role name" link. It will be opened in a new tab.

Now in a newly opened window. click on "Add permissions" and then "Attach policies".

a new window will be opened. Search "s3" in that and select "AmazonS3FullAccess" policy. and click on the "add permissions" button at the bottom.

Now go to lambda Function.

We will be testing If it's working or not.

Click on the "Test" Section. Select options like below and add below JSON which is our test payload.

{
  "body": {
    "url": "https://anujdube.com/"
  }
}

Click on "Save" and then click on the "Test" button.

After Successful execution, you will see the output like below. To see detailed logs you can click on the "logs" link.

Now Our lambda function is ready we are going to add an API gateway to this lambda function.

Step 3: Attaching API gateway to lambda function

Search API Gateway in the search bar then click on the first result as shown in the image.

after opening it. Click on the "build" option in HTTP API.

In "Intergrations" select our lambda function as shown below. and add name to API.

click on next. in routes. add a POST route like below.

Now Click on next. keep everything default. and click on the "Create" button.

You will see your API created like below.

Now open your API. On the left-hand side click on the section after "VPC links".

and in there you can see the invoke URL. this is your API URL.

You can use this URL and do a POST request using POSTMAN. It will be successful.

But if you try to use this URL in any application you might get CORS errors like below.

To resolve these errors click on CORS on the left-hand side. and click on "configure"

Add values as below and click on "Save"

Access-Control-Allow-Origin : *

Access-Control-Allow-Headers : Content-Type, x-requested-with

Access-Control-Allow-Methods : OPTIONS, POST

At the moment, we are permitting access from all origins. However, for enhanced security in a production environment, it's advisable to restrict access to just your domain name.

Click on the "save" button.

Now You will not face any CORS Errors While using your API in any app.

Step 4: Integrating the Created API into a Website

Now we will be trying to use the created API in a simple app.

Click on the below link and clone the repo

https://github.com/Anujsd/qr-code-front-end

it just has 3 files.

open script.js

and on line number 1. update your API endpoint

now if you open index.html

you will see UI like below

enter any link to shorten and you will see a QR code generated for that link

you can test if the QR code is correct or not.

Now You can host this website online if needed.

In this way, We developed our custom QR Code generation API utilizing AWS Lambda. If you encounter any issues or spot any errors in the article, please don't hesitate to contact me.

Ansible Overview

Anuj Dube — Sun, 22 Oct 2023 17:21:10 +0000

In the Field of IT, Automation and Configuration management have become indispensable. As systems grow in complexity, the need to manage and provision them in a reproducible and scalable manner is crucial. Here Ansible can help it's a simple, agentless IT automation tool that has become a go-to solution for many administrators and developers.

What Is Ansible?

Ansible is an open-source automation tool that helps in software provisioning, configuration management, and application deployment. Developed in Python, it uses a YAML to describe system configurations, making it readable and easy to implement. With its agentless architecture, all you need is SSH access and Python installed on the target nodes. In the case of Windows, It can use Winrm to connect to the Windows host and execute automations. Not only servers you can manage network devices like routers, switches etc with Ansible.

Ansible Architecture

Ansible's design is straightforward and Easy to understand:

Control Node: This is the machine where Ansible is installed and from which all tasks and playbooks are run.
Managed Nodes: These are the systems that are managed by the control node. No agents are installed on these systems.
Inventory: A file that lists all managed nodes.
Playbooks: Written in YAML, these define the desired states of systems, describing tasks to be executed on managed nodes.
Modules: These are units of code Ansible executes. With over a thousand modules, Ansible can manage almost any IT component.

Ansible communicates with its managed nodes using SSH. but, for Windows nodes, it utilizes WinRM.

One of the standout features of Ansible is its agentless nature. This means there's no need to install any additional software on the managed nodes. Instead, Ansible is only installed on the control node. The configurations/Automation Steps are stored in files called playbooks written in YAML, while the list of nodes it manages is documented in an inventory file, typically in INI format. To ensure security, the credentials used to connect to the managed nodes are stored securely in the Ansible Vault, where they are encrypted.

When it's time to execute tasks on the managed nodes, Ansible converts these tasks from the playbooks into small Python scripts called modules. These modules are then dispatched to the managed nodes for execution. This process showcases Ansible's "push" mechanism, where configurations are actively sent to the managed nodes.

For IT devices like networking equipment that have limited computing power and constraints, Ansible runs modules on the control node instead of directly on the devices. These modules then produce specific commands tailored for each device or initiate API requests to automate various tasks.

Automation Execution is idempotent means if you run an Ansible playbook once or multiple times, the end state of the target system will remain the same. This means repeated playbook runs won't unintentionally change systems, making Ansible's automation dependable.

Why use Ansible?

Open Source : Being open-source, Ansible benefits from collective expertise, allowing for rapid improvements.
Simplicity : Ansible's declarative language (YAML) means that even those unfamiliar with programming can understand and write playbooks.
Agentless : There's no need to install any software on the managed nodes. This reduces overhead and potential attack vectors.
Versatility : From managing configurations and deploying applications to orchestrating complex workflows, Ansible can handle it all. It can even manage Cloud Infrastructure like AWS, Azure, Google Cloud, VMware, etc.
Community Support : A large, active community ensures that new modules are continually added, and existing ones are maintained.
Extensibility : With its modular architecture, it's easy to extend Ansible to support custom or niche requirements. Lots of pre-created modules are present for various use cases. You can write your custom modules using languages like Python, bash, PowerShell, etc., returning JSON.

Community Ansible Vs RedHat Ansible Automation Platform

Community Ansible is open-source, while the Red Hat Ansible Automation Platform is a commercial offering by Red Hat, which builds on the foundation of Community Ansible.

Community Ansible is perfect for those looking for a free, open-source automation tool with a broad ecosystem.

Red Hat Ansible Automation Platform Provides Professional support, A curated collection of certified modules, Regular updates, better integration with other Redhat products and additional features such as advanced analytics, logging, and integrations making it suitable for enterprises with critical workloads and requirements.

Ansible vs Puppet vs Chef

Comparison of Ansible with other automation tools.

Conclusion

In the ever-evolving realm of IT, the importance of efficient and scalable management cannot be overlooked. Ansible emerges as a robust solution, offering a blend of simplicity, versatility, and security in its automation capabilities. it's a game-changer in the landscape of IT automation.

Configuring Ansible For AWS Management

Anuj Dube — Sun, 08 Oct 2023 13:18:25 +0000

Amazon Web Services (AWS) offers a scalable environment for deploying applications, but as the infrastructure grows, so does the complexity of managing it. Organizations need an easy way to manage this infrastructure. they are using different tools to keep the infrastructure as code. Ansible is such an open-source tool. it can help you to create and manage your infrastructure in different cloud environments.

In this article, we will look into how to create and manage AWS ec2 instances using Ansible.

Creating Ansible Control Node

Step 1: Create an Ubuntu EC2 Instance on AWS

After the Creation of the instance SSH into it. we will be using this as a control node to create and manage other AWS resources

Step 2: Install Ansible

Update Repository and Upgrade Packages

sudo apt update
sudo apt upgrade

Sometimes You will be asked to restart

sudo reboot

Install Ansible

sudo pip install ansible

Step 3: Install amazon.aws Ansible Collection

Install amazon.aws ansible collection

ansible-galaxy collection install amazon.aws

amazon.aws Ansible Collection needs boto3 and botocore packages to connect to AWS for creating and managing infrastructure

First Install PIP

sudo apt install python3-pip

Install boto3 and botocore packages

pip install boto3 botocore

Step 4: Install and Configure AWS CLI

sudo apt install awscli

Configure AWS credentials using the AWS CLI

aws configure

You will be prompted to enter some details below

To Get These Details First go to AWS.

On the Right side, Click on your Profile. In Drop Down Menu You will see an option called "security credentials". Click On it

On this page if you move down You will see an option called "Access Keys".

Click on it and select first option "Command Line Interface"

Then Just Click next and create your access key.

Now You have "Access Key ID" and "Secret Access Key".

Please do not use keys from images They will not work I will be deleting this user and all his keys😉

To Get the Default Region name click on Drop Down on the left side of the profile. I have taken the region Mumbai so the name will be ap-south-1

After Entering all the details, We will check if we can connect to AWS or not

Run Below Command It will show information about your EC2 instances

if the below command fails then you must have made some mistake while entering configuration information please try again

aws ec2 describe-instances

Now Our Control Node is ready

Creating and Managing EC2 Instances

Example 1: Create a Single Ubuntu EC2 Instance with a Public IP Address

For This, we are going to use the module "ec2_instance"

Create a YAML File called create_ec2.yml

---
- name: Create Single Ubuntu EC2 Instance
  hosts: localhost
  connection: local
  gather_facts: no
  tasks:
    - name: Create Ubuntu EC2 instace with public ip
      amazon.aws.ec2_instance:
        name: "Created Using Ansible"
        key_name: "newUbuntu"
        vpc_subnet_id: subnet-0bd155c67558454b1
        instance_type: t2.micro
        security_group: sg-076a5ea097f046d8e
        network:
          assign_public_ip: true
        image_id: ami-0f5ee92e2d63afc18
        tags:
          Environment: Testing
      register: ec2_node_info
    - name: Display information
      debug:
        msg: "{{ec2_node_info}}"

"key_name": SSH Key you will be using to login into the created server, You can use the same key that we are using to log into the control node

You Can use the Same details as we are using in the control node

Image_Id: In Details Tab

to create another type of instance you can go to "AMI catalog" and you can get image_id from there.

vpc_subnet_id: In Networking Tab

security_group: In the Security tab

instance_type: In Details Tab

To Get other instance types go to the "Instance Types" tab

To Run the playbook use the below command

ansible-playbook create_ec2.yml

we are running it in localhost those options are mentioned in yml. so you can ignore warnings.

You will see output like the below as we are printing created instance details

Now Go To AWS EC2 Console You will see a new instance with the name "Created Using Ansible" will be created.

Example 2: Get a List of all running Instances

For This, We are going to "ec2_instance_info"

Create a YAML File called get_all_ec2.yml

---
- name: Get All Running EC2 instances
  hosts: localhost
  connection: local
  gather_facts: no
  tasks:
    - name: Gather information about instances in states "running"
      amazon.aws.ec2_instance_info:
        filters:
          instance-state-name: "running"
      register: ec2_node_info
    - name: Get Running Instances Count
      debug:
        msg: "Total running instances: {{ ec2_node_info.instances | length }}"
    - name: Display information
      debug:
        msg: "{{ec2_node_info}}"

Run Using Command

ansible-playbook get_all_ec2.yml

You will see output like below giving all running ec2 instances details

Example 3: Terminate Every Instance From the Current AWS Region

This will terminate the control node also.

Create a File called terminate_all_ec2.yml

---
- name: Terminate Every Instance in Current AWS Region
  hosts: localhost
  connection: local
  gather_facts: no
  tasks:
    - name: Terminate every running instance in a region. Use with EXTREME caution.
      amazon.aws.ec2_instance:
        state: absent
        filters:
          instance-state-name: running

Now Run this using below command

ansible-playbook terminate_all_ec2.yml

You will see That Your Control node is also terminated so you might be disconnected

You can Manage and control lots of other AWS Resources like S3, VPC, Lambda, RDS etc. using Ansible.

You can learn more about that using the below link

AWS Ansible Documentation

Happy automating! 🤖

Setting Up a Practical Ansible Lab on AWS: A Step-by-Step Guide

Anuj Dube — Sun, 01 Oct 2023 14:58:28 +0000

Ansible is a powerful, easy-to-use and easy-to-learn configuration management tool. If you are trying to learn Ansible then the first hurdle for you will be setting up a lab environment to practice Ansible.

In this article, we are going to create a lab environment on AWS. You can use any other cloud platform like Azure, or Google Cloud or you can create vm on Virtual Box. I would suggest using AWS or Azure (here we will use AWS) as they offer 12 months of free trial you can play with lots of different OSs also. not suggesting Google Cloud as they don't offer a Windows server on a free trial.

Now Let's get started, We are going to create 3 servers. All of them will be Ubuntu servers. We are going to use one server as the control node and the remaining two as target nodes.

we can use Windows servers as the target node but not as the control node. here for setting up the lab, we are going to use Ubuntu servers only, as using a Windows server as a target has some extra steps. you can learn how to set up a Windows server for Ansible management here.

1. Get an AWS Free Tier Account

To get this account you might need to provide them with a credit/debit card. No need to worry they will not charge but remember to put a billing alert and close all resources after using them so you don't get any charges.

2. Create EC2 Instances

Search EC2 on the Search Bar and click on that

you will see windows like the above. Click on Instances(running)

now click on the Launch Instances Button at the right-top corner. and enter details below

Name and tags: Server1

Application and OS Images: Select Ubuntu do not change anything

Instance type: Keep Default Free Tier instance type

Key pair: Click on "Create new key pair" and create a new key it will be downloaded automatically after creation keep it safe

Network settings: allows SSH, HTTP, HTTPS

Keep Every other thing Default.

on the right-hand side, you will see the summary. In "Number of instances" enter 3. then click on "Launch instance".

Now change their names as above

3. Setting Up EC2 Instances

I am using Windows So I will be using MobaXtrem as an SSH Client. If you are on Mac or Linux you don't need to install anything SSH Client is pre-installed on those OS.

I have Stored My Key Pair named "newUbuntu.pem" which was generated while creating instances on location "C:\temp\aws".

First, we will go through the steps for Mac/Linux users and then for Windows.

Steps For Mac/Linux Users

Go to the location where you have stored your key pair and open the terminal there

Now On the AWS console, Select "Client" Server and Click on the Connect button at the top. then Go to SSH Client and copy the example SSH command

Paste That command in the terminal and press enter. it will ask for yes/no enter yes.

Now you are logged in to the control node. Now logout enter "exit"

We are going to copy our private SSH Key into the control node. for that use the below command. do not forget to update the command with your values

# scp -i path/to/your/private/key.pem path/to/your/local/file user@public-dns-name:/path/to/destination/directory
scp -i newUbuntu.pem newUbuntu.pem ubuntu@ec2-3-110-197-196.ap-south-1.compute.amazonaws.com:~/

now again log into the server and you can see your private key is copied like below

Now Type the below command to ensure your key is not publicly viewable

chmod 400 newUbuntu.pem

now we will try to see if our control node can SSH into server1 and server2 or not.

Go to AWS Console select server1, click on the connect button, and copy the ssh command like below.

ssh -i "newUbuntu.pem" ubuntu@ec2-3-110-197-196.ap-south-1.compute.amazonaws.com

now paste the command into the terminal of the control node and press enter

it might ask for yes/no. press yes. now you are in server1.

now type "exit" to exit from server1 to the control node.

Copy the SSH command for server2 also and try SSH into it.

You will be able to SSH into server2 also.

now you can directly go to the "4. Installing Ansible" step.

Steps For Windows Users

Download MobaXterm From here and Install it.

When MobaXterm is first started click on Settings -> Configuration

Change Home and Root Directory to the location where the key is stored. in my case it is C:\temp\aws. Click on OK. It will ask for a restart Click on Yes. MobaXStream will restart.

Now Click On Start Local Terminal

When we run "ls" it should show your key in the current directory

Now On the AWS console, Select "Client" Server and Click on the Connect button at the top. then Go to SSH Client and copy the example SSH command

Paste That command in the terminal and press enter. it will ask for yes/no enter yes.

Now you are logged in to the control node. On the left-hand side, you will see SSH Browser.

Upload your private key

Now Type the below command to ensure your key is not publicly viewable

chmod 400 newUbuntu.pem

now we will try to see if our control node can SSH into server1 and server2 or not.

Go to AWS Console select server1, click on the connect button, and copy the ssh command like below.

ssh -i "newUbuntu.pem" ubuntu@ec2-3-110-197-196.ap-south-1.compute.amazonaws.com

now paste the command into the terminal of the control node and press enter

it might ask for yes/no. press yes. now you are in server1.

now type "exit" to exit from server1 to the control node.

Copy the SSH command for server2 also and try SSH into it.

You will be able to SSH into server2 also.

Windows Users can also use SCP to copy private SSH key to control the server. But Using MobaXterm is much easier.

As Learning Experience you can try SCP also.

4. Installing Ansible

Update Package list and upgrade Packages

sudo apt update
sudo apt upgrade

Sometimes you may be asked to reboot then you can reboot using the below command.

sudo reboot

Install Ansible

sudo apt install ansible

Now Let's Create a Folder Called "ansiblePractice" and go into that folder

mkdir ansiblePractice
cd ansiblePractice

Create a File called "inventory" and enter the public IP address of server1 and server2 in there like below

nano inventory

Ctrl + O to Save -> Press Enter -> Ctrl + X to exit.

Now we will execute our first ansible ad-hoc command. Sometimes it can ask for yes/no. press yes.

ansible all --key-file newUbuntu.pem -i inventory -m ping

Now Your Lab Environment Is Ready You can start learning Ansible.

// Detect dark theme var iframe = document.getElementById('tweet-1702900896705970451-655'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1702900896705970451&theme=dark" }

Happy automating! 🤖

Setting up a Windows Server for Ansible Management

Anuj Dube — Sun, 24 Sep 2023 10:31:53 +0000

Ansible, a widely used configuration management tool, is not just for Unix-like systems. With the right setup, it can also be used to manage Windows servers. This article will guide you through the process of setting up a Windows server to be controlled using Ansible.

For this demonstration, we'll be setting up two instances of "Microsoft Windows Server 2022 Base" as child nodes. Additionally, we'll create an Ubuntu instance to serve as our control node.

Ansible Uses SSH for connecting to Linux systems, And for Windows systems it uses Winrm.

Steps to Configure a Windows Server

Open Powershell in Administrator mode and run the below command. it will run a script that will do all the necessary configurations. yes, that's it on the Window side.

Invoke-WebRequest -Uri https://raw.githubusercontent.com/rallabandisrinivas/winrm_ansible/main/README.md -UseBasicParsing | Select-Object -ExpandProperty Content | Invoke-Expression

You can refer to the official documentation below if you face any issues

Ansible Windows Setup Link

Steps to Configure a Ubuntu Server

Step 1: Update Package list and upgrade Packages

sudo apt update
sudo apt upgrade

Sometimes you may be asked to reboot then you can reboot using the below command.

sudo reboot

Step 2: Install python3-pip and then install pywinrm package.

sudo apt install python3-pip
pip install pywinrm

Step 3: Let's install ansible

sudo apt install ansible

Step 4: Now create an inventory file and add IP and authentication info about Windows servers in it.
inventory

 [windows]
 server1 ansible_host=65.2.122.214 ansible_user=Administrator ansible_password=3zA)--5TI$4pFng6*=qnAVLudgYtTqRP
 server2 ansible_host=15.206.194.23 ansible_user=Administrator ansible_password=3zA)--5TI$4pFng6*=qnAVLudgYtTqRP

 [windows:vars]
 ansible_connection=winrm
 ansible_winrm_server_cert_validation=ignore

Yes, Storing Passwords in plain text is a security risk we can use ansible-vault. But for the sake of the demo, we are storing it in plain text.

In Production environments, We can Use Ansible Vault to securely store and use passwords.

Step 5: Now We will be running the below Adhoc command to test the connectivity

ansible windows -m win_ping -i inventory

if you have created instances on the cloud and have not opened Winrm ports you might get the below error.

Open those ports and run the command again

Successful execution will look like the below

Now, with all the Configurations in place, you can fully automate a myriad of tasks on your Windows servers.

Happy automating! 🤖

Ansible Conditionals

Anuj Dube — Sun, 17 Sep 2023 04:36:10 +0000

In Ansible, conditionals offer control in playbooks, allowing tasks, plays, or even entire roles to be executed or skipped based on certain conditions. Here's an overview of the most used conditionals and their uses:

1. when

The most commonly used conditional. It determines if a task or play should be executed.
You have to write an expression that evaluates to a boolean (true/false).

Example:

You want some tasks to be executed only on some specific servers.

---

- hosts: all
  become: true
  tasks:
  - name: update repository index Ubuntu
    apt:
      update_cache: yes
    when: ansible_distribution == "Ubuntu"

  - name: update repository index Centos
    yum:
      update_cache: yes
    when: ansible_distribution == "CentOS"

Here while executing the task. Ansible will first check the when condition. If it's true only then it will execute the main task.

You can see it's skipping if the condition is not true.

From where we are getting this "ansible_os_family" variable?

while running every Ansible playbook. By default, Ansible runs the gather_facts task. that task gathers all information about servers. which gives us access to these variables.

2. changed_when

Modifies the behavior of a task to report a "changed" status.
You have to write an expression that evaluates to a boolean (true/false).

Example:

Updating the repository is a basic task Suppose you don't want to consider that as a change then you can write it like below

---

- hosts: all
  become: true
  tasks:

  - name: update repository index
    apt:
      update_cache: yes
    when: ansible_distribution == "Ubuntu"
    changed_when: false

  - name: update repository index
    yum:
      update_cache: yes
    when: ansible_distribution == "CentOS"
    changed_when: false

It will give you a changed=0

3. failed_when

Modifies the behavior of a task to report a "failed" status based on custom conditions.

Example: You want to check disk usage and want to mark it as a failure if usage is more than 90%.

---

- hosts: all
  vars:
    disk_usage_threshold: 90
  tasks:
    - name: Check root filesystem disk usage
      command: df -h /
      register: df_output
      changed_when: false
      failed_when: >
        (ansible_distribution == "CentOS" and (df_output.stdout_lines[-1].split()[4]|replace('%',''))|int > disk_usage_threshold) or
        (ansible_distribution == "Ubuntu" and (df_output.stdout_lines[1].split()[4]|replace('%',''))|int > disk_usage_threshold)

The output will be like the below

as my disks are not filled let me fill the disk of one server.

Now you can see that for one server script has failed showing disk is filled 94%.

4. until

Used for retrying a task until a certain condition is met or a certain number of retries is reached.

Example: Retry a task until a file exists.

---
- hosts: all
  tasks:
    - name: Check if file exists
      stat:
        path: "/tmp/file.txt"
      register: file_status
      until: file_status.stat.exists
      retries: 5
      delay: 3

You can see playbook is trying to check if file exist at that location or not. while it's running i logged into one server and created file. you can see it's showing file found for one server. it kept trying for another server.

5. assert

Used for failing a play if certain conditions are not met.

Example: Fail if the free memory is less than 10 GB.

---
- hosts: all
  tasks:
    - name: Get filesystem details for /
      ansible.builtin.setup:
        filter: ansible_mounts

    - name: Assert minimum disk space requirements
      assert:
        that:
          - item.size_available >= 10000000000 # 10 GB in bytes
        fail_msg: "Not enough disk space on {{ item.mount }}. At least 10GB required."
        success_msg: "{{ item.mount }} has sufficient space."
      with_items: "{{ ansible_mounts }}"
      when: item.mount == "/"

Output

You can see in the above image it failed for one server having insufficient space and was successful for another.

6. block and rescue

block groups tasks together.
rescue defines tasks to be executed if any task in the block fails.
It's like try and catch in javascript, java etc.

Example: Attempt to install a package and handle failures.

---
- hosts: all
  become: yes
  vars:
    package_name: example_package
  tasks:
    - name: Attempt to install a package
      block:
        - name: Install the "{{ package_name }}"
          package:
            name: "{{ package_name }}"
            state: present
      rescue:
        - name: Handle the failure
          debug:
            msg: "Failed to install {{ package_name }} . Check if the package name is correct or if there's a repo issue."
      always:
        - name: This always gets executed
          debug:
            msg: "This message will always be displayed, regardless of success or failure in the block above."

As you can see above script will fail to install a package called example_package as no such package exists. still, we are not shown any failure(failed=0). You can see the task in rescue is executed correctly.

if we put package_name=wget. which is actually a real package name. task will succeed. It will look like below.

7. Conditional imports and includes

Conditionally import playbooks or include tasks/roles based on certain conditions.

Example: install web server depending on user input

web_setup.yml

---
- hosts: all
  become: true
  vars_prompt:
    - name: webserver_type
      prompt: "Which web server do you want to install? (apache/nginx)"
      private: no

  tasks:
    - name: Include tasks based on web server choice
      include_tasks: "install_{{ webserver_type }}.yml"
      when: webserver_type in ['apache', 'nginx']

install_apache.yml

---
- name: Install Apache
  package:
    name: apache2
    state: present
  when: ansible_distribution == "Ubuntu"

- name: Install httpd
  package:
    name: httpd 
    state: present
  when: ansible_distribution == "CentOS"

install_nginx.yml

---
- name: Install Nginx
  ansible.builtin.package:
    name: nginx
    state: present

so depending on user input it will select playbook and run it.

for apache as input you can see it ran install_apache.yml

For nginx as input, you can see it ran install_nginx.yml

In conclusion, Ansible's conditionals provide a powerful way to bring logic and decision-making into playbooks, ensuring that infrastructure automation is both flexible and precise. Like any tool, the key is understanding the available options and applying them judiciously.

Happy automating! 🤖

Copying Files From Local To Remote Using PowerShell

Anuj Dube — Wed, 16 Aug 2023 14:31:23 +0000

Let's say you want to copy files/folders from a local device to one or more remote Windows servers. We have lots of ways to do this but in this article, we will be looking into how to do that using PowerShell. In your local and remote at both places, you need to enable PowerShell remoting.

Step 1: Check and set the execution policy

Get-ExecutionPolicy

if the exception policy is not RemoteSigned set it to RemoteSigned.

Set-ExecutionPolicy RemoteSigned

Step 2: Enable PowerShell remoting

Please keep in mind, When the SkipNetworkProfileCheck parameter is specified, it bypasses the check for network profiles during the configuration process. This allows PowerShell remoting to be enabled even if the network profile is not classified as private or domain.

Enable-PSremoting -SkipNetworkProfileCheck

Now to the main part

Method 1: Using Copy-Item with session

The most simple and best way. just use the ToSession parameter provided within copy-item.

$sourcePath = "C:\temp\files"
$destinationPath = "C:\temp\files"
$serverName = "<server name>"
$username = "<username>"

$credentail = Get-Credential -Message "Enter Credentials" -UserName $username
$session = New-PSSession -ComputerName $serverName -Credential $credentail

Copy-Item -Path $sourcePath -Destination $destinationPath -Recurse -ToSession $session

in the above script, we created a PowerShell session to that remote Windows server and then we are copying the whole folder from source to destination.

Method 2: Using Copy-Item with PS Drive

Here we are creating a ps-drive and then doing the copying part. this method helps to interact with the remote location in a way that resembles working with a local drive.

$sourcePath = "C:\temp\files"
$destinationPath = "C:\temp\files"
$serverName = "<server name>"
$username = "<username>"
$driveName = "R"

$credentail = Get-Credential -Message "Enter Credentials" -UserName $username

New-PSDrive -Name $driveName -PSProvider FileSystem -Root $destinationPath -Credential $credentail

Copy-Item -Path $sourcePath -Destination $driveName:\ -Recurse

Remove-PSDrive -Name $driveName

Method 3: Using Robocopy

Robocopy (Robust File Copy) is a command-line utility built into Windows that provides advanced options for copying files and directories.

While using robocopy easier way is to create a local drive and then use that for copying files.

keep in mind drive name should be a single character like A, R, X etc. Not names like "newDrive", "test" etc.

$sourcePath = "C:\temp\files"
$destinationPath = "C:\temp\files"
$serverName = "<server name>"
$username = "<username>"
$driveName = "R"

$credentail = Get-Credential -Message "Enter Credentials" -UserName $username

robocopy $sourcePath $driveName:\ /E /Z /COPYALL /R:3 /W:1

Remove-PSDrive -Name $driveName

Meaning of copy options we have passed

/E: copies subdirectories, including empty ones.

/Z: enables restartable mode.

/COPYALL: copies all file information.

/R:3: specifies 3 retries on failed copies.

/W:1: sets the wait time between retries to 1 second.