DEV Community: Rahul Kumar Sharma

How To Set Up Multi-Factor Authentication for SSH on Ubuntu?

Rahul Kumar Sharma — Fri, 15 Aug 2025 16:15:08 +0000

Securing SSH access is a top priority for any Linux server, and relying on passwords alone is no longer sufficient against modern attacks like credential stuffing and brute force attempts. Adding multi‑factor authentication (MFA) to SSH hardens access by requiring something known (key or password) plus something possessed (a one‑time code or hardware key).
Ubuntu supports common MFA methods through PAM (Pluggable Authentication Modules), including app‑based TOTP(Time-based One-time) codes (Google Authenticator/Authy/FreeOTP) and FIDO/U2F security keys (YubiKey, Nitrokey).

Why MFA for SSH is Needed?

Reduces risk from password reuse and brute‑force attacks by requiring a second factor even if a credential is compromised.
Integrates cleanly with existing SSH setups using PAM without replacing SSH keys or existing workflows.
Ubuntu includes enhanced support for FIDO/U2F, enabling hardware‑backed second factors for stronger, phishing‑resistant authentication

Prerequisites:

Ubuntu 20.04 server with SSH access and sudo privileges.
An authenticator app (Google Authenticator) on a phone if using TOTP.
A backup console or out‑of‑band access in case of misconfiguration to avoid lockout

Steps:
Step 1 — Install the Google Authenticator PAM module

Update and install the PAM module:

sudo apt update && sudo apt install -y libpam-google-authenticator

Step 2 — Enroll each user with google-authenticator

For every account that will use SSH with MFA, run:

google-authenticator

Recommended prompts:

time‑based tokens -> (y)
update the ~/.google_authenticator file -> (y)
disallow reuse -> (y)
keep default window (-> n)
enable rate limiting -> (y)

Scan the displayed QR code in the authenticator app and store the emergency scratch codes securely

Step 3 — Enable PAM integration for SSH

Edit PAM config:

sudo vi /etc/pam.d/sshd

Add the Google Authenticator line (place it near the top so it’s evaluated early): auth required pam_google_authenticator.so
If using SSH keys with MFA and not passwords, comment out the Un*x password include to avoid password prompts:

#@include common-auth

Notes:Using “nullok” makes the factor optional for users without enrolment; remove it later to enforce MFA globally.

Step 4 — Configure sshd to prompt for the second factor

Back up and edit SSH server config:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak && sudo vi /etc/ssh/sshd_config

Ensure these are set:

UsePAM yes
ChallengeResponseAuthentication yes

If using SSH keys + MFA, explicitly require both:

AuthenticationMethods publickey,keyboard-interactive

Save and restart SSH:

sudo systemctl restart sshd.service

Step 5 — Test login in a second terminal

Connect and verify that after key authentication (or in addition to password if configured), SSH prompts for a “Verification code”.
Use -v for verbose SSH output to confirm publickey then keyboard‑interactive flow.

Best Practices and Recovery:

Keep a break‑glass account or console access before enforcing MFA server‑wide to prevent lockouts during testing.
Use “nullok” initially if migrating gradually, then remove it to mandate MFA for all users once enrolled.
Store emergency scratch codes securely in a password manager or vault.
For key+MFA deployments, use AuthenticationMethods publickey,keyboard-interactive to cryptographically require both factors.

Conclusion:
Adding MFA to SSH on Ubuntu significantly elevates security by combining something possessed (TOTP code) with something known or inherent (SSH key or password). The TOTP route via libpam‑google‑authenticator is quick and widely compatible. With PAM and a few sshd settings, Ubuntu delivers a robust MFA setup that meaningfully reduces the risk of unauthorized SSH access.

Let's Connect!

VPC Security: Building Fortress-Like Network Architecture

Rahul Kumar Sharma — Sat, 19 Jul 2025 15:22:03 +0000

In the ever-evolving landscape of cloud security, our Amazon Virtual Private Cloud (VPC) serves as the foundation of our network defence strategy. I've learned that VPC security isn't just about checking boxes—it's about building a digital fortress that actually works under pressure.

This comprehensive guide will walk you through advanced VPC security configurations that transform our cloud infrastructure into an impenetrable network architecture.

Why Your Current VPC Security Probably Isn't Enough ?

Most of the time we make the same mistake: set up basic security groups, maybe throw in a NACL or two, and call it secure. Then reality hits. A misconfigured application suddenly needs database access. A new microservice requires communication with three other services. Before you know it, security rules become a tangled mess of "temporary" fixes that somehow became permanent.

A well-architected VPC security strategy operates on multiple layers of defence, each serving a specific purpose in our overall security posture. Unlike traditional on-premises networks, AWS VPCs offer unprecedented granular control over network traffic, but this power comes with the responsibility of proper configuration.
The core principle of VPC security revolves around the concept of "default deny"—nothing should be allowed unless explicitly permitted. This approach ensures that even if one security control fails, multiple layers of protection remain in place to safeguard our resources.

Network Segmentation: The Foundation of Fortress Architecture

Multi-Tier Subnet Strategy

The cornerstone of robust VPC security lies in strategic subnet segmentation. Design your network with clear boundaries between different application tiers and security zones.

Public Subnets should house only resources that genuinely need internet access, such as load balancers, NAT gateways, and bastion hosts. Keep these subnets minimal and heavily monitored.

Private Subnets contain your application servers, databases, and internal services. These subnets should never have direct internet access, routing outbound traffic through NAT gateways or NAT instances in public subnets.

Isolated Subnets provide the highest level of security for sensitive data stores and critical infrastructure components. These subnets have no route to the internet gateway, ensuring complete isolation from external networks.

Cross-VPC Communication Patterns

For organizations managing multiple VPCs, implement secure communication patterns using VPC peering or Transit Gateway. Design these connections with specific route tables that limit cross-VPC traffic to only necessary communication paths.

Consider implementing a hub-and-spoke model where a central security VPC manages shared services like DNS resolution, centralized logging, and security monitoring. This approach provides better visibility and control over inter-VPC traffic flows.

Security Groups: Your Application Firewall

Security Groups function as stateful firewalls operating at the instance level. Their stateful nature means that return traffic for allowed inbound connections is automatically permitted, simplifying rule management while maintaining security.

Advanced Security Group Patterns

Layered Security Groups involve applying multiple security groups to a single instance, each serving a specific purpose. For example, one security group might handle SSH access for administrators, while another manages application-specific ports. This modular approach enhances maintainability and reduces the risk of overly permissive rules.

Reference-Based Rules leverage security groups as sources and destinations rather than IP ranges. This approach creates dynamic relationships that automatically adapt as instances are launched or terminated within referenced security groups. For instance, allow database access only from instances in the "application-tier" security group.

Port Range Minimization requires opening only the specific ports your applications need. Avoid common shortcuts like opening ranges 1-65535 or using 0.0.0.0/0 as source unless absolutely necessary for public-facing services.

Security Group Governance

Implement naming conventions that clearly indicate purpose and ownership. Use tags to categorize security groups by environment, application, and responsible team. Regular audits should identify unused or overly permissive security groups, with automated tools flagging rules that deviate from established baselines.

Network Access Control Lists: The Perimeter Defence

NACLs operate at the subnet level, providing stateless filtering that evaluates both inbound and outbound traffic independently. This stateless nature requires explicit rules for both directions of communication, offering more granular control but requiring careful planning.

Strategic NACL Implementation

Default Deny Approach starts with NACLs that deny all traffic, then explicitly allows necessary communication. This approach ensures that forgotten or misconfigured rules don't create security gaps.

Ephemeral Port Management requires understanding how your applications communicate. Since NACLs are stateless, you must account for return traffic on ephemeral ports (typically 1024-65535 for Linux, 49152-65535 for Windows). Consider using more restrictive ranges based on your operating system and application requirements.

DDoS and Attack Mitigation can be enhanced through NACL rules that block known malicious IP ranges, suspicious traffic patterns, or protocols not used by your applications. While not a complete DDoS solution, NACLs provide an additional layer of filtering.

NACL Rule Ordering and Optimization

NACL rules are processed in numerical order, with lower numbers taking precedence. Design your rule numbering system with gaps (100, 200, 300) to allow future rule insertion without renumbering. Place more specific rules before general ones to ensure proper traffic handling.

Advanced Network Isolation Strategies

VPC Endpoints for Service Communication

VPC Endpoints enable private communication with AWS services without traversing the public internet. Gateway endpoints for S3 and DynamoDB provide secure, cost-effective access to these services. Interface endpoints, powered by AWS PrivateLink, support numerous AWS services and third-party applications.

Implement endpoint policies to control which resources can access specific services through the endpoint. These policies add another layer of access control beyond IAM permissions, ensuring that network-level restrictions complement identity-based controls.

DNS Security and Resolution

Configure VPC DNS settings to prevent DNS-based data exfiltration and ensure proper name resolution. Enable DNS resolution and DNS hostnames for your VPC to support proper functionality of AWS services and applications.

Consider implementing Route 53 Resolver for hybrid environments, allowing secure DNS resolution between on-premises networks and your VPCs. Create resolver rules that direct specific domains to appropriate DNS servers while maintaining security controls.

Flow Logs for Network Visibility

VPC Flow Logs capture network traffic metadata, providing crucial visibility into communication patterns and potential security issues. Configure flow logs at the VPC, subnet, and network interface levels to capture comprehensive traffic information.

Analyze flow logs to identify unusual traffic patterns, unauthorized communication attempts, and optimization opportunities. Integrate flow log data with security information and event management (SIEM) systems for real-time monitoring and alerting.

Monitoring and Threat Detection

Network-Level Security Monitoring

Deploy AWS GuardDuty to leverage machine learning for threat detection across your VPC infrastructure. GuardDuty analyzes VPC Flow Logs, DNS logs, and CloudTrail events to identify malicious activity, compromised instances, and reconnaissance attempts.

Implement custom CloudWatch metrics and alarms for network-related security events. Monitor metrics such as rejected connections, unusual traffic volumes, and connections from unexpected geographic locations.

Automated Response and Remediation

Create automated response workflows using AWS Lambda and CloudWatch Events to respond to security incidents. For example, automatically isolate compromised instances by modifying their security group rules or launching replacement instances in clean subnets.

Develop runbooks for common network security scenarios, including DDoS response, compromise containment, and emergency access procedures. Test these procedures regularly to ensure effectiveness during actual incidents.

Implementation Best Practices and Common Pitfalls

Planning and Design Phase

Begin with a comprehensive network design that accounts for current needs and future growth. Map out traffic flows between application tiers, external dependencies, and administrative access requirements. This upfront planning prevents security gaps and performance issues later.

Document your security group and NACL strategy, including naming conventions, rule approval processes, and regular review schedules. Maintain an inventory of network resources with clear ownership and purpose documentation.

Common Configuration Mistakes

Avoid overly permissive rules that grant broader access than necessary. Regularly audit rules that use 0.0.0.0/0 as a source or destination, ensuring they're justified and properly secured through other means.

Don't rely solely on security groups while ignoring NACLs. While security groups provide excellent instance-level protection, NACLs offer additional subnet-level controls that can prevent lateral movement and provide defence in depth.

Continuous Improvement and Maintenance

Establish regular review cycles for all network security configurations. As applications evolve and new threats emerge, your network security posture must adapt accordingly. Use infrastructure as code tools like CloudFormation or Terraform to maintain consistent, version-controlled network configurations.

Implement security scanning and compliance checking tools that continuously monitor your VPC configuration against security best practices and regulatory requirements. Address identified issues promptly and update your baseline configurations to prevent similar problems in the future.

Conclusion: Building Your Network Fortress

Creating a fortress-like VPC architecture requires careful planning, layered security controls, and ongoing vigilance. By implementing comprehensive network segmentation, properly configured security groups and NACLs, advanced isolation strategies, and robust monitoring, you create a network infrastructure that can withstand sophisticated attacks while supporting your business objectives.

Remember that network security is not a one-time configuration but an ongoing process of monitoring, analysis, and improvement. The techniques and strategies outlined in this guide provide the foundation for a secure VPC architecture, but they must be adapted to your specific requirements and regularly updated as threats evolve.

Your network is only as strong as its weakest component. By applying these advanced VPC security practices consistently across your entire infrastructure, you build not just a fortress, but an intelligent, adaptive defence system capable of protecting your most valuable digital assets in the cloud.

Let's Connect!

Reliable S3 Data Replication: Automatically Mirror New Files Without Worrying About Deletions

Rahul Kumar Sharma — Sat, 17 Aug 2024 07:58:16 +0000

Ensuring the security and redundancy of your data is crucial in today's data-driven environment. Duplicating your data over several storage places is a good approach to protect it. We'll discuss how to automatically copy data from a primary S3 bucket to a backup bucket in this blog article. This will make sure that your important files are always safe and backed up, even in the event that the primary bucket is filled with garbage.

A primary S3 bucket, event notifications, an AWS Lambda function, and a backup S3 bucket are the main elements of the architecture. These elements cooperate in the following ways to provide smooth data replication:

Primary Bucket: You upload your stuff to the primary S3 bucket here. An event notice is sent whenever a new file is added to this bucket. Although it is merely a portion of the solution, this bucket serves as the source of truth for your data.
Event Notification: You can set up AWS S3 to send out notifications for particular events, such the uploading of a new file. Here, the event notification is configured to be triggered each time a file is added to the main bucket. This notice initiates the subsequent stage in the process; it doesn't carry out any action on its own.
AWS Lambda Function: An AWS Lambda function is automatically called when an event notification is received. A little serverless piece of code that executes in response to events is called a lambda function. The freshly uploaded file is copied from the primary bucket to the backup bucket by this function in our setup. The backup bucket is always updated with the most recent files thanks to this nearly fast process.
Backup Bucket: The replicated files are kept in the backup S3 bucket. The backup bucket is set up to keep files even when they are removed from the primary bucket, in contrast to the latter. This implies that the backup copy stays secure and undamaged in the backup bucket even if a file is unintentionally or purposely deleted from the primary bucket.

How S3 Replication Work?

Source:AWS

Why do we need this?

Data Redundancy: Every file uploaded to the primary bucket is immediately replicated to a backup site thanks to this architecture. This redundancy, which offers a backup copy of your data that you can rely on in the event that the primary bucket's data is lost, is essential for disaster recovery and data security.
Protection Against Deletion: The backup bucket does not synchronize deletions, which is one of this setup's most notable characteristics. A file stays in the backup bucket even after it is removed from the primary bucket. Because you can always restore the file from the backup bucket, this is especially helpful in preventing inadvertent data loss.

Steps:
Step1: Create Two S3 Buckets: primary-bucket and backup-bucket.
Step2: Create an IAM Role for the Lambda Function

Go to the IAM service in the AWS Management Console.
Click on "Roles" in the left-hand menu, then "Create role".
Select "AWS service" and choose "Lambda".
Click "Next: Permissions".
Click "Create policy" and go to the JSON tab.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::primary-bucket-7",
        "arn:aws:s3:::primary-bucket-7/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::backup-bucket-7/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "sns:Publish"
      ],
      "Resource": "arn:aws:sns:ap-south-1:965519929135:s3DataBackUpSNS"
    }
  ]
}

Attack the policy to Lambda role.

Step3: Create an SNS Topic and Subscribe Your Email.
Step4: Create a Lambda function.

import boto3
import urllib.parse

s3 = boto3.client('s3')
sns = boto3.client('sns')

def lambda_handler(event, context):
    for record in event['Records']:
        source_bucket = record['s3']['bucket']['name']
        source_key = urllib.parse.unquote_plus(record['s3']['object']['key'])
        destination_bucket = 'backup-bucket-7'
        copy_source = {'Bucket': source_bucket, 'Key': source_key}

        try:
            # Copy the object to the backup bucket
            s3.copy_object(CopySource=copy_source, Bucket=destination_bucket, Key=source_key)
            print(f'Successfully copied {source_key} from {source_bucket} to {destination_bucket}')

            # Send SNS notification
            sns.publish(
                TopicArn='arn:aws:sns:ap-south-1:965519929135:s3DataBackUpSNS',
                Subject='File Uploaded to Backup Bucket',
                Message=f'The file {source_key} has been successfully uploaded to {destination_bucket}.'
            )
            print(f'Successfully sent SNS notification for {source_key}')
        except Exception as e:
            print(f'Error copying {source_key} from {source_bucket} to {destination_bucket}: {str(e)}')
            raise e

The source bucket in the Lambda function is dynamically determined based on the event that triggers the function. This means that we don't need to hard-code the source bucket name in the Lambda function code. Instead, the source bucket is extracted from the event record whenever a new object is uploaded to the primary-bucket.

Step5: Configure S3 Event Notifications for the Primary Bucket

Go to the S3 service in the AWS Management Console.
Select the primary-bucket.
Go to the "Properties" tab.
Scroll down to "Event notifications" and click "Create event notification".
Configure the event to trigger on s3:ObjectCreated:* and select the Lambda function S3ReplicationFunction.
Save the event notification.

Step6: Configure S3 Event Notifications for the Backup Bucket

Go to the S3 service in the AWS Management Console.
Select the backup-bucket.
Go to the "Properties" tab.
Scroll down to "Event notifications" and click "Create event notification".
Configure the event to trigger on s3:ObjectCreated:* and select the same Lambda function S3ReplicationFunction.
Save the event notification.

Happy Blogging!

Let's Connect

Simplify EC2-S3 File Access with Instance Roles

Rahul Kumar Sharma — Sun, 07 Jul 2024 15:47:38 +0000

Access all the buckets:

Create an IAM Role for the EC2 Instance:

Go to the IAM console in AWS and create a role.
Select "AWS service" as the trusted entity and choose "EC2." Click "Next: Permissions."
Attach the policy “AmazonS3ReadOnlyAccess” to access the S3 bucket.
Click "Next: Tags" (optional) and then "Next: Review."
Give the role a name and click "Create role."

Attach the IAM Role to the EC2 Instance:
Go to the EC2 console.

Select the instance that you want to grant S3 access.
Click on the "Actions" button, navigate to "Security" and then "Modify IAM Role."
Choose the IAM role you created in the previous step and click "Update IAM role."

Testing:

SSH into the instance to verify.
Install awscli into the instance.

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"  
unzip awscliv2.zip  
sudo ./aws/install

Access specific S3 bucket:

Create a Custom Policy for S3 Access:

Click "Create policy" to define a custom policy that grants list access to all S3 buckets and read access to a specific S3 bucket.
Click "JSON" and paste the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}

Create a New Role:

Click on "Roles" in the left sidebar, then click "Create role."
Select "AWS service" as the trusted entity type.
Choose "EC2" under the "Use case" section, then click "Next” and attach the policy which you created.

Attach the IAM Role to the EC2 Instance:

Go to the EC2 console.
Select the instance that you want to grant S3 access.
Click on the "Actions" button, navigate to "Security" and then "Modify IAM Role."
Choose the IAM role you created in the previous step and click "Update IAM role."

Testing:

Let's Connect!

Automated AWS Bill Reminder System using Serverless Architecture with AWS Lambda

Rahul Kumar Sharma — Sat, 23 Mar 2024 20:52:08 +0000

The aim is to develop a serverless system using AWS Lambda to notify users when their AWS bill is pending, to ensure timely payment and avoid disruptions to AWS services.

Components:

AWS Lambda Functions: Use Lambda functions to periodically check the status of AWS billing and send notifications to users when their bills are pending.
AWS Billing and Cost Management: Utilize AWS Billing and Cost Management services to retrieve billing information and check the status of pending bills.
Amazon SNS: Use Amazon Simple Notification Service (SNS) to send email notifications to users when their AWS bills are pending.
AWS CloudWatch Events: Set up CloudWatch Events to trigger Lambda functions at regular intervals for checking the billing status.
AWS SDKs: Leverage AWS SDKs (e.g., Boto3 for Python) to interact with AWS services programmatically and automate billing notifications.

Steps to Implement:

Set up IAM Roles:

To send notifications via Amazon SNS and use AWS Billing and Cost Management services, create an IAM role with the necessary permissions.

Write Lambda Function:

Develop a Lambda function that retrieves billing information using the AWS SDK.
Implement logic to check if the bill is pending and trigger a notification if necessary.

Configure CloudWatch Event Rule:

Set up a CloudWatch Event rule to trigger the Lambda function at regular intervals (e.g., daily, weekly) to check the billing status.

Implement Notification Logic:

Use the SNS service to send email notifications to users when their AWS bills are pending.
Customize the notification message with relevant billing details and instructions for payment.

Handle Error Cases:

Incorporate error handling into the Lambda function to handle exceptions, including failed service requests and incorrectly retrieved billing data.
Configure appropriate logging and monitoring using CloudWatch to track function executions and errors.

Testing:

Test the Lambda function and CloudWatch Event rule to ensure they trigger notifications correctly based on the billing status.
Verify that users receive notifications as expected and that the notification content is accurate.

Deployment:

Deploy the Lambda function and CloudWatch Event rule in the AWS account where billing notifications are required.
Configure necessary permissions and IAM roles for the Lambda function to access AWS Billing and SNS services.

Monitor and Maintain:

Monitor the execution of Lambda functions and CloudWatch Events to ensure they run as scheduled and handle any failures or exceptions promptly.
Periodically review and update the system to accommodate changes in AWS billing policies or user requirements.

Lambda Function:

import json
import boto3

def lambda_handler(event, context):
    # Initialize AWS services
    sns_client = boto3.client('sns')
    billing_client = boto3.client('ce')  # AWS Cost Explorer

    # Retrieve billing information
    response = billing_client.get_cost_and_usage(
        TimePeriod={
            'Start': '2024-01-01',
            'End': '2024-03-01'  # Modify the date range as needed
        },
        Granularity='MONTHLY',
        Metrics=['UnblendedCost']  # You can customize the metrics as needed
    )

    # Extract the total cost for the current billing period
    total_cost = float(response['ResultsByTime'][0]['Total']['UnblendedCost']['Amount'])

    # Check if the bill is pending (total cost > 0)
    if total_cost > 0:
        # Send notification
        topic_arn = 'arn:aws:sns:ap-south-1:965519929135:awsBillNotification'  # Replace with your SNS topic ARN
        message = f'Your AWS bill for the current month is pending. Total amount due: ${total_cost:.2f}'
        subject = 'Action Required: Your AWS Bill is Pending'

        sns_client.publish(
            TopicArn=topic_arn,
            Message=message,
            Subject=subject
        )

        print('Billing notification sent successfully')
    else:
        print('No pending bills found')

    return {
        'statusCode': 200,
        'body': 'Billing notification process completed'
    }

Testing:

Output:

Find me Online!

Exploring ChatGPT's Magic in the Linux Command Line

Rahul Kumar Sharma — Thu, 07 Sep 2023 14:52:28 +0000

Integrating ChatGPT into the Linux terminal opens a world of possibilities for natural language interactions and automation. By combining the power of OpenAI's ChatGPT with the flexibility of the Linux terminal, users can harness the capabilities of this language model for a wide range of tasks and applications.

ShellGPT:

Users can communicate with the AI chatbot on their Linux terminal using ShellGPT, a command line interface for ChatGPT. It is based on the GPT large language model from OpenAI.

Based on our text input, ShellGPT can provide intelligent suggestions and recommendations and even run shell commands. Additionally, it gains knowledge from our exchanges and improves with time. Users no longer need to input lengthy commands or memorize difficult Linux Terminal commands thanks to the ChatGPT application included into the command line. By having ChatGPT handle part of their tedious labor on their behalf, they may reduce errors while saving critical time.

Prerequisites to install ChatGPT in Linux:

AWS account
Create an EC2 Instance(Ubuntu).

Steps:

a. Create an EC2 instance and SSH into it.

b. Check if Python is installed in it or not.

c. Install PIP and check the version.

d. Install venv.

e. Setup ShellGPT to use ChatGPT.

Setup environment.
Create a directory.

Create a virtual environment.

Activate the env.

f. To get OpenAI API Key, navigate to OpenAI’s website.

g. Click on profile and select View API Keys.

h. Click on Create new Secret key.

i. Copy the Key and save it.

j. Now, create an environment variable for this API key with the command below. In Linux, you can create an environment variable using the “export” command.

Replace placeholder with the actual API key you generated to use ChatGPT in the Linux terminal.

export OPENAI_API_KEY=sk-YHhk0****************RMVLVZdqQjiNZ

k. To verify if the export is successfully use env.

l. To store variable permanently, open .bashrc file and add the API key.

m. To make commands in effect, run the command: source .bashrc .

n. Install ShellGPT.

o. Now that we have install ShellGPT, we can use it.

Syntax:

Options	Description
--temperature	Changes the randomness of the output
--top-probablity	Limits to only the highest probable tokens or words
--chat	Used to have a conversation with a unique name
--shell	Used to get shell commands as output
--execute	Executes the commands received as output from --shell option
--code	Used to get code as output

In Action:

Let's Connect!

Unlocking Real-Time Insights: Harnessing Lambda and SES to Supercharge S3 Bucket Alerts

Rahul Kumar Sharma — Sat, 12 Aug 2023 20:21:30 +0000

In the age of data-driven decision-making, real-time insights play a pivotal role in ensuring the smooth operation of business processes. Amazon Simple Storage Service (S3) offers a robust storage solution, while AWS Lambda provides the power of event-driven computing. When combined with Amazon Simple Email Service (SES), this trio becomes a formidable toolset for creating a highly efficient alert mechanism. In this blog post, we will delve into the process of leveraging Lambda and SES to supercharge S3 bucket alerts, enabling you to gain real-time insights and proactively respond to critical events.

Amazon Simple Storage Service (Amazon S3) is a versatile and highly scalable cloud storage solution provided by Amazon Web Services (AWS). Designed to handle large amounts of data and seamlessly integrate with various AWS services, S3 has become a foundational component for countless applications, from simple file storage to complex data analysis and backup solutions.
S3's features and capabilities include:

Features and Capabilities of Amazon S3:

Scalability: S3 scales effortlessly to accommodate your growing storage needs. You can start small and expand to petabytes of data without any disruption.
Data Security: S3 offers various security mechanisms, including access control policies, identity and access management (IAM), and encryption options (both in transit and at rest) to keep your data safe.
Data Management: With S3's lifecycle policies, you can automatically transition objects between storage classes (Standard, Intelligent-Tiering, Glacier, etc.) based on access patterns and cost considerations.
Versioning: S3 enables you to maintain multiple versions of an object, allowing you to recover from accidental deletions or changes.
Event Notifications: S3 supports event triggers that can invoke AWS Lambda functions, enabling real-time data processing and event-driven architecture.
Cross-Region Replication: You can replicate objects across different regions for disaster recovery or to reduce latency for global users.
Data Analytics: S3 integrates seamlessly with AWS analytics services like Amazon Athena, Amazon Redshift Spectrum, and Amazon EMR for data processing and analysis.
Static Website Hosting: S3 can host static websites, allowing you to serve web content directly from your bucket.
Multipart Upload: For large files, S3 supports multipart uploads, making it efficient and resilient even in less reliable network conditions.
Data Access Control: S3 offers fine-grained control over access permissions, allowing you to specify who can access your objects and how.

AWS Lambda is a serverless compute service provided by Amazon Web Services (AWS) that enables you to run code without provisioning or managing servers. It's designed to help you build and deploy applications quickly and efficiently by allowing you to focus solely on your code and its functionality, while AWS handles the underlying infrastructure, scaling, and maintenance.

Features of AWS Lambda include:

Event-Driven Computing: Lambda functions are triggered by various events within AWS services, such as changes in data in Amazon S3 buckets, updates in DynamoDB tables, HTTP requests via Amazon API Gateway, or even custom events. This event-driven model promotes a reactive and highly scalable architecture.
Scalability: Lambda functions scale automatically in response to the volume of incoming events. This means your code can handle a few requests or millions, without manual intervention, ensuring optimal performance.
Pay-as-You-Go Pricing: With Lambda, you only pay for the compute time your code consumes. You're billed based on the number of requests and the time it takes for your code to execute. This cost-effective model eliminates the need to pay for idle server time.
Language Flexibility: AWS Lambda supports a variety of programming languages, including Python, Node.js, Java, C#, and more, allowing developers to use their preferred language for building functions.
Stateless Execution: Lambda functions are stateless, meaning they don't retain data between executions. This encourages the use of external data stores (like databases or Amazon S3) for persisting data.
Integration with AWS Services: Lambda can easily integrate with other AWS services, enabling you to create powerful workflows and automation. For example, you can process data, generate reports, or trigger actions across various services.
Customizable Configuration: You can configure various parameters for your Lambda functions, including memory allocation, timeout duration, and environment variables. This allows you to optimize performance and resource usage.
Versioning and Aliases: Lambda supports versioning and aliases, allowing you to manage and deploy different versions of your code and direct traffic to specific versions.
Testing and Debugging: AWS provides tools and resources for testing and debugging Lambda functions locally before deploying them to the cloud.
Security and Access Control: Lambda integrates with AWS Identity and Access Management (IAM) for fine-grained control over who can invoke your functions and what resources they can access.

Amazon Simple Email Service (Amazon SES) is a cloud-based email-sending service provided by Amazon Web Services (AWS). It offers a reliable, scalable, and cost-effective solution for sending transactional, marketing, and notification emails, allowing businesses to communicate effectively with their customers and users.

Features of Amazon SES include:

Email Sending: Amazon SES enables you to send a wide range of email types, including one-to-one transactional emails, marketing campaigns, and automated notifications. It supports both HTML and plain text emails.
High Deliverability: Amazon SES leverages Amazon's proven email infrastructure to help ensure that your emails are delivered to the recipient's inbox. It provides tools for monitoring and improving email deliverability.
Scaling and Resilience: SES automatically scales to handle large volumes of email traffic, making it suitable for businesses of all sizes. It also offers built-in redundancy and fault tolerance.
Flexible Integration: Amazon SES integrates seamlessly with other AWS services, making it easy to incorporate email functionality into your applications. It can be used in conjunction with AWS Lambda for event-driven email sending.
Customizable Templates: You can create and manage email templates in Amazon SES, allowing you to maintain consistent branding and design across your email communications.
Bounce and Complaint Handling: SES provides feedback loops to help you manage bounces and complaints, allowing you to maintain a clean and engaged recipient list.
Content Filtering: SES includes options for content filtering, allowing you to add custom headers or apply rules to filter out certain types of content.
Sending Statistics: SES provides detailed sending statistics, including delivery rates, bounce rates, and complaint rates, which can help you monitor and improve your email campaigns.
Cost-Efficiency: Amazon SES offers pay-as-you-go pricing, meaning you only pay for the emails you send. It provides a cost-effective alternative to building and maintaining your email infrastructure.
Data Security and Compliance: SES offers encryption options for data in transit and at rest. It also provides mechanisms to comply with email-related regulations, such as CAN-SPAM and GDPR.

Architecture Diagram

Steps to implement this:
One AWS Console in your browser, once you are logged in.

Create an S3 bucket.

Create an IAM role to perform this.
Set up an SES domain or email, verify the identities before we move to next step.
Create a lambda function and choose a runtime, I will be using python3.7.
Set up an S3 bucket trigger for the lambda function. This trigger will execute the lambda function, whenever any object is uploaded in the S3 bucket.

Add the below code in a lambda file, which consists of HTML, CSS, and Python to send mail.

import json
import boto3
import os

ses_client = boto3.client('ses')

def send_email(bucket, key, recipient, event_type, sender_name, sender_email):
    sender = f'{sender_name} <{sender_email}>'
    subject = f'S3 Event Notification - {event_type}'
    body = f"""
        <html>
        <head>
        </head>
        <body style="text-align: left; font-family: Arial, sans-serif; color: black; background-color: #e3f2fd; padding: 5%">
            <h1 style="color: red; text-align: center">S3 Alert!</h1>
            <p>Hello {recipient},</p>
            <p>A {event_type} event occurred in the bucket {bucket}.</p>
            <p>File key: {key}</p>
        </body>
        </html>
    """
    response = ses_client.send_email(
        Source=sender,
        Destination={
            'ToAddresses': [recipient],
        },
        Message={
            'Subject': {
                'Data': subject,
            },
            'Body': {
                'Html': {
                    'Data': body,
                },
            },
        }
    )
    print('Email sent to:', recipient)

def lambda_handler(event, context):
    recipients = ['user1@gmail.com', 'user2@gmail.com']
    sender_name = os.environ['SENDER_NAME']
    sender_email =  os.environ['SENDER_EMAIL']
    try:
        for record in event['Records']:
            bucket = record['s3']['bucket']['name']
            key = record['s3']['object']['key']
            event_type = record['eventName']

            for recipient in recipients:
                send_email(bucket, key, recipient, event_type, sender_name, sender_email)
        return {
            'statusCode': 200,
            'body': json.dumps('Emails sent successfully')
        }

    except Exception as e:
        print('Error sending emails:', e)
        return {
            'statusCode': 500,

            'body': json.dumps('Error sending emails')

        }

Select Configuration in the Lambda function and add the sender’s name and email.

To test it, upload a file into the bucket, you will receive a mail.

You can connect with me.