DEV Community: Wonder Agudah

Analysing Drivers of Digital Transformation in Corporate Innovation Capacity Using Amazon SageMaker Studio and Kaggle API

Wonder Agudah — Tue, 30 Dec 2025 14:46:56 +0000

The discourse on digital transformation has received a myriad of contributions on how the advent of technological tools, cloud computing, a culture of digitalisation, and especially in recent times; how the adoption of Artificial Intelligence(AI) can be used to drive performance, innovation, growth and development in business organisations and corporate entities in general. Economists, social scientists, business and technological analysts have propounded and relied on a number of key indicators to evaluate the phenomenon of Corporate Digital Transformation. This includes metrics that span across the spectrum of operations, innovation, and organisational structure to mention but a few.

An example of a dataset that captures these metrics will be extracted from Kaggle(a data science and machine learning community platform) to conduct a data ingestion and analysis project using python programming language and a ML service offered by AWS known as Amazon Sagemaker.

The measure of the value and importance of data analysis and visualization is not just restricted to gathering information and insights on the data. These insights can drive innovation by tools such as AI & ML to train models that can make important predictions based on the data that was analyzed. A useful service offered by AWS that helps to achieve this is Sagemaker. Amazon Sagemaker is a managed ML service offered by AWS that is used to test, train and build models in the cloud. The essence of performing data analysis and visualisation in an Amazon sagemaker notebook is that it presents an opportunity to have a central hub for your data analytics workload to also be leveraged by ML engineers to train and build models that can make insightful predictions based on this data. To get started with creating a Sagemaker notebook, follow this steps provided in the Sagemaker developer guide:

docs.aws.amazon.com

NB: In the configuration window of your sagemaker notebook instance. Select the VPC option with an internet gateway attached to enable internet access for the underlying EC2 instance to have access to the kaggle api to pull the dataset used in this project. However, it’s important to note that for security purposes and unique workload requirements use the No VPC, only VPC options to prevent exposure from external traffic.

Additionally, you must have an account on Kaggle to generate API access keys to pull the dataset that I will use for the demonstration in the notebook below.

This simple analysis of digital transformation metrics dataset is just a brief demonstration of how the integration of kaggle into your Amazon Sagemaker studio notebook can aid data ingestion and analysis, by having ready access to a plethora of datasets to conduct any data analysis project and also train and build models that can make valuable predictions that your project requires.

Utility Scripts to Expedite Machine Learning(ML) Adoption Leveraging AWS ML Service APIs.

Wonder Agudah — Tue, 07 Jan 2025 17:25:26 +0000

This article seeks to articulate a number of common machine learning features and capabilities that can be integrated into applications, data pipelines, business intelligence tools and many other projects depending on your peculiar use case by leveraging utility scripts that invoke the Application Programming Interface(API) of managed ML services provided by AWS.

Prerequisites:

AWS credentials: Configure AWS Access Keys via the Command Line Interface(CLI) >> Environment Variables can equally be utilised to ensure secure and persistent access to your AWS account.
AWS Software Development Kit(SDK) for Python: Boto3 pip install boto3

Utility scripts can be very useful when considering an idea to test and develop for a specific feature of a program without necessarily having a full stack application developed yet. Some utility scripts to perform cloud operations can be found in this GitHub repository which I compiled for both Python and Bash scripts:

Wonder717 / Utility-Scripts

Below are some utility scripts to implement specific machine learning features that you can use in your projects.

A. Language Translation Utility

Use Case: Translate text files or strings into various languages.

AWS Service: Amazon Translate

Implementation:
Automate the translation of text or files into target languages.

Utility Script:

import boto3

translate = boto3.client('translate')

text = "Utility Scripts to Expedite Machine Learning(ML) Adoption Leveraging AWS ML Service APIs.
"
response = translate.translate_text(
    Text=text,
    SourceLanguageCode='en',
    TargetLanguageCode='fr'
)
print(response['TranslatedText'])  
# Output: 'Scripts utilitaires pour accélérer l'adoption de l'apprentissage automatique (ML) en tirant parti des API de service AWS ML.'

B. Text/Data Extraction Utility

Use Case: Extract key data from invoices (Example: invoice numbers, totals) and classify them into categories for financial processing.

AWS Service: Amazon Textract is a machine learning service offered by AWS that reduces the manual efforts and automates the process involved in the extraction of any kind of data such as forms, tables, and texts from scanned documents, making it expedient to derive important information from different sources. I wrote an article that explores this service in detail.Check it out here:

Implementing an Organisational Cloud Resource Tagging Strategy Using Amazon Textract, AWS Lambda and Boto3.

Wonder Agudah for AWS Community Builders ・ Feb 29 '24

#machinelearning #aws #community #lambda

Utility Script:

import boto3
import os
import csv
import re

s3 = boto3.client('s3')
textract = boto3.client('textract')

def upload_files_to_s3(folder, bucket_name):
    for root, _, files in os.walk(folder):
        for file in files:
            if file.endswith(('.pdf', '.jpg', '.png')):
                filepath = os.path.join(root, file)
                s3.upload_file(filepath, bucket_name, file)

def extract_text(bucket_name, document_name):
    response = textract.analyze_document(
        Document={'S3Object': {'Bucket': bucket_name, 'Name': document_name}},
        FeatureTypes=['FORMS']
    )
    return response

def parse_invoice_data(textract_response):
    data = {}
    for block in textract_response['Blocks']:
        if block['BlockType'] == 'KEY_VALUE_SET' and block['EntityTypes'] == ['KEY']:
            key = ''.join([
                item['Text']
                for item in block.get('Relationships', [])
                if 'Text' in item
            ]).strip()

            if key in ['Invoice Number', 'Total', 'Date']:
                data[key] = ''.join([
                    item['Text']
                    for item in block.get('Relationships', [])
                    if 'Text' in item
                ]).strip()
    return data

def save_to_csv(data, csv_file):
    with open(csv_file, 'a', newline='') as csvfile:
        writer = csv.writer(csvfile)
        for item in data:
            writer.writerow([item['Invoice Number'], item['Date'], item['Total']])

# Main function
def main():
    folder_path = './invoices'
    bucket_name = 'your-invoice-bucket'
    csv_file = 'invoice_data_output.scv'

         # Upload files to S3
         upload_files_to_s3(folder_path, bucket_name)

         # Process each file in the S3 bucket
         processed_data = []
         for obj in s3.list_objects_v2(Bucket=bucket_name).get('Contents', []):
             file_name = obj['Key']
             print(f"Processing file: {file_name}")

             # Extract text using Textract
             textract_response = extract_text(bucket_name, file_name)

             # Parse relevant data
             parsed_data = parse_invoice_data(textract_response)
             if parsed_data:
                 processed_data.append(parsed_data)

         # Save extracted data to CSV
         save_to_csv(processed_data, csv_file)
         print(f"Processed data saved to {csv_file}")

     if __name__ == "__main__":
         main()

C. Text Classification and Analysis Utility

Use Case: Analyze text data for sentiment and language detection.

AWS Service: Amazon Comprehend

Utility Script:

import boto3

comprehend = boto3.client('comprehend')

text = "Education is necessary for development!"
response = comprehend.detect_sentiment(Text=text, LanguageCode='en')
print(response['Sentiment'])  

# Output: 'POSITIVE'

D. Image Classification and Processing Utility
Use Case: Automate image analysis for object detection and facial recognition.

AWS Service: Amazon Rekognition

Utility Script:

import boto3
import os
import csv

s3 = boto3.client('s3')
rekognition = boto3.client('rekognition')

def upload_to_s3(folder, bucket_name):
    for root, _, files in os.walk(folder):
        for file in files:
            if file.endswith(('.png', '.jpg', '.jpeg')):
                filepath = os.path.join(root, file)
                s3.upload_file(filepath, bucket_name, file)

def label_images(bucket_name, output_csv):
    with open(output_csv, 'w', newline='') as csvfile:
        writer = csv.writer(csvfile)
        writer.writerow(['Image', 'Labels'])

        for obj in s3.list_objects(Bucket=bucket_name).get('Contents', []):
            response = rekognition.detect_labels(
                Image={'S3Object': {'Bucket': bucket_name, 'Name': obj['Key']}},
                MaxLabels=10
            )
            labels = [label['Name'] for label in response['Labels']]
            writer.writerow([obj['Key'], ', '.join(labels)])

folder_path = './images'
bucket_name = 'your-bucket-name'
output_csv = 'image_labels.csv'

upload_to_s3(folder_path, bucket_name)
label_images(bucket_name, output_csv)
print(f"Labels stored in {output_csv}")

The utility scripts in this article are by no means conclusive! Consider them a starting point to explore different managed machine learning services offered by AWS that you can integrate into your applications and projects. You can go through the official documentation for Boto3:

docs.aws.amazon.com

to explore more code snippets you can customise and tailor to your unique case scenarios for implementing machine learning capabilities by leveraging AWS managed ML services.

Architecting a Notification Platform on AWS and how to Implement Personalization of Delivery Channels

Wonder Agudah — Sat, 14 Dec 2024 22:19:03 +0000

Design Overview:

This design serves as a simple and efficient architecture for a standard notification platform that can be expanded during the lifecycle of the system, as and when extra specific, use case requirements are introduced for further improvements.

The notification platform leverages a number of AWS serverless technologies, which can be either deployed with CloudFormation or its extension for serverless applications and resources; AWS SAM.

The entry point to this system is designed with API Gateway and interfaced by AWS WAF to filter incoming requests by providing protection against common web attacks based on defined conditions. Filtered requests are then authenticated by API Gateway through Cognito for external services and via IAM for services that might be accessing the platform internally.

The design module employed for the notification platform leverages an event driven architecture(EDA)—in effect, all authenticated requests call an API Gateway endpoint that sets the system in motion by either invoking a lambda function for individual requests or routing batch requests through SQS. The notification gateway as indicated in the attached architectural diagram is responsible for this initial process of handling individual and batch requests. In the event, where real time data processing of notifications are required, Amazon Kinesis can be leveraged instead.

The preliminary requests(individual and batch) made in the notification gateway are then routed accordingly using SNS and SQS to the notification processing and distribution service where AWS Lambda functions are invoked and tasked with custom business logic to validate, format and schedule notification content, as well as to interact with a DynamoDB table that stores contact information and delivery channel preferences of users(subscribers).

After the notification distribution service is done executing, an event bus is provisioned with Amazon EventBridge to route processed notifications using event rules that push these notification events to queues designated for each delivery channel to be polled and processed in concurrency. Each
notification event will be tagged with a specific category for respective delivery channels in order to be routed to its corresponding queue. A number of Lambda functions here will be provisioned to run specific delivery jobs that poll their assigned queues and fetch notification templates from a S3 bucket
to forward notifications to integrated delivery channels configured with the subscriptions of consumers. Amazon SES will be used for delivering emails, while Amazon SNS will be used for both SMS and mobile push notifications. A fourth queue and delivery job to be customised is provided to interact with the API of any third party service channel that needs to be connected in the future. Amazon Pinpoint can be used to replace these services if campaigns and a more interactive session is required with the users to be notified.

The status of notification delivery jobs(sent/failed) as well as delivery channels(delivered/failed) will be logged into a DynamoDB table.

CloudWatch Metrics will be collected and analysed from all applicable services.

The following points are some features of the AWS services used that ensure non-functional requirements of the notification platform are met

Scalability
● Amazon S3: Provides virtually unlimited storage for logs, tracking data, and notification
templates, automatically scaling as data grows.
● Amazon SQS/SNS: Scales to handle millions of messages per second.
● AWS Lambda: Scales by creating instances for high concurrency workloads.
● Amazon DynamoDB: On-demand scaling for unpredictable workloads.
● Amazon Kinesis: Shard-based horizontal scaling for real-time data streams.

High Availability
● Amazon S3: Designed for 99.99% availability, with data replication across multiple AZs.
● SQS, SNS, DynamoDB: Multi-AZ replication ensures fault tolerance.
● AWS Lambda: Auto-retries failed executions and operates across multiple AZs.
● Amazon Pinpoint: Multi-AZ operations for continuous message delivery.

Security
The recent feature of API gateway to provide support for TLS(API Gateway now supports TLS 1.3) satisfies the requirement for data encryption in transit. Integration with Amazon Cognito will ensure authentication and authorization is maintained by verifying access for services that need to access the platform using their application users directory(user pool) or by leveraging an identity provider(identity pools) IAM entities, specifically roles can also be assigned to internal services for authentication. AWS services roles will be capitalised with specific policies to provide limited access for specific operations and API calls in a bid to strengthen the security posture of the platform.
● Encryption in Transit:
● All services, including S3, SQS, SNS, API Gateway, and Lambda, support TLS
(Transport Layer Security) to encrypt data during transmission.
● Amazon S3:
● Bucket Policies and IAM: Restrict access to sensitive data like logs and templates.
● Encryption at Rest: Uses server-side encryption (SSE-S3, SSE-KMS) or client-side
encryption.
● IAM Roles/Policies: Fine-grained access control ensures secure access to resources.
● API Gateway Security: Supports Cognito, IAM, and custom Lambda authorisers for
authentication.
● AWS WAF: Protects APIs from DDoS, SQL injection, and other threats.

Monitoring and Logging
● Amazon CloudWatch:
● Metrics: Monitors key metrics such as SQS queue depth, SNS delivery success
rates, Lambda execution times, DynamoDB throughput, and API Gateway requests.
● Alarms: Sends alerts for anomalies or performance issues (e.g., high queue backlog,
failed deliveries).
● Service Lens: Provides an integrated view of performance and availability across
services, combining metrics, logs, and traces for end-to-end observability.
● AWS X-Ray:
● Traces requests through the system to identify performance bottlenecks and
dependencies.
● Provides detailed insights into latencies, errors, and service interactions.
● CloudWatch Logs:
● Aggregates runtime logs from Lambda, API Gateway, and other services.
● Supports detailed querying for debugging and analysis.
● CloudTrail:
● Tracks all API actions for audit and compliance, ensuring a secure and traceable
system.
● Dead Letter Queues (DLQs):
● Captures failed or undeliverable messages from SQS or SNS for analysis and
troubleshooting.
● Amazon DynamoDB:
● The status of notification delivery jobs(sent/failed) as well as delivery
channels(delivered/failed) will be logged into a DynamoDB table.
● Amazon S3:
● Stores logs, archived notifications, and tracking data for long-term analysis and
compliance reporting.

The notification system can be enhanced with machine learning features using AWS ML services to optimise operations, improve user experience, and enable personalisation. Here are some ML features to include, along with the AWS services that can support them:

A. Personalization

ML Feature: Predict the most effective delivery channel or time for each user.
AWS Service: Amazon Personalize
- Trains custom ML models to recommend delivery preferences based on user behaviour and history (E.g: channel preference, response patterns).

B. Intelligent Scheduling

ML Feature: Analyze user behaviour and historical engagement data to send notifications at the best time (E.g: optimal open rates).
AWS Service: Amazon SageMaker
- Build models to predict engagement and dynamically schedule notifications.
- Use Amazon Forecast for time-series predictions, such as predicting the best times for notifications.

C. Content Optimization

ML Feature: Automatically analyze and improve the effectiveness of notification content (E.g: subject lines, message body) for higher engagement.
AWS Service: Amazon Comprehend
- Perform sentiment analysis and key phrase extraction on notification content.
- Tailor content to match user preferences and sentiment.

D. Fraud Detection

ML Feature: Detect and prevent fraudulent activities or malicious notifications (E.g: spam detection or unusual activity in the system).
AWS Service: Amazon Fraud Detector
- Automatically identifies anomalies in notification delivery patterns or content based on historical data.

E. Audience Segmentation

ML Feature: Automatically segment users based on behavior, demographics, or engagement levels for targeted notifications.
AWS Service: Amazon SageMaker or Amazon Pinpoint
- Train ML models for advanced clustering or use Pinpoint's built-in segmentation features for simpler implementations.

F. Tracking and Analytics Insights

ML Feature: Use ML models to analyze delivery metrics and provide actionable insights, such as trends in user engagement or areas for improvement.
AWS Service: Amazon Lookout for Metrics
- Automatically detects anomalies in delivery rates, open rates, or click-through rates, enabling faster troubleshooting.

The following section will provide guidelines to implement personalization by sending delivery notifications via the most effective channel for each user based on their historical preferences and behaviour.

Steps to Implement Personalization

1. Collect Data

You need a dataset to train the recommendation model. Include the following attributes:

User ID: Unique identifier for the user.
Channel Used: Delivery channel.
Engagement Metrics: Open rates, click rates, or read receipts.
Timestamp: When the notification was sent.
Context Data: Device type, location, time of day, etc.

Data Source:

Export data from Amazon Pinpoint, Amazon S3 (logs), or your database.

2. Prepare and Store Data

Use AWS Glue to clean and transform the raw data.
Store the processed data in Amazon S3 as input for ML models.
Use AWS Glue Data Catalog to create metadata for structured access to your data.

3. Train a Recommendation Model

Use Amazon Personalize:
1. Create a Dataset Group: Upload your data to Amazon Personalize (user-interaction data).
2. Choose a Recipe: Select the Personalized Ranking recipe for this use case.
3. Train the Model: Use your data to create a machine learning model. Personalize automatically tunes the model using AutoML.

4. Deploy the Model

Deploy the trained model in Amazon Personalize as a real-time recommendation engine.
Generate a recommendation for each user by providing their User ID and optional context data .

5. Integrate with the Notification System

Real-Time Inference:
- Use the Amazon Personalize Runtime API to query recommendations (E.g: which channel to use).
- Example Input: User ID + Context (E.g: time of day, location).
- Example Output: Ranked list of channels (For instance, Email > SMS > Push).
Lambda Function for Decision Making:
- Trigger AWS Lambda from your notification gateway.
- The Lambda function calls the Personalize API, selects the top-ranked channel, and routes the notification appropriately.

6. Monitoring and Feedback Loop

Use Amazon CloudWatch to track the performance of key metrics of the recommendation system.
Collect feedback: Track if users engage with the suggested channel and store this data back in Amazon S3.
Regularly retrain the model in Amazon Personalize with updated interaction data for continuous improvement.

I hope this document is helpful. Please let me know if you have any other questions in the comment section. You can use the link below to optionally view this presentation on YouTube:

AWS CB Swag Kit-Year 2🎉

Wonder Agudah — Fri, 02 Aug 2024 15:21:52 +0000

My second year AWS CB swag kit arrived this week & I must say: it’s quite neat!😎 I look forward to continue my active participation and contribution in the AWS community. Special thanks to @jasondunn for the opportunity and incentives to motivate community builders. To learn more about the program, click this link https://aws.amazon.com/developer/community/community-builders/
Happy Friday Everyone & Cheers to the weekend 🎉

AWS re:Invent—All Builders Welcome Grant

Wonder Agudah — Wed, 31 Jul 2024 17:37:05 +0000

As part of efforts by AWS to reduce disparity due to financial constraints and increase accessibility towards training & educational programs offered by re:Invent—-The All Builders Welcome Grant provides an opportunity for early career technologists who are still in the first five years of their careers to be afforded an all expense paid trip that covers the essentials such as travel and accommodation costs to attend re:Invent.The deadline is on the 2nd of August so it’s not too late to put in an application!

Apply here: https://reinvent.awsevents.com/all-builders-welcome/

Implementing an Organisational Cloud Resource Tagging Strategy Using Amazon Textract, AWS Lambda and Boto3.

Wonder Agudah — Thu, 29 Feb 2024 15:00:04 +0000

Asset management is imperative in the execution of a cloud governance framework. The process of applying key value pairs known as tagging that contains important information and constitutes the metadata of resources in a cloud environment(An AWS account in this case) helps achieve this goal. Services such as Tag Editor and Resource Groups in AWS offer the ability to perform this action(tagging). However, this process can quickly get cumbersome. Especially when there is a large number of tags to be collected from a variety of sources and applied across a wide array of resources and workloads provisioned in an AWS account.

I recently had to perform this operation at work and I encountered the challenge that arises when there are many tags to be applied on many resources in an AWS account.

In pursuit of the company's cloud governance objectives. Forms were distributed to each department under the company to collect tag info. To appreciate the vast amount of data to be collected, an itemised list of some departmental units that are operative in any large scale organisation and business have been outlined below.

Human Resources (HR):
- Personnel Management
- Recruitment
- Training
- Employee Relations
- Compliance with Employment Laws
Finance and Accounting:
- Financial Planning
- Budgeting
- Accounting
- Financial Reporting
- Financial Analysis
Marketing:
- Marketing Strategy
- Advertising
- Branding
- Market Research
Sales:
- Selling Products or Services
- Customer Relationship Management
- Sales Target Achievement
Information Technology (IT):
- Technology Infrastructure Management
- Hardware and Software Management
- Network Management
- Cybersecurity
Operations:
- Day-to-Day Business Operations
- Process Optimization
- Supply Chain Management
- Production Oversight
Customer Service:
- Customer Inquiry Handling
- Customer Support
- Complaint Resolution
Research and Development (R&D):
- Research for New Products
- Improvement of Existing Products
- Innovation
Legal:
- Legal Counsel
- Contract Management
- Compliance with Laws and Regulations
- Management of Legal Risks
Administration:
- Office Management
- Facilities Management
- Administrative Support
Quality Assurance (QA) or Quality Control (QC):
- Ensuring Product or Service Quality
- Compliance with Quality Standards
Supply Chain or Logistics:
- Goods Movement
- Inventory Management
- Overall Supply Chain Oversight
Public Relations (PR):
- Communication with the Public
- Media Relations
- Stakeholder Communication
Project Management:
- Project Planning
- Project Execution Oversight
Strategic Planning:
- Long-Term Goal Setting
- Growth and Success Planning
Compliance:
- Ensuring Adherence to Laws and Regulations
- Industry Standards Compliance
Internal Audit:
- Internal Auditing
- Compliance Audits
- Financial Practice Audits
Training and Development:
- Employee Training Programs
- Professional Development Initiatives
Cost Center
Application Owner

After the painstaking effort involved in the manual collection and entry of tag information across all departments of an organisation into one comprehensive document. A more cumbersome task is involved in manually entering this tags on all development, QA and production EC2 instances and their associated EBS volumes as well as snapshots after they have been scanned and stored in a digital format.

Amazon Textract is a machine learning service offered by AWS that reduces the manual efforts and automates the process involved in the extraction of any kind of data such as forms, tables, and texts from scanned documents, making it expedient to derive important information from different sources.

Combining the above service with Boto3; the Python SDK for AWS and AWS Lambda, I architected a solution to ease and automate this effort using Amazon Textract. By automating the process of extracting this info I was able to write a script that applied this extracted data to all EC2 instances and further replicated them to their associated EBS volumes and snapshots.

This project will be demonstrated in a two part series. This article will demonstrate how to perform this action using
the Amazon Textract console to ensure that users who are just starting out with AWS can get comfortable implementing this solution using the AWS console. A second article as part of this series will also be put out and focus will be centered on how to perform this entire solution programmatically by using the AWS Boto3 documentation for Amazon Textract targeted for intermediate cloud practitioners in AWS and with considerable experience in programming.

Extracting Text from Documents with Amazon Textract in the AWS Console

Since tags are essentially key value pairs, I drafted a sample document as attached below and in the cover image of this article showing varying field entries and their associated keys in this scenario in order to adhere with compliance and security standards as the actual scanned tagging document contains company's info.

A. Sign in and Access Textract:

Sign in to the AWS Management Console at https://aws.amazon.com/.
Search for "Amazon Textract" and select it.

B. Choose a Document:
* Upload document: Upload your own document in PNG, JPEG, or PDF format.

C. Analyze the Document:

In the right pane, select the type of information you want to extract:
- Raw text: Extracts all text using Optical Character Recognition (OCR).
- Lines: Extracts individual lines of text.
- Words: Extracts individual words.

D. Explore the Extracted Data:

Explore the other tabs for further analysis options:
- Blocks: Identifies and categorizes text blocks based on their location and content.
- Tables: Extracts data from tables within the document.
- Forms: Identifies form fields and extracts their corresponding values.

E. Download Results (Optional):

Click the "Download JSON" button to download the extracted data in JSON format for further processing.

PS: This tutorial provides a basic overview. For detailed information and advanced functionalities, refer to the official Amazon Textract documentation: https://docs.aws.amazon.com/textract/

At this point, as depicted in the above process on using Amazon Textract to extract the key value pair information that was manually entered in the sample form for collecting tag information, the extracted tags can be applied to the specified EC2 resources. Considering the fact these are just a few instances, the tag information can be applied using AWS Tag Editor. The second part of this article will perform this action programatically to cater for use cases were there are a myriad of instances.

After applying the tags to DEV-EC1, DEV-EC2, DEV-EC3, and DEV-EC4, a lambda function with runtime; Python 3.9 can be created using the code block below to replicate the applied tags to all EBS volumes attached to the respective EC2 instances as well as the captured snapshots. Also attach the IAM policy;AmazonEC2FullAccess to the permissions of the role of the AWS Lambda function.

import boto3

def lambda_handler(event, context):
    # Get a list of all AWS regions
    ec2_regions = [region['RegionName'] for region in boto3.client('ec2').describe_regions()['Regions']]

    # Iterate through each region
    for region in ec2_regions:
        print(f"Working in region: {region}")

        # Initialize the EC2 client for the current region
        ec2_client = boto3.client('ec2', region_name=region)

        # Get all instances with their tags
        instances_info = ec2_client.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])

        # Prepare a mapping of instance ID to its tags
        instance_tags_map = {}
        for reservation in instances_info['Reservations']:
            for instance in reservation['Instances']:
                instance_tags = [tag for tag in instance.get('Tags', []) if not tag['Key'].startswith(('DEV:'))]
                instance_tags_map[instance['InstanceId']] = instance_tags

        # Get all EBS volumes and their snapshots in this region
        volumes_info = ec2_client.describe_volumes()
        snapshots_info = ec2_client.describe_snapshots(OwnerIds=['self'])

        # Iterate through each volume and its associated snapshots
        for volume in volumes_info['Volumes']:
            # Get the volume ID
            volume_id = volume['VolumeId']

            # Check if the volume has attachments
            if 'Attachments' in volume:
                # Get the instance ID to which the volume is attached
                instance_id = volume['Attachments'][0]['InstanceId']

                # Get tags of the associated instance
                instance_tags = instance_tags_map.get(instance_id, [])

                # Apply instance tags to the volume
                if instance_tags:
                    ec2_client.create_tags(
                        Resources=[volume_id],
                        Tags=instance_tags
                    )

                # Tag the snapshots associated with the volume
                for snapshot in snapshots_info['Snapshots']:
                    if snapshot['VolumeId'] == volume_id:
                        ec2_client.create_tags(
                            Resources=[snapshot['SnapshotId']],
                            Tags=instance_tags
                        )

        print(f"Tags copied from EC2 instances to their associated EBS volumes and snapshots in {region}")

    print("Script completed in all regions.")

In the above code I listed a few instances to perform this process on by indicating the instance name prefix “DEV”. To apply this same process on a fleet of EC2 instances with a naming convention for Dev,Test and Prod, the logic can be modified to perform this action on all instances with a prefix of Prod for instances rather than manually entering the instance ids of all EC2 instances.

Since AWS Lambda has a default timeout of 3 seconds, increment this value accordingly to accommodate the number of instance tags to be replicated across attached EBS volumes and captured snapshots. The execution time of Lambda has a threshold of 15 minutes. Thus, ensure that the processing logic for your use case is written to be as efficient as possible.

Applications Opened For The AWS Community Builders Program!

Wonder Agudah — Tue, 16 Jan 2024 17:59:42 +0000

Good day everyone. Applications for the 2024 AWS Community Builders Program is open! This is a great opportunity for all AWS enthusiasts who are passionate about sharing knowledge in the community and creating awareness on how cloud services can be used to achieve digital transformation whiles addressing the pressing problems of businesses and society in general. You can apply for the program using this link: https://pulse.aws/application/ZDVCOXKA

Leveraging Amazon CodeWhisperer for Agile Cloud Development and IaC.

Wonder Agudah — Tue, 28 Nov 2023 16:49:04 +0000

The influx of tools that facilitate managing cloud resources especially at an abstract level leveraging Infrastructure as Code(IaC) has been very instrumental in the growth and adoption of cloud computing. More recently, the widespread use and application of generative AI has also redefined the way and manner in which we approach instrumenting code to implement cloud resource functionalities in a given workload. There is no doubt that documentation on how to use a given library, framework or SDK is invaluable when programming in general and specifically too when writing code for a cloud resource or service. Nonetheless, a much more efficient approach has been introduced with the inception of generative AI tools that render a coding environment or IDE sufficient to code without the constant switch between an IDE and your browser. AWS recently launched one such tool know as Amazon CodeWhisperer.

Amazon CodeWhisperer is an AI-powered code development tool that helps developers write better code faster. It can provide code completion, code generation, and other suggestions based on your current context.

Getting Started with AWS CodeWhisperer

Prerequisites

An AWS account
An IDE supported by CodeWhisperer (VS Code, IntelliJ IDEA, AWS Cloud9, Lambda, JupyterLab, or Amazon SageMaker Studio)
The AWS Toolkit for your IDE installed and configured
An AWS Builder ID or IAM credentials

Setting Up CodeWhisperer

Choose your IDE.
Install or update your IDE (if applicable). Make sure you are using the latest version of your IDE to ensure compatibility with CodeWhisperer.
Install or update the AWS Toolkit. The AWS Toolkit is required for CodeWhisperer to function properly. You can install the toolkit from the IDE's plugin marketplace. For this demonstration, I will be using VSCode as my IDE of choice.

As indicated from the screenshot above, I have installed the AWS Toolkit from the "Extensions" panel in VSCode.

After clicking on the AWS Toolkit icon in VSCode a number of options are presented to you as shown below from the "DEVELOPER TOOLS" menu list. We will proceed to click on Amazon CodeWhisperer and select an authentication method.

Choose your authentication method. You can authenticate with CodeWhisperer using an AWS Builder ID, IAM Identity Center, or IAM credentials.
Set up your Builder ID, IAM Identity Center, or IAM credentials. Follow the instructions in the AWS CodeWhisperer documentation for your chosen authentication method.

For this demonstration I will be using the Builder ID authentication method

Set up your AWS Builder ID:

Go to the AWS Builder ID portal and sign in with your AWS account credentials.
Click on the Create a new Builder ID button.
Enter your email address and choose Next.
Enter your name and choose Next.
AWS will send an email to the address you provided. Open the email and click on the Verify button.
Your Builder ID is now created. Copy the Builder ID code from the portal.

Click "Allow" after authentication to enable Amazon CodeWhisperer to have access to code you write in your IDE of choice.

As shown above, I have been able to successfully authenticate CodeWhisper in VSCode using the AWS Builder ID method.

Using CodeWhisperer

Open your IDE and start coding.
Position your cursor in the code where you want CodeWhisperer to provide assistance.
Type the Alt + Space keyboard shortcut (Windows and Linux) or Cmd + Space keyboard shortcut (macOS).
CodeWhisperer will display a list of suggestions for completing your code.
Select the suggestion you want to use and press Enter or Tab.

The screenshot below provides a short demonstration of how CodeWhisperer aided me in writing code to list all S3 buckets in a region, simply by just commenting about the action I want to perform.

Extra Points
The following are some advantages of using Amazon CodeWhisperer:

Increased productivity: CodeWhisperer can help you write code faster by providing code suggestions, generating code from comments, and detecting and remediate security vulnerabilities. This can free up your time to focus on other tasks.

Reduced risk: Increase security posture of your codebase by leveraging security scans powered by CodeWhisperer to detect security vulnerabilities such as hard coded credentials and by providing code suggestions that are aligned with security best practices.

Improved code quality: CodeWhisperer can help you write higher-quality code by providing code suggestions that are idiomatic and follow best practices. It can also help you to identify and refactor code that is difficult to maintain.

Greater adherence to coding best practices: CodeWhisperer can help you to adhere to coding best practices by providing code suggestions that are aligned with industry standards. It can also help you to identify and refactor code that is not in compliance with best practices.

Enhanced collaboration: CodeWhisperer can help you to collaborate more effectively with other developers by providing a shared platform for generating code suggestions and discussing code changes.

Here are some additional merits of using CodeWhisperer:

It supports over 15 programming languages.
CodeWhisperer can be used to complete code snippets, generate code from comments, and provide code recommendations.
It is available in both personal and professional editions.
It is easy to use and integrate with your existing development workflow.

Troubleshooting
If you are having trouble getting started with CodeWhisperer, refer to the AWS CodeWhisperer documentation or contact AWS support.

I hope this document is helpful. Please let me know if you have any other questions in the comment section.

Mistakes I Made Deploying an IaC Project in GCP Using Terraform (and How I Fixed Them)

Wonder Agudah — Wed, 12 Jul 2023 09:19:54 +0000

Introduction:

I recently deployed an IaC project in GCP using Terraform. The project was a simple web application, but it was a good learning experience. In this blog post, I'll discuss some of the mistakes that I made deploying the project, and how I fixed them.

Mistake #1: Not using modules

One of the biggest mistakes I made was not using modules. Modules are a way to reuse code in Terraform, and they can make your code much more maintainable. For example, I had a set of resources that I used to create a database. I could have put these resources in a module, and then I could have reused the module in other projects.

How I fixed the mistake:

I fixed this mistake by starting to use modules. I created a module for the database resources, and then I reused the module in other projects. This made my code much more maintainable, and it made it easier to deploy the project.

Here are some of the benefits of using modules:

Reusability: Modules can be reused in multiple projects, which can save you time and effort.
Maintainability: Modules make your code easier to maintain, because you only have to make changes in one place.
Consistency: Modules can help you to ensure that your code is consistent across different projects.

Mistake #2: Not using variables

Another mistake I made was not using variables. Variables are a way to pass data into Terraform, and they can make your code more flexible. For example, I had a set of resources that I used to create a web server. I could have used variables to pass the web server's name and port number into Terraform.

How I fixed the mistake:

I fixed this mistake by starting to use variables. I created variables for the web server's name and port number, and then I passed the variables into Terraform. This made my code more flexible, and it made it easier to deploy the project to different environments.

Here are some of the benefits of using variables:

Flexibility: Variables can make your code more flexible, by allowing you to change the values of the variables without having to change the code itself.
Reusability: Variables can make your code more reusable, by allowing you to use the same code in different environments with different values for the variables.
Documentation: Variables can help you to document your code, by providing a clear explanation of the values that the variables can take.

Mistake #3: Not using Terraform Cloud

I also made the mistake of not using Terraform Cloud. Terraform Cloud is a hosted version of Terraform, and it can make it easier to manage your Terraform projects. For example, Terraform Cloud can help you to track your changes, and it can help you to collaborate with others on your projects.

How I fixed the mistake:

I fixed this mistake by starting to use Terraform Cloud. I created a Terraform Cloud account, and I imported my Terraform projects into Terraform Cloud. This made it easier to manage my projects, and it made it easier to collaborate with others on my projects.

Here are some of the benefits of using Terraform Cloud:

Centralized management: Terraform Cloud provides a centralized location for managing your Terraform projects.
Collaboration: Terraform Cloud makes it easy to collaborate with others on your Terraform projects.
Auditing: Terraform Cloud provides auditing features that can help you to track changes to your Terraform projects.

Mistake #4: Not testing my code

Finally, I made the mistake of not testing my code. Testing is important to make sure that your code works as expected. For example, I could have written some unit tests to test the database resources.

How I fixed the mistake:

I fixed this mistake by starting to test my code. I wrote some unit tests to test the database resources, and I also wrote some integration tests to test the entire project. This made me more confident that my code worked as expected, and it made it easier to find and fix bugs.

Here are some of the benefits of testing your code:

Confidence: Testing your code can give you confidence that your code works as expected.
Bug fixing: Testing your code can help you to find and fix bugs.
Documentation: Testing your code can help you to document your code.

Conclusion:

I made a lot of mistakes when I was deploying the IaC project in GCP using Terraform. However, I learned from my mistakes, and I was able to fix them. I hope that this blog post will help you to avoid making the same mistakes that I made.

Here is the link to the full post detailing how I completed this project:https://dev.to/wonder717/deploying-an-infrastructure-as-code-project-in-gcp-using-terraform-25jn

Thanks for reading!

About Me
My name is Wonder; a cloud architect/engineer doubling in AWS and GCP but more inclined to AWS 😉. Excited to embark on this journey with you lot and hopefully we learn from each other so we can grow collectively. Here are my socials;
LinkedIn: https://www.linkedin.com/in/wonder-agudah-784268183
Dev community: https://dev.to/wonder717
GitHub: https://github.com/Wonder717

Applications Opened For The AWS Community Builders Program!

Wonder Agudah — Fri, 07 Jul 2023 07:25:24 +0000

Happy Friday Everyone. Applications for the I AWS Community Builders Program is open! This is a great opportunity for all AWS enthusiasts who are passionate about sharing knowledge in the community and creating awareness on how cloud services can be used to achieve digital transformation whiles addressing the pressing problems of businesses and society in general. You can apply for the program using this link: https://lnkd.in/e74CKxmB

AWS Community Builders Welcome Kit🎉

Wonder Agudah — Fri, 30 Jun 2023 17:38:21 +0000

I’m glad to have received my 1st ever AWS CB welcome kit🥳. I look forward to continue my active participation and contribution in the AWS community. Special thanks to @jasondunn for the opportunity and incentives to motivate community builders. To learn more about the program, click this link https://aws.amazon.com/developer/community/community-builders/
Happy Friday Everyone & Cheers to the weekend 🎉

I am an AWS Community Builder!💪🏾🥳

Wonder Agudah — Mon, 27 Feb 2023 17:46:15 +0000

I’m beside myself with joy to have been recognized and accepted into the AWS community builders program(ML) as an active builder of the community. The efforts made to learn and understand disruptive tech like Cloud, AI & ML, and Programming is undoubtedly exciting! However, a more significant impact of this knowledge is to make and effect meaningful contributions to the community. A very effective way to do this is by sharing knowledge with the community, that is, for both builders, end users and all stakeholders. The AWS community builders program was designed for this exact purpose and I am delighted to contribute in this cause that gives back to a community that is continually providing excellent resources to deepen our understanding & use of AWS! I’m grateful for the opportunity AWS & I look forward to create more awareness in the community about the innovation and monumental strides that can be achieved with cloud services that are powered by AWS.