DEV Community: Leonard Hermawan

How to use DMS services

Leonard Hermawan — Fri, 31 Jan 2025 15:20:26 +0000

DMS is a fully managed service that enables migration and continuous data replication from on-premises, cloud, or hybrid databases to AWS with minimal downtime. It supports heterogeneous and homogeneous migrations.
Setting Up a Replication Instance
First, we must set up a replication instance. One advantage of AWS DMS is that it does not require a license—we only pay for the virtual machine (VM) used.
For production purposes, we can choose a multi-AZ (Availability Zone) deployment to enhance availability and fault tolerance.

Configuring the Source Endpoint
Next, we must create a source endpoint for AWS DMS to connect to. In my case, the source is located in an AWS VPC, but we can also use an on-premises server or another cloud provider.

Once AWS DMS successfully connects, the source endpoint status will change to Active. We must also create a target endpoint and ensure that both the source and target endpoints have an Active status.

To verify connectivity, we can use the Test Endpoint feature in AWS DMS.

Creating a Migration Task
After configuring the endpoints, we need to set up an AWS DMS migration task by selecting the source and target databases.

For the migration type, we can choose from:

Full Load – Migrates all data in one go.
Full Load and CDC (Change Data Capture) – Performs a full migration and continues replicating changes in real-time.
CDC Only – Captures and replicates only the incremental changes. I used Full Load and CDC for real-time replication of delta changes. It is recommended to enable CloudWatch monitoring to track the performance of the replication task.

Filtering and Migrating Data
We can filter and specify which schemas or tables to migrate to AWS and which ones to exclude.

Once everything is set up, the migration task will start, and the data will begin replicating to the target database.

Amazon Q in Quicksight

Leonard Hermawan — Mon, 27 Jan 2025 05:03:29 +0000

Amazon Q in QuickSight is a natural language querying feature in Amazon QuickSight BI tools that allows users to ask questions about their data in plain language and receive visual responses instantly. This feature helps users create visual diagrams and analyze datasets with the assistance of a large language model (LLM) in the background.
To start using QuickSight, you must first sign up via the QuickSight AWS Console. You can log in to QuickSight using SAML, an IAM user, or IAM Identity Center.
Setting up an IAM Role for QuickSight
You need to allow access to the resources you plan to use with QuickSight. For this example, I used a dataset stored in an S3 bucket.
Once the user is successfully created, you can proceed by uploading your dataset to S3, which serves as the repository. QuickSight supports many different dataset types as sources.
After successfully uploading the dataset, it becomes visible in QuickSight. For this demonstration, I used a dataset related to the sales of home materials at offline retail stores.

My user successfully created.

Now we put our dataset in the S3 as a repository. Quicksight support many different datasets as a source.

After dataset upload successfully, we can see our data in the Quicksight.
I used data set about sales of home material at retail offline stores.

Optimizing the Dataset in QuickSight
In this stage, you can create synonyms and remove unused labels to optimize the dataset for both the QuickSight dashboard and QuickSight Q (Generative AI for BI). This step is particularly useful when enabling the generative AI capabilities later.
QuickSight User Roles and Licenses
To create a dashboard from scratch and enable generative AI features in QuickSight, you need an AUTHOR PRO license.

Using Generative AI in QuickSight
With generative AI in QuickSight, you can issue prompts to generate visualizations based on your source dataset using the "Build a Visual" command. This feature allows you to ask questions about your dataset, create diagrams, and gain deeper insights for analysis.

Auto Remediation Action with AWS Config and AWS System Manager to Make Your Life Easier

Leonard Hermawan — Tue, 09 Jan 2024 03:55:55 +0000

  It can be challenging to monitor the EC2 instances that developers or admin teams launch. Sometimes, these launches may exceed the budget allocated to an organization. To address this, I will demonstrate how to set up automatic remediation actions using AWS Config, AWS Systems Manager and AWS Lambda. This setup ensures that only EC2 instances of type 't3.medium' are considered compliant. Any instance launched with a type other than 't3.medium' will be identified as non-compliant by AWS Config and will be immediately stopped by an automatic remediation action. This approach helps in maintaining budget control and ensuring compliance with organizational standards for resource usage.

In this case we will make custom rules with help from AWS Lambda in AWS config. Before start, this is my network topology diagram for real life testing.

Add new rule:
I will choose custom lambda rule to make custom AWS config rule.

For lambda code,this is lambda code I configured:

AWS Config detects a change in an EC2 instance's configuration, it will triggers this Lambda function. This function evaluates the instance and reports back to AWS Config whether the instance is compliant or not.
Configure the rule, and fill lambda ARN with ARN lambda we just created.

I configured trigger type with “when configuration changes” and resource= “AWS ëc2 instance”

Check review for one last time:

After creating rules, we will make remediation action IF our rule is not compliant.

We go to AWS System Manager to use Automation Document to help us with remediation automatic action.

We will use automation document named : AWS-StopEC2Instance. This is the content of automation document in SSM.

After this, we can continue setting automatic remediation action in AWS config.

AWS Config also offers manual remediation, or manual remediation with a human approval process. We can choose based on our requirements.

We can setup retries for this script to take action, I setup just 1 time. Resource id parameter I fill with InstanceID.
For remediation action, choose the document we choose at AWS System Manager.
Our rules and automatic remediation action have now been completed. It's time to test if our script and design are functioning correctly.
I am attempting to launch a noncompliant EC2 instance with the type “t3.micro”. After the EC2 instance starts and passes the status check, it won't be long before our SSM automation initiates action to stop this instance.

The EC2 instance will be stopped automatically.

Summary of aws config detail page:

Our remediation action status is “Äction executed successfully “.
Lets try it again to launch another ec2.

Only ec2 type t3.medium running well. Not comply ec2 already stopping by SSM automation action .
(Unused ec2 already terminated by me after stop)
And one of important to run these lab is IAM role and policy.
This is IAM role to make SSM run automation process to stopping ec2 that not compliant. We can change policies ”EC2fullaccess” to have more least privilege level in our production /development level.

With trust relationship to SSM.

Source to read:

https://docs.aws.amazon.com/config/latest/developerguide/remediation.html

How to Configure AWS Managed Microsoft Active Directory

Leonard Hermawan — Mon, 08 Jan 2024 05:08:34 +0000

AWS Managed Microsoft Active Directory (AD) provides a fully managed Active Directory service in the AWS cloud. It's designed to enable AWS resources and applications to use standard AD features without setting up and managing your own Active Directory infrastructure.
We can create this using Directory Service window:

Fill in the Directory DNS name, admin password for this new domain.

Next we configure where VPC and subnet that this AWS Managed Microsoft AD will launch.

And review it for one last time:

And then create the directory.

After we create the directory, we will configure directory administration EC2 instance.

We can configure Remote access CIDR /which subnet is allowed to do RDP to this instance.

SSM Will help to create our resources .

Management instance will be up:

We will check Management Instance properties:

If we already joined with the domain, computer name will similar as below:

If the admin EC2 still not joined with the domain, we can do manually to join this Management intsance ec2 to leonard.example.com domain. Configure manually DNS name, and domain name in the properties.

We will install features to support AWS Managed Microsoft AD in our Management instance.
Install the feature we need for AD:

IAM Role that needed to run this service:

With trust relationship as below:

Security group enabled: RDP port

And try log in again to Management instance using user and password domain:

I successfully log in and Management instance already connected with AWS Managed Microsoft AD (domain leonard.example.com) :

We can add new user /group from this Management instance.

We can custom password policy. Expired password time , etc. or when someone left the company.

And we can try to log in using this new username to another device that already joined domain .

Source:

https://docs.aws.amazon.com/directoryservice/latest/admin-guide/directory_microsoft_ad.html

Live Sport Broadcast using AWS Elemental Service

Leonard Hermawan — Thu, 05 Jan 2023 05:30:25 +0000

World Cup Time finally over! I really enjoyed this World Cup!. I watched official live stream in my Windows Laptop, sometime on my another mobile phone devices Android or an IOS device. I wonder. Is there any service that I can use for live broadcasting with AWS?
Yes it is! AWS provided us with AWS Elemental service.

This lab i demonstrate using above topology diagram.
AWS Elemental MediaLive will encodes video in real time, compressing live broadcast sources into streams for delivery to broadcast TV , mobile devices (Android or IOS ) or another internet connected devices .
AWS Elemental MediaPackage prepares, distributes our live / on demand video content to a broad range of connected devices ( Windows, Android, IOS, etc ). The service can take a single video input from an encoder such as AWS Elemental MediaLive.
Amazon CloudFront will deliver our content all around the world.
Live streaming using Real time Messaging Protocol, and will use AWS MediaLive and AWS Elemental MediaPackage

For simplify my lab , I substitute live broadcasting devices with S3 Bucket. I put my free sample soccer video I got from pixabay.com/ to my S3 bucket.

And then we can setup our AWS Elemental MediaLive
We must create channel first .

Create IAM Role

Fill input source in INPUT SOURCE A:

Add input attachments:
With input sources our S3 url wehre we put our free video.

Medialive channel succesfully create:

I will setting the output video first using AWS Elemental MediaPackage.

Choose mediapackage:

Create a new channel

Create apple output video. This is for viewer who owned IOS devices.

I create three endpoints for three different outputs devices. For windows viewer using laptop or PC. For Android and IOS viewer using mobile devices , smart TV or Laptop.

After we created all three endpoint, we can preview our video using preview button under each our origin endpoint. If our video appear, we successfully configure our AWS Elemental services.

I successfully configure using Elemental MediaLive and MediaPackage for sport live broadcasting service. Now we can distribute our live streaming globally using AWS CloudFront!

Source:

https://aws.amazon.com/medialive/faqs/
https://aws.amazon.com/mediapackage/faqs/

How to Choose & Configure Appliance for AWS Gateway Load Balancer

Leonard Hermawan — Thu, 24 Nov 2022 09:14:43 +0000

This article is continuation from my previous article.

My Topology for this lab same as previous one:

Choose and configure 3rd party appliance for Gateway Load Balancer in AWS Marketplace.
When we create security Gateway load balancer, AWS give us many choice using their 3rd party partner security appliance. I tried using Checkpoint product for this demo. We can use another 3rd party brand as well.

At AWS marketplace I found 2 of these product and subscribed it. One is for Security Gateway load balancer and one is for their management server

After we subscribed it we can deploy in using EC2 instance.

We can use cloudformation template from checkpoint to deploy our security management server and Security Gateway load balancer.
In this stack template we chose which vpc for our Management server and Security GWLB , which subnet we using. I created new VPC called security VPC for this purpose.

After the stack complete, we can access our security GWLBs using our Management Server
And we can ssh to our management server using our private key file using putty.

we must set up username and password for our Management Server before we can access it with WEBGUI.

If we want monitor our Security Gateway Load Balancer in our local laptop, we can install Smart Console from checkpoint website.
After installed and verify our account, it will be automatically detected our security gateway and our Management server in AWS . Seamless integration with our AWS environment.

My reference for this lab:

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk174447 ->AWS cloudformation example template for gateway load balancer (we can custom all the parameters freely)
https://www.youtube.com/watch?v=f4DduW2M5WI -> AWS official short video about Gateway Load Balancer

Setup and Configuring AWS Gateway Load Balancer

Leonard Hermawan — Mon, 31 Oct 2022 04:13:01 +0000

Gateway load balancer works at Layer 3 OSI layer. Gateway Load Balancers use Gateway Load Balancer endpoints to securely exchange traffic across many VPC we have. In example we have 2 VPC. Security VPC and HQ VPC .

This is network topology diagram for this lab.

I used 2 ip subnets for HQ VPC. One subnet for our web servers, and one subnet for GWLB enpoint in HQ VPC.

First step we must choose and do initial configuration to launch third party security appliance.
We can use Cloudformation template to help us do initial configuration for Gateway LoadBalancer. We must fill the important parameters ourself such as which VPC our security Gateway Load Balancer will reside, which subnet will we use, and if we want to install security management server directly,and configured our security group to permit udp port 6081 for allowing traffic from GWLB.

I configured my Gateway Load Balancer in my Security-VPC with 2 of availabilty zones( refer my topology diagram).

I turn to ‘false’ connection acceptance required.

I am using checkpoint AMI with C5.Xlarge EC2 (this is EC2 type recommendation). (I will added another post how to choose security gateway from AWS Marketplace).For this lab I created 2 Security Gateways Checkpoint AMI .

After our security gateway instance is up , we must create target group

After that we must create Gateway load balancer endpoint located in our HQ VPC

Before creating endpoint, we must create endpoint services first and choose our Gateway Load Balancer we created earlier.

After that we can create endpoint . We must copy our endpoint service name from previous activity, and select subnets which our endpoint services located.

Last step we must configure our route tables . Outgoing route table for redirect outgoing traffic from our web server to our endpoint and Ingress route table for redirect incoming traffic from internet that will go to our web server. All traffic will be check by our security gateway in our security VPC.

All outgoing traffic will go to our security gateway first via our endpoint.

and Ingress route table for redirect incoming traffic from internet that will go to our web server.

We can use add 1 more route table for outgoing traffic to the internet from our HQ-VPC.
We can check our Endpoint Metric in AWS CloudWatch too. (Configured at Cloudformation Stack). Thankyou for your time reading this!

My reference for this lab:

https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk174447 ->AWS cloudformation example template for gateway load balancer (we can custom all the parameters freely)
https://www.youtube.com/watch?v=f4DduW2M5WI -> AWS official short video about Gateway Load Balancer