DEV Community: Steve Pe

Private Access to S3 Data from two AWS regions

Steve Pe — Tue, 28 Nov 2023 19:29:13 +0000

What is AWS Private Link?

AWS Private Link provides private connection between VPCs, AWS services, and your on-premises networks, without exposing your traffic to public internet.

Network traffic that use AWS Private Link doesn't travel the public interent
Reduce exposure to attacks
Regulatory compliance
Hybrid cloud
Cost saving

On this blog I am setting up two VPCs in two AWS regions ( Ohio, US and Tokyo, Japan) with two different CIDRs range and use VPC endpoint for Amazon S3 to access the data.

VPC endpoints for Amazon S3 simplify access to S3 from within a VPC by providing configurable and highly reliable secure connections to S3 that does not require and Internet Gateway(IGW) or/& Network Address Translation (NAT) device. When you create a S3 VPC endpoint, you can attach an endpoint policy to it that controls access to Amazon S3.

Step 1: Create a VPC in the Ohio Region:

In the AWS console go to VPC, select Ohio region and create VPC named as Ohio-VPC with CiDR of 10.10.0.0/16

Create 4 subnets in the Ohio-VPC. 2 public subnets and 2 private subnets.
No Internet Gateway (IGW) nor NAT Gateway in this Ohio-VPC.

Step 2: Create a VPC in Tokyo Region:

Go to Tokyo region and create the VPC named Tokyo-VPC with CIDR of 10.20.0.0/16.
Create 4 subnets in the Tokyo-VPC. 2 public subnets and 2 private subnets.
No Internet Gateway (IGW) nor NAT Gateway in this Tokyo-VPC.

Step 3: Create S3 bucket in the Ohio, Region:

In the AWS Console, go to the S3 service and create a bucket in the Ohio region.

Step 4: Create Interface endpoint in the Ohio region:

In the VPC console, in the left panel, click on Endpoint under Virtual Private Cloud and it will appear Endpoints console.
In the Endpoints console, click on Create Endpoint and new screen will pop asking to give it a name and select AWS Service for wihi we want to create endpoint. As we are creating for S3, we will filter with S3 service and you will get available endpoints for S3 which includes gateway and interface endpoints. S3 outposts and access point interface are available and we will choose S3 interface endpoint.
Choose right VPC with private subnets you created earlier. Create new security group with no inbound with default outbound.
S3 interface endpoint is created and its available. Grab the DNS name and we would replace * with below appropriate valuses:
Example of S3 bucket: bucket.vpce-xxxxxxxxxx1a295b3-zshjrg35.s3.us-east-2.vpce.amazonaws.com

Step 5: VPC Peering between Ohio-VPC and Tokyo-VPC

Create VPC peering between Ohio and Tokyo VPCs, accept the peering connection at acceptor VPC and add the routes of other VPC CIDR in the route tables.
Then accept the Peering Connection at Tokyo VPC
Add a route in the route table with destination as the VPC CIDR of peered VPC and target as peering connection.

Step 6: Create EC2 instance in the Ohio Region:

Create an EC2 instance in the private subnet in the Ohio region and attach necessary roles (SSM and S3)
Connect private EC2 instance at Ohio region using SSM and list S3 buck
Use S3 interface endpoint which we created earlier and run below command
(Sample: aws s3 --region us-east-2 --endpoint-url https://bucket.vpce-xxxxxxxxxxxxxxxxx-zshjrg35.s3.us-east-2.vpce.amazonaws.com ls s3://{Your Bucket at Ohio}/

Step 7: Create EC2 instance in the Tokyo Region:

Create an EC2 instance in the private subnet in the Tokyo region and attach necessary roles (SSM and S3)
Connect private EC2 instance at Tokyo region using SSM and list S3 buck at Ohio region
Use S3 interface endpoint which we created earlier and run below command
(Sample: aws s3 --region us-east-2 --endpoint-url https://bucket.vpce-xxxxxxxxxxxxxxxxx-zshjrg35.s3.us-east-2.vpce.amazonaws.com ls s3://{Your Bucket at Ohio}/

Conclusion:
S3 Interface endpoint helps us to access S3 buckets privately and we can access cross region S3 buckets privately if VPC peering is enabled between the VPCs.

BEST METHOD TO CONNECT TO AWS EC2 INSTANCE

Steve Pe — Wed, 08 Nov 2023 19:42:24 +0000

AWS Transit Gateway Peering 2 VPCs in different AWS Regions

Steve Pe — Tue, 18 Jul 2023 12:52:03 +0000

A transit gateway is a network transit hub that you can use to interconnect your virtual private clouds (VPCs) and on-premises networks. As your cloud infrastructure expands globally, inter-Region peering connects transit gateways together using the AWS Global Infrastructure.

Enabling peering between multiple VPCs on AWS can be difficult. It can be much more difficult if they are in different regions.

Here is digram about AWS VPC Peering vs. AWS Transit Gateway

I will walk you through the process of enabling peering between multiple VPCs.

I have 2 VPCs and Subnet information.

US East 1: 10.1.0.0/16 - 3 Public & 3 Private subnets /20

US West 2: 10.2.0.0/16 - 3 Public & 3 Private subnets /20

Two Route tables each VPC. Public route table is using Internet Gateway to access outside and private route table has just local. Here are their screenshots

US East 1 Region:

US West 2 Region

Also I created Two EC2 instances each region. One public instance and one private instance. I am going to use Ping, ssh & telnet as my test cases using private ip addresses.

US East 1 Region

Create Transit Gateway with unique ASN: TGW-US-East-1-TG1
Create Transit Gateway Attachment: TGW-US-East-1-VPC and select all available subnets.

Verify Transit Gateway Route tables (Make sure there is entry at Propagations & Routes tabs

Update Route Tables (both Public & Private Route tables) at US East 1 VPC. Add recently created Transit Gateway with US West 2 VPC CIDR address 10.2.0.0/16

Copy US East 1 Transit Gateway ID & Save somewhere:

US West 2 Region

Create Transit Gateway with unique ASN: TGW-US-West-2-TG2
Create Transit Gateway Attachment: TGW-US-West-2-VPC and select all available subnets.
Update Route Tables (both Public & Private Route tables) at US West 2 VPC. Add recently created Transit Gateway with US East 1 VPC CIDR address 10.1.0.0/16
Create Transit Gateway attachment Peering Connection at US West 2 region (Need to paste US East 1 region Transit Gateway ID - copied from earlier step)
Sent Peering request to US East 1
At US East 1 Region Accept Peering request
After 20 minutes Peering was available
Back to Transit Gateway route tables under Transit Gateway and create static route
At US East 1 Region Transit Gateway route tables add US West 2 CIDR
Click "Create static route" enter US West 2 CIDR and choose Peering type
At US West 2 Region Transit Gateway route tables add US East 1 CIDR
Click "Create static route" enter US East 1 CIDR and choose Peering type

US East 1 Region EC2 service

Security group for EC2 instance. Didn't open for anywhere 0.0.0.0/16. Just opened for US West 2 VPC/CIDR at inbound rules
From US East 1 Region EC2 instance Ping/SSH/Telnet using US West 2 Region EC2 instance's private ip address (successful)
From US West 2 Region EC2 instance Ping/SSH/Telnet using US East 1 Region EC2 instance's private ip address (successful)

Now we are successfully peered two VPCs (different regions) using AWS Transit Gateway method.

Here is solution digram

Clean up hint: Shutdown or terminate ec2 instances, delete security group, delete transit gateway attachments from both regions (vpc & peering) then delete transit gateway from both regions. _

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Building AWS Managed Microsoft Active Directory: Step-by-step

Steve Pe — Sat, 11 Mar 2023 11:02:54 +0000

This is the solution I created for account I supporting. Using AWS managed Microsoft Active Directory (AD) less headache for administrator and team because no need to manage license and it is pay as you go model. Also it is Highly Available (HA) configuration and managed service so no need to patching or easy to add more AD nodes.

Major benefit of using AWS managed AD is on-board and off-board user easily and handle outside of AWS console. Also can use AD connector to connect Corp AD.

Here are step by step building AWS Management Active Directory service and integrate with AWS IAM.

Login to your AWS Console with Administrator permission
Under Security, Identity & Compliance service select Directory Service.
Click Set up Directory
Select AWS Managed Microsoft AD and click Next
Select Standard unless require to setup larger business
Give any name at Directory DNS name (Not Public DNS, inside your VPC) See example from below screenshot
Enter Directory NetBIOS name (See example from below screenshot)
Enter Admin password and make sure write down somewhere safe because it will require. Click next
Choose VPC you want and select at least two different Subnets. Click Next
Review and Create Directory (~ 86 dollars per month and 30 days free trial) This step will take around 45 minutes.
Creating

*Let's setup IAM Role and policy *

Under Security, Identity & Compliance service select IAM
Click "Role" from left under Access Management and click 'Create Role'
At next page select AWS Service and EC2 then click Next
At next page search AmazonSSM and select AmazonSSMDirectoryServiceAccesss & AmazonSSMManagedInstanceCore then click Next
At Role detail page give meaningful Role name and description. Then click Create Role.
Create another role for this blog/demo purpose only. In reality you will need to create multiple roles.
Click "Role" from left under Access Management and click 'Create Role'
At next page select AWS Service and EC2 then click Next
At next page search ViewOnlyAccess and select ViewOnlyAccess policy
Click Next and give meaningful name and create role.

Back to Directory Service page

Verify AD service
It is HA at two subnets (Same Region)
Under Add a trust relationship you can add external / Corporate AD

*Provision Windows base for add/join Domain and manage users
*

At EC2 console launch Windows 2019 base (Not Core Base) t2.micro with IAM role "EC2DomainJoin" earlier created. Also select AD you created. See below screenshot as example
Login (Remote Desktop) with Domain Users & Domain Admin password created above. (Hint: Domain\User)
You can verify with Open Terminal windows and run "whoami" & "set" commands
Go to Windows "Server Manager" Dashboard and click "Add roles and features" link click Next until 'Features" is selected.
Expand "Remote Server Administration Tools" and select 'AD DS and AD LDS Tools'
Click Next and Install. (It will take 2 to 3 minutes)
Click Close
Go to Windows Administrative Tools and Open "Active Directory Users and Computer and select Users
Create new user(s)

Give permission to users

Go back to AWS AD service and select Directory Service you created and select again "Application and Management"
Go to AWS Management Console session and under action select enable.
Go to next session "Delegate console access" click AD-ViewOnlyUser (or whichever permission you created earlier)
Click the group you want to add user(s)and add
Copy the Console url from AWS Apps and Services session and go to different browser or private browser and login
That way users can login with permission you give at AD Service. No need to add at AWS IAM page.
Now you are successfully created AWS Managed AD and integrated with AWS Console login.

Securely Access Window Bastion host using System Manager Port Forwarding method

Steve Pe — Sun, 28 Aug 2022 10:23:33 +0000

This secure solution I introduced to one of the largest financial institution in US to access their linux vm from private / restricted subnets. It's the best practice to host critical infrastructure in restricted subnets.

Requirement was must not open known firewall / security group port open for bastion host.

Prerequisites

For this post, I used MacOS version 12.5.1 workstation

Install AWS Command Line Interface (AWS CLI tool) to your workstation.
Install AWS Session Manager Plugin to your workstation.
Create Custom IAM role for Session Manager, for this post I am not using KMS key and S3 bucket configuration.
For this demo I created new AWS Identity and Access manager (IAM) user with programmatic access to AWS Account.
For this demo I already created Linux vm in private / restricted subnet.

Solution Overview

I created a new Amazon Elastic Compute Cloud EC2 security group with no single port open at inbound rules. Then lunched new Window instance at public subnet and attached above step #3 IAM role and no port opened security group. If require see below for detail step for lunching basing host. See below screenshot for example

Above Linux and Windows bastion host must be in same AWS Virtual Private Cloud (Amazon VPC). On Linux instance security group inbound rule you can update with Windows bastion host private ip address that way no other host from same vpc can connect. No IAM role need to attach to Linux vm or open for the whole vpc cidr. See below sample screenshot.

Here are detail steps for provision bastion host (for this demo/blog)

On the Amazon EC2 console, choose the Windows 2019 base image
Choose the t2.micro instance class
For Subnet, choose a public subnet with auto assign ip but not going to use dns/public ip address
For IAM role choose the IAM role created above
Choose Review and Launch.

Before create remote session better to verify AWS CLI and SSM plugin on your workstation.

Here are commands and sample output.

AWS CLI Version
Command: aws --version
Sample output: aws-cli/2.4.5 Python/3.8.8 Darwin/21.6.0 exe/x86_64 prompt/off

AWS Session Manager Plugin Verification
Command: session-manager-plugin
Sample output: The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.

AWS Session Manager Version
Command: session-manager-plugin --version
Output: 1.2.339.0 (latest version for now)

Create a remote RDP port forwarding session

In this session, I created a port forwarding session to remote host using AWS Systems Manager and connecting to RDP instance.

Store AWS Credential file using AWS Configure command. You can view detail how to setup at https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
Create a Systems Manager Session to RDP/bastion host
aws ssm start-session --target <your bastion instance id>
--document-name AWS-StartPortForwardingSession `
--parameters portNumber="3389",localPortNumber="13389"

You should see something similar like this
Starting session with SessionId: demo-0ec9fd2af3cce14fd
Port 13389 opened for sessionId demo-0ec9fd2af3cce14fd.
Waiting for connections...

Leave that session open and use Microsoft Remote Desktop to connect to RDP instance.. with hostname as localhost:13389 and enter user name and password. Do not use RDP DNS nor public IP address to connect.

On my solution I configured SSH Agent forwarding on RDP/bastion host. You should never store aws key pair in bastion host. I followed this blog "Secure Connect to Linux Instances Running in a Private Amazon VPC" to setup SSH Agent forwarding.

Conclusion
Using my securely connect to bastion host with SSM port forwarding and ssh agent forwarding methods we are supporting securely servers at Private VPC. Here is high level digram.

Here is my short video about this securely access rdp server solution .

Thank you!!

How to setup AWS Application Migration Service

Steve Pe — Mon, 20 Jun 2022 11:25:34 +0000

AWS Application Migration service is easiest and quickest way to migrate your virtual, physical or cloud-base servers to AWS, minimal business disruption.

Here is how to setup and migrate your Linux server(s). This setup no ‘Web Proxy’ nor VPN.

Prerequisite

At least IAM role with “AWSApplicationMigrationFullAccess” Attached. Also both aws console and cli access. s
TCP port 443 open from source server/environment to AWS Region
Source server will use TCP port 1500 for data transfer to replication servers in Staging area subnet.
SSH sudo permission on source server.

Here is Network Architecture diagram

Mark as "Ready for cutover"

What is AWS Elastic Beanstalk and how to deploy simple HA web application

Steve Pe — Mon, 28 Mar 2022 14:01:45 +0000

AWS elastic beanstalk (EB) is developer centric view of deploying an application using aws components like ec2, asg, elb, rds and etc. EB is free service and only pay for the underly infrastructure. Fast simple and customizable. EB support multiple languages like Go, Java, .Net, Node js, PHP, Python, Ruby and so on. It is aws fully managed service.

Before any eb deployment make sure create IAM role called aws-elasticbeanstalk-service-role with these eb policies:

AutoScalingFullAccess
ElasticLoadBalancingFullAccess
AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy

Here how to deploy sample web ha app using aws elastic beanstalk:

Go to AWS Elastic Beanstalk console: https://console.aws.amazon.com/elasticbeanstalk and click Create Application.

Give your application name at Application name area & choose the platform at platform section.

In the Application code section you have to choose a sample application and the click on Configure more options

Select High Availability at Configuration Presets section area

At Capacity section click edit and enter desire number of instances at Auto Scaling Group and click Save at bottom.

At Network section click edit and choose VPC, Load balancer settings and Instance subnets and click Save. (Only of not using default VPC) this example is not using default VPC.

At the Load balancer section and click Edit and select Application Load Balancer at Load balancer type area, then click Save at bottom.

At Security section click edit and enter "ec2 keypair" if require to login (ssh) to ec2 instances.

Finally, at Rolling updates and deployments section click edit and update desire update Deployment policy. Click Save at bottom.

After all configuration updates Click Create app at bottom.

Here are events and at the end Green check mark with environment url.

Environment overview page

Congratulations you are successfully deployed Web App using AWS Elastic Beanstalk!