DEV Community: Samir

Implementing OSRM Map Backend on AWS Elastic Container Service (ECS)

Samir — Thu, 06 Jul 2023 16:31:26 +0000

Introduction:

The Open Source Routing Machine (OSRM) is a powerful routing engine designed for calculating shortest paths in road networks.

In this blog, we will explore the process of deploying OSRM on AWS Elastic Container Service (ECS), allowing you to leverage the scalability and flexibility of containerized environments.
We will cover the steps to install Docker, create an OSRM Docker image, push the image to AWS Elastic Container Registry (ECR), and create an ECS cluster, task definition, and service, then publish OSRM service through AWS Application loadBalancer.

Step1: Installing Docker and Creating an OSRM Docker Image:

Install Docker on Linux:
1- Run the following commands in your Admin Server:

sudo yum install -y docker
systemctl enable docker
systemctl start docker

2- Create a demo folder and navigate to it:

mkdir demo && cd demo

3- Pull the OSRM Docker image from DockerHub:

docker pull osrm/osrm-backend

4- Create a Dockerfile:

vi Dockerfile

5- Add the following content to the Dockerfile:

FROM osrm/osrm-backend:latest

RUN mkdir /data
WORKDIR /data
ADD https://download.geofabrik.de/asia/gcc-states-latest.osm.pbf /data
RUN /usr/local/bin/osrm-extract -p /opt/car.lua /data/gcc-states-latest.osm.pbf && \
    /usr/local/bin/osrm-partition /data/gcc-states-latest.osrm && \
    /usr/local/bin/osrm-customize /data/gcc-states-latest.osrm

CMD [ "/usr/local/bin/osrm-routed", "--max-table-size", "100000", "--algorithm", "mld", "/data/gcc-states-latest.osrm" ]

Step2- Create an AWS ECR Repository and Push the OSRM Docker Image:

1- Go to AWS ECR and click on "Repositories."
2- Create a new repository with the name "osrm" and set the visibility to "Private."

Push the Docker image to AWS ECR:
Execute the following commands in your Admin Server:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 612843029448.dkr.ecr.us-east-1.amazonaws.com
docker build -t osrm .
docker tag osrm:latest 612843029448.dkr.ecr.us-east-1.amazonaws.com/osrm:latest
docker push 612843029448.dkr.ecr.us-east-1.amazonaws.com/osrm:latest

Step3- Creating an ECS Cluster, Task Definition, and Service:

Create an ECS Cluster:
Provide a name for the cluster (e.g., "osrm-demo").
Choose the desired VPC and private subnets.
Select "Amazon EC2 instances" as the infrastructure type.
Create a new Auto Scaling group (ASG) with the following settings:
Operating system/Architecture: Amazon Linux 2
EC2 instance type: t3.large
Desired capacity: Minimum (1), Maximum (1)
Enable optional monitoring using Container Insights.

Create a Task Definition in JSON format:
Use the provided JSON template to define the OSRM task with the necessary configurations.

{
    "taskDefinitionArn": "arn:aws:ecs:ap-south-1:012345678901:task-definition/osrm:1",
    "containerDefinitions": [
        {
            "name": "osrm",
            "image": "012345678901.dkr.ecr.ap-south-1.amazonaws.com/osrm:latest",
            "cpu": 2048,
            "memory": 4096,
            "portMappings": [
                {
                    "name": "osrm-5000-tcp",
                    "containerPort": 5000,
                    "hostPort": 0,
                    "protocol": "tcp",
                    "appProtocol": "http"
                }
            ],
            "essential": true,
            "environment": [],
            "mountPoints": [],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "/ecs/osrm",
                    "awslogs-region": "us-east-1",
                    "awslogs-stream-prefix": "ecs"
                }
            }
        }
    ],
    "family": "osrm",
    "taskRoleArn": "arn:aws:iam::012345678901:role/ecsTaskRole",
    "executionRoleArn": "arn:aws:iam::012345678901:role/ecsTaskExecutionRole",
    "networkMode": "bridge",
    "revision": 1,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
        },
        {
            "name": "ecs.capability.execution-role-awslogs"
        },
        {
            "name": "com.amazonaws.ecs.capability.ecr-auth"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
        },
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role"
        },
        {
            "name": "ecs.capability.execution-role-ecr-pull"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.29"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EC2"
    ],
    "requiresCompatibilities": [
        "EC2"
    ],
    "cpu": "2048",
    "memory": "4096",
    "runtimePlatform": {
        "cpuArchitecture": "X86_64",
        "operatingSystemFamily": "LINUX"
    },
    "registeredAt": "2023-01-01T16:18:09.874Z",
    "registeredBy": "arn:aws:iam::012345678901:user/myuser",
    "tags": [
        {
            "key": "Name",
            "value": "osrm"
        }
    ]
}

Create an ECS Service:
1- In either the Task Definition or Cluster console, click on "Create Service."
2- Select the existing ECS cluster "osrm-demo".
3- Choose "Capacity provider strategy" under "Compute options."
3- Set the task definition to "osrm" with revision 1 (or the latest revision).
3- Specify the desired number of tasks (e.g., 1).
4- Configure the deployment options and failure detection as per your requirements.
5- Set up an Application Load Balancer (ALB) for load balancing.
6- Create the service.

Creating an AWS Application Load Balancer (ALB):
1- Provide a name for the ALB (e.g., "osrm-demo-ALB").
2- Set the scheme to "Internet-facing" and IP address type to "IPv4".
3- Choose your VPC and configure mappings for two Availability Zones with one public subnet per zone.
4- Associate the ALB with an existing security group (e.g., "Demo-SG").
5- Configure the ALB listener for port 5000 and protocol HTTP.
6- Create a new target group (e.g., "osrm-tg") and set the health check path to "/nearest/v1/driving/13.388860%2C52.517037?number=3&bearings=0%2C20" (URL-encoded format).
7- Save the ALB configuration.

Step4- Testing the OSRM Deployment:

You can test your OSRM deployment by accessing the URL generated by your ALB, for example:

http://osrm-demo-alb-012345678901.us-east-1.elb.amazonaws.com:5000/nearest/v1/driving/13.388860%2C52.517037?number=3&bearings=0%2C20

Links:
OSRM: https://map.project-osrm.org/
osrm-backend in GitHub: https://github.com/Project-OSRM/osrm-backend/tree/master
osrm-backend image in Dockerhub: https://hub.docker.com/r/osrm/osrm-backend/
URL Encoding Reference: https://www.w3schools.com/tags/ref_urlencode.ASP

Conclusion:
By following the steps outlined in this blog, you can successfully deploy OSRM on AWS Elastic Container Service (ECS). Leveraging the power of containers and AWS services like ECR and ECS, you can easily scale and manage your OSRM routing engine to handle routing requests efficiently. Enjoy exploring the possibilities of OSRM and building applications that leverage its robust routing capabilities!

Choosing the Right AWS Region: Considerations for Cost, Latency, and Service Availability

Samir — Tue, 04 Jul 2023 12:07:48 +0000

Introduction

When deploying your applications on AWS, selecting the appropriate region is an essential decision that can impact factors such as cost, latency, and service availability. In this blog post, we will explore how to make an informed choice by considering these critical aspects.

Cost Considerations:

Determining the cost of running your application in different regions is crucial for optimizing your expenses.
AWS provides a pricing calculator that enables you to estimate the costs of various resources.
For example, you can use AWS calculator to evaluate the pricing for an EC2 m5.2xlarge instance in multiple regions.
https://calculator.aws/

Latency Considerations:

Reducing latency is crucial for ensuring optimal application performance. To evaluate latency from different regions, you can check links such as https://aws-latency-test.com/ and CloudPing.info
These platforms allow you to measure network latency and determine the best regions based on your specific requirements.
By leveraging these tools, you can gain insights into network performance and select regions that offer lower latency, thereby enhancing the user experience for your application's target audience.

Service Availability by Region:

AWS's global infrastructure is spread across multiple regions, each providing a different set of services. To identify the availability of specific AWS services in different regions, you can refer to the official AWS Regional Services List.
https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/
By exploring this resource, you can ensure that the services required by your application are accessible and supported in the regions you are considering. This information allows you to make informed decisions and avoid potential limitations or constraints that may arise from service unavailability in specific regions.

Conclusion:

Choosing the right AWS region involves considering several factors such as cost, latency, and service availability. By utilizing tools like the AWS pricing calculator, aws-latency-test.com, and the AWS Regional Services List, you can gather the necessary information to make informed decisions.

Remember to evaluate your specific requirements and priorities when selecting an AWS region. By doing so, you can optimize costs, minimize latency, and ensure that the required services are available to support your application's successful deployment and performance.

Creating and Mounting EFS on AWS EC2 with system manager runbook

Samir — Tue, 04 Jul 2023 11:17:26 +0000

Introduction

In this blog post, we will walk you through the steps to create and mount an Amazon Elastic File System (EFS) on an AWS EC2 instance using the AWSSupport-CheckAndMountEFS system manager runbook. EFS provides scalable and shared file storage that can be accessed from multiple EC2 instances simultaneously.

Systems Manager Automation runbook

AWS Systems Manager provides predefined runbooks. These runbooks are maintained by Amazon Web Services, AWS Support, and AWS Config. The runbook reference describes each of the predefined runbooks provided by Systems Manager, AWS Support, and AWS Config.

AWSSupport-CheckAndMountEFS

The AWSSupport-CheckAndMountEFS runbook verifies the prerequisites to mount your Amazon Elastic File System (Amazon EFS) file system and mounts the file system on the Amazon Elastic Compute Cloud (Amazon EC2) instance you specify.
This runbook supports mounting your Amazon EFS file system with the DNS name, or using the mount target’s IP address.

Step 1: Create EFS from AWS Console

Log in to the AWS Management Console and navigate to the EFS service.
Click on "Create file system" and provide a name for your EFS.
Choose the VPC where your EC2 instance resides.
Select the "Regional" EFS option for high availability.
(Optional) In the advanced options, you can choose the storage class. For example, you can select the "Standard" storage type instead of "Infrequent Access" (IA) if desired.

Step 2: Configure Network in EFS

After the EFS creation, navigate to the EFS dashboard and go to the "Network" tab, then click on "Manage". Copy the security groups listed under "Mount targets" and navigate to the EC2 service, then go to "Security Groups".
Add the EC2 instance's security group as a source under the EFS security group to allow access.

Step 3: Install EFS Utils

Connect to your EC2 instance using SSH or AWS SSM.
Run the following command based on your Linux distribution: For SUSE: zypper install aws-efs-utils For Amazon Linux 2: sudo yum install -y amazon-efs-utils

Step 4: Create EFS with AWS SSM Document

Navigate to the AWS Systems Manager (SSM) service and go to "Documents".
Search for "AWSSupport-CheckAndMountEFS" and select the corresponding document.
Click on "Execute automation" in the top right corner.
Choose "Simple execution".
Provide the necessary input parameters: Choose your EC2 instance (only one instance allowed at a time). EfsId: Enter the EFS ID obtained from the EFS console. MountPoint: Specify the desired mount point (e.g., /usr/test). Region: Select your desired region. Action: Choose "CheckandMount".
add your tags.
Click on "Execute" to run the automation.

Step 5: Keep Mounted File After Reboot

In your EC2 server, navigate to the /etc/fstab file.
Add the following line, replacing the EFS ID and directory (/usr/test) with your own:

fs-0c1cf955858757bc7.efs.us-east-1.amazonaws.com:/ /usr/test efs defaults,_netdev 0 0

Step 6: Test EFS Mounting

To unmount the EFS, run the command: umount /usr/test.
Use the command "mount -a" to read the /etc/fstab file and mount all file systems listed in it.
Verify the mounted EFS by running the command: df -hT

Conclusion:

By following the above steps, you can create and mount an Amazon EFS on your AWS EC2 instance using the AWSSupport-CheckAndMountEFS system manager runbook. This enables you to leverage scalable and shared file storage for your applications.

Links:
https://docs.aws.amazon.com/systems-manager-automation-runbooks/latest/userguide/automation-awssupport-check-and-mount-efs.html
https://docs.aws.amazon.com/efs/latest/ug/automount-with-efs-mount-helper.html

Step-by-Step Guide: Creating an AWS Client VPN Connection with Peered VPC

Samir — Tue, 04 Jul 2023 09:45:54 +0000

Introduction:

In this blog post, we will guide you through the process of setting up an AWS Client VPN connection with a peered VPC.
This step-by-step tutorial will walk you through the necessary prerequisites and configuration steps to establish a secure VPN connection between your client and an AWS environment, while also enabling connectivity to resources in a peered VPC.

Prerequisites:

Before we begin, make sure you have the following prerequisites in place:

Download Open VPN from the following link: https://openvpn.net/client-connect-vpn-for-windows/
Refer to the AWS Documentation here for detailed instructions: https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/client-authentication.html

Step 1: Generate AWS Certificates

To start, follow these steps to create the required certificates:
(For Windows)

Download the "EasyRSA releases" from the AWS documentation and extract the files to your Desktop (not on a drive).
Open the command prompt as an administrator and execute the following commands:
.\EasyRSA-Start.bat
./easyrsa init-pki
./easyrsa build-ca nopass
./easyrsa build-server-full server nopass
./easyrsa build-client-full client1.domain.tld nopass
exit
Open the AWS Certificate Manager (ACM) and import two certificates: one for the server and one for the client.
Server files
Certificate body: EasyRSA-3.1.2 > pki > issued > server.crt
Certificate private key: EasyRSA-3.1.2 > pki > private > server.key
Certificate chain: EasyRSA-3.1.2 > pki > ca.crt
Client files
Certificate body: EasyRSA-3.1.2 > pki > issued > client1.domain.tld.crt
Certificate private key: EasyRSA-3.1.2 > pki > private > client1.domain.tld.key
Certificate chain: EasyRSA-3.1.2 > pki > ca.crt

Step 2: Create an AWS Client VPN

Now, let's create the AWS Client VPN with the following configuration:

Name: My-VPN
Client IPv4 CIDR: Define an address range that does not overlap with the target network, VPC address range, or any associated routes.
Server certificate ARN: Choose the server certificate you imported earlier.
Authentication options: Select "Use mutual authentication."
Client certificate ARN: Choose the client certificate you imported earlier.
Enable log details on client connections: Yes
CloudWatch logs log group name: Create a new CloudWatch log group.
Client connect handler: keep default
DNS server 1 & 2: keep default
Transport protocol: TCP
Enable split-tunnel: Yes (to maintain local internet connectivity)
Security group IDs: Create a new security group allowing all traffic from 0.0.0.0/0
VPN port: 1194
Enable self-service portal: Not allowed for mutual authentication.
Enable client login banner: "Welcome to companyVPN".

Step 3: Configure AWS Client VPN Service

Next, configure the AWS Client VPN service as follows:

Target network associations: Associate the target VPC and subnets you want to connect to.
Authorization rules: Add an authorization rule allowing all users from the VPC CIDR.
Route table: The route table will be added automatically.

Step 4: Download Client Configuration

Download the client configuration file from the AWS console.
Open the file in a text editor (e.g., Notepad or VSCode) and make the following changes:

Add the client certificate file "client1.domain.tld.crt" in line 2: Contents of client certificate (.crt) file
Add the client key file "client1.domain.tld.key" in line 2: Contents of private key (.key) file
Add a subdomain (e.g., "company") before ".cvpn" in line 4: remote company.cvpn-endpoint-0366a1a56fdf0ef53.prod.clientvpn.ap-south-1.amazonaws.com 1194

Note:
After completing the configuration, share the client configuration file with your team members who will be using the VPN to access the AWS environment. They can then connect by uploading the file to OpenVPN and clicking "Connect."

Connect Client VPN to Peered VPC:

If you want to connect your Client VPN to another VPC, follow these additional steps:

Step 1: Create a VPC peering between both VPCs:

Go to "Peering connections" and create a peering connection.
Choose the "Requester" and "Accepter" VPCs, and fill in the required details.
Accept the peering request.

Step 2: Configure VPN Client-to-Server (C2S):

In the Client VPN configuration, go to Authorization rules and add an authorization rule for the peered VPC.
Create a route for the other VPC CIDR and choose a subnet in the Client VPN VPC for the target network association.

Step 3: VPN Route Table:

Create a route in the VPN route table for the destination VPC CIDR.
Add the subnets in the Client VPN VPC as the target network association.

Step 4: Subnets Route Table:

In all the subnets connected with the Client VPN in the Client VPN VPC, add a route for the full destination VPC CIDR with the VPC peering as the target.
In all the subnets connected with the peered VPC, add a route for the full Client VPN VPC CIDR with the VPC peering as the target.

Step 5: Security Groups:

For the server or service you want to connect to using the Client VPN, add an inbound rule allowing all traffic from the Client VPN VPC CIDR as the source.
By following these steps, you can establish a VPN connection between your Client VPN and another VPC, enabling secure communication between the two environments.

Remember to modify the instructions and configurations according to your specific requirements and AWS region.

Conclusion:
In this blog post, we have provided a comprehensive step-by-step guide to creating an AWS Client VPN connection with a peered VPC. By following these instructions, you can set up a secure VPN connection and enable communication between your client and the AWS environment, as well as connect to resources in a peered VPC. Enjoy seamless and secure connectivity to your AWS resources!