This is the third part of the Deploy Rails in Amazon ECS post. It's part of a broader series called More than 'hello world' in Docker. The series will get you from hello world in Docker to having your application deployed in AWS.
- Build Rails + Sidekiq web apps in Docker
-
Deploy Rails in Amazon ECS
- Concepts
- Push an image to ECR
- Create the RDS database, Task Definition, and Load Balancer - we are here
- Create the ECS Cluster and connect it all together
- Configure Sidekiq
- Automate Deploys with AWS CodeDeploy
- Advanced ECS Concepts:
- Service Discovery and Auto Scaling - coming soon
- Fargate - coming soon
In this post, we will set up three AWS services we need to run a containerized applications on AWS: a database, a task definition, and a load balancer.
6 | Creating an RDS database
Docker containers should be stateless: they can be turned off and on because they don't hold any data or states (or at least not hold states that cannot be retrieved from another source). Databases, by definition, are stateful: they contain data and are considered to be the source of truth. If the data disappears from the database, there's no other trusted source we can count on to get it back. Since containers should be stateless, we should not host our databases in containers.
In a previous post, we use docker-compose to set up a PostgreSQL server in a Docker container. That's okay for development since the data there can easily be reproduced. We cannot do that on a production database because if the container disappears, the data goes with it.
Database-on-containers proponents argue that you should use docker volumes instead so when the container dies, the data does not disappear. Well, you can, but that's added complexity you have to manage. Better to just use a managed database service and not worry about engineering reliability into a solution you can just pay someone else to do.
For the database, we will create an RDS PostgreSQL database.
(6.1) On the services tab, search for RDS and click it. Then, click "Create Database".
(6.2) On the next page, choose PostgreSQL as your database of choice and Free Tier from the template. On the Settings tile, add the DB instance identifier of ruby-docker-database, and the password of your choice. Leave the DB instance size as t2.micro. For connectivity, leave the VPC as the default VPC (take note of the id of this VPC for section 8).
Then, click "Create Database". You will see your database is being created on the next screen:
(6.3) When the database is done creating, click on the database name. On the next page, take note of the endpoint. We will use it to access the database later on.
7 | Creating a Task Definition
A task definition is required for us to run a task in ECS. The task definition defines the resources available to a task. It also holds information about containers that are part of the task. In OOP terms, think of a task definition as the class and a task as an instance of a class: to instantiate a task, you must get information from the task definition.
(7.1) On the services tab, search for ECS and click it. Then, click "Create new Task Definition".
(7.2) Then, choose the EC2 launch type compatibility and click next.
(7.3) Let's name our task definition as docker-rails-app. For the task size, let's give it 512mb of memory and 512 units of CPU. CPU units are a simplified way of quantifying compute capacity. This gives 512mb and 512 CPU units per task spawn. The containers inside one task share this resource.
(7.4) Scroll down and click "Add container". A task definition defines a single task. A task can contain many containers inside it. You can add as many containers as you like, but I recommend just having one per task.
(7.5) Next, add the URI of the Docker image we just uploaded in section 5.4 to the Image field, and "web" in the name field. Also, make sure to add host port as 0 and container port as 8080. Setting the host port as zero allows many containers to reside in one EC2 instance.
Each container inside the host instance has its own port range, and the host instance also has its own port range. On its own, the container and its port range cannot be accessed from outside the instance. By setting the host port as 0 and the container port as 8080, we are exposing the port 8080 of each container to any, random port in the host instance. So imagine if we had 3 containers in the host instance with the IP 10.0.0.123. Each instance will have its own port 8080 but they will be mapped to a different port on the host instance:
-
container_one:8080-->10.0.0.123:16217 -
container_two:8080-->10.0.0.123:11394 -
container_three:8080-->10.0.0.123:2390
This feature is called dynamic port mapping. It makes each container accessible via a unique ip_address:port combination. Hence, we can register it to the load balancer and have traffic directed to it. AWS handles it all in the background for us. All we have to do is put the host port as zero.
In the example, it adds 10.0.0.123:16217, 10.0.0.123:11394, and 10.0.0.123:2390 to the load balancer so traffic can be directed to each container.
The load balancer performs health checks on each container to test if they're still functioning as they should. If a container fails a certain number of health checks, the load balancer deregisters the container so traffic won't flow to it anymore. ECS kills the problematic container and starts a new one.
(7.6) Scroll down to the environments section. For the command, put puma,-C,config/docker_puma.rb,-p,8080. This is the command that Docker will run when starting your container. The command starts Rails' default application server, Puma and routes traffic to the port 8080.
Add the following environment variables:
-
POSTGRESQL_HOST: << the endpoint you took note of in 5.3>> -
POSTGRESQL_USER_NAME: postgres -
POSTGRESQL_PASSWORD: << password you entered on 5.2 >> -
POSTGRESQL_DB: ruby_docker_database -
RAILS_ENV: staging -
RAILS_MASTER_KEY: << if you cloned my repo, usee451d94494f606afa27a4cae3dea3948. if you made your own, use the value inconfig/master.key>> -
RAILS_LOG_TO_STDOUT- ENABLE
The POSTGRESQL_ prepended variables will be used in the database.yml we made in step 4.1. The RAILS_ENV variable is to ensure we are using the staging environment. The RAILS_MASTER_KEY is required for Rails 5 to run.
(7.7) In the storage and logging section, click on Auto-configure Cloudwatch Logs. This will set up Cloudwatch logs to collect application logs from Rails, but the logs have to come from standard out. Enabling RAILS_LOG_TO_STDOUT in 7.6 outputs the rails logs to standard out.
(7.8) Then, click "Add" to close the container definition window. And then, click "Create". You should see something like this:
Since this is the first time we have created a task definition, this is going to be the first version. If you click "Create new revision" from the image below, you will have an opportunity to change any of the settings we made earlier. When you save your changes, it's going to be placed on a new version, version 2.
8 | Create an Application Load Balancer
We talked extensively about dynamic port mapping in section 7.5. Essentially, each container in a host instance will be assigned a random port on the host instance. Host 10.0.0.123 can contain three containers accessible via 10.0.0.123:1911, 10.0.0.123:7421, 10.0.0.123:5819. AWS ECS will take care of registering these ip_address:port combinations to the load balancer. Once registered, the load balancer will distribute incoming traffic to the containers registered to it.
(8.1) On the services tab, search for EC2 and click it. Then, click Load Balancers on the left-hand side menu and click "Create Load Balancer". On the next page, click Application Load Balancer.
(8.2) Let's name our load balancer, rails-docker-alb. Next, choose the VPC we chose in 6.2, and select all of its subnets. Then, click next. Skip "Step 2: Configure Security Settings" by clicking next.
(8.3) Let's create a security group on the next page with the name rails-docker-alb-sg. This security group defines what traffic can enter our load balancer.
Make sure that port 80 is opened for 0.0.0.0/0 (everyone). This will make sure our app will be accessible to the open internet. Then, click next.
(8.4) Create a new target group called default-target. Then, click next. Skip "Step 5: Register Targets" by clicking on next.
(8.5) Review your load balancer, then click create.
That's it! You've created a database, a task definition and a load balancer.
What's next?
On the next post, you will create the ECS cluster and we will deploy our Rails app there. We will finally be able to see our Rails app deployed!
Latest comments (5)
by
you mean:
"the endpoint you took note of in 6.3"
"password you entered on 6.2"
right? :)
Seems to me that POSTGRESQL_HOST matches the image URI, correct ?
Hi Raphael!
Is there a way of configuring the task definition to allow greater http body input?
I mean like if we were configuring nginx I would be doing something like
http {
client_max_body_size: 20M
}
Thank you!
Thank you for your detailed guide. It helped me a lot with ECR/ECS operations.
I think when you package your code into the Docker image,
master.keyfile is included, so you don't need to setRAILS_MASTER_KEY.And I'm also thinking that we should only deploy code committed to git, instead of packaging all code in current PC into the image. Maybe using
git cloneinside Dockerfile instead ofCOPY .?Hi Phan, I'm glad you found the article useful. :D For the
master.key, I explicitly addedRAILS_MASTER_KEYto make sure this works for people who would be cloning my GitHub repository instead ofrails newand starting their own project from scratch. I also think it would be one less step to worry about for people who would want to set up a dedicated build server for building their docker images. But point take that if you already havemaster.keyon your local, then this step will be redundant (not to mention master.key andRAILS_MASTER_KEYwould have different values)I haven't considered using
git clonefor a project like this. I think that could work but my problem with that is you would have to fetch the entire repository every time you build your Docker image. This could take long for big projects. Also, most projects are private so you would have to find a way to pass your SSH credentials to docker while it is building your image.