Introduction
On a cold and dark evening in December 2022, a good friend of mine calls me and says: "Nicolas, I am creating a product tha...
For further actions, you may consider blocking this person and/or reporting abuse
Using EKS on a app with only 2 components and 2 people working on it definitely seams like overkill to me. Any reason why you didn't opt to run it on something like Beanstalk or Heroku?
Definitely, for many reasons actually:
ECS seems better
ECS still doesn't support ConfigMaps (well, the equivalent) which quickly becomes a nuisance.
@nick well you have many choices: AWS secret manager or use env file from s3: docs.aws.amazon.com/AmazonECS/late...
I'm surprised client VM (+ VPN) is still a thing, with session manager allowing RDP as well. Much more secure and simpler thing would be to have proper IAM controls around session manager with some "client VM" like AMIs.
How do you provide direct access to private resources for developers?
Via the AWS console. you can have a Linux or Windows VM (whatever the prefer) with desktop clients like workbench/pgadmin/kubectl. The Dev eks doesn't really need to have a private endpoint unless it's a specific requirement, and provided you haven't networked Dev and prod VPCs, as Kubectl is easier to have locally as well as using docker for MySQL/postgresql, thus guiding developers to develop everything as code.
Give fleet manager/session manager a go. The "client VMs" don't need to be in the public subnet either, see how you could use it for your use case, it does make like easier.
I bet if you used Google Cloud. Cloud Run for the service. Something like Firestone for the DB. You can get crazy low bills, like sun $50/m and have just as much power.
GCP has the best free tier. If cost is a major factor.
Each time I add codebuild and codepipeline into the mix my deploy times go through the roof. I always think I may be doing something wrong. I come from a RoR background and my Capistrano deploys take all of 30 seconds. Can you share how long it takes from the time code is commited in repo, to server with new code being up and running? With codepipeline and ecs this always takes about 10 minutes for me, even with dead simple apps which IMO is unacceptable.
Hi Augusto,
Do you suffer from a delay before the pipeline starts? or the build and deployment times take 10 minutes from the moments the pipeline starts execution?
Delay is on the pipeline / codebuild side. Basically it takes a LOONG time for codepipeline to spin up the new code version Fargate ECS containers into the cluster, have them as available, and then remove the previous code versions.
I realize you're not using Fargate but EKS. However, I'd like to know in your opinion, what would an "acceptable" deploy time for containerized deploys be, as compared to more traditional deploy methods.
@augustosamame I think I know what the problem is, if the ECS tasks are part of a Target Group, navigate to this Target Group and modify the de-registration delay to 10 seconds or an even smaller number.
This will reduce the time.
I like this post a lot!!
Would be happy to see a tutorial of all that.
It can be gold for any learner or tech lead, like your friend, who wants to create a
Reliable infrastructure to his team and product.
Thank you for your kind words @amitkad. I already have a free Udemy course (Introductory course). Please check it out and leave a feedback: udemy.com/course/intro-fullstack-d...
I hope to create more tutorials in the future.
Great write up. You said they have a mono repository using nx for the build system. Do you mind sharing some details of how you structured the IAAC with the AWS CDK for the mono repo? I have a project that uses a mono repo and NX as well and I have been struggling to figure out what a good approach is for separating my CDK code from my application code. My current approach now is having an infra/ directory at the root of the repository with multiple nested folders for my different infra to keep my build and deployments separated. E.g., infra/frontend and infra/backend.
Hi @christiankaseburg,
I don't think there is one solution that fits all. I started out similar to your method, by including a folder in the root directory containing subfolders and files.
But eventually, you would want to separate this from developers to avoid unwanted changes or errors.
In my opinion à quick solution would be to either create a separate repository for it, or store it in some file storage such as S3.
Have the pipeline download it before executing commands.
I hope this answers your question.
Interesting read, if I read correctly, scale and possible future complexity are the main drivers.
Did you use the serverless RDS? And fargate eks? Possible reduced cost and complexity
Could ECS be used, or do they need an exit strategy off Aws?
Could scaleway (same tech, postgres and k8s) be used to reduce costs?
(Just wondering)
Hi Dave! How are you?
Yeap, you're right, the main driver behind this infra is to minimize headaches in the future.
Yes I used Aurora. No I did not use Fargate Eks, but definitely worth investing in it.
I avoided ECS for many reasons, especially lock in, and lack of portability. Once the application is stable on kubernetes, I can easily redeploy it anywhere (such as on premise)
Do you have cost breakdown of this solution? 500$/month seems reasonable but looking at the picture, it looks quite expensive. I am wondering how much cost comes from each component? Are there usage charges like egress bandwidth etc.?
Did you try to Amplify Hosting?
Unfortunately, Amplify will not meet all the requirements stated in the article.
What did you end up using for observability? Essentially the tracing part.
Also did you authored any IaC for infrastructure orchestration?
Hi Adeel,
Cloudwatch for Metrics. Tracing was not implemented yet, as it needs development effort. But in my opinion, either use X-Ray, or a custom built mechanism that integrates with Cloudwatch logs.
Thank you! Any pointer on how to learn Kubernetes in the fastest way?
Hi,
There are multiple courses and tutorials. However, in my opinion, Kubernetes is an advanced and complicated tool. The only way to learn is to actually be involved in a large project that uses Kubernetes heavily and endure all the headaches and head-banging until everything makes sense :))) That's how I learned it. Painful, but worth it
Nicely written.
How front end communicating with back end? Are you using API gateway?
Hi Nisam,
The backend deployed on Kubernetes is fronted by an internet-facing ALB. The frontend application communicates with this ALB, which in turn redirects the traffic to the backend service.