I have, of course, looked at Amazon Web Services (AWS) a number of times over the last few years. I currently use S3 as cheap storage for some database backups and I've tried spinning up EC2 and RDS instances (and, yes, like most people, I've been stung by large costs when I've forgotten to take them down again).
But last week I was at the 2019 AWS Summit in ExCel, London and I realised two things. Firstly, I really need to make an effort to get to grips with how Amazon's cloud services work and, secondly, in order to do that I'm going to need to move one of my real projects into the cloud. And I've decided that I'm going to blog about my experiences here.
Let me tell you about the project that I'm going to be using as an example. It's a web site called Line of Succession. The site has one aim - it enables you to see the line of succession to the British throne on any date back to the start of Queen Victoria's reign (the plan is to take this back to 1701 when the current succession rules were enacted - but that needs me to add a lot more data). It has a few moving parts:
- There's a database (currently MariaDB) which contains details of the royal family tree which can be used to calculate the line of succession on any date.
- There's a web application (written in Perl using the Dancer2 framework) which takes a request for the date, does the calculations and presents the correct line of succession to the user.
- The calculated lines of succession rarely change, so once we've calculated one, we can store it and not have to calculate it again. Currently, this pre-calculated data is stored in memcached.
- There's a WordPress blog where I write (hopefully) interesting articles about the royal family.
- And there's nginx which ties the whole thing together, ensuring that HTTP requests are forwarded to the right place.
This is all running on a single server that I rent in a data centre. But that's all very last millennium and I want to do something a bit more up to date with it. Currently, this is what I'm thinking.
- The database will become an Amazon RDS instance.
- The web app will be containerised and run on either Amazon ECS or Amazon Fargate.
- The cache will become Amazon ElastiCache
- The blog will be another container - presumably based on a standard container supplied by the WordPress team.
- I'm not sure what will happen with nginx. Which brings me neatly to...
Some questions that I need to find answers for:
- Should I switch from MariaDB to a different database? How hard would that be?
- How do I choose between ECS and Fargate? What's best for my use case?
- What ElastiCache options should I be considered?
- Is the blog a single container? Is the WordPress database a separate container or, perhaps, another RDS instance?
- Will nginx be another container? Or does AWS have something better I can use?
In fact, the whole routing thing is a big unanswered question for me. I need to do a lot more research into how AWS handles that. And I'm slightly concerned about the switchover. My site has visitors (ok, not many visitors, but still...) and once I have the site working on AWS, how do I seamlessly move the HTTP requests to the new system?
I'm also thinking about improvements. As I said above, the data rarely changes - only when someone is born or dies, or when I add more data. So perhaps building the cached data should be completely divorced from web requests. Perhaps a background job (an AWS Lambda?) could monitor the database for changes and rebuild and invalidate only the required bits of the cache.
So that's the plan. I hope to be able to blog about my progress every couple of weeks or so. If you have any advice about what I'm doing then I'd love to hear it.
Top comments (1)
RDS has a Mysql version but RDS is expensive for personal stuff I think. DynamoDB can be used to store relational data - here's a guide docs.aws.amazon.com/amazondynamodb...
Fargate is just another way of running containers with ECS. It takes away the need for you to run your own EC2 instances.
I'd run the app as a docker container and the blog as a docker container within ECS so each can be deployed separately. Although if you don't care so much for comments and stuff, you could host the blog as a static website in S3 very cheaply. There's also Lightsail that can host a Wordpress blog cheaply but doesn't help with the learning part!
You could replace Nginx with Cloudfront. You can tell it to send certain requests to ECS and others to S3.
If you're doing all this you should also do it all in code rather than using the console. There's a devops group called Cloudposse on Github have have some great Terraform modules you can use as building blocks to build this stuff github.com/cloudposse.