I wish I could give each of these replies multiple likes, because they're so, so good. Thank you @ahferroin7
for the phenomenal explanation.
Some follow on questions to your comment:
I've never fully understood Cloudflare...I understand they're a CDN, but do they cache / speed up access to ENTIRE sites, or images only, or JS/CSS etc?
Why would I need DNS hosting, like Route53? Aren't DNS services free??
What are some use-cases for EC2 (elastic compute capacity)? Is it essentially a replacement of a VPS like Linode? I did take a look at this, btw.
Given that I'm on a VPS (Linode), does it make sense to be using that, and AWS for:
I'm a Systems Reliability and DevOps engineer for Netdata Inc. When not working, I enjoy studying linguistics and history, playing video games, and cooking all kinds of international cuisine.
I've never fully understood Cloudflare...I understand they're a CDN, but do they cache / speed up access to ENTIRE sites, or images only, or JS/CSS etc?
It depends on how you configure it, but usually it's used for entire sites. The big thing for them (and AWS CloudFront) though is geo-caching. That is, they have dozens of endpoints around the world, and requests through them get routed through the geographically closest endpoint, so you have minimal latency for stuff that's already cached.
Why would I need DNS hosting, like Route53? Aren't DNS services free??
If, for example, you wanted to use your own domain name to serve data from. AWS also provides domain registrar services through Route 53, so you can use them to register your own domain as well (and it's essentially at cost for that, they have very little markup). Some VPS offerings like Linode provide really basic free DNS, but you have to go through their specific naming scheme. S3 has a similar thing where you don't need a domain to serve data from it, but you have to live with their domain layout.
What are some use-cases for EC2 (elastic compute capacity)? Is it essentially a replacement of a VPS like Linode? I did take a look at this, btw.
The big thing with EC2 is that unlike all-in-one offerings like Linode (or AWS Lightsail), billing is itemized per resource. IOW, you pay separately for the storage for the VM (note that this is different from the pricing for S3), the network usage for the VM (which is the same as for S3), and the compute time plus memory for the VM. EC2 has four major benefits over the all-in-one offerings:
If you have a workload that only needs one out of the three in large amounts, it will often be a bit cheaper than an all-in-one offering. For example, if you mostly need bandwidth and not storage capacity or computing power, you can pay for only a small disk, a very lightweight VM, and a lot of bandwidth. You have to have a very asymmetrical workload for this to pay off though.
It offers much higher capacities than many all-in-one VPS offerings. Linode is the only one that I know of that comes close (with it's 48-core offering) to the biggest EC2 instances (with 96 cores). There are also much higher memory options (up to 3/4 of a TB of RAM).
It provides better options for automatic scaling of infrastructure. This is really the biggest selling point for EC2, as you can have limits set up to automatically scale up or down by starting or stopping extra instances of a VM as your actual usage changes. Because of how the bill is itemized, this is much easier to keep track of when you actually pay the bill at the end of the month.
On a similar note to scaling, EC2 provides a rather neat feature called 'spot instances', which are essentially a way to say 'run X many instances when the price of running them is less than Y per hour'. Essentially, you end up renting spare computing power, which varies in price depending on actual utilization of resources in the region.
Note, however, that if your needs closely match up with a specific offering from an all-in-one provider, EC2 will usually be more expensive than going with the all-in-one offering.
SES + SNS for emails
Probably not. SES and SNS are really designed for very large scale messaging management, and the interfaces for working with them really reflect that. Unless you're talking about dealing with hundreds of thousands of emails to send, they're probably overkill.
S3 for storage and serving media files
Probably, but I'd suggest looking very closely at pricing. There's a nice calculator web app provided by AWS that can help you figure out what your costs would look like. Most likely, it will be worth it for your usage, but keep in mind that your monthly bill will fluctuate based on actual usage.
Amazon CloudFront for the CDN?
Possibly. Even if you're using S3 for storage, I'd generally recommend CloudFlare over AWS CloudFront in most cases. Keep in mind that you will probably need your own domain though for either of them. Route 53 is what I'd recommend for that, as it will effectively only cost a few USD a month plus the registration fees for whatever domain you pick (which are billed yearly, and typically only cost 10-20 USD depending on what TLD you're using).
@ahferroin7
Austin, phenomenal explanation, thank you so much for this.
I finally understand a lot about what these services do, thanks to you. Also, thanks for the tip about Cloudflare vs Amazon Coudfront. It shows you've got a ton of experience in this domain, and you're exactly the type of person whose opinion I needed.
I read your explanation twice.
Route 53: I already have a domain (renewal costs $15/yr), and DNS set up with Linode (costs $10/mo). The domain (and all it's subdomains) resolve fine. So, I don't see the need for Route 53....unless I am missing something.
Is there a downside to using SES + SNS for emails, if the number of emails is in the hundreds only (small numbers)? I ask because I already have it configured and set up (which was quite painful if I'm being honest).
I'm a Systems Reliability and DevOps engineer for Netdata Inc. When not working, I enjoy studying linguistics and history, playing video games, and cooking all kinds of international cuisine.
Route 53: I already have a domain (renewal costs $15/yr), and DNS set up with Linode (costs $10/mo). The domain (and all it's subdomains) resolve fine. So, I don't see the need for Route 53....unless I am missing something.
Ignoring the transfer fees, Route 53 is probably less expensive unless you're seeing many millions of queries. It's 0.50 USD per hosted domain per month, and 0.10 USD per million standard queries per month. The registration costs are probably the same (though worth looking into, because they might not be). I think there's a one-time transfer cost if you want to switch registration to AWS (which would make other Route 53 setup easier), but I'm not sure.
So, unless you're really busy, you'd probably end up paying less than 1 USD a month through Route 53 (provided you don't use any of the fancy features like monitoring and failover). It may also be marginally faster for people who aren't located close to Linode datacenters (AWS has a lot more datacenters, and they automatically route the queries through the closest datacenter).
Is there a downside to using SES + SNS for emails, if the number of emails is in the hundreds only (small numbers)? I ask because I already have it configured and set up (which was quite painful if I'm being honest).
If you've already got it set up and it works for you, there's probably not much of a downside unless you can find a less expensive alternative. The biggest issue with it is how much effort it takes to set up.
Thanks @ahferroin7
! It definitely looks like I should give Route 53 a go at some point in the future when I have more time. It's low onthe priority list, though.
All said and done, your tips have been a great help. Following you on DEV, and I hope you don't mind the odd question on AWS every now and then..
I'm a Systems Reliability and DevOps engineer for Netdata Inc. When not working, I enjoy studying linguistics and history, playing video games, and cooking all kinds of international cuisine.
I wish I could give each of these replies multiple likes, because they're so, so good. Thank you @ahferroin7 for the phenomenal explanation.
Some follow on questions to your comment:
Given that I'm on a VPS (Linode), does it make sense to be using that, and AWS for:
It depends on how you configure it, but usually it's used for entire sites. The big thing for them (and AWS CloudFront) though is geo-caching. That is, they have dozens of endpoints around the world, and requests through them get routed through the geographically closest endpoint, so you have minimal latency for stuff that's already cached.
If, for example, you wanted to use your own domain name to serve data from. AWS also provides domain registrar services through Route 53, so you can use them to register your own domain as well (and it's essentially at cost for that, they have very little markup). Some VPS offerings like Linode provide really basic free DNS, but you have to go through their specific naming scheme. S3 has a similar thing where you don't need a domain to serve data from it, but you have to live with their domain layout.
The big thing with EC2 is that unlike all-in-one offerings like Linode (or AWS Lightsail), billing is itemized per resource. IOW, you pay separately for the storage for the VM (note that this is different from the pricing for S3), the network usage for the VM (which is the same as for S3), and the compute time plus memory for the VM. EC2 has four major benefits over the all-in-one offerings:
Note, however, that if your needs closely match up with a specific offering from an all-in-one provider, EC2 will usually be more expensive than going with the all-in-one offering.
Probably not. SES and SNS are really designed for very large scale messaging management, and the interfaces for working with them really reflect that. Unless you're talking about dealing with hundreds of thousands of emails to send, they're probably overkill.
Probably, but I'd suggest looking very closely at pricing. There's a nice calculator web app provided by AWS that can help you figure out what your costs would look like. Most likely, it will be worth it for your usage, but keep in mind that your monthly bill will fluctuate based on actual usage.
Possibly. Even if you're using S3 for storage, I'd generally recommend CloudFlare over AWS CloudFront in most cases. Keep in mind that you will probably need your own domain though for either of them. Route 53 is what I'd recommend for that, as it will effectively only cost a few USD a month plus the registration fees for whatever domain you pick (which are billed yearly, and typically only cost 10-20 USD depending on what TLD you're using).
@ahferroin7 Austin, phenomenal explanation, thank you so much for this.
I finally understand a lot about what these services do, thanks to you. Also, thanks for the tip about Cloudflare vs Amazon Coudfront. It shows you've got a ton of experience in this domain, and you're exactly the type of person whose opinion I needed.
I read your explanation twice.
Route 53: I already have a domain (renewal costs $15/yr), and DNS set up with Linode (costs $10/mo). The domain (and all it's subdomains) resolve fine. So, I don't see the need for Route 53....unless I am missing something.
Is there a downside to using SES + SNS for emails, if the number of emails is in the hundreds only (small numbers)? I ask because I already have it configured and set up (which was quite painful if I'm being honest).
Thank you, again!
Ignoring the transfer fees, Route 53 is probably less expensive unless you're seeing many millions of queries. It's 0.50 USD per hosted domain per month, and 0.10 USD per million standard queries per month. The registration costs are probably the same (though worth looking into, because they might not be). I think there's a one-time transfer cost if you want to switch registration to AWS (which would make other Route 53 setup easier), but I'm not sure.
So, unless you're really busy, you'd probably end up paying less than 1 USD a month through Route 53 (provided you don't use any of the fancy features like monitoring and failover). It may also be marginally faster for people who aren't located close to Linode datacenters (AWS has a lot more datacenters, and they automatically route the queries through the closest datacenter).
If you've already got it set up and it works for you, there's probably not much of a downside unless you can find a less expensive alternative. The biggest issue with it is how much effort it takes to set up.
Thanks @ahferroin7 ! It definitely looks like I should give Route 53 a go at some point in the future when I have more time. It's low onthe priority list, though.
All said and done, your tips have been a great help. Following you on DEV, and I hope you don't mind the odd question on AWS every now and then..
Always glad to share my knowledge!
Thank you!