DEV Community

Discussion on: So your boutique dev shop needs to handle Super Bowl ad traffic?

Collapse
 
erebos-manannan profile image
Erebos Manannán

Just as a minor point - I really hate it when people keep bringing up the database issue with the typical "yadi yadi yada MySQL can scale, you don't need X", and completely miss the point.

Everyone is 100% perfectly aware you CAN scale with MySQL, PostgreSQL, hell even with SQLite just as well. The issue isn't the ability to scale, it's the effort and limitations of scaling. With MySQL and similar after your basic caching layers stop being the only solution needed (and you need those in practically any large popular web application anyway) you end up trying to actually scale MySQL, a matter that is not by any means simple.

You've basically got the options of

a) vertical scaling (i.e. a bigger, faster server), that takes you a bit further

b) switching to a multi-master Galera Cluster, which comes with limitations you probably didn't take into account when you set up your site, and switching from your current MySQL setup means downtime and a lot of effort

c) adding read slaves, which ALSO has significant issues and limitations. Not the least is that the replication will just randomly break and you have to manually fix it at 3am on a Friday night after a giant celebration party and you're out of your mind drunk.

d) sharding, which you will have to do manually, in your application logic. You will no doubt do it wrong since there are no good guides out there, and you will spend significant amounts of effort trying to first get your data to shard at all, and then later lose most of your hair when you realize how bad your original design was and how much work it is going to be to change to a new way of sharding

e) buy one of those really expensive MySQL cluster products and hope it solves all of your problems, before realizing that they come with even more limitations and will break your application logic completely and require significant architectural redesign

For most small startups upfronting the cost and time of trying to figure out their possible scalability headaches in the far future, so unless you've got a guy who's intimately experienced with these things you will no doubt have a bad time when you realize you need to scale.

Now, for most people this SHOULD sound like an extremely bad idea, and no SQL database out there really solves this in an affordable way. Also if Galera was "the solution" it should be the default as well, but it's not.

Some alternatives exist, e.g. NoSQL databases.

Now a bunch of NoSQL databases have solved a lot of these growth pain issues in nice ways, and depending on your needs you can e.g. use Cassandra (or the ScyllaDB fork), Riak, MongoDB, or many of the other ones. Just try to understand that ALL databases come with downsides, and you should google for them in advance.

If your NoSQL database of choice doesn't provide you with the searching tools you need that's ok, you probably shouldn't be using SQL for search anyways, try something like ElasticSearch instead.

If you're lucky enough to be able to pick your tools 100%, maybe pick Google Cloud Platform and their Datastore, or if AWS is your poison maybe DynamoDB might be for you. Hell, maybe Azure Cosmos DB is your thing. Either way, any of these tools will significantly reduce the growth pains you will experience.

Collapse
 
ben profile image
Ben Halpern

Yep, definitely agree with this.