Here are some of my thoughts on the future of backend/server-side development. Feedbacks welcome!
Proposition 1 - Thin Server, Rich Platform
Now that scalability and security are mostly handled by the infrastructure and database layers themselves, the backend code is becoming more vanilla than ever. The complete decoupling of the front-end, native-like performance on the client-side platforms and modern frameworks and cloud services are allowing us to keep our backends thin. Serverless, microservices, containerization, and asynchronous data processing are a few latest big changes in server-side development. Rich infrastructures on top of these technologies are going to make traditional style backend development obsolete in the near future.
Proposition 2 - Dynamic Language or DSL
One common conception of object-oriented programming or programming, in general, is that our code should resemble the real world to make it easy to develop and manage. However, this particular philosophy leads to a deep coupling of the business logic and the code and makes it unnecessarily hard to make fast changes. The computing world is way more flexible and dynamic than the real world. So using real-world constructs in programming makes it artificially limited. The 1:1 modeling approach used to work really well when the software did not really need frequent changes. In the new world, where specs pivot every day, we should aim for dynamicity. This is already repopularizing the usage of newer dynamic languages for backend development.
Proposition 3 - CMS and Simulation
Decoupling the business rules from the code is super important. Everything that’s configurable should be coming from some sort of configuration management system (CMS). The code should interpret the configurations and work like simulators. Softcoding is not an anti-pattern anymore. The CMS can handle the versioning of configurations. The same configuration can be shared amongst multiple micro-services and client applications. Product managers would get an intuitive interface to make quick product changes. Coupled with an A/B testing framework with some gradual rollout mechanism we can release simple changes pretty quickly across all applications.
Proposition 4 - Resiliency via Technical Intelligence
Instrumenting the code as important as unit testing or any other quality assurance measures. Build a culture of build-measure-learn from the engineering perspective. Modify the platform to automatically log all important metrics like exceptions, API execution time, queue size, etc. Log all unusual situations by default, over-logging is better than under logging. The worst feedback loop is waiting for your customers to complain about what’s not working. Have real-time technical/operational insights, be proactive at discovering and fixing problems, and make your system resilient by automating your runbooks.
Now that you know my take on this, please share your opinion.
Oldest comments (7)
Interesting thoughts!
I'm not sure where you get the idea that dynamic languages are faster and/or more flexible. The type system may be 'more flexible' but that does not equate to flexibility in expressing a domain.
I would contend that decoupling business rules from code makes for a muddled mess in the code. The clearer the business rules are encoded into code the more easily you can change it rather than building a fragile CMS system designed for only a few use cases (CRM / blog / shopping cart / niche business unit). CMSes are not a panacea.
I hardcore believe in logging/monitoring/observability. It's the only info we really get from prod.
Serverless and microservices are more buzz than real change. Asynchronous processing also not a something new. So, in near (and not so near) future traditional backend development will remain largely unchanged, although internally there will be big shift towards asynchronous processing.
I worked at a small creative agency that switched from WordPress to serverless solutions. Webapps, websites and one pagers were not the main product but there were a decent amount of clients who requested them.
Because of the focus the dev team was extremely small (three people), the switch was really nice because front-enders only needed to know some nodejs to get traditional back-end stuff to run. With some services CI, auth and a CMS are covered as well. I think it could be a game changer for small agencies like that.
I'm pretty sure that there are use cases for serverless, especially at initial stages or for (functionally) small backends. But this is just a tiny piece of huge IT sector. From this point of view serverless is just slightly relaxes constant very high demand for developers able to do traditional backend development.
A lot of this sounds like ideas that were expressed in The Pragmatic Programmer 20 years ago.
I definitely don't think that traditional backend development is going to go away - serverless and microservices are each specific engineering tradeoffs, which come with their own set of pros and cons. I definitely don't believe that they're the right answer for every solution.
I think you're on-point regarding the business rule separation, though doing that well seems like an extremely complex problem.
Also, definitely agree with the logging. I think, if nothing else, the observability tools coming out of this shift to microservices is a massive win for developers as a whole. That stuff makes the debugging orders of magnitude easier!
Two points missing:
Microservice and over logging in cases even without microservice architecture leads to harder to remove data. Even if you delete user data like images, DB records and what else, they stay in backups, app logs, network logs... So you see the point a lot of GDPR issues lately and such things. It might revert to having less logging and less "microservice" approach. Especially DB per microservice approach where some companies have thousands of sercices = thousands of DBs. But you do have to agree to terms & conditions when using software so I might be badly wrong about this one.
It's not cheap setting up microservices that are well logged, message drivent etc etc. Even serverless for that matter. It looks cheap. It's not. Ever tried "renting" Kinesis, Event Grid/Hub, or setting up Apache Pulsar or Apache Kafka and checking how much price goes up? I did and my case was it's not easy to have unless you have money. A lot of events happen, lot of stuff stored. Try having to build amazing system with budget meant for less than a simple blog :D. I'm a bit blowing it out of proportion but 2000$ per month cost is simple development environment for things I do. And it's something I smashed together to have less services running around and using free subdomains and load balancer from cloud providers. Now imagine what happens when I split it up in at least 10 APIs and each has it's own full blown database. Not gonna happen, I'll keep it for now "API driven architecture" where I have services accessing same DB server even same database on that server until we figure out stuff with this. Paying for providers that would lower the cost per hour seams like 2x the price as they charge for backups, licenses for support and such, where I set scripts to backup full DB disks and just cry afterwards.
So no, I don't think backend will be "vanilla". Imagine multi tenant systems running thousands of users for each tenant. You create each user in DB? What if, as in my case, users can cross jump tenants? How do you deal with access rights in DB when using something like OAuth2 and it's roles? You check the roles from token or ask the service to decide weather user has access rights for something usually. How do you fit in DB here or even infrastructure, do you update OIDC token with roles and then have all DBs for all microservices update user access? Many cases where it can't happen. I think this was all mainly good for small to mid sized projects. Massive amount of software is still monolith and with all the effort cannot be replaced with micro services not because "it's tightly coupled" or bad architecture but because it's a domain that fits well with having everything so connected it makes no sens to split it. If you have no class makes no use for you to be able to still drive a car there and wait. Why would you do it? Wait until classes are available back again? A lot of people have been reverting back to monolith or switching to "API approach" where they don't follow all the rules of microservices nor have to have ESB as in SOA.
Just mine point of view on it from other peoples experiences but also mine.
Almost 2 years down the line, I wonder what is your opinion now