First off, in an attempt to avoid any flame wars, nothing I say is definitive. I'll just give a few opinions and hopefully some are helpful.
Second, trends in tech are hard to spot. And can be quick to change. The industry "bible" on trending tech is the Technology Radar, put together by ThoughtWorks. These people know a thing or two about tech, so I will expand on some of the relevant "adopts" for this topic.
Lets pick a few technologies I think you should be using/trying/reading about if you are interested in building scalable web apps right now.
TL;DR
Looking at the strong adoption of cloud compute, microservices and serverless products in scalable web applications I think moving forward we will see more focus on the technologies that suit these patterns the most. A few to look at, in my opinion, are:
Typescript: Adding static type checking to JS is a very good idea as frontend applications become more and more complex code bases.
Terraform: With the adoption of microservices and cloud solutions comes IaC. Terrafrom are doing a great job in this space.
Micro Frontends: Be careful with this one. As with microservices on the backend, fools rush in! A balanced and calm approach to the adoption of micro frontend patterns could be a great option for scalable web application clients. SPAs are hard!
golang: Making a come back after a drop in popularity, golang is a great fit inside serverless. This one is worth learning (until Rust knocks it off its pedestal)
Web Assembly: As we have seen with Web Components, W3c initiatives take time! Keep an eye on web assembly, it could open up a world of opportunity to browser based applications. I already have a colleague playing with this stuff and it has great potential. Native speed applications written in the language of your choice that run in the browser. Let's just leave that out there for a moment...
Aurora Serverless: Scaling you middle tier had become less hands on with the progression of cloud compute. A trend to nosql has lead to developers being responsible for building back in all that is lost from sql database management applications. But nosql scales massively and with less effort than a postgresql cluster. Not with serverless db services. Aurora is an example, and could lead a charge back to sql as more developers see the advantages of relation based data schemas. This is not a move away from nosql, rather a move towards a combination of DB types. The right tool for the right job.
A deeper dive
Tools
Before we get started on web app specifics, let's think about some of the tools you should be investing in.
Typescript
It seems to me if your journey in to development was through web applications it is possible that your first experience of a programming language was JavaScript. If so, you will have been spared exposure to a system called strongly typed languages that facilitate a thing called Type Checking. This is done at compile time or, even better, by you code editor!
Others who made the journey from, for example, C# or Java into the world of web, will often be found lamenting the loss of the ability to statically declare the type of the data intended to be pointed to in a variable. Like the philosopher in the allegory of the cave, there is not going back.
Something that is clear now is that cloud compute is the solution for almost all infrastructure requirements in the modern era. There are use cases for owning and managing your own tin, but even then it will be running something like VSphere and internally consumed as cloud compute anyway.
As the microservice flavour of SOA shows more and more that it can be a great choice for scalable web applications, the adoption of Automated Infrastructure through Infrastructure as Code (IaC) propositions will be on your radar. Right now, Terraform is becoming a strong contender in this space, mostly due to its large and active community. At Kano we leverage a lot of Amazon services and in conversations with their engineers it seems that Terraform is even quicker to react to changes in the AWS service catalogue than the Cloud Formation team (anecdotal only, I offer no data on this point!).
So check out Terraform, it will be important as you build you web application infrastructure.
Micro Frontends
Anyone who has built (or more horrifyingly, taken over) a single page application (SPA) will be able to tell you tales of code splitting and routing delights. There are tools dedicated to helping with some of the optimisations that are essential for any SPA of more than a little complexity. I am interested in how to open up access to the wealth of web applications we now have to developing markets (see below). So we need to reduce the size of downloads, we need to think about data transfer and low powered devices running our apps.
Micro Frontends do two things to help with this. The separation of your client application code into services will allow targeted application loading. But it also will set up a mindset of building out components that communicate through published APIs, and message passing. This facilitates the leveraging of other emerging patterns, such as agent based UI designs that allow you to launch certain UI rendering tasks in web workers and get back the state of a UI component from the render process asynchronously. I think. Maybe?
Golang for serverless
I built a lot of middle tier i nodejs using a framework called express. SO much so I wrote an opinionated library to build them out in the pattern I was using.
However, as I started to use AWS' Serverless product I started to see that maybe node is not the best solutions for a lambda.
The popularity of node can be attributed to a range of things, but the value proposition is its non-blocking IO implementation. This allows for a single threaded application to accept and handle potentially thousands of connections (as long as you are using it for the right things). The design of lambda means that each invocation of a function is tied to one request. Once the request is completed it becomes available for the next request. Any additional requests that arrive to a serverless deployment are handled by a separate lambda in its own deployment (container). So the main advantage of node being build on the V8 engine is lost.
Now we want to keep the number of concurrent lamda invocations down, as they each cost money. SO we want the function running inside our lambda to be as fast as it can be, often from a cold start.
Compare Javascript and python to Golang in a head to head race on getting stuff done and you will see why I think golang is a good fit for my lambda code! So give it a go.
Plus, career wise, it is a good tech to ad to your polyglot CV.
Web Assembly
This one could be good for web apps. But as of yet I have not done a lot of work with it. But watch this space. Plus there is a tool that combines web assembly and typescript: AssemblyScript
AssemblyScript compiles strictly typed TypeScript (basically JavaScript with types) to WebAssembly using Binaryen. It generates lean and mean WebAssembly modules while being just an npm install away.
What I would like to happen
I have always been interested in how we can take all the richness and complexity in web applications of today and make them available to the billions of people who have a digital device but do not have the latests versions nor the highest bandwidths. I was hopeful that Progressive Web Apps and We Components would be a path to achieve this. So please keep those in mind as you move forward with all this.
What are some of the key trends you see as important to highly scalable web apps in the next few years?
Hi there Ben. Thanks for engaging with my #ama :)
First off, in an attempt to avoid any flame wars, nothing I say is definitive. I'll just give a few opinions and hopefully some are helpful.
Second, trends in tech are hard to spot. And can be quick to change. The industry "bible" on trending tech is the Technology Radar, put together by ThoughtWorks. These people know a thing or two about tech, so I will expand on some of the relevant "adopts" for this topic.
Lets pick a few technologies I think you should be using/trying/reading about if you are interested in building scalable web apps right now.
TL;DR
Looking at the strong adoption of cloud compute, microservices and serverless products in scalable web applications I think moving forward we will see more focus on the technologies that suit these patterns the most. A few to look at, in my opinion, are:
A deeper dive
Tools
Before we get started on web app specifics, let's think about some of the tools you should be investing in.
Typescript
It seems to me if your journey in to development was through web applications it is possible that your first experience of a programming language was JavaScript. If so, you will have been spared exposure to a system called strongly typed languages that facilitate a thing called Type Checking. This is done at compile time or, even better, by you code editor!
Others who made the journey from, for example, C# or Java into the world of web, will often be found lamenting the loss of the ability to statically declare the type of the data intended to be pointed to in a variable. Like the philosopher in the allegory of the cave, there is not going back.
And then we got Typescript. You should use it.
Terraform
Something that is clear now is that cloud compute is the solution for almost all infrastructure requirements in the modern era. There are use cases for owning and managing your own tin, but even then it will be running something like VSphere and internally consumed as cloud compute anyway.
As the microservice flavour of SOA shows more and more that it can be a great choice for scalable web applications, the adoption of Automated Infrastructure through Infrastructure as Code (IaC) propositions will be on your radar. Right now, Terraform is becoming a strong contender in this space, mostly due to its large and active community. At Kano we leverage a lot of Amazon services and in conversations with their engineers it seems that Terraform is even quicker to react to changes in the AWS service catalogue than the Cloud Formation team (anecdotal only, I offer no data on this point!).
So check out Terraform, it will be important as you build you web application infrastructure.
Micro Frontends
Anyone who has built (or more horrifyingly, taken over) a single page application (SPA) will be able to tell you tales of code splitting and routing delights. There are tools dedicated to helping with some of the optimisations that are essential for any SPA of more than a little complexity. I am interested in how to open up access to the wealth of web applications we now have to developing markets (see below). So we need to reduce the size of downloads, we need to think about data transfer and low powered devices running our apps.
Micro Frontends do two things to help with this. The separation of your client application code into services will allow targeted application loading. But it also will set up a mindset of building out components that communicate through published APIs, and message passing. This facilitates the leveraging of other emerging patterns, such as agent based UI designs that allow you to launch certain UI rendering tasks in web workers and get back the state of a UI component from the render process asynchronously. I think. Maybe?
Golang for serverless
I built a lot of middle tier i nodejs using a framework called express. SO much so I wrote an opinionated library to build them out in the pattern I was using.
However, as I started to use AWS' Serverless product I started to see that maybe node is not the best solutions for a lambda.
The popularity of node can be attributed to a range of things, but the value proposition is its non-blocking IO implementation. This allows for a single threaded application to accept and handle potentially thousands of connections (as long as you are using it for the right things). The design of lambda means that each invocation of a function is tied to one request. Once the request is completed it becomes available for the next request. Any additional requests that arrive to a serverless deployment are handled by a separate lambda in its own deployment (container). So the main advantage of node being build on the V8 engine is lost.
Now we want to keep the number of concurrent lamda invocations down, as they each cost money. SO we want the function running inside our lambda to be as fast as it can be, often from a cold start.
Compare Javascript and python to Golang in a head to head race on getting stuff done and you will see why I think golang is a good fit for my lambda code! So give it a go.
Plus, career wise, it is a good tech to ad to your polyglot CV.
Web Assembly
This one could be good for web apps. But as of yet I have not done a lot of work with it. But watch this space. Plus there is a tool that combines web assembly and typescript: AssemblyScript
What I would like to happen
I have always been interested in how we can take all the richness and complexity in web applications of today and make them available to the billions of people who have a digital device but do not have the latests versions nor the highest bandwidths. I was hopeful that Progressive Web Apps and We Components would be a path to achieve this. So please keep those in mind as you move forward with all this.
Oh, and Flutter of course.