DEV Community: Aidan Gee

5 Tips to Scale your App

Aidan Gee — Mon, 18 Oct 2021 08:34:46 +0000

Original Post: https://aidangee.dev/blog/5-tips-to-help-scale-your-app-scale

Intro

You want your multichannel application to grow and have more users. But as this grows you will need to be able to handle the increased traffic whilst also balancing the cost of infrastructure. If you get that big brand customer you don't want them to take down your service for themselves and other customers! (seen this first hand!)

The quick solution is often to vertically or horizontally scale the compute, but costs can quickly get out of control.

I have worked on both small and extremely high traffic applications and wanted to share some techniques I have seen successfully used.

1. Serverless

Serverless Architecture is ideal for allowing an initially small web application to scale over time. Serverless infrastructure is both auto-scaling and pay per use. So whilst your app is small you will not need to pay for idle compute costs and when the time comes you have the capacity for significant growth in the future.

This fits well with applications that have a 'spikey' traffic pattern. For example ticket sales or product launches, in these use cases the user traffic is generally condensed into a short window of time and serverless compute like lambda or cloud functions allow for the instant scale needed to support this. Whilst keeping costs low.

This doesn't just end with compute, there is a growing number of serverless database solutions as well. DynamoDB, MongoDB, FaunaDB, Aurora these all offer a serverless way to store your data. Meaning there is little to no infrastructure for you to worry about managing across your stack.

Example of a Serverless Application built on top of AWS:

What to check out: Serverless Framework, Lambda, Cloud Functions, Cloud Run, Cloudflare Workers, Netlify

2. Client side rendering / Pre-rendering / Hybrid

Depending on the user experience you need to provide and your SEO requirements, the way you render your UI can make a big difference to how your application scales.

If your application is purely server side rendered, it might be worth asking yourself the question "does it need to be?". Moving some server-side work to the user's browser or pre-rendering it at build time could save on some resources. For example, in the past I have moved an e-commerce website's checkout process to an SPA which cut down the traffic hitting their web servers on average by 30%. This made a big difference to infrastructure costs and scale at high traffic events. With little change to the users experience.

This is a varied subject based on your use case, I have written a whole post on rendering options with JavaScript if you want to learn more.

Theres also a fantastic talk recently from Rich Harris (Svelte Maintainer) on the trade-offs between server rendering and SPA's.

3. Queues & Asynchronous Workers

Queues and background workers can help improve the scalability of your application by offloading slow or intensive tasks. This is commonly by decoupling a user action from some processing that could cause a backlog and effect performance.

Some common examples of this might be uploading an image, generating a report or processing a video. If we take the example of generating a report for a large dataset, this might take a few minutes for a user. You do not want this process using up a large amount of resources and holding open connections on the same infrastructure that is dealing with the requests of your other users. You risk an extra large report or a few reports running simultaneously to start effecting performance or cost.

Shifting this work to the background can allow this to be dealt with predictably. Sending these requests to a queue that can push (or pull) the work to a set of worker processes allows you to control the scaling of infrastructure and not risk the affecting other requests coming in.

What to check out: AWS SQS, Cloud Tasks, render.com background workers, Heroku workers, Go Worker

4. Spot / Preemptible Instances

If you have auto-scaling servers, costs can quickly rack up as you grow. Spot instances (preemptible VMs on GCP) make use of unused compute for upto 90% savings on the usual on-demand price.

Integrating with spot instances has been made much easier the past year or so with integrations with EC2 auto-scaling and Spot fleet. But there can still be some responsibility to handle interruptions and rebalancing if your application is not 'fault-tolerant'. To take away these concerns services like Spot can guarantee a high availability and abstract away some of the complexities.

What to check out: Spot intances, Preemptible VM instances

5. Event Driven Architecture

Applications usually start with monolithic design. A singular running process to do everything from logging & metrics to sending emails. If part of that fails, or you get a long running task, then this could clog up the system.

An event driven approach allows you to decouple these different pieces into seperate services that can scale & fail independently of one another. This system uses events to trigger and communicate between decoupled services.

What to check out: AWS video series, Event Bridge, Eventarc

Bonus: Monitoring!

Something that I think is often overlooked is the power of performance monitoring. How do you know if a new release is using significantly more resources, is it from your metrics or when you get the big infrastructure bill at the end of the month?

Theres a ton of Application monitoring solutions out there with integrations for all types of languages. New Relic & DataDog are 2 popular examples. They allow you to get much more fined grained than just CPU, Memory usage & scaling. You can see information such as :

External API performance
Code-level visibility e.g. inspecting a particular methods performance
Traces for slow performing request

And way more !

Performance monitoring can help you stay scalable whilst you continue to make changes.

Quick Tip - JavaScript Rendering

Aidan Gee — Tue, 06 Jul 2021 18:20:37 +0000

Originally Posted - https://aidangee.dev/blog/quick-tip-javascript-rendering

Intro

Popular JavaScript frameworks like Next.js, Nuxt.js and SvelteKit come with a number of rendering options included. But what does SSR, ISR, SSG and all the other fancy acronyms mean?

Client Side Rendering

Minimal static HTML is served back to the user, this will most likely only contain links to scripts and CSS files. The JavaScript is in charge of generating the HTML in the browser.

Because there are no servers needed you will often see platforms that host static websites for free with a generous amount of network bandwidth e.g. Render, Firebase Hosting, Vercel, Netlify. Or you could run this yourself in AWS storing the files in S3 and backing with a CloudFront CDN for a very low cost (often a few cents a month).

Pros

Simple deployments, just an index.html file and built JavaScript
Easy to scale with static files, that requires no server side compute to serve to the user.

Cons

SEO requirements can be more complicated (good video series about that on Google search YouTube channel)
Some performance metrics can be affected, for example CLS & FCP
All JavaScript is shipped to the client, so it must not contain any secrets / private data

Ideal for

Applications that require authentication to use
Applications without SEO requirements
Applications that receive spikes in traffic (static HTML does not need compute that has to scale)

Static Generation (SSG)

HTML is generated at build time and the full static HTML will be served over the network to the user.

This generates static HTML files, which means much of the same low cost hosting solutions as the client side rendering example can be used. The difference being that with Static Generation you will have a HTML file per page generated, rather than just an index.html.

Pros

Easy to scale with static files, no servers needed
Faster response times than if the file was generated on the fly
Full HTML content served to the user which benefits SEO, FCP, CLS over client side rendering

Cons

Longer build times, that can increase as content on an app increases
Will often have to be rebuilt to update page content
Cannot contain personalised content, the same generated page is served to all users

Ideal for

Applications where content / data is not required to be updated frequently
Applications with high performance requirements
Applications that receive spikes in traffic (static HTML does not need compute that has to scale)

Server Side Rendering

HTML is generated on request and the full static HTML will be served over the network to the user.

As it implies in the name, this requires a server side component. Each request will need to use some compute to generate the HTML (if you are not using a cache). You could use a serverless platform here like Begin, Vercel or Netlify to mitigate having to manage any servers.

Pros

Full HTML content served to the user which benefits SEO, FCP, CLS over client side rendering
Data can be dynamic on each request

Cons

Each request to the origin requires some server side compute resource
Slower response time than static generation

Ideal for

Applications where content / data is updated often
Applications with personalised content
Applications with strict SEO requirements

Hybrid

This can be considered a mixture of the above approaches. Frameworks like Next.js, Nuxt.js & SvelteKit (to name a few) have excellent APIs to achieve this.

To demonstrate this let us look at a simple example scenario with SvelteKit. Imagine we are building a blog with the following specification -

A static welcome homepage
A blog page that lists posts with content from a CMS

We could split these pages into different categories:

The homepage is static and won't change, so we can generate this at build time
The blog listing page, well that depends. We could generate the list page at build time with static generation but if the data source for blogs is often being updated then it might make sense to use SSR which would allow the page to update as the content updates.

This sounds like it might be complicated to mix and match, but the frameworks make this easy.

Homepage (pages/index.svelte)

<script context="module">
    // exporting this variable is all you need to do
    export const prerender = true;
</script> 

<svelte:head>
    <title>Homepage</title>
    <meta name="description" content="My homepage">
</svelte:head>

<main>
    <!--  content goes here -->
</main>

Blog List (pages/blog/index.svelte)

<script context="module">
     // export a load function to grab data server side
    export async function load({ page, fetch, session, context }) { 
        const blogs = await fetch('https://mycms.io').then((response) => response.json());

        return {
            props: {
                blogs
            }
        }
    })
    // we have static generation disabled
    export const prerender = false;
</script> 

<script>
    export let blogs;   
</script>

<main>
    <BlogPosts blogs={blogs}/>
</main>

If you wanted to switch this to being statically generated like discussed above, you could just set prerender to true. (be aware, in svelte the load function is run on the client and server)

Incremental Static Regeneration (ISR)

One more I wanted to include under the hybrid list is a feature of Nuxt.js and Next.js they call Incremental Static Regeneration (ISR). This can be viewed as a middle ground between SSR and SSG, if you use ISR then the page will be generated at build time like it would if you use static generation. But with ISR you specify a duration, and after this duration passed the page will be regenerated.

With this, you get the benefits of static generation but the increased frequency of updates that you get from SSR. This would actually be a good solution to our blogs list page from above. ISR can allow us to have a pre-rendered page but will update frequently enough to support any new blogs being added to the CMS.

Tip

Unfortunately, there is not one answer to how you should render your application. It is highly specific to what you are building. The good news is that hybrid rendering makes this specificity possible to allow best mix in your application.

For the best performance and low cost, static generation is recommended. I find myself saying 'can I pre-render this?' more and more, and often when I have something on the page that is dynamic like comments on a blog post. I'll mix in a component that grabs and renders that data client side before reaching for SSR. Why? This allows the initial content for the user to be pre-rendered with the dynamic part sprinkled on top in the client.

Getting Started with Machine Learning Models in the Browser with TensorFlow.js

Aidan Gee — Wed, 30 Jun 2021 21:06:29 +0000

Originally posted here: https://aidangee.dev/blog/getting-started-with-tensorflow-in-the-browser

Intro

There were a great set of talks this year at Google IO 2021, one that piqued my interest was this talk on machine learning & TensorFlow. There is a lot great new stuff here, but I'll summarize some key points from a web perspective.

TensorFlow Lite models can now directly be run on the web in the browser 🎉
Supports running all TFLite Task Library models for example image classification, objection detection, image segmentation and NLP

So I wanted to see how viable it is to use ML models on device in the browser.

TensorFlow.js & Pre-trained Models

TensorFlow.js is a library for machine learning in JavaScript and can be used both in the browser and Node.js. We can use this library to build, run and train supported models.

What is great for starters in the ML world (like me), is that this library comes with a number of pre-trained TensorFlow.js models. So anyone can jump in and start using things like image object detection or text toxicity detection without the huge barrier to entry that is model training.

Let's take a look at how the code looks for running object detection on an image.

// Note: Require the cpu and webgl backend and add them to package.json as peer dependencies.
import('@tensorflow/tfjs-backend-cpu');
import('@tensorflow/tfjs-backend-webgl');
import { load } from '@tensorflow-models/coco-ssd';

(async () => {
  const img = document.querySelector('img');

  // Load the model.
  const model = await load();

  // Classify the image.
  const predictions = await model.detect(img);

  console.log('Predictions: ', predictions);
})();

So in just a few lines of JavaScript we have managed to load and run a ML Model in the browser on an image 🎉. This is not even restricted to images, the detect method will accept a canvas element, video element and a 3D tensor shape. So quite quickly here, we could do something like track objects as a video is playing:

// Note: Require the cpu and webgl backend and add them to package.json as peer dependencies.
import('@tensorflow/tfjs-backend-cpu');
import('@tensorflow/tfjs-backend-webgl');
import { load } from '@tensorflow-models/coco-ssd';

(async () => {
  const videoEl = document.querySelector('video');

  // Load the model.
  const model = await load();

  // Classify the frame of the video.
  // timeupdate is a quick way to run this as the video plays
  videoEl.addEventListener('timeupdate', async () => {
    const predictions = await model.detect(videoEl);
    console.log('Predictions: ', predictions);
  })

})();

The predictions you get back from the detect function look like this:

  [{
    bbox: [x, y, width, height],
    class: "person",
    score: 0.8380282521247864
  }, {
    bbox: [x, y, width, height],
    class: "sports ball",
    score: 0.74644153267145157
  }]

Note : The position (bbox) variables you get back will be based on the original video resolution.

You could use this data to detect some context of what was in a particular video or track certain objects in the video as it plays ... all in the browser.

Thoughts

I could not believe how easy this was to get going with. The pre-trained models are a breeze to use and I would definitely recommend checking out the full list.

Depending on how you plan to use this functionality, something to keep in mind is the download times of models and how this effects the UX. For example, I found the Coco SSD model to take about 10 seconds to download on a solid Wi-Fi connection. So if your application relied on this, you are going to have extremely long start up times and probably frustrated users. Loading them in the background before the user needs them would be a nicer solution.

I am excited to see this space develop over the next few years. I think we all know about the growth of AI / ML but having this be available to run so easily with JavaScript in the browser can only help accelerate it's usage.

Deno Deploy Beta - First look & start up times

Aidan Gee — Tue, 29 Jun 2021 14:38:00 +0000

Originally Posted : https://aidangee.dev/blog/deno-deploy-beta-first-look

What is Deno

If you haven't heard of Deno (pronounced 'dee-no'), it is a JavaScript and TypeScript runtime by the creator of Node.js Ryan Dahl.

In a nutshell Deno allows you to run JavaScript on the V8 engine much like Node.js does, but there are a few key differences:

Supports Typescript out of the box
No centralised package manager like NPM
Aims to have a browser compatible API (e.g. fetch and web workers)
Is 'secure' by default, you must explicitly enable network access, file access etc.
Built in tools for code formatting, linting, test running and more...
Has a set of standardised modules reviewed by the Deno team

Ryan Dahl himself has spoken about these decisions in a number of talks. I would recommend taking a look at this talk he gave '10 Things I Regret About Node.js'.

If you want to dive deeper into Deno there are a great set of resources on the “awesome deno” GitHub that you can use.

So what is Deno Deploy then?

From Ryan Dahl himself :

Deno Deploy is a multi-tenant JavaScript engine running in 25 data centers across the world. The service deeply integrates cloud infrastructure with the V8 virtual machine, allowing users to quickly script distributed HTTPS servers. This novel “serverless” system is designed from the ground up for modern JavaScript programming.

Ok, but what does this mean? Deno Deploy is wanting to be the way you deploy your server-side Deno code. By using the service you get fast CI/CD and serverless deployments optimised for Deno.

For an in depth look at the features of Deno Deploy, checkout their latest blog post or documentation.

First look

Reminder, this is the first version Deno Deploy and is a beta. So I wouldn't expect this to be the final product, but it is still fun to see what is already available.

On that note, the initial public beta for Deno Deploy is free to use. So it is a great time to jump in and try it, they have a list of limits that apply during the beta here.

When you sign up and login to Deno Deploy you will be asked to create a project to house the Deno services you intend to deploy. You will also be met with a couple of examples ready to deploy at the click of a button.

Project Dashboard:

Let's look at that code for the Hello World example:

addEventListener("fetch", (event) => {
  event.respondWith(new Response("Hello world"));
});

Now if you have been using Node.js with express or running on AWS lambda this might look a bit alien. What I find interesting about this example is that this is not code you can just pop into Deno (yet) to run a server locally. It's Deno Deploy sprinkling some of that platform magic on top that allows you to use the Fetch Event API that you would use in a Service Worker in your browser. So I'm already thinking this is going to be more of an all encompassing Deno platform rather than just a hosting service.

Deploying

There are a couple of ways you can deploy your code.

Connect your GitHub repo
Provide a URL to a repository

The first is almost a requirement of a hosting platform nowadays, you can connect a repository from GitHub and have it build and deploy. What is nice to see is the inclusion of preview deployments. This will create a deployment whenever you push to a branch. I love this feature, it makes testing and pull request reviews just that bit faster.

The second point however, is very ... Deno. It fits in with the theme of decentralised packages and importing via a URL. I can see this making it really easy to share your open source service with others and letting them easily host it themselves, a nice touch.

Impressive Start Times

We believe Deploy is the fastest serverless system available. We hope to nail down this bold claim with performance benchmarks in future releases.

This is a quote from their blog that made me want to test out this beta. It is a bold claim to say the least, but I think performance needs to be something every developer has in their mind when building a modern Web App. Especially with the push from Google on core web vitals, and it's effect on your websites SEO. The option of a fast easy to use serverless platform is right up there on my Christmas list.

So, what I wanted to look at was the speed of the platform itself. The simple hello world app is perfect for a basic test, how fast does it respond with a simple hello world? For these tests, I compared the TTFB over a number of requests.

Deno Deploy

TTFB of Cold Start: 575 ms (avg of 5 requests)

TTFB once warmed : 44ms (avg of 50 requests)

For a Beta, I think these are impressive numbers. To give some perspective, I also tested out Netlify who as part of their platform provide Netlify Functions. A similar easy to use serverless deployment experience but for Node.js. if you put the same 'Hello World' example on Netlify Functions (deployed in Europe) the same tests looked like :

Netlify

TTFB of Cold Start: 812 ms (avg of 5 requests)

TTFB once warmed : 138 ms (avg of 50 requests)

Now I don't think or claim these tests are an exact science and I definitely don't think this means you should be choosing Deno Deploy over Netlify just yet. But I believe what we are really seeing here is the difference in compute on the edge vs a data centre. And the speed boost Deno Deploy is getting here is probably mostly due to that. Compute on the edge is becoming more common nowadays, popular options like Lambda@Edge and Cloudflare Workers have steadily been getting better over the passed few years and much more accessible to developers. Deno Deploy having this tech from the get-go does give them a speed advantage over some existing platforms.

Thoughts

I think there are some good early signs here. Firstly, there is a free open beta that anyone can try out and give feedback. This is great that anyone in the community can check it out and potentially help shape it.

The tech is there to provide a great experience for developers and end users already. Even at this early stage it is easy to use and get up and running. The edge compute is a nice performance inclusion.

I have my reservations about some of the platform magic that seems to be in there at the moment. The Deno deploy homepage specifically lists 'No Vendor Lock In' as one of its aims. But it looks like there is already some features in there that would make it hard to move away from the platform. Just look at Broadcast Channels. So this will be something I keep an eye on.

The Deno Team are aiming for the end of this year to enter General Availability. So keep an eye out for updates and that all important pricing model.