DEV Community: Kumar Swapnil

Cache miss : Escaping the Storm

Kumar Swapnil — Tue, 07 Jul 2020 18:50:36 +0000

In pursuit of speeding up database driven applications at scale by alleviating the database load, we use in memory caching tools like Memcached and Redis.

During peak hours when there is heavy load on the Server, if a cache expires whose computation is expensive (or if we version Cache so that new computation would be required for each of the new keys) and there are significant number of Concurrent clients, all the clients will start slamming the database to compute the Cache value. This will put an unnecessary load on the Database and the latency would suffer causing a spike every time it happens (and may cause Congestion collapse in worst case scenario). This is known as Cache Stampede, or Cache Miss Storm.
To prevent multiple simultaneous computations and avoid Cache stampede several of the well known approaches includes Probabilistic Early Expiration, External re-computation and locking.

In locking mechanism, when a Cache miss is encountered, it will set a global lock to recompute the Cache value. Once the value is computed, the lock is released. The threads which lost the race to acquire the lock are left with the following array of choices (but not limited to):

short sleep, then re-fetch.
show the user a page without this expensive content
Keep a stale item in the cache to be used while the new value is recomputed
Wait until the value is recomputed

The last point is more efficient as here we are not speculating on when the recomputation will finish(sleep based solution) or the solution does not deal with multiple moving parts (stale item and its handling). This approach is easy but a bit tricky to implement. Let’s see how can we achieve it:

private static Dictionary<string, Task> _pendingTasks;

private async Task<T> Get<T>(string key, TimeSpan cacheDuration, Func<Task<T>> dbCallback)
{
    var cacheObject = await _client.GetAsync<Object>(key);
    if (cacheObject != null)
        return (T)cacheObject;
    else
    {
        if(AcquireLock(key))
        {
            var task = CreateMemcacheObjectAsync(key, dbCallback, cacheDuration);
            _pendingTasks.TryAdd(key, task);
            var res = await task;
            _pendingTasks.Remove(key);
            return res;
        }
        else
        {
            await _pendingTasks[key];
            return (T)_client.GetAsync<Object>(key);
        }

    }
}

In the above implementation, when there is Cache miss and the thread lost the race to recompute the Cache value, it will pull out the already enqueued task from the Dictionary which is computing the Cache value and then wait on that task instead of doing a redundant computation. Once, its computed, we simply return the Cached value.

However, If it acquires the lock, then it simply proceeds with the computation using createMemcahedObject(). This way, we end up calling database only once per Cache miss.
To keep a track of pending tasks which are performing computation, I have simply used a Dictionary. The AcquireLock() function looks something like below:

private bool AcquireLock(string key)
{
    return _client.Store(StoreMode.Add, key + "_lock", "lock", TimeSpan.FromMinutes(1));
}

The above method describes the use of Memcached’s add feature to acquire a “ghetto lock”. add(key, value) operation succeeds only if the cache wasn’t already holding a value for the key and so, only one process will acquire this lock with a lease time of one minute. One minute should be more than the estimated maximum time for the lengthy operation. This avoids the lock being held forever if ever things go really bad such as your server crashing. And once the Computation is done, we can simply release this lock.

That’s it. With this small change we can avoid a Cascading failure that may arise when the server is pushed to limits and keep the latency consistent across varying loads.

Optimizing React SSR Performance: Part II

Kumar Swapnil — Sun, 14 Jun 2020 10:30:06 +0000

In the earlier post, we have seen how we can split the JavaScript execution cost from one long running tasks to multiple smaller tasks and improve the perceived performance of the page. Now, this approach can be fine tuned further by leveraging Code Splitting.
Code-Splitting is a feature supported by bundlers like Webpack which allows you to create smaller chunks which you can then “lazily load” on demand. React provides React.lazy to render a dynamic import as a regular component. Let us look at code below,

const LazyComponent = React.lazy(() => import('./LazyComponent'));

This will create a different chunk for LazyComponent. Now, we can use this to lazy load our components straight away, but there is a catch. React.lazy and Suspense are not yet available for server-side rendering. They recommend loadable components to Code Split our app this way if we want Code splitting with server rendering.

Now, let’s recap the two aspects which we will be focusing on:

We can bail out of hydration phase using the dangerouslySetInnerHtml trick.
We can lazy load our chunks using Code Splitting with loadable components.

Now, the question is, when do we fetch these chunks and hydrate the Component. One possible solution is, just before they are entering viewport. This is referred as Progressive hydration. In Google I/O ’19, we get a gist of what it is and how we can implement this today (they did mention about React’s roadmap to ship this). So, let’s look at the code as how can we achive it.

import React from 'react'
import ReactDOM from 'react-dom'
import loadable from '@loadable/component';
import { InView } from "react-intersection-observer";

/**
 *
 * @description lazily loads the JS associated with a component and hydrates it based on its distance from viewport
 * @tutorial {@link https://youtu.be/k-A2VfuUROg?t=960} - Rendering on the Web: Performance Implications of Application Architecture
 * @tutorial {@link https://medium.com/@luke_schmuke/how-we-achieved-the-best-web-performance-with-partial-hydration-20fab9c808d5} - Partial Hydration
 */
class Hydrator extends React.Component
{
    constructor(props)
    {
        super(props);
        this.Child = null;
        this.state = {
            renderChild : false
        }
    }

    shouldComponentUpdate(nextProps, nextState)
    {
        return nextState.renderChild;
    }

    handleInviewChange = async (inView) => {
        if (inView) {
            const {load, id} = this.props;
            const root = document.getElementById(id);
            this.Child = loadable(load);
            this.setState({
                renderChild : true
            });
        }
    };

    render()
    {
        if(typeof window !== "undefined")   // Avoid client side hyration
        {        
            if(this.state.renderChild)
            {
                return React.createElement(this.Child, this.props);
            }
            else
            {
                return (
                    <InView
                        rootMargin="640px"  // to avoid janky experiency in case the user scrolls fast
                        triggerOnce={true}
                        dangerouslySetInnerHTML={{ __html: "" }}
                        suppressHydrationWarning={true}
                        onChange={this.handleInviewChange}
                        {...this.props}
                    />
                );
            }
        }
        else    // Server side rendering
        {
            return(
                <div dangerouslySetInnerHTML={{__html: this.props.serverMarkup}}>
                </div>
            );
        }
    }
}

export default Hydrator;

Since the Server side rendering is going to happen in a single pass, you need to handle your component’s markup differently on the server side. You can simply create the component independently and the pass this as a prop to the root component. The Code would look something like below:

private string SSR(componentProps, progressiveComponentProps = null)
{
    componentProps.ProgressiveComponent  = RenderProgressiveComponents(progressiveComponentProps);            
    var reactComponent = Environment.CreateComponent(componentName, componentProps, containerId, clientOnly);
}

private static string RenderProgressiveComponents(componentProps)
{
  IReactComponent reactComponent = Environment.CreateComponent("LazyCarousel", componentProps, "article-detail__carousel-wrapper");
  return reactComponent.RenderHtml();
}

So, once we handled the Server rendering (and else block of render method). Let’s have a closer look at the if block of render() method now. When the first time render() will be called on client side, it will wrap the component with InView and will make use of dangerouslySetInnerHtml to avoid hydration cost. The InView component makes use of Intersection Observer to observe an element’s visibility.
Now, once the user is about to enter viewport, the handleInviewChange will trigger which will fetch the component chunk making good use of loadable and then toggle the component state to trigger rendering. This way, now the component is ready to handle user interactions.

Results: In addition to what we achieved from previous post, now we are able to save on initial network cost to boot up the app. Here, along with Network cost, we are also saving on the JavaScript parse /compile time which was unavoidable in the last implementation.

Conclusion: In my opinion, both idle-until-urgent and progressive hydration can complement each other to build a better progressive web app.
While Progressive hydration makes sense on below the fold components which can significantly (like carousels) reduce the initial chunk size required to boot the app but it comes up with an overhead of special Server Side handling, on the other hand Idle-Until-Urgent is pretty quick and easy to implement(making it a better candidate for light weight components).

Optimizing React SSR Performance : Part I

Kumar Swapnil — Sat, 13 Jun 2020 17:06:22 +0000

Without burdening you with the intricate details of the cost of Javascript, anywhere up to 30% of the page load time can be spent in JavaScript execution.

It can create an annoying User Experience if the main thread is busy for long duration rendering the site unresponsive.

In React, when we render a page server side, we get the benefit of a better First Contentful Paint where the user does not have to wait for the Javascript to boot up and render the page. But, we still need to hydrate in order to make the page interactive. This Client side hydration is slow and can get really slow if there are a lots of components on the page.

To tackle this, strategies like Progressive Hydration and Selective Hydration is already on the React roadmap and I hope we get to use these awesome strategies real soon. But for the time being, I have tried to implement a pattern known as idle-until-urgent which basically helps to break the hydration from one long task into smaller tasks which can be executed during Browser’s ideal periods or When the user interacts with it. Okay, enough with the words, let’s see some code.

import React from 'react';

export function idleUntilUrgent(WrappedComponent, ComponentId) {
    class IdleUntilUrgent extends React.Component {
        constructor(props) {
            super(props);
            this.renderChild = false;
            this.firstRender = true;
            this.callbackId = null;
        }

        shouldComponentUpdate(nextProps, nextState) {
            return (
                this.props != nextProps || (nextState && nextState.renderChild)
            );
        }

        // to prevent calling setState on an unmounted component
        // and avoid memory leaks
        componentWillUnmount() {
            this.callbackId && cancelIdleCallback(this.callbackId);
        }

        enqueueIdleRender = () => {
            if (typeof requestIdleCallback !== "undefined") {
                // https://caniuse.com/#search=requestIdleCallback
                this.callbackId = requestIdleCallback(() => {
                    const root = document.getElementById(ComponentId);
                    this.setState({
                        renderChild: true
                    });
                });
            } else {
                setTimeout(() => {
                    const root = document.getElementById(ComponentId);
                    this.setState({
                        renderChild: true
                    });
                });
            }
        };

        urgentRender = () => {
            this.setState({
                renderChild: true
            });
        };

        render = () => {
            if (typeof window !== "undefined" && this.firstRender) {
                this.firstRender = false;
                this.enqueueIdleRender();
                return (
                    <div
                        dangerouslySetInnerHTML={{ __html: "" }}
                        suppressHydrationWarning={true}
                        onClick={this.urgentRender}
                    />
                );
            } else {
                // Cancel the already scheduled render, if any
                this.callbackId && cancelIdleCallback(this.callbackId);
                return <WrappedComponent {...this.props} />;
            }
        };
    }
    const wrappedComponentName =
        WrappedComponent.displayName || WrappedComponent.name || "Component";
    IdleUntilUrgent.displayName = `IdleUntilUrgent (${wrappedComponentName})`;
    return IdleUntilUrgent;
}

Let’s understand the above code one byte at a time:

During the hydration phase, we use this neat trick of passing an empty string to dangerouslySetInnerHtml which will bail the component out of the hydration phase (React doesn’t try to manipulate the tree of a dangerouslySetInnerHTML node on the client. Even if it is wrong.), thus saving us the hydration cost for the component.
Also, it will call enqueueIdleRender which will render the component later at some idle time.
Before the component is interactive, if a user interacts with it, it will render immediately (making it interactive) and in this process will cancel the already scheduled render to avoid multiple renders.
This way, we can simply wrap the components with this Higher Order Component in order to split the hydration cost into multiple smaller tasks rather than one long task keeping the app responsive.

Results: With this approach, the initial hydration cost dropped by ~45%, from ~128 ms to ~70ms. These are really impressive results and the difference and benefits will only grow up when the Components will grow on the page.

Before Idle-Until-Urgent
After Idle-Until-Urgent
Conclusion: This is one of the ways we can make progressive enhancement to our page to improve the overall User Experience. This will help you bring down the Maximum First Input Delay and Total Blocking Time of the page. So, hopefully, this post will convince you to think about the way you were hydrating your react app until now.

Gzip is good, Brotli is better!

Kumar Swapnil — Sun, 12 Jan 2020 14:20:30 +0000

Compressing the data before sending it to the client is almost a fundamental web development aspect. The lesser the number of bytes transferred down the wire, the faster the client would receive it. The most common compression schemes include gzip and Deflate.
Recently, I bumped into a relatively younger compression algorith, Brotli. After reading this post by linkedin, I was pretty much excited and thought to see the results for myself.

I created a demo project and am putting the comparison snapshot below.

The Browser Support for Brotli is quite good but not 100%. You could easily fallback to gzip for the Browsers that does not support Brotli.

Sending lesser number of bytes would surely improve the latency but remember that the Browser will still parse and evaluate the same number of bytes. So, it would be better if you do not rely upon compressing the data very much and ship the user the best version of yourself for a good user experience.

A step towards a faster Web: Early flushing in c#.net

Kumar Swapnil — Sun, 12 Jan 2020 08:40:26 +0000

A simple demonstration to Early flushing in dotnet.

Flushing, Early flushing, head flushing or Progressive HTML is when the server sends the initial part of the HTML document to the client before the entire response is ready. All major browsers start parsing the partial response.If done correctly, the browser won't sit idle after requesting your page, rather, it can start process other important things in the meantime, like requesting static assets which would be used later on the site. It could give a significant perceived performance gain.
In this given example, I have used Thread.Sleep(200). This could be the time where your page does heavy database calls and other computation.

HomeController.cs

public ActionResult Index()
{

  PartialView("/Views/Shared/_HeadPart.cshtml").ExecuteResult(ControllerContext);
  Response.Flush();
  Thread.Sleep(200);
  return PartialView("/Views/Home/Index.cshtml");
}

_HeadPart.cshtml

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <script src="https://code.jquery.com/jquery-3.4.1.min.js"></script>
    <title>Hey - My ASP.NET Application</title>
</head>

Index.cshtml

<body>
    <div class="row">
        Hey there, How u doin'?
    </div>
</body>
</html>