DEV Community: Subhasish Kundu

OSS Contributions: 16th-23rd August 2021

Subhasish Kundu — Sun, 22 Aug 2021 17:20:46 +0000

After a long time, I started contributing to open-source software. For a long time, I had my eye on Deno. This week, I merged my first PR in Deno Lint. Following is the detailed post about the issue.

Issue 425

Category: Refactoring

Description of the Issue

If we go through the issue, we could see that in the tests of each rule, there is a repeat of messages and hints. For example, from *\ban_ts_comments.rs from a previous commit, we can see that messages and hints are being repeated

    r#"// @ts-nocheck"#: [
            {
              col: 0,
              message: DirectiveKind::Nocheck.as_message(),
              hint: DirectiveKind::Nocheck.as_hint(),
            }
          ],
    r#"/// @ts-nocheck"#: [
            {
              col: 0,
              message: DirectiveKind::Nocheck.as_message(),
              hint: DirectiveKind::Nocheck.as_hint(),
            }
          ],
    r#"//@ts-nocheck"#: [
            {
              col: 0,
              message: DirectiveKind::Nocheck.as_message(),
              hint: DirectiveKind::Nocheck.as_hint(),
            }
          ],
    r#"// @ts-nocheck    "#: [
      {
        col: 0,
        message: DirectiveKind::Nocheck.as_message(),
        hint: DirectiveKind::Nocheck.as_hint(),
      }

The test cases for each rule in Deno Lint could be grouped together based on common messages and hints. This issue talks about that. Macro assert_lint_err is responsible for running test cases.

Solution

In this commit, I have extended this macro with new rules, where we could use it to run multiple test cases with the same messages and hints. Following is the modified assert_lint_err macro. I have added two new rules, where we could group tests based on messages and hints.

Challenges

I did not know about Rust. I had to learn about macro_rules! to solve this issue. I have gotten interested in it. Let me see, how much I make progress.
As this is the first time, I was contributing to Deno, I was not aware of all the formalities. Yusuke Tanaka from the Deno team helped me a lot. They have been very receptive and active. Their help has encouraged me to take up my new issue as well.
This is not a challenge, but learning. I could not totally comprehend Rust-style styling. The reviewer pointed those out to me. From next time onwards, I have to be better with that.

Follow up

In this commit, I have covered ban_ts_comment.rs and
no_await_in_loop.rs. I need to cover other files as well. If someone reads this post and feels good to contribute, I would encourage them to do that 😀😀. Connect with me or anyone in Deno team for help/any questions. We will be happy to help.

I will write a detailed post on various aspects of Deno and Deno development and contributions in subsequent blog posts.

Best Practices of Exception Handling

Subhasish Kundu — Sat, 17 Jul 2021 06:15:24 +0000

Exception handling is one of the most important aspects of software engineering. There are many articles on this topic. Yet, I have seen other developers and myself making mistakes and not exiting gracefully. In my experience, I have encountered many exceptional scenarios and failure reports. I have written a lot of codes and made mistakes a thousand times. The following quote by Malcolm Muggeridge summarises exception handling pretty well.

Few men of action have been able to make a graceful exit at the appropriate time.

In this article, I am documenting my share of learnings along the process.

Note: I am using Java in this article as a reference. It is my primary language of coding. But, the principles are language agnostic. I will cover the things that are out of the scope of coding as well. As I learn new things about the topic, I will keep this post updated. I have used unchecked and runtime exceptions interchangeably.

I am dividing this article into two sections

Handling exceptions inside code
Handling exceptions outside code

Handling exceptions inside code

1. Use Exceptions for Exceptional Scenarios

Exceptions should be used for exceptional scenarios only. In Effective Java, author Joshua Bloch is extremely vocal about this. If he is so vocal about this, it is an issue. I hope none of us do this anymore. Following is an example from the book

// Horrible abuse of exceptions. Don't ever do this!
try {
    int i = 0;
    while(true)
        range[i++].climb();
} catch (ArrayIndexOutOfBoundsException e) {
}

Try to guess the answer! The above code is clever but lacks readability, intent, and performance. JVM does not apply some optimization as well seeing try-catch.

The simpler version is readable, understandable, and lets JVM apply optimizations.

for (Mountain m : range)
    m.climb();

2. Never Ignore Exceptions in Code

It is obvious but often forgotten. I have seen a similar code as follows.

try {
     executeCommand();
} catch (IllegalStateException e) {
    //Highly suspicious. Anything can happen here
    //without your knowledge.
}

This is not a good practice at all. In the future, if something happens, you will never know. I am also a criminal of ignoring an exception altogether (by mistake) 😔. Later I fixed it while refactoring. I still don't know about the impact 🥶. And it still bugs me!

3. Make Exceptions Visible When Occurs

Make sure you have ways to know that exception has occurred. It might be some metrics or UI-based log aggregator for capturing all logs. But it is a necessity. Otherwise, you will not know if there is anything wrong with the system.

4. Don't Use "Log and Then Throw" Exception Strategy

If you log and throw another exception, the catcher of exceptions might again log details about exceptions. So, you will have redundant information for the same exception. It will pollute error logs. In turn, it will make your life uneasy while debugging.

try {
      executeCommand();
} catch (IllegalStateException e) {
      // Following log is redundant information.
      // It will pollute your error log.
      log.error("Exception at doSomething:", e); 
      throw new BadRequest("Exception has occurred.", e);
}

5. Log Errors with Exception Details

This is a common practice. Let me warn you! If you are not logging exception details, you are digging a hole for the future. It will be hard to debug an actual issue. Your on-call will keep looking for issues without knowing where to look. Having a proper stack trace will help him to decide where to look. It will help root cause the issue faster.

6. Use Exception Translator

Dependencies or library calls throw exceptions. Those exceptions can be specific to the callee. Those are seldom related to the caller. But we need to handle exceptions at the caller end. It is better to map specific exceptions from the callee to exception types of the caller. Exception Translator is one of the best tools available in this case. Another way can be propagating status codes such as success or failure, of the operation. In the following example, CommandArgumentException conveys better meaning relative to the caller.

try {
      executeCommand();
} catch (IllegalStateException e) {
      throw new CommandArgumentException("Arguments 
             to command are not right.", e);
}

Also, I have worked on monolith software written in C (codebase size was 2.2GB). In C, we don't have a proper exception handling framework like in Java. In such cases, exception translator and framework help a lot. Following are a couple of ways

It simplifies the handling of exceptions by calling call-back functions. In case of some null pointer exception, you should hand over the control to the exception handler. You cannot call exit(). For debugging, you need to collect reports.
If you don't capture stack trace in Exception Handler, you will not be able to debug the issue. The stack trace will help you create the test case. It will be your friend for the entire debugging.

7. Be skeptical about Runtime Exception

Compilers force us to try-catch checked exceptions. We often miss runtime exceptions. We should be skeptical about runtime exceptions especially while calling dependencies. In large-scale distributed systems, failure is common. We once encountered a dependency call that does not have any checked exceptions. After careful inspection, we wrapped it up inside try-catch for runtime exceptions. It allowed us to exit gracefully.

8. Consider catching Generic Exception

As I work in distributed system development, I generally follow this. It helps to exit with grace. But I make sure to log this and capture such exceptions. In case of a surge of generic exceptions, we should investigate the issue. There might be some new checked exception in the callee side or there is some runtime exception. In any case, we should check the issue.

9. Don't catch Throwables

Throwables are the superclass of everything including Errors. Errors are used by JVM to indicate a serious system-related problem like StackOverflowError etc. Applications are not supposed to handle these errors. When JVM throws an error, there is some problem. Go check it! 🧐

10. Strive for Failure Atomicity

This one is interesting. Let me elaborate on failure atomicity. In case of a failed operation on an object, we should roll back all the changes to the previous state. The object should not have partial updates. So, we can check if an operation can fail or not. We can use a checker function for this. JDK has some good examples of this. Iterator interface in Java has a checker hasNext() method. If it fails, we generally don't call the next() method. We can use this pattern as well when needed.

11. Document All Exceptions

Most of the developers dislike documentation. As we write a piece code, we leave our legacy. Your documentation ensures your successor understands your code. Secretly, we all hope - when the next developer sees our code, he says

We should document all the exceptions including runtime exceptions. In Java, the "throws" clause forces to check checked exceptions. For covering all the bases, we should document runtime exceptions as well.

We should document all exceptions including checked and unchecked using the JavaDoc '@throws' tag. Don't just mention Exception or Throwable. The below example does not express anything.

/**
 * JavaDoc Exception sample documentation
 *  @throws Exception in case of exceptional scenarios. 
 */
public void doSomething() throws AnException, AnotherException {
    ...
}

We should only include checked exceptions in the throws clause, but not unchecked exceptions. It lets developers know that an exception mentioned in JavaDoc, but missing in the throws clause is unchecked.
Document the condition for which exception is thrown. For the above example, it is desirable to include conditions for which each exception is thrown.

12. Always cover Exceptions in Unit Tests

This will give you confidence that your code works during exceptional scenarios. Check for outputs during exceptions. If it is not the same as you expected, anything can fail in production.

13. Add Your Counters in Unit Tests

In case you use some counter or special logs during the exception, your unit tests for exceptions should cover those. Unit tests will fail in case someone changes counter or logs. And seldom, we have reasons to change these. If there is any reason, the developer should know why it was there in the first place. Your tests should be like documentation. It should cover all the aspects of your code including exceptional scenarios.

Handling exceptions outside code

Operations are one of the big issues in large software systems. It includes a large-scale distributed system as well as a standalone desktop or mobile app. Sustenance of our service is one of the most important goals. We should have a proper mechanism for handling error scenarios. Following are my two cents on this

1. Indicators and Logs Should have Meaning

Your on-call might have to debug a production issue at midnight. The last thing he wants at that time is confusion. Good logs and indicators (specific to your use case) can help him find the root cause of the issue faster. Ambiguity and complexity are hindrances to the root cause of the issue.

Now food for thought, we have two code segments.
Option 1

String serviceA = "Something has happened in serviceA";
String serviceB = "Something has happened in serviceB";
//somewhere else
log.error(serviceA);
//somewhere else
log.error(serviceB);

Option 2

String errorStatement = "Something has happened in ";
String serviceA = "serviceA";
String serviceB = "serviceB";
//somewhere else
log.error(errorStatement + serviceA);
//somewhere else
log.error(errorStatement + serviceB);

Suppose, we got an exception at midnight and we are getting "Something has happened in serviceA" as exception signature. Which one do you think will help to find code faster? I think Option 1. Single copy-pasting is easier than creating strings from multiple variables. Obviously, with the signature of exception, you can fast-track your problematic point. I still find Option 1 easier for on-call.

Note: Have you checked the last blank space at the end of errorStatement definition? These small spaces can give you nightmares.

2. Don't Ignore Metrics

Add metrics wherever you think an error might occur. And add alarms on this. This will let you know if something is wrong with your system. In the monolith that I worked on, we incorporated AddressSantizer to collect data about runtime exceptions. It was worth it. It helped a lot for finding a lot of long-term issues. I can still remember the days of debugging a problematic pointer! 😫

3. Plan Your Action

Metrics and alarms without action items are meaningless. It is the same as a fire brigade team without protocol. Update the run-book with all relevant details and action items for important exceptions. Also, if required, add metrics to the dashboard.

4. Keep your Counters Updated

You don't want a false alarm in the middle of the night. If you need to update an alarm, update it without delay. Getting regular wrong alarms can make you lose your trust in your system.

Generic Guidelines for Handling Exception

Here are some general guidelines for handling exceptions.

1. Always Be Skeptical

Skepticism is better than optimism for handling errors and exceptions. Even if you have done everything, exceptions will occur. If you have a proper mechanism in place, you can debug an issue faster.

2. Aim for Sustenance

If you build your system for sustenance, you will be better prepared for unknowns. This is one of the signs of mature developers.

3. Exceptions are not Your Mistake

Failure is seldom a single developer's mistake. From coding a solution to deploying that involves a lot of steps. Any mistake at any stage can cause failure. We should aim for reducing failures by owning the process. There might be some issues unknown to the team. The developer and reviewers might miss it. Lack of testing might have missed the bug. We should look forward to improving the process. A root-cause analysis might help in this regard.

Conclusion

Exception handling is a well-known misunderstood aspect of programming. In summary, we

Should aim for sustenance and share knowledge across the way. It will help a lot.
Do not ignore exceptions. Handle it properly. Some principles mentioned might not be applicable to you. Break those and make yours. And share it with others.
Do not play blame games. It will not help anyone.

We know how to use try-catch, but we don't know how it can affect our system. This article is my attempt to capture my thoughts on this topic. Please like the post if you enjoyed reading it. Let me know in the comments if you have any thoughts or questions.

Happy Learning!! 😀

JavaScript Event Loop in Depth

Subhasish Kundu — Mon, 28 Jun 2021 04:01:19 +0000

Sometime back, I started learning about JavaScript and React JS internals. My goal is to understand the internals of JavaScript and frameworks. In my last article, I wrote about ReactJS internals. To understand more about React Fiber, we need to understand some preliminary JavaScript concept implementation. In the next three posts, I will be explaining in detail the following three concepts

JavaScript Event Loop
requestAnimationFrame and rendering cycle
requestIdleCallBack

In this article, I will attempt to document my learning about Event Loop. This topic might be particularly interesting to people who want to learn how NodeJS and Deno internals work. The next two topics are interesting to me for understanding how React works internally (for code base understanding). So, let's dive deep.

JavaScript Execution Environment

JavaScript is a single-threaded language. It executes one method at a time. Following high-level components are integral to its execution environment -

Call Stack
Job Queues
Rendering Step
Microtasks
Event Loops

Note: I have used runtime environment and execution environment interchangeably. At some points, I have used EcmaScript references. I am finding EcmaScript standards quite interesting. So, I thought to include those as well.

Call Stack

Like other languages like C, C++, etc., JavaScript uses a Call Stack to execute methods. Whenever it decides to execute a method, it wraps the method in a stack frame and pushes the frame into the Call Stack. For any eventual sub-method calls, it keeps on pushing sub-methods into the stack (LIFO order). Once a method is executed, the corresponding stack frame is popped. Call Stack is an integral part of JavaScript for synchronous programming.

test1();
function test1() {
    console.log("Test 1 entry");
    test2();
    console.log("Test 1 exit");
}
function test2() {
    console.log("Test 2 entry");
    console.log("Test 2 exit")
}

For the above script, the result is as same as predicted:

Test 1 entry
Test 2 entry
Test 2 exit
Test 1 exit

But, JavaScript supports asynchronous programming. We can use setTimeout to execute something in the future like the below example. We could use the following code snippet that might trigger an alert and console.log() after 10s.

    test1();
    function test1() {
        console.log("Test 1 entry");
        setTimeout(test2, 10000);
        console.log("Test 1 exit");
    }
    function test2() {
        console.log("Test 2 entry");
        alert("Test 2 execution");
    }

As expected, the result is

Well, there was an alert as well. Following is the timeline of events

We can see the "Install Timer" at the bottom left. This is where the timer was installed to be fired after 10 seconds (10847ms).

It seems some thread is running in the background. Hold on! Did I not mention JavaScript is single-threaded? Let's see what could happen if JavaScript is multi-threaded. Suppose, in our hypothetical model, for the above JavaScript code execution, we are allowed to spawn a new thread for executing a single setTimeout. So, for one thousand setTimeout calls, we have a thousand threads. Each thread might attempt to change DOM as well independently. Now, if DOM is a shared resource for all the threads, multiple threads are bound to create chaos reading it at the same time. We would have spent most of our time debugging race conditions rather than building rich applications 😛. So, yes, JavaScript is single-threaded and it has served JavaScript and us well.

How come JavaScript schedule setTimeout callback after 10s then if there is no thread? The answer is Job or Task Queue. I am using both names interchangeably.

Job Queue

With Es6, the concept of Job Queue has been introduced. All the asynchronous tasks such as Mouse click, setTimeouts, etc., are pushed into the Job Queue for execution. JavaScript engine picks up tasks one by one and executes them using Call Stack (FIFO order).

What part of the JavaScript engine picks up tasks from Task Queue?

The Event Loop. It runs in a loop throughout the life cycle of the app. It picks up tasks from the Task Queue and executes them with the help of Call Stack.

Note: My guess is the browser might maintain different queues for different kinds of event handling such as button clicks, mouse clicks, setTimeout calls, etc. Each browser can have different prioritization for each queue. It is just my guess. I am looking for ways to confirm that. I might write about that in the future.

What does the Event Loop do when there is nothing in Call Stack and Task Queue?

You guessed it right - it runs in a loop.

How does the Event Loop know if it needs to execute something from Task Queue?

The Browser engine can enqueue a task in the relevant queue. The Event Loop, in the next rotation, can dequeue the task and execute it.

What happens if we encounter an infinite loop?

    <body>
        <button id="demo">Trigger Alert</button>
    </body>
    <script>
        document.getElementById("demo").addEventListener("click", () => {
            console.log("Clicked");
            while(true);
            console.log("After loop is executed");
        });
    </script>

In the above example, the button click will enqueue the above synchronous script in Task Queue. In the next rotation, the Event Loop starts executing the task. Now, our script is stuck with infinite while loop. As a result, the Event Loop is stuck in the same position because of our infinite loop.

Rendering Step

Let's look into another aspect of the JavaScript runtime environment - the Rendering Step. Let's take the following example

<body>
        <p id="test_paragraph"> Test JS Hello</p>

        <button onclick="changeParagraphColor()">Trigger Alert</button>
    </body>
    <script>
        function changeParagraphColor() {
            document.getElementById("test_paragraph").style.background = "red";
            var top = 100, nested = 1000;
            for(var i=0; i<top; i++) {
                for(var j=0; j<nested; j++) {
                    console.log(i + ":" + j);
                }
            }
            console.log("Changing color");
            document.getElementById("test_paragraph").style.background = "blue";
        }
    </script>

In the browser, we can see the following output.

In the above example, the background color of the paragraph is changed to blue after the execution of the loop. The red background never takes effect. If I remove line 21, I can see the red paragraph, but after the execution of the script (including the loop). I expected to see first a paragraph with red background and then a blue. But I only saw, blue background.

Here, we are seeing the UI effect for the last line, but not the previous UI change. Why is that? That is because of the Rendering Step. The browser does not repaint the screen after the execution of each JavaScript code. It executes the above JavaScript code and collects all styling and UI changes. It applies final change (in this case line 21) in a single shot afterward, in the Rendering step. It is a step in the Event Loop that is executed independently. And initially, I thought the changes are happening so fast that my eyes could not catch them 😤. I will dive into the Rendering step in my next post.

Microtasks

For now, I am associating Microtasks with JavaScript Promises. Promises are used to perform asynchronous operations in JavaScript. If you want to know more about Promises, visit this page for more details. The tasks executed inside Promise are called Microtasks. Following is an example

    <body>
        <button id="demo">Trigger Alert</button>
    </body>
    <script>
        document.getElementById("demo").addEventListener("click", () => {
            Promise.resolve().then(() => console.log("Micro Task 1"));
            console.log("Task 1");
        });
    </script>

The above example will print the following output as expected. Microtask will be executed after all the synchronous tasks are executed.

Task 1
Micro Task 1

Let's look into another interesting example

    <body>
        <button id="demo">Trigger Alert</button>
    </body>
    <script>
        document.getElementById("demo").addEventListener("click", () => {
            Promise.resolve().then(() => console.log("Micro Task 1"));
            console.log("Task 1");
        });
        document.getElementById("demo").addEventListener("click", () => {
            Promise.resolve().then(() => console.log("Micro Task 2"));
            console.log("Task 2");
        });
    </script>

Try guessing the output (take 5 seconds). If you guess the following output, it's perfectly alright! It is natural, intuitive, and we are human.

Task 1
Task 2
Micro Task 1
Micro Task 2

But the answer is

Task 1
Micro Task 1
Task 2
Micro Task 2

If you thought of the above answer, Congratulations! 😀 👍

How microtasks are executed?

Micro tasks are executed in two situations

At the end of the synchronous script execution.
If the Call Stack is empty.

For handling microtasks, JavaScript maintains another queue - Microtask Queue. For each call to Promise, an entry for each microtask is pushed into Microtask Queue. So, for the above example, the following will be the order

Task 1 got pushed into Call Stack for synchronous execution. Also, Microtask 1 is enqueued in the Microtask Queue.
After execution of Task 1, the Call Stack gets empty. So, it's the microtask time (condition 2 above)! If there were any other synchronous calls, Microtask 1 would have been picked up after execution of those.
Microtask Queue is dequeued and the Microtask 1 gets executed.
After step 3, the second handler gets called by Event Loop. The same thing repeats for the second microtask.

What are Event Loops?

I know you might be thinking what am I talking about for this long. Nowadays, all browsers create a separate event loop for each browser tab. Collectively these are called Event Loops. At first, Chrome started doing this. Others followed soon.

Impact of Event Loops

I have seen one unresponsive page causing the entire browser to get stuck in Firefox's earlier version. I have never seen this issue with Chrome. Chrome has a separate JavaScript environment for each tab from early versions I believe. The browser might require more resources to do this. But I think, it is worth the better and richer experience.😌

Interesting Use case: Opening a page in a separate new tab

Let's assume, for the below three examples, target_page contains an infinite loop.

<a href="target_page" target="_blank">New Target Page</a>

For the earlier version browser, for the above scenario, the target_page and current page used to share the same Event Loop for all the JavaScript tasks. As a result, if the target_page is containing an infinite loop, both the current and target_page got stuck in an infinite loop as well. Both the pages used to become unresponsive. The workaround proposed

<a href="target_page" target="_blank" rel="noopener">New Tab</a>

Upon setting "rel=noopener", each page gets separate environment. It was introduced to reduce the impact of untrusted pages containing malicious JavaScript, which might affect the current page. Now, all the browsers have started implementing this behavior by default ( relevant MDN doc). I have tested with the latest Chrome. It is implemented there.

Also, for this example,

<a href="target_page">New Tab</a>

If you open target_page in a new tab (by right click on the mouse), both the pages share the same JavaScript runtime environment. So, if target_page has an infinite loop, both the pages get stuck.

Conclusion

The Event Loop is an interesting and critical component for understanding any JavaScript run time environments. I am building up my knowledge on React JS and Node JS internals upon this. If you have enjoyed the article, do give me thumbs up. In case of any questions, let me know in the comments.

Happy learning! 😀

References for further reading

EcmaScript Standard - EcmaScript standard has documentation around how each component such as Job Queues, Call Stack should be implemented. I find these details particularly interesting.

Abstract Introduction to React Internals

Subhasish Kundu — Sat, 05 Jun 2021 15:50:25 +0000

React is a rich JavaScript framework. I think it's necessary to know a high-level internal working of a language or a framework to use it effectively. React's performance comes from its internal data structures and algorithms. In this article, I will not touch upon the actual codebase, data structures, or algorithms. I will share, on a high level, how React 16 renders and updates the UI. So, let's get started!

Smart data structures and dumb code works a lot better than the other way around. - Eric S. Raymond

Internal Representation of Components

During parsing, React creates an in-memory representation of components for easy rendering and updating. Typical React components can have multiple children and sibling components. We have multiple options to represent a component.

N-ary tree.
Multi-linked doubly linked list.

Sample Component

Representation of The Component

The following diagram shows the internal representation for the above component using both data structures.

For both the data structures,

we can represent the parent-child relationship vertically.
we can represent sibling relationships horizontally.
all the relationships are bidirectional for easy traversal.

One thing to note - I am taking a component for simplicity. I am not taking the entire page into the scope.

Rendering and Updating React Component

There can be two types of rendering - simple rendering without any updates or rendering page components after updates in a dynamic UI. Let's see how each scenario works.

Simple Rendering without Updates

In this case, React can use a combination of Depth-First and Breadth-First Traversal to render the entire component (for both types of representation).

Handling UI Changes

In a dynamic user interface, we will have periodic updates as well as other high-priority activities like network calls. It is possible while rendering current changes, some other high-priority tasks come up. In that case, React may have to do a context switch. And that's where React shines. How does React handle that? Let's jump into that.

[I will use representation and DS (for data structure) inter-changeably.]

Rendering Phase

Let's call the original representation current copy. Let's go step by step.

Engine starts cloning the current copy into another DS by traversal. Let's call it work-in-progress copy.
If there is any change in any Node, it can tag the updated Node during traversal. It can store the updates in a separate internal DS like a list. Let's call it effects list.

Let's assume, for our example, Logo, About Us, and Products sub-components have changed after this operation. In that case, the following can be the effects list.

Why not update the UI while traversal?
Let's suppose, the user creates an animation with tasks A and B. React is supposed to render A and B consecutively for creating a rich experience. Now, a high-priority task has arrived between A and B. Engine cannot ignore it. If the engine has to perform it between A and B by stopping B, the user will see a glitch between A and B. It is not desired user experience.

Why collect updates in a separate DS?
As a high-priority task may arrive in between, rendering may not be as important as that task. In that case, React should switch context. After finishing that task, it can resume the traversal again. This collection of updates will help React to start the operation easily again.

Commit Phase

After the collection of updates in a DS, it's time for applying the changes. It has to go in one shot. Otherwise, the user will see glitches and partial updates.

We can traverse the effects list and apply the updates in the actual UI. It has to be synchronous operation.
Once everything is done, React needs to store the updated DS in current copy to represent the current UI. How to do that? Yes, you guessed it right - point current copy to work-in-progress copy.

The above steps complete the entire update cycle. React engine applies various optimizations. We will explore React internals in detail in the next post. I promise I will touch on React data structures and codebase.

Notes

React uses the Multi-Linked Doubly Linked List representation internally.
I have used nomenclature used by React teams like current copy, work-in-progress copy, and effects list. Naming is always harder. They did a good job with these names. So, I just used the names.
In React 15, React used to perform partial updates as I mentioned in Why not update the UI while traversal? section. In React 16, they have changed the algorithm.
Using an effects list kind of DS in compiler is quite interesting. It is worth a separate post. I have seen this pattern to execute a bunch of assembly code for showing the final updates to the user for low-level language compilation. It is a great example of the power of simple data structures like Linked List. Let me know if you want to know more about this technique in the comments. I will try to create a post on that particularly.
Check out my article about React Virtual DOM.

If you enjoyed the article, please don't forget to like it. In case you have any doubts, ask in the comment section.

Happy Learning! 😀

How Virtual DOM is Powering React?

Subhasish Kundu — Wed, 02 Jun 2021 20:19:09 +0000

I have been doing front-end web development for last year only. Lately, I started playing with ReactJS. I have worked on vanilla JavaScript and jQuery for quite some time. My expert colleague at Amazon suggested I explore React. My only sin was I said, "Can't we have an abstraction over jQuery?"

I have heard about React for a long time. The overwhelming number of tutorials on React made my learning very easy. I apologize to myself for not starting early. But better late, than never.

I have worked on compilers for a long time. So, while exploring a new language, I try to think how an interpreter or compiler might do the trick. After learning syntax, I started exploring how React is improving performance. And I encountered Virtual DOM.

[In this article, I will try to explain how virtual DOM might work with respect to a compiler. I will not go into any specific details. I am still learning how React is implemented. I will keep posting details.]

Document Object Model or DOM

A tree of individual components that the browser creates upon receiving an HTML response from a server. JavaScript provides us ways to manipulate DOM to get the desired effect like displaying a div on clicking a button (Codepen sample).

The above operation is expensive as the browser manipulating the DOM tree each time some changes are applied. I believe the browser has to do some smart computation for identifying the changed element and making the changes (like suppose, keeping a reference to each element in a hash map. I am still exploring it.). It is definitely an expensive operation - time and memory-wise. Also, the browser has other tasks to do like displaying the page.

What is Virtual DOM?

"The virtual DOM (VDOM) is a programming concept where an ideal, or “virtual”, representation of a UI is kept in memory and synced with the “real” DOM by a library such as ReactDOM." - React Documentation .

I did not even try to define it with my words. It is such a concise definition. What I have understood so far - React virtual DOM is an in-memory optimized representation of DOM. And React engine deals with low-level APIs to apply the changes.

How React brings Performance into Picture?

Now, with the above concept, let's bring the compiler into the picture.

Every compiler stores an entire program in some form of representation (Abstract Syntax Tree). I am mapping Virtual DOM to AST (I am still learning how React optimizes). As virtual DOM is an in-memory representation, React engine or compiler can access it fast. Now, let's go step by step.

Now, if there is some change - React changes virtual DOM.
React engine compares the virtual DOM with real DOM (aka "diffing") and applies the changes to the changed part only. It finds out the changes.
It applies the changes using low-level APIs directly (like the JavaScript engine does for our code).
React engine can apply optimizations in all three steps because virtual DOM representation is in its control. React engine is actually a very advanced JavaScript engine. With syntax and semantics of React, it can fasten the performance.

Now, in JS, the browser is doing all the work. Also, to some extent, it is dependent on how code is being written. In the case of React, React engine is abstracting (remember the word I mentioned at the beginning) out all optimization.

This is one of the reasons, React is faster than traditional HTML JS webpages. I know technically everything I mentioned might not be true and I have not touched upon a lot of things. As I keep exploring, I will keep posting my findings in detail. I will see how my concepts get cleared on the way. I hope you enjoyed the article.

Happy learning!