DEV Community: Jennie

3 facts of Web copy & paste you may not know

Jennie — Thu, 27 Feb 2025 20:10:07 +0000

One day I accidentally opened a Pandora's box while trying to fix an ancient bug. The issue involved copy & paste functionality for items in our template editor (JSON under the hood) in Safari. My fix solved the original problem but created new ones in Windows and introduced other quirks in Safari. This fascinating journey made me relearn the complexity of copy & paste APIs—something I'd previously dismissed as just a single line of code throughout my career. 🙈

We know that there are 2 ways to trigger copy & paste diagrammatically now:

document.execCommand() - widely known but deprecated for years
navigator.clipboard APIs

No matter in flexibility, security, and content support, navigator.clipboard appears to be a better choice now, and the browser support looks very promising.

However, there are 3 things you may need to know when you want to work with it.

Copy & paste event aren’t triggered

In MDN it confuses us with:

Events are fired as the result of cut, copy, and paste operations modifying the clipboard.

However, reading and writing content with navigator.clipboard APIs won’t trigger the copy or paste events. Here is a demo (open console):

This makes sense as we are firing the copy & paste programmatically instead of interacting with the UI.

The unremovable? “Paste” menu in Safari

Running the above example on Safari, you may find this interesting “Paste” menu popup on top of our “Paste” button if you copy the text with shortcut or context menu, and ONLY trigger the click event if you dismissed this menu:

What makes things more fun is that if you copied by my “Copy” button, then click on “Paste” button, this menu won’t appear. ☹️

Here is an reference of this issue: https://developer.apple.com/forums/thread/133001

System will modify your content

Another fun fact was found when I copied content including line break and use as a source for comparison. That is Windows does modify your written text!

Try this on Mac and Windows Chrome (Does not work on Edge because of security policy):

If you copy text with \n symbol on Windows, it will replace them with \r\n:

The reason of this behavior was that \r\n is the valid new line symbol on Windows instead of \n . I guess the difference made some complains on Windows and then it decided to convert the content when user copy text.

To be fair, this won’t create a big problem unless you are the unlucky person like me trying to do some comparison.

Lastly

Thanks for reading and hope these information make your day better.

Relocate to Australia and re-discover engineering culture

Jennie — Sat, 08 Feb 2025 04:48:23 +0000

Time flies—it has been 2.5 years since relocating from Singapore to Australia and 1.5 years at an Australian SaaS company. Though I had experience moving from China to Singapore 10 years ago, this transition took longer to adapt to. It was not only because I became a parent, but also because the cultural differences between Australia and Asia are more significant.

A Challenging Time

My husband received a job offer in Sydney just before Covid while I was working at the fast-growing Shopee. Back then, without our little one, everything seemed simpler, and we eagerly decided to embark on our next adventure.

However, Australian visa processing was suspended for over two years. Like many couples, we welcomed a new family member during this special time.

When we finally arrived in Sydney, we faced multiple crises: soaring rental prices, doubled petrol costs, and a shortage of childcare spots due to increased demand from the childcare subsidy policy, labor shortages, and higher birth rates. This forced me into an unwanted year as a housewife, watching our savings dwindle each month.

When I finally secured childcare and started job hunting, both the market and I were at the low time. I had two years of broken sleep and minimal coding practice. Layoffs were still common and opening head counts were still little.

SafetyCulture offered me an opportunity to excel. Later I learnt that I was their only front-end hire in that year.

On-boarding Challenges

Our engineering team is divided into customer-facing teams and platform teams. All the teams are quite lean. Inevitably, a team sometimes lacked dedicated platform engineers when people were sick, on leave, or positions remained unfilled.

I was replacing an engineer who had returned to Europe. She could only onboard me remotely in my evening hours or asynchronously. Other the engineers were either lacked context about the six-year-old messy front-end code base or on leave.

Moreover, the outdated documentation and legacy code created unnecessary confusion. After all, the on-boarding documents hadn't needed updates for a year company-wide and over two years in our team.

Before this, I was in a large web team. On-boarding ran smoothly with well-maintained documentation and plenty of mentor resources.

Fortunately, a kind engineer from the platform team appeared unexpectedly to help. I soon learned that people here readily assist others, even at the cost of their own time—my first culture shock.

In Asia, engineers tend to be reserved, especially those who aren't tech leads or managers. Most prefer coding to talking. Cross-team assistance was rare, and sacrificing time to help others wasn't celebrated in the organizational culture—only delivering impressive features was.

Encouraged by the feedback-welcomed environment here, I took some notes and chat to the platform engineers. Only a few months after, the massive mono-repo documentation was beautifully revamped, earning praise widely from new hires.

The Flexible (or Messy) Process

Another aspect that confused me during on-boarding was the process. Each lean engineering team reports to an engineering manager, who reports to a director. The organization gives teams maximum flexibility to establish processes based on their business needs. Everyone is encouraged to shape the team through initiatives that gain team consensus. While this increases ownership and enables quick adaptation, it can disorient newcomers.

In my first few months, I often felt uncertain about next steps, and no one seemed able to clearly define which situations called for which actions.

Surprisingly, everything functioned well, with everyone displaying strong self-motivation. This contrasted with my previous experience, where managers or leads would define most processes, refining them over years based on feedback to reduce ambiguity. Most engineers would simply follow established procedures.

Here, when dealing with ambiguity, we're not only encouraged to discuss with others but expected to contribute our own perspectives. Though decision-making takes longer, it leads to better solutions for various scenarios and ensures all voices are heard. This workplace values active communication over static documentation.

Re-understanding Engineering Manager and Leadership Roles

While many leading IT companies started to separate engineering and management career tracks years ago, this was my first experience with such a system.

I happily chose the engineering track due to my exhausting experience of managing technical responsibilities alongside team members without actual managerial authority.

Yet it wasn’t a simple split.

Task assignment and workload balancing become shared responsibilities between engineers and managers based on individual interests, particularly when multiple engineers maintain the same platform and team goals aren't assigned to specific individuals.

Engineers need to put in significant effort to demonstrate the value of technical work when competing for priority against tech debt and product roadmap items. This is especially important since managers, even those with engineering backgrounds, may not fully understand the technical details of our work.

The glue work is not reduced from our responsibilities. Instead, these essential activities - like mentoring others, improving documentation, or facilitating technical discussions - are actively recognized and celebrated within the organization. This represents a significant shift from environments where such work might go unnoticed, as the company culture here places high value on these contributions that help strengthen team cohesion and knowledge sharing.

Re-understanding the Product Manager Role

Product managers here operate differently. They focus on high-level product strategy rather than details— determining the product roadmap, priorities, and scope.

Instead of PM, engineers will take the lead, carving detailed plans including milestones, tasks, acceptance criteria, edge cases, and handling options.

Initially, I struggled to identify the right person for discussing issues and solutions. It wasn't a simple if-else scenario: UI/UX for designers, business logic for PMs, resources for project managers, and technical issues for tech leads/managers.

After all, people in the same role work differently—some PMs focus on high-level strategy and delegate details to designers, while others take a more hands-on approach when the team needs decisions. A crucial skill is identifying the right "team" for efficient collaboration without disturbing others unnecessarily.

Re-understanding the Quality Engineer Role

We use a quality engineer assistance model where multiple teams share one QE. This means QEs don't verify everything or conduct team regression testing. Engineers take greater responsibility for testing and rely more on automation. We rely on CI/CD pairing with diff coverage for faster delivery, while QEs help identify essential test cases and establish quality goals and metrics.

The shift from strict, process-heavy quality control to this trust-based system affected me deeply—like a child's first independent outing. Each PR brought anxiety: Should I request QE testing? When? Are my test cases sufficient? What if I miss critical edge cases and cause production issues?

Reviewing other engineers' PRs made the responsibility feel even more daunting. And eventually it pushed me to write better tests and accelerate development by removing QA bottlenecks. When issues arise in production, we have clear processes for both urgent incidents and non-urgent customer reports, encouraging quick responses and continuous improvement.

Re-understanding Engineering

Back-end developers, SREs, and database engineers regularly create technical design documents with detailed architecture diagrams, data structures, and API designs. However, for the Front-end work, documentation typically appears only for ambitious projects, usually with rough diagrams beforehand and explanatory documents afterwards.

I had no experience writing Front-end technical designs or RFCs. This stems partly from my Chinese education background, where a notable book criticized the lack of training in research and proposal writing.

Frontend developers often lack of motivation to craft formal technical design in favor of quick feedback, moving straight from proof of concept to implementation, and expectation of fast delivery.

After several technical discussions, issue investigations, conversations and observations with our talented Back-end engineers, I've grown to appreciate how technical design and investigation documents help solve complex problems, document decisions, track progress, and support career advancement in cross-functional teams.

The struggling writing process was time consuming but allowed me to dry-running ideas in all scenarios and refine again and again rather than implementing and refactoring code with double, triple or even more efforts. And I find it pushed me to learn new things everytime, much more than coding.

Here are some common documentation topics I found easy to start with:

comparing and choosing third party library, tool chain
naming conventions
learnings

And some often skipped and sometimes more difficult to write:

common utility or component design
state structure design
refactor and migration plan
complex implementation with trade-offs, other options and why

Afterwords

Fundamentally, all the work remains, but instead of having dedicated project managers, we've distributed those responsibilities across multiple roles. Engineers now occupy a more central position in the organization with expanded capabilities and responsibilities—though this comes with the trade-off of spending less time coding. That updates my understanding of engineering culture.

This has been a fascinating learning journey, though it took me considerable time to find the right words to describe my observations. After 10 years in the industry, I'm delighted to see that ways of working continue to evolve.

Web component tutorial for React devs

Jennie — Thu, 25 Apr 2024 02:47:16 +0000

Web component was a breaking news but sank over the years. I never paid attention util one day I read about a new open-sourced micro front-end framework is based on web component 🤯.

Reading MDN and some beginner blogs hardly answered my doubts. Being a React dev for so many years, I knew I would learn better with a React mindset.

What is the foundation of a React component?

A custom JSX tag
Internal state
Properties and children
Life cycle

Let’s write some code!

NOTE that enable to learn progressively, demo code are ignoring good practices.
Full demos are here.

Starting with a classic counter

In React, we may create a counter with very little code:



export function Counter() {
  const [count, setCount] = useState(0);
  const increase = useCallback(() => {
    setCount((n) => n + 1);
  }, []);
  return (
    <div>
      Current count: {count}
      <button onClick={increase}>Increase!</button>
    </div>
  );
}

The web component version is a bit longer (try here):



<my-counter></my-counter>

<script>
class Counter extends HTMLElement {
  constructor() {
    super();

    const shadow = this.attachShadow({ mode: 'open' });
    shadow.innerHTML = `
      <div>
        Current count: <span class="counter-number">0</span>
        <button class="counter-increase">Increase!</button>
      </div>
    `;

    let count = 0;
    const countNode = this.shadowRoot.querySelector('.counter-number');
    this.shadowRoot
      .querySelector('.counter-increase')
      .addEventListener('click', () => {
        countNode.innerText = ++count;
      });
  }
}

customElements.define('my-counter', Counter);
</script>

To create the web component, we declared a class extends HTMLElement first, and define a custom HTML element with customElements.define() API that binds my-counter HTML tag to the class.

💡 Note that there are a lot of constrains with the custom HTML tag.

Next, we enriched the component with a Shadow DOM by this.attachShadow({ mode: ‘open’ }) , and operates the DOM from this.shadowRoot.

Shadow DOM explained

Shadow DOM, just like the document in iframe, is a HTML fragment that contains a DOM tree named as shadow tree. It is attached to a regular DOM element named as shadow host.

When the mode of Shadow DOM is set to open, we can access the shadow tree from the shadow host like this:



shadowHostElement.shadowRoot.querySelector()

When the mode of Shadow DOM is set to closed, the shadow tree will get hidden from the document.



console.log(shadowHostElement.shadowRoot); // null

As it is a separate HTML fragment, the styles are isolated. That means shadow DOM won’t inherit or apply any styles from the document, and vise versa.

To style the shadow DOM, we need to place the style tag in the shadow tree like this:



const shadow = this.attachShadow({ mode: 'open' });
shadow.innerHTML = `
<style>
.my-div { background: red; }
</style>
<div class="my-div"></div>
`;

The style isolation and the shadow root access created a layer of encapsulation that is very important in sharable components and micro front-end architecture.

Must we use shadow DOM? In some cases, but not in this one! It is generally recommended to use shadow DOM in a web component to have the layer of encapsulation. One of the case that must use shadow DOM will be demonstrated later.

State in web component

In the counter demo above, we did tedious DOM operations for the count state in our web component version. Do we have state management in web component?

The answer is yes and no. It is quite different (try demo here):



<with-state></with-state>

<script>
class WithState extends HTMLElement {
  constructor() {
    super();

    const shadowRoot = this.attachShadow({ mode: 'open' });
    shadowRoot.innerHTML = `
      <div>
        <input type="checkbox" id="check" />
        <label for="check">
          Tick and untick me!
        </label>
      </div>
      <style>
        // ONLY IN SAFARI WITH PREVIEW ON
        :state(--checked) {
          background: green;
        }
      </style>
    `;

    const { states } = this.attachInternals();

    const label = shadowRoot.querySelector('label');
    shadowRoot.querySelector('input').addEventListener('change', (e) => {
      if (e.target.checked) {
        states.add('--checked');
      } else {
        states.delete('--checked');
      }
      label.innerText = states.has('--checked') ? 'Untick me!' : 'Tick me!';
    });
  }
}

customElements.define('with-state', WithState);
</script>

In this WithState component, we enabled states with this.attachInternals() first. Then add or delete state accordingly.

There are a few interesting details:

This state is a Set - it can only act like a boolean.
The state require a ugly double dash - if not, browser simply throws error although some of MDN document is not specifying this. Well, this is still tolerable considering to avoid conflicts with tag names in CSS.

This “state” is kind of unhelpful. The only game changer is probably the :state() CSS selector which is only supported in Safari with preview FF on 👀. This selector allows the state work similar to input disabled, focus.

Creating components with Template

Demos above attach the DOM tree to web component with innerHTML . In fact, they can be more flexible, scalable and readable with <template/> and <slot/> instead.

Let’s look at a simple Card component:



export function Card({ children }) {
  return (
    <div
      style={{
        background: '#ccc',
        borderRadius: '8px',
        padding: '14px',
      }}
    >
      {children}
    </div>
  );
}

A card component in React is a container with children prop. While in web component, this is achievable with <template/> and <slot/> (try here):



<my-card>
  <span slot="card-content">Span slot of the card.</span>
</my-card>

<template id="card-template">
  <div style="background: #ccc; border-radius: 8px; padding: 14px">
    <slot name="card-content"></slot>
  </div>
  <style>
    ::slotted(span) {
      background: red;
    }
  </style>
</template>

<script>
class Card extends HTMLElement {
  constructor() {
    super();

    const template = document.getElementById('card-template');

    const shadow = this.attachShadow({
      mode: 'open',
    });
    shadow.appendChild(template.content.cloneNode(true));
  }
}

customElements.define('my-card', Card);
</script>

The component class simply attached the cloned DOM tree from a <template/> node. Styles and the “slot” for content noes were all specified in a template.

We use a <slot /> tag with a name attribute to declare slot, and we may insert any tag with an attribute slot and the name we declared.

Although the HTML markups in <template/> is not rendered anywhere, it is accessible and mutable like any other regular DOM elements. Hence, we’d better clone the template DOM tree every time instead of using it directly.

What happens to any content out of slot? I tried with below code and the browser perfectly ignored them 😄.



<my-card>
  Card Children
</my-card>

How about using the same slot multiple times? It works! 🤣 The elements were added into card-content slot from top to down.



<my-card>
  <span slot="card-content">Span slot of the card.</span>
  <div slot="card-content">Div slot of the card.</div>
</my-card>

Remember that we talked about shadow DOM may not be necessary, not here! It turns out the <slot /> tag only works in the web component context with shadow DOM.

The life cycle of a web component

To pass in properties and trigger effects when the property changes, we need to work with the life cycle methods.

There are 4 life cycle methods other than constructor:

connectedCallback
disconnectedCallback
adoptedCallback
attributeChangedCallback

Let’s try this demo:



<button id="with-attribute-demo-toggle">Start demo</button>
<div id="with-attribute-demo-container"></div>

<template id="with-attribute-demo">
  <input
    id="with-attribute-input"
    type="text"
    placeholder="Change value here"
  />
  <with-attribute value="Initialized"></with-attribute>
</template>

<script>
class WithAttribute extends HTMLElement {
  static observedAttributes = ['value'];

  constructor() {
    super();
    console.log('Constructor.');
  }

  connectedCallback() {
    const input = document.getElementById('with-attribute-input');

    input.addEventListener('change', (e) => {
      this.setAttribute('value', e.target.value);
    });
    console.log('Connected.');
  }

  disconnectedCallback() {
    console.log('Disconnected.');
  }

  adoptedCallback() {
    console.log('Adopted');
  }

  attributeChangedCallback(name, oldValue, newValue) {
    console.log(`Attribute "${name}" changed ${oldValue} to ${newValue}.`);
  }
}

customElements.define('with-attribute', WithAttribute);

const toggleBtn = document.getElementById('with-attribute-demo-toggle');
toggleBtn.addEventListener('click', () => {
  const container = document.getElementById('with-attribute-demo-container');
  if (container.hasChildNodes()) {
    container.innerHTML = '';
    toggleBtn.innerText = 'Start demo';
  } else {
    toggleBtn.innerText = 'End demo';
    container.appendChild(
      document.getElementById('with-attribute-demo').content.cloneNode(true)
    );
  }
});
</script>

Don’t get terrified by the length. The WIthAttribute component only logs the life cycle methods. Other than that, there are a little input and a button to help mount, unmount the component and change the component attribute. Additionally, a compulsory static observedAttributes is added for observing the attribute change.

Click Start demo button, here is the console log:



Constructor.
Attribute "value" changed null to Initialized.
Connected.

Something interesting here: Right after the constructor, the attributeChangedCallback was triggered to change my value attribute from undefined to “Initialized” that I assigned directly on the HTML markup, and connectedCallback follows. That is suggesting connectedCallback is actually a better place for initializing a component.

When we click the End demo button, a new log appended:



Disconnected.

Clicking Start demo button again, you will see following log again:



Constructor.
Attribute "value" changed null to Initialized.
Connected.

I was expecting Constructor not showing up here but nah…

The last adoptedCallback I didn’t figure out with the MDN doc. Then I looked stackoverflow for help and found this:

The adoptedCallback() method is called when a Custom Element is moved from one HTML document to another one with the [adoptNode()](https://developer.mozilla.org/en-US/docs/Web/API/Document/adoptNode) method.

🤯what? You can do that? #TIL.

Using web component in React

If you are maintaining a React repo but wondering whether you could introduce web component. The answer is yes! Implementing a web component into a React component is just like any other HTML markups (try here):



export function App() {
    return (
        <Card>
      <my-counter></my-counter>
    </Card>
    );
}

However, placing a React component into a web component is not that feasible as the React components need to connect with the React app root.

Wrapping a React app in web component

Though React component in web component is not likely to work (and not necessary?), it is possible to encapsulate the entire React app in a web component (try here).

First, modify the React app to allow passing in the app root:



export function initApp(container = document.getElementById('react-app')) {
  const root = createRoot(container);

  root.render(
    <StrictMode>
      <App />
    </StrictMode>
  );
}

Next, create a web component wrapper like this to initialize the React app:



import { initApp } from '../react-app';
class ReactAppWrapper extends HTMLElement {
  connectedCallback() {
    const shadow = this.attachShadow({ mode: 'closed' });
    initApp(shadow);
  }
}

customElements.define('react-app-wrapper', ReactAppWrapper);

That is exactly what micro front-end needs!

Last but not least

Nobody likes to manually update DOM when state updates with the convenience of modern frameworks, but that could be resolved by integrating another framework or library. For example, Lit, Stencil, etc. Yep! Another framework!🙈

Refactoring code reused across multiple teams

Jennie — Wed, 20 Mar 2024 03:34:10 +0000

The tech debt

I found an interesting ticket concerns about a legacy domain component being reused across 4 teams by importing source code path and coping necessary logic because

The copied logic lied in other domain is not under my team’s maintenance.
Applying updates requires more efforts.
Importing source code path works in the current mono-repo setup temporary. But it may break easily if the package published without source code or changed internal path.
The code was not designed for reuse at all.

It was not hard to guess the major cause of the ticket is the limit of resource - a norm of a fast developing tech company. It becomes a tech debt and the complexity grew rapidly. Devs started to alter the copied logic for their needs, and more teams follow the efficient copy over way that worked.

Eventually, we need much more context to ensure the quality than the time it created to pay back the debt.

My team never feel the pain as before adding any new stuffs there, no extra works for the team. Teams reused these code didn‘t feel the pain as they avoid the long waiting from my team and get things delivered quickly. Util one “lucky” developer feel the pain and decided to cut it off.

Well, I was lucky enough to become that person and here is some notes and thoughts for such case.

The refactor

Preparations

There were a lot of challenges for a refactor like this, including:

Limit of resources - Completing the refactor will take long as we have to deal with other higher priority stuff.
Wide impact range - Every change has risks to cause incident all over the place.
Lack of code context - No one in the team knows the part of code well.
Questionable testing coverage - There was decent unit test coverage but almost no e2e coverage. As refactor changes implementation, unit tests often become invalid. Hence no guarantees of things work as expected after refactoring.

Base on these challenges I identified some necessary preparations, like:

Break the task down into the granularity that could fit both the development and review works into our tight schedule.
Instead of changing any exisiting business logic code, create new and migrate one by one.
Learn the code well before jumping in.
Fill the test coverage gap before any changes.
Pair with the teams for the migration.

The plan

Based on all the information and conditions, I tried to create small frequent iterations that could get delivered in a few days and resilient of interruptions from higher priority works.

I formed a rough plan in my head:

1. Move the reused files into a shared package

Ideally moving the reused files shall be done at the last step to ensure a smooth and gradually transition like the diagram above. However, the effort of creating the new reusable hook was too large to fit in the schedule, and completing the migration could span across several months. Therefore, I decided to bring this piece of work up front that takes relatively short time and increase the awareness of the reused attribute of this component.

2. Make small clean ups

Reducing unnecessary properties, states and callbacks are very helpful in the start of the refactoring work. These small clean ups are not only helping to reduce the complexity of refactoring and the following up migration, but also helping me to understand the implementation and usage.

3. Create a new encapsulation

There were a few duplicated and modified code sitting within the host components. To reduce the redundancy, and to avoid modifying exisiting logic and causing incidents everywhere, creating a new encapsulation was a better choice. The new encapsulation may reuse any existing code and extend the logic while not affecting anyone before fully tested and migrated.

It ends up to be a new hook allowing the hosts to control the component, and exposing some necessary states.

4. Migrate usage to the new encapsulation one by one

With the new hook ready and tested. I can finally help the hosts to import the new hook, and clean up the copied and modified logic.

5. Final clean up

For a smooth migration, I may create some intermediate stuff. After the complete migration, these code would only cause confusion. This last step was to clean up and reduce the possible confusion.

The actual work carried out

Before starting touching the code, I only created 4 tickets and added more during the actual work.

With the rough understanding of the code and plans, I reached out to a Front-end developers from one of the 4 teams. Unsurprisingly, importing our source code from the private space and maintaining the duplicated messy logic was not his concern, but initializing the extra Redux state in their domain was. This was completely out of scope and I promised to take a look later.

As planned, I moved the related code to a shared package first. To reduce the impact scope, I relocated the source code but keep the original path and export everything from the new place. The change took only few days but the code review took weeks thanks to the long list of the files.

The next piece of work started at 4 months later. Without writing down the plan, I almost forgot what I wanted to do. Luckily, my brain picked up quickly after scanning through the related code.

I started with removing a prop that could retrieve from the another prop instead, and making another prop more generic.

Then I happily discovered a piece of my work done in this 4 months gap removed a blocker of retiring the necessary Redux state. And I did it. The clean up was huge but straightforward enough to demand relatively small efforts from each team to review and merge.

Next, I created a new encapsulation but went a longer way as the long iteration span made me forgot my initial plan again 🙈 . Furthermore, I found a block of complex logic in the reused code that only required in the inspections list. As a result, I created an intermediate implementation to help the transition.

I thought I am ready to start the migration with the intermediate hook. Soon I realize it was not good enough when I tried migrating. Therefore, another better hook was born based on the learnings from the failure trail. The new hook required minimal code and provides intuitive and only essential callbacks for the host component to implement.

When I finally get back on the track of migration, the new hook that I was confident about crashed a page. Interestingly, it was neither a bug in the new hook nor in the host component of that page. It was an implementation in the hook could not work well with the host component. To resolve this, I adjusted the implementation of the hook one more time.

Eventually, I created 3 more small PRs and migrated everywhere except the most complicated one, and I got pulled into another high priority feature development expecting months of effort.

My mistakes

There were various mistakes I made:

Planed in head. Since the time span for the project took very long, I kept forgetting my initial plan and spent extra time to recompose one. This could be avoided if I wrote them down.

Created large PRs that was possible to break down. The PR I created for relocating files, retiring Redux state was large. These PRs were possible to break down for a safer transition, but I didn’t figure it out.

Breaking down PRs with little conflicts requires lots of planning ahead as it has to ensure everything works as expected with the small subset of changes. While the large PRs, could contain incomplete changes and polish later util reach the point of perfection.

Although more planning effort required, I really love the quick feedback loop of the small PRs. The reviewer were happy to review and verify the change in a few hours. Furthermore, my confidence of the deployment was higher as the quality of a small scope were easier to guarantee. Additionally, the maintenance effort before merging is much lower as it gets merge quicker, and the conflict range of each PR is small.

Simple migrations didn’t inform all the stakeholders in advance. Engineers love to surprise reviewers with the PRs without having a proper conversation in advance. I am one of them. After the first and only conversation with one team that I found little value, I skip the other conversations. Thankfully everything works well so far. But this has to be corrected as this is how this tech debt started and I tried to avoid.

Other take aways

Pace it slow is not a bad thing. As mentioned above, I had a 4 months gap between the first and second piece of work. During this gap, a blocker of the Redux store pain point was removed in a non-related task. What an existing surprise! Sometimes the problem goes away after a while.

Break the tech debts task down to minimal is the key of delivering when resource is precious. Other than the fast feedback loop, the higher quality confidence, and the less maintenance overhead mentioned above. It was easy to find a chance between the high priority job since each subtask took up to 3 days of initial work. More over, the small scope of the subtask makes it less overwhelming to drop and pick up after a while.

Complexity of patching tests across domain is very high and may not worth the effort. To ensure the quality, it is better to have complete test coverage before refactoring in theory (I did so before and it worked really well). While in reality, the complexity of crossing the domains doubles or even triples the effort that may not worth the value created. Thus, in this refactoring, I chose to rely on the existing tests and doing more manual tests to help speeding up the process. Meanwhile, in the test of the host components, I mocked my component and hook to reduce the testing overhead.

After thoughts

Although large refactor was satisfying, I would rather it never happen. However, this is inevitable if an app keeps growing after some time. Then how can we make things better and easier when before reaching that point or avoid it happen?

First of all, follow the good practices whenever possible. For example, creating components or functions, classes that take care of (little but not too little) logic about the same feature, same category. This practice maximizes the reusability of the code, make them flexible enough for composing in different cases.

Secondly, refactor often and small. Overtime, good practices change, meanwhile our knowledge grows and the “flawless” design reveals flaws. If we continuously tweaking the code to reduce the flaws, it might never grow into a big refactor.

Next, remember if we want to reuse other team's code, discuss with the code owner first to gain some context and make short to long term actionable plans together. It seems take more time initially but speeds everything up after.

Lastly, if we want to prevent tech debts like this and help others to reuse our code, we may consider:

Open a public channel for others reaching out. Sometimes people gave up discussion because they didn’t find a easy way.
Define standards for the sharing code so that everyone could follow and make it easy to find. The simplest thing we could do is clarify where to put the shared files especially in a mono-repo structure.
Assist proactively. Helping the developer to understand the context, review the design and code takes time in a short-term but ensure things go well and causing less headache in a long run.

Setup VSCode and WSL

Jennie — Wed, 31 May 2023 06:56:27 +0000

Just like most of the Front-end folks, I was a loyal MacBook user for the last 7 years. Only util recent months I started coding on my Windows PC with VSCode.

Although the development experience is significantly improved since Windows 10 introduced the Linux subsystem(WSL), I still met problems one after one. Eventually, I found that I went in the wrong direction and the quick solution was right in front of me!🤦‍♀️

In short: Remote development in WSL with VSCode.

Here are the simple steps and my exploration with other setups. I hope it helps saving your time.

The setup in short

Before we start, please ensure that you have upgraded your Windows 10 to at least version 2004 (According to the official WSL tutorial) for WSL to work.

Enable the subsystem

There are 2 ways to enable the subsystem.

From GUI in the “Turn Windows Features on and off”:

Or in the command line tools with administrator according to the official WSL tutorial:



wsl --install

Then you shall be able to find the “WSL” app in the Windows Start Menu.

Update Ubuntu version

In my experience, the default Ubuntu installed is v16, which is so outdated that I could only install Node.js v14.

With command wsl --list --online, you may find a more updated version like 22.04 LTS in the following screenshot I took few weeks ago:

You may install it with command:



wsl --install -d Ubuntu-22.04

Then you shall be able to find the “Ubuntu 22.04 LTS” app in the Windows Start Menu.

VSCode remote development

Next, install VSCode and the WSL extension, and you will be able to find “Connect to WSL using Distro” in the command palette:

Select the Ubuntu-22.04 you have just installed:

A new VSCode window will open in a remote mode working on the Ubuntu file system, and you may happily coding there just like working locally. The only noticeable latency I found so far is in the starting up.

Why remote development in WSL

I started local development with WSL as a terminal because quite a few things couldn’t work well under Windows. And soon the nightmares came to me:

Splitting terminal failed
Extension like Jest debugger, prettier spoiled
Watching mode broken
HMR died

Maintaining VSCode extensions was still tolerable, while it was nearly impossible for me to work on a web app.

I noticed the root cause of these malfunctions was path. Afterall, Windows and the subsystem Ubuntu are running on 2 different file systems no matter how seamlessly WSL looks:

All the Windows drives will be mounted under /mnt/ folder in WSL.
All the localhost ports will be forwarded without extra configurations.

However, if we operate the VSCode in Windows, the IDE and their extensions expect working with Windows file system not the Linux’s. As for stuff like HMR not working, I haven’t dived deep into it but I guess something to do with the port forwarding setups.

I believe most people will choose to code in Linux but play in Windows just like me :) (Or you can just buy a powerful but expensive MacBook).

You may wonder why not using a docker image for the remote development? Well, docker image is a great choice if you are not planning to run another docker instance in it. In my case, I sometimes setup things such as DB with docker image.

And I tried to walk away from the environment issue with web IDEs such as GitHub Workspace, Replit, Stackbliz, CodeSandbox before thinking of remote. They are the fast-acting drug worth to take a shot in my mind. However, the web IDE crashed frequently when my experiment grew bigger, and some IDEs demand for a “pro” upgrade to get better experience while the Copilot I paid is not available there.

Final thought

I made a mistake of trying to live with the situation and soon it backfires. It is important to get the tools work right enable to get the job done.

The inspiring extension systems design overview (Chrome & VSCode)

Jennie — Thu, 18 May 2023 02:04:58 +0000

A few years ago, I discovered the power of Chrome and VSCode extensions in improving productivity. I created my own extension if my desired feature was not available. The extension systems allow me to experiment my wild ideas with less effort thanks to their graceful design.

While they have similarities, there are also some significant differences determined by the nature of each app. In this blog post, I will provide an overview of the extension systems of Chrome and VSCode and share what I have learned from them.

Development
- Initialization
- APIs
- The script triggers
- Activating extension
- The extension parts and interfaces
- Events
- The communication between parts
- UI/UX
Testing
- Testing a Chrome extension
- Testing a VSCode extension
Distribution
- Distribution of Chrome extension
- Distribution of VSCode extension
Summary of what I have learnt

Development

Initialization

Both extension systems have an entry point - a JSON file. Chrome extension starts from a manifest.json file, while VSCode extensions extend the package.json file, which can sometimes become lengthy and contain too much information. I am worried about conflicts arise from the file.

In the JSON, developer need to provide essential information such as icon for display and script to initialise with. Each parts of a Chrome extension require a script file or a html file, while VSCode only need an entry script.

Here is an example of a manifest.json file for a Chrome extension:



{
  "name": "Broken Background Color",
  "version": "1.0",
  "description": "Fix an Extension!",
  "permissions": ["activeTab", "scripting", "storage"],
  "options_page": "options.html",
  "background": {
    "service_worker": "service-worker.js"
  },
  "icons": {
    "16": "images/icon-16.png"
  },
  "manifest_version": 3
}

And here is an example of a package.json file for a VSCode extension:



{
    "name": "cat-coding",
    "description": "Cat Coding - A Webview API Sample",
    "version": "0.0.1",
    "publisher": "vscode-samples",
    "activationEvents": ["*"],
    "main": "./out/extension.js",
    "contributes": { ... },
    "devDependencies": { ... }
}

In addition to these files, Chrome extensions also require careful declaration of browser permissions to enable their APIs. I totally understand the importance and difficulty of guarding browser security, but providing excessive information that can be accessed through automation is unreasonable.

APIs

The extension APIs in both Chrome and VSCode are well-categorized based on the functionality of the app. This categorization allows developers to explore the possibilities by following the names of the app modules. For example:



chrome.cookies.get({ url: "https://example.com", name: "cookie_name" }, (cookie) => {
  if (cookie) {
    console.log(cookie.value);
  }
});

However, there are possibilities that we have no clues about the name at all 🤦‍♀️ such as the text decoration in VSCode as mentioned in this blog.

Being a Microsoft product, the coding style of VSCode extensions can sometimes be too OOP style. For instance, to implement a tree view, we must declare a class extends the TreeDataProvider class and provides methods getTreeItem() and getChildren():



export class DepNodeProvider implements vscode.TreeDataProvider<Dependency> {

    private _onDidChangeTreeData: vscode.EventEmitter<Dependency | undefined | void> = new vscode.EventEmitter<Dependency | undefined | void>();
    readonly onDidChangeTreeData: vscode.Event<Dependency | undefined | void> = this._onDidChangeTreeData.event;

    constructor(private workspaceRoot: string | undefined) {
    }

    refresh(): void {
        this._onDidChangeTreeData.fire();
    }

    getTreeItem(element: Dependency): vscode.TreeItem {
        return element;
    }

    getChildren(element?: Dependency): Thenable<Dependency[]> {
        //...
    }
}

Plus the “traps” I mentioned above in Chrome, a well structured and clear document with rich example codes become vital.

The script triggers

There are three ways to trigger our scripts:

From the extension activation;
User interaction on the extension interface;
Internal events.

Activating extension

When a user enables an extension in Chrome, the background script of the extension starts working immediately.

Chrome has migrated the extension background script to the Service Worker in manifest v3, making the extension work more like a web app.

In VSCode, the whole extension initializes from one JS file, and the main JS file need to export an activate() function. Developers shall declare the activation conditions in the package.json file (see the example above).



// out/extension.js
export function activate(context) { /* ... */ }

It seems that Chrome tries to align the pattern with web development, and VSCode adopts another style commonly seen in serverless development. As a web developer, the way of activating script in the Chrome extensions is more natural to me.

The extension parts and interfaces

A Chrome extension may contain 4 parts of scripts or HTML pages for different purposes:

a popup page for the popup activated from the menu next to the address bar;
panel pages in the developer tool;
a content script to interact with the web page content;
background scripts or page for more interactions.

They are just web pages or scripts allow to access Chrome APIs with some constrains:

We must declare the corresponding permissions in the manifest.json;
Some APIs may only work in certain scope (e.g, we can only access user tab document element from the content script).

VSCode defines too many parts to list in a blog (see Contribution Points). Some parts (or we should call contribution points) are so tiny that all they need are displaying texts and actions.

As multiple contribution points often result into the same action, VSCode took a command approach. Command is a scoped key name binding to a registered script.

We may define our own command in the package.json:



{
  "contributes": {
    "commands": [
      {
        "command": "myExtension.sayHello",
        "title": "Say Hello"
      }
    ]
  }
}

And register the command in the activate() function:



export function activate(context) {
  const command = 'myExtension.sayHello';
  context.subscriptions.push(
        vscode.commands.registerCommand(command, () => console.log('Hello!'))
    );
}

We may trigger either internal command or our customised command from its name:



vscode.commands.executeCommand('editor.action.addCommentLine');

Or a Uri (for text editor or webview):



const commandUri = vscode.Uri.parse(`command:editor.action.addCommentLine`)

When everything relies on name, all the problems we had debated for CSS class come across. It may not cause any issue at all when the extension is small, but it is hard to tell when the extension grows big.

In fact, more names and complexities come with the command Taking one of the menu contribution point in my extension as example:



{
    "contributes": {    
        "view/title": [
        {
          "command": "gitlabSnippets.addHost",
          "when": "view =~ /^gitlabSnippetsExplorer-(mine|all)$/",
          "group": "navigation"
        }
      ]
    }
}

The configuration above adds an icon button to the title of 2 view panels. VSCode requires:

A when clause expression mixing a bit flavor of bash and regular expressions;
ID (like the view name “gitlabSnippetsExplorer-mine” above);
Position name (like the “group” above).

In return, our design needs get minimized. All we need to provide are the condition to display, the icon and the description text.

In addition, thinking in command forces developers to split part of the workflow and reuse as much as possible. For example, in my GitLab Snippets Explorer extension, there is a addHost command showing up as a command palette menu item and an icon button in the view panels.

In the meantime, another publishSnippet command simply jump to this adding host action if user select the “Adding new …” option below:

Events

Both Chrome and VSCode provide a set of events to interact with.

We may subscribe to a Chrome event like following:



chrome.devtools.network.onNavigated.addListener(
  callback: function,
)

In VSCode, we may listen to an event with a slightly shorter path, like this:



vscode.workspace.onDidChangeConfiguration(callbackFunction);

We may fire some event manually. For example, in the VSCode TreeView, it is necessary to inform data changes to with this.onDidChangeTreeData.fire()in the TreeDataProvider :



export class ExampleTreeProvider implements vscode.TreeDataProvider<Dependency> {

    private _onDidChangeTreeData: vscode.EventEmitter<Dependency | undefined | void> = new vscode.EventEmitter<Dependency | undefined | void>();
    readonly onDidChangeTreeData: vscode.Event<Dependency | undefined | void> = this._onDidChangeTreeData.event;

    refresh(): void {
        this._onDidChangeTreeData.fire();
    }

    // ...
}

This looks ugly and unbelievably long to me (even worse in Typescript). I thought the tedious method binding in the React class component was the worst thing I knew till I see this.

The communication between parts

In Chrome extensions, communication between different parts is achieved through messaging. It is similar to the messaging between browser main thread and workers:



// Send a message
chrome.runtime.sendMessage({ message: 'hello' }, function(response) {
    console.log(response);
});
// Receive a message
chrome.runtime.onMessage.addListener(function(request, sender, sendResponse) {
    if (request.message === 'hello') {
        sendResponse({ message: 'hello from background script' });
    }
});

Instead of messaging, VSCode extension take advantage of the command design except for webview. I think it may be more reasonable for webview to execute command instead of posting message if the message type is complex.

UI/UX

The command design in VSCode also provides a convenient developer experience and a consistent user experience. Developers only need to provide an icon, label, and description for most scenarios. However, this convenience can sometimes limit creativity and the overall user experience.

In contrast, Chrome extensions offer more freedom for developers in terms of UI/UX design.

Testing

Testing a Chrome extension

Testing a Chrome extension is very similar to testing a web page. There are a few differences:

You will need to manually load the compiled folder in chrome://extensions and update a background script will bring you back there for a manual update.
You have to find the inspection window for a background script from the extension detail page or the service worker list.
You may have no idea of why things do not work due to a wrong manifest configuration such as permission missing 😖.
You have to remove the developer version to revive the live installation.

Testing a VSCode extension

Testing a VSCode extension requires more learning efforts.

I confess that I am not motivated in adding automation tests in my small extensions since the VSCode APIs are everywhere.

The manual testing of a VSCode extension runs on the debug system of VSCode. It is my favorite tool of debugging Node.js. Additionally, the test runs on an isolated environment and the marketplace installation settings are not affected.

However, I find it hard to trouble shoot the root cause sometimes as:

Details of the network requests are invisible.
Error messages from the message popup or the log panel are not helpful enough, and very little tracing information is offered.
Passing the local testing does not mean the published one will work perfectly.

Distribution

Distribution of Chrome extension

To distribute a Chrome extension, we need to:

Register a Google account and top-up $5 to become an eligible Chrome Web Store Developer.
Pack the extension to a .zip manually as required.
Fill out a crazy long form to market the extension and declare the purpose of requesting for the permissions (I mean, each permission).
Wait for several days to pass or fail the review.

I was horrified by the long form at my first submission. I didn’t expect it would force me to bring out my design tool to get the screenshot size right and the file size small. I thought we have image processing plugins for decades?

What made things worse was that I waited for a long review time due to a public holiday. The result was an short Email of submission failure. It took me some time to figure out the problem was requesting some unnecessary permission and filling in unclear information.

Once successfully submitting the extension, we may see the basic download times and user review comments.

To understand more of the extension usage, it is possible to integrate analytic tools like Google Analytics. I forgot the details but it was not as straightforward as integrating in a normal web app.

Google allow to publish an extension without showing up in their search engine and the web store. User may only access this kind of extension from a URL. This is a perfect simple work around for internal extensions.

Distribution of VSCode extension

In contrast, the process of publishing a VSCode extension is incredibly simple and efficient:

Register an Azure account and get a personal token with marketplace access.
Install the latest version of CLI tool vsce.
vsce login with the account info.
Compile the code and vsce publish.
Usually within few hours will receive an Email of success or fail.

Well, I took one more step to summarise the process, but…

No form fillings for the extension. All the necessary information could be found in the package.json, README.md just like npm do.
vsce does some simple review to fail fast and prevent meaningless waiting.
The review process was so short that I suspect it is mostly automated as well.

However, I did meet a review failure without helpful information due to a “risky” API usage. And I also did find my extension not working as expected after publishing due to packing wrongly.

Similar to Chrome extension, once successfully publishing a VSCode extension, we may see the basic download times, user review comments, and a handy statics graph of the recent downloads.

Again, it is far from enough to understand user behaviors and figuring out how to improve from these data. Most of the Node.JS monitoring tools are focusing on Node.JS server, not native tools with interfaces like this. It seems to me that I have to setup the metrics and monitoring system by my own.

For the internal extensions, someone had experimented and created an extension: Private Extension Manager. One of my brilliant teammate did fork and made some small changes to create an internal extension marketplace.

Summary of what I have learnt

First of all, here are the essentials for an extension system based on my observation:

An entry point. JSON is perfect for it, but we need to design wisely. I think it is great to reuse some information like name, version, description from the package.json but there is conflict risk to extend the file designed for another system.
Super clear document about the extension concepts, the APIs, the configuration of extension parts, the distribution guide, etc.
Isolated runtime environment for the whole extension or even for each part of the extension.
A communication channel between the parts of the extension. I love the command design from VSCode, but it would be better to avoid trapping us in the naming battle and costing us to learn new expressions.
APIs categorised by feature, consistent in style and similar to the existing standards to flatten the learning curve.
App events subscription.
Complete testing support. It is complicated to make a good one, but we may try to use the mature system from browsers.
A marketplace for user to discover and developer to publish, analyse the extension usage.
A review workflow. Automate as much as we can, request as little information as we can.

The enhancement I believe is small but significantly improved developer experience is: publishing tool like vsce. We may fail faster with it.

It is also nice to have:

Helpful and traceable error messages.
Official solution for internal extensions.
More informative statistics.

And there are dark and bright sides to choose: design freedom or programming efficiency and complex configuration. No matter which choice we make, extracting actions is a great choice for code reusability.

Last but not least, I will try my best to keep away from:

Asking for declaring permission instead of finding out from the code.
Names, a lot of names.
Creating new rule expressions.

Combining async requests in the background

Jennie — Tue, 09 May 2023 23:55:42 +0000

When I was in Shopee, there was an interesting case about a React component triggering some API requests and a long list of this component often appears in a page. As a result, client-side send many requests in a short time and not only slowed down the interface but also occupied server resource significantly.

To improve this, my smart teammate created a "batching" solution that I loved so much and used the idea in my interview later.

How did we achieve this?

Let me recap the situation with some detailed setups first:

There is a <ListItem/> component sends a API GET /item under certain event (e.g, mouse enter) with compulsory parameters like item ID, like this:

import React, { useCallback, useState } from 'react';

// Proper error handling is ignored here...
const fetchItem = id => fetch(`/item?id=${id}`).then(res => res.json());

export function ListItem({ id }) {
  const [item, setItem] = useState({});
  const onMouseEnter = useCallback(() => {
    fetchItem(id).then(setItem);
  }, [id]);
  return <div onMouseEnter={onMouseEnter} >{item.name}</div>;
}

The <ListItem/> component is used to display a long list, like this:

import React from 'react';
import { ListItem } from './ListItem';
export function List({ ids }) {
  return (
    <div>
      {ids.map(id => 
        <ListItem id={id} />
      )}
    </div>
  );
}

Now you may question that isn't the solution just sending a list request like GET /items?ids=1,2,3 in <List/> component instead? Well, it is not the case here as the request is only sent when user triggers a specific event on the item. But you are getting closer to the point - instead of GET /item?id=xxx repeatedly with different sets of parameters, it is better to find a way to send GET /items?ids=1,2,3 instead.

As these requests are sent at different times, here is an important decision to make: under what condition shall we combine the requests?

Here are 2 ideas:

Combine the requests triggered within a certain period;
Combine the requests when the request queue reaches a certain length.

Timing is the first factor to consider as if the interface is waiting for the response data from the request, we shall not keep user waiting for too long.

Queuing request length however, depends more on the server side. How many items can they process at one time (considering each item may require extra information to process)? Is there any request URL (for GET requests) or body size limit (load balancer like Nginx has a default setting, and sometimes BE framework sets limit as well)?

Combining by time

Speaking of time and requests, there are 2 well-known existing strategies: throttle and debounce.

Debounce delays the function invocation by a defined period of time to avoid unnecessary invocations. It works well with button clicks and key events.

Throttle invokes the callback function at regular intervals as long as the event trigger is active. It suits more in the continuous events.

Here I will demonstrate with a strategy inspired from throttle.

First of all, let's add a fetchItems() helper:

const fetchItems = ids => fetch(`/item?ids=${ids.join(',')}`).then(res => res.json());

Then tweak the original fetchItem() to store the single set of parameters into a queue:

const queue = [];
const fetchItem = id => {
    queque.push(id);
    // TODO: flush the queue
};

And handover the parameters to the fetchItems() helper with a timer delaying 200ms.

const queue = [];
const fetchItem = id => {
  queue.push(id);
    if (queue.length === 1) {
      setTimeout(() => {
        fetchItems(quque);
        queue.length = 0;
      }, 200);
    }
}

You may notice that I only created a timer when the parameters start to queue up, and clear the queue once the batch request fetchItems() is sent.

Now it should work for the requests not looking forward to responses (e.g. recording user behaviours). If we would like to process with the responses, we may create a Promise and cache it for later like this:

const queue = [];
let task;
const fetchItem = id => {
  queue.push(id);
    if (queue.length === 1) {
      task = new Promise((resolve) => {
            setTimeout(() => {
            fetchItems(quque);
            queue.length = 0;
          }, 200);
        });
    }
    return task;
}

The promise is wrapping the timer, and each single fetchItem() "request" is returning this promise. As a result, you may get all the items in the same batch once this promise resolves:

fetchItem(1).then(items => items[0]);
fetchItem(2).then(items => items[1]);
fetchItem(3).then(items => items[2]);

Furthermore, we may abstract this logic for future usage:

function batchRequests(request, paramMerger, period = 200) {
  const queue = [];
  let task;

  return (params) => {
    queue.push(params);
    if (queue.length === 1) {
      task = new Promise((resolve) => {
        setTimeout(() => {
          const params = paramMerger(queue.slice());
          queue.length = 0;
          resolve(request(params));
        }, period);
      });
    }
    return task;
  };
}
const fetchItem = id => batchRequests(
  fetchItems,
  ids => ids
);

Here is the CodeSandbox to try:

Combining by count

Combining requests by count alone is quite straightforward. Let's reuse the fetchItems() created above:

const MAX_QUEUE = 5;
const queue = [];
const fetchItem = id => {
  queue.push(id);
    if (queue.length === MAX_QUEUE) {
    fetchItems(quque);
    queue.length = 0;
    }
}

However, considering count only is a bad idea as fetchItem() may never send the request till it collects enough parameters. Therefore, we shall consider with time together, like this:

const MAX_QUEUE = 5;
const queue = [];
let timer;
const flushQueue = () => {
    clearTimeout(timer);
    fetchItems(quque);
  queue.length = 0;
};
const fetchItem = id => {
  queue.push(id);
    if (queue.length === 1) {
      timer = setTimeout(flushQueue, 200);
    } else if (queue.length === MAX_QUEUE) {
    flushQueue();
    }
}

And oops, it gets tricky if we expect the response data from the function:

const MAX_QUEUE = 5;
const queue = [];
let timer;
let flushQueue = () => null;
const fetchItem = id => {
  queue.push(id);
  if (queue.length === 1) {
    const timer = setTimeout(() => flushQueue(), period);
    task = new Promise((resolve) => {
      flushQueue = () => {
        const params = paramMerger(queue.slice());
        queue.length = 0;
        clearTimeout(timer);
        resolve(request(params));
      };
    });
  } else if (queue.length === MAX_QUEUE) {
    flushQueue();
  }
  return task;
};

This is because it is trying to resolve the promise for both 2 conditions, but we only create the timer and promise once for each batch. It looks quite messy and risky. We will discuss alternatives later.

Similar as above, we may extract the logic for future usage:

function batchRequests(request, paramMerger, period = 200, maxQueue = 5) {
  const queue = [];
  let task;
  let flush = () => null;
  if (maxQueue < 2) {
    throw new Error("Max queuable requests must be more than 1!");
  }

  return (params) => {
    queue.push(params);
    if (queue.length === 1) {
      const timer = setTimeout(() => flush(), period);
      task = new Promise((resolve) => {
        flush = () => {
          const params = paramMerger(queue.slice());
          queue.length = 0;
                    clearTimeout(timer);
          resolve(request(params));
        };
      });
    } else if (queue.length === maxQueue) {
      flush();
    }
    return task;
  };
}

Here is the CodeSandbox to try:

The alternative

When a logic gets tricky, it is highly likely we are doing things wrongly or simple alternative exists.

Here for getting the fetchItems() response I believe state manager such as Redux is a better alternative. With the state manager, we just need to update the items state in our modified fetchItem(). Then the "reactive" framework will notify the change and render the latest data, and resolving promise is no longer expected from the fetchItem().

The risk

Everything looks pretty nice before user close the tab without sending the request and your important data (e.g. data relates to advertisement charges) get lost! It is necessary to consider moving this part of the logic into ServiceWorker to avoid losses.

Words at last

To wrap up, handling async requests is an essential Front-end skill nowadays, and sometimes it is more than just applying aysnc/await and Promise APIs.

How to write a helpful library document

Jennie — Fri, 01 Jul 2022 08:30:46 +0000

When we start to try a new tool, library or framework, what will we first look for? Document!

When we are implementing a feature and forget how, what will we look for help? Document!

When different fellows enquiry similar questions and we don't have time to hang around with them, what can we offer them as an alternative? Document!

Yes, document plays a vital role in our life as a developer. However, writing a document is not an easy job. If the document is not good enough, it may shut our potential users out of the door and bring more hassle to us.

Several years ago, I created a little formatting library which was widely used in the following years. However, instead of enjoying the happiness of the success, I did suffer from my poor document (as I mentioned in "Writing library is not cool at all") until I received very helpful suggestions from a peer.

Thanks to the poor document, I have to deal with my daily job while hanging on the internal chat app to support my precious users. Their questions mainly were like "How to format xxx?" as if they had never spent time reading my document. But, did they?

Yes, they did read. There is nothing wrong with our users. When this happens, it must be the document is not doing its job well.

The structure of a proper document
- The getting started guide
- Common usage guides and advance guides
- API reference
- Contribution guide
Write the understandable content
Improve continuously

The structure of a proper document

Like writing anything else, documents require a clear and user-friendly structure to help new or existing users find the information they want.

You may get some clues from those popular libraries, tools or frameworks.

The rough document structure of React:

Key features;
A list of all the tools can help new users start their journey;
A getting started tutorial;
Guides for common usages;
Advanced usage guides, including a testing guide;
API reference;
Contribution guide;
FAQ.

The rough document structure of Lodash:

A getting started guide;
Why Lodash;
API reference;
Advanced guide for FP;
Related links, including contribution guide, etc.

The rough document structure of Vite:

Key features;
Why Vite;
A getting started guide;
Guides for common usages;
Similiar tool comparisons;
Migration guide;
API reference, config and plugin details;
Related links.

We may find some patterns in these documents:

There is an attractive landing which is full of why new users should try this product. If similar competitive products are available in the market, compare and show the advantages.
Followed by a neat and simple getting started guide to try the significant features without knowing many backgrounds.
Then, another set of guides to explain the common usages.
And probably a set of advanced guides folding the complex, less applied use cases.
Finally, a searchable place to list all the API details.
Do not forget the contribution guide if you are looking for contributors.
Also, a migration guide if there are multiple major versions.

The getting started guide

Once potential users are attracted by our fantastic landing or others (advertisements, user recommendations, etc.), the getting started guide is the key to open the door to a product.

Remember, a getting started guide ought to be as simple as no brainer. Users follow the steps, and they will get an express tour of the significant features.

It simply starts with an installation command with npm/yarn/pnpm or some script tags with the latest CDN links.

Next, it presents the command to scaffold a new app if the product setup contains multiple steps.

Lastly, it offers a minimal working example in just few lines of code.

Common usage guides and advance guides

A robust product may support various use cases that will confuse new users if we list them together. Therefore, it would be better to group the guides into 2 groups: one for common usages, the other for complex usages, and less applied usages. And place the common usage guides below the getting started guide for users who want to learn more.

In comparison to the getting started guide, these guides shall be a bit juicier. By slipping in context little by little between the steps, we may gradually demonstrate the concept of this product.

Furthermore, we may help users sort the guides from what they may require first and next, naming the guide with a minimal title containing keywords.

API reference

Apart from the guides, API reference is essential for all the libraries, frameworks and tools opening for custom integrations. It is meant for experienced users to find in-depth information as fast as possible. As a result, the APIs must be searchable in specific ways, such as Ctrl + F or a built-in search engine.

To keep the document up to date and minimize the effort for maintenance, I suggest generating the API reference from the code comments (e.g., like JSDoc). Since the comments live closely with the code, developers tend to forget less about updating the document while updating the API.

There are multiple choices for document generating in the market: Storybook, TypeDoc, JSDoc 3, etc. Finding one that fits your project and takes time to learn is definitely worth your effort!

However, a generating tool still requires extra input other than the source code. A description sentence for the constants, functions, classes, etc. and their parameters, properties, and returns would make the document more understandable. It is even better if we provide one or two code examples.

Contribution guide

If we would like to attract some contributions, a comprehensive contribution guide is necessary. It should include:

How to set up the project from scratch?
How to run the project from local?
How to test the project?
What is the process of fixing a bug and developing a new feature?
Is there any other ways to contribute? Like providing suggestions and filing bugs.
How to communicate with the maintenance team?

Additionally, it would be better to provide clues on where to start with the different kinds of tasks for large projects. Here are some great examples:

Write the understandable content

With a clear structure ready, we have accomplished half of the job. The other half, however, could be tedious yet crucial.

Just like how we alter our code to optimize its readability, a remarkable document craves our patience to modify words and sentences, even formats, over and over. There is no universal standard but a few well-accepted rules. And here is a great learning resource from Google to recommend: https://t.co/ioo5hnHMSX.

Except for the wording, it is worthy of mentioning context and repetition.

No matter how good is a writing expression, people get confused without sufficient context. As a result, we usually suggest avoiding acronyms and other unfamous terms. If we cannot escape the shortened expressions, we may insert links in our document to help. Read more about "Why Is Context Important in Writing?" here: https://www.masterclass.com/articles/why-is-context-important-in-writing#4-types-of-context-in-writing.

And repeat. As a developer, we hate repeating, almost like having OCD. It is about the golden rule - reusability in programming but not the one in writing. To enhance the understandability of the document, we shall repeat some explanations anytime, anywhere, if it is necessary.

Improve continuously

Last but not least, a document needs our care along with the application. If this version does not work well, we shall seek feedback and suggestions to improve it.

Though the effort of maintenance is often hidden, it brings us benefits the other way around, like attracting more users for our creations and avoiding trapping ourselves in the supporting chats, contributing to personal branding, soft skill improvements, etc.

Identifying code with Typescript compiler

Jennie — Sat, 18 Jun 2022 23:03:07 +0000

Since Typescript gets popular and typed code is easier to be identified, I found it is a good idea sometimes to do code automation with Typescript compiler instead of basing on the AST.

For example, when I tried to find out all the React function components with AST, I may require help from tool like Babel as following simplified code shows:

import { transformSync, Visitor } from "@babel/core";
import { Identifier } from "@babel/types";

const funcs = [];

transformSync(content, {
  parserOpts: {
    plugins: ["jsx", "typescript", "classProperties", "optionalChaining"],
  },
  plugins: [
    () => ({
      visitor: {
        ExportDefaultDeclaration({ node: { declaration } }) {
          switch (declaration.type) {
            case "FunctionDeclaration":
            case "ArrowFuntionExpression":
              funcs.push("default");
              break;
          }
        },
        ExportNamedDeclaration({ node }) {
          const { declaration } = node;
          if (declaration) {
            switch (declaration.type) {
              case "FunctionDeclaration":
                if (declaration.id) {
                  funcs.push(declaration.id.name);
                }
                break;
              case "VariableDeclaration":
                declaration.declarations.forEach((d) => {
                  if (!d.id.name) return;
                  switch (d.init?.type) {
                    case "ArrowFunctionExpression":
                      funcs.push(d.id.name);
                      break;
                  }
                });
                break;
            }
          }
        },
      },
    }),
  ],
});
console.log(funcs);

Here my little VSCode plugin Babel AST Explorer helped me saved a lot of time.

You may found the above Visitors could only recognize following code:

export default function f() {}

export default () => {};

export function f() {}

export const f = () => {};

When they are assigned to variables, it gets even harder to identify. For instance:

function f() {}
export { f };
export const f2 = f;
export default f;

This brought me to the Typescript compiler.

However, the document of Typescript compiler is kind of unhelpful. So here are some notes of the usage of Typescript compiler that I tested in a small project. Hopefully, it could save you some time.

The basic definitions
Initiating the type checking utility
Finding the exports of an ES module
Printing the closest type name
Identifying the type syntax with flags
Identifying the Node type with ts helpers
Get Type object of a function

The basic definitions

The Typescript compiler defines several basic objects, and I often worked with these 3: Node, Symbol, Type.

In my understanding,

Symbol is the major data structure that Typescript use to communicate in the compiler.
Node is the "superset" of AST node, which handles the token, the character locations, etc.
Type is the Typescript "type" to diagnose type safety.

Therefore, we should always get the Symbol first, and request for Node or Type if necessary.

For example, we could get Symbol from source code like this:

checker.getSymbolAtLocation(source);

You may wonder what is the "checker" here, we will talk about it in the next section.

With the Symbol, we could find the corresponding Nodes according to the node type, like following:

symbol.declarations;
symbol.valueDeclaration;

If we require to work with type then we may get it with Symbol and Node:

checker.getTypeOfSymbolAtLocation(mySymbol, myNode);

Or get from the type Node directly:

checker.getTypeFromTypeNode(myNode);

Initiating the type checking utility

To figure out the type of a declaration, we will need to initiate a Typescript type checker and parse our source code to the Typescript Symbol first.

import ts from "typescript";
import assert from "assert/strict";

const program = ts.createProgram({
  rootNames: [file], // A set of root files
  options: {}, // The compiler options
  // There are 3 other options available but I am not sure what they are used for yet
});
const source = program.getSourceFile(file);
assert(source);
const checker = program.getTypeChecker();
const moduleSymbol = checker.getSymbolAtLocation(source);
assert(moduleSymbol);

Finding the exports of an ES module

With the type checker and symbol we get above, continue analysing from the ES module exports are a lot easier than with Babel:

const exports = checker.getExportsOfModule(moduleSymbol);
exports.forEach((ex) => {
  ex.declarations.forEach((d) => {
    // ...
  });
});

Printing the closest type name

The type checker could provide the closest type name with helper:

const myType = checker.getTypeOfSymbolAtLocation(
  mySymbol,
  mySymbol.valueDeclaration!
);
checker.typeToString(myType);

Well, this name is honestly not very helpful in identifying whether this identifier is the right type we are looking for as we often use union or extended types. Unfortunately, I haven't found a better way yet. One possibility I could think of is to create a piece of code and request Typescript to diagnose it with the whole project.

Identifying the type syntax with flags

Typescript compiler defined several flags including SymbolFlags, SyntaxKind, ModifierFlags, etc. I found the most helpful one is SyntaxKind. SyntaxKind could help to identify the type keywords from a Node in the following way:

if (myNode.kind === ts.SyntaxKind.NullKeyword) {
  // This is a null type, do something
}

While some flags like ModifierFlags require a bit more operations:

if (ts.getCombinedModifierFlags(myNode) & (ts.ModifierFlags.Export !== 0);) {
  // I am exported, do something
}

Identifying the Node type with ts helpers

The compiler provides a set of helpers to identify what kind of code we are dealing with. For example:

ts.isFunctionDeclaration(myNode);
ts.isArrowFunction(myNode);
ts.isVariableDeclaration(myNode);
ts.isIdentifier(myNode);
// ... and you can get more from auto-complete

These helpers work with Node, and look quite straightforward from their name if you are familiar with the syntax definitions. However the list of helpers too long to find the right one sometimes as Typescript tried to work with ECMA, JSDoc and its typings.

Get Type object of a function

In some cases, we may want to know the parameters and return type of a function or construct. To achieve this, we will start from the Signature of them which could get from the Type object:

checker
  .getTypeOfSymbolAtLocation(mySymbol, mySymbol.valueDeclaration)
  .getConstructSignatures()
  .map((signature) => ({
    parameters: signature.parameters.map((paramSymbol) =>
      checker.getTypeOfSymbolAtLocation(
        paramSymbol,
        paramSymbol.valueDeclaration
      )
    ),
    returnType: signature.getReturnType(), // returns Type object
  }));

Gain and retain users for your project with user survey

Jennie — Wed, 25 May 2022 08:55:30 +0000

Survey was part of our life, but I was not aware of its importance till I read the book "Hacking Growth" and learnt about AARRR model. The book suggests that survey will not only help us to understand what our product should improve, but may also promote the product as a "side effect".

In comparison to user experimentation(that I mentioned in "Gain and retain users for your project with user experimentation"), user survey could get more people involved, more sets of data, yet not certainly helpful information.

One of my first few trials was surveying both our front-end developers and the backend developers about the backend API integration experiences. Our team was facing the pain of working with multiple backend teams reporting to different managers and not applying the same practices.

The survey aimed to:

collecting the data and expectations of API integration from both sides;
providing actionable and convincible suggestions based on data;
finding out the common patterns and providing tools base on them;
advertise some existing practices and benefits of adopting them.

With above goals in mind, I carefully designed the questions and options, distributed with the helps from each sub-team PICs. As a result, I gain over 70% response rates from FEs, and responses from all the BE teams we were working with.

The data I collected and summarized helped me to clarify the situation, and helped others to make some technical decisions(like what kind of API schema should internal tools relates to API integration adopt). However, the result of "side effect" was not obvious in a short term, as it was designed to be conducted periodically to reflect the situation.

The other interesting survey I tried was altering a regular feedback form for a team sharing activity. I found the feedback collected from that form was not helpful in improving my sharing or the project I shared. Therefore, I carefully chose questions to keep the form short, provided options to reduce effort on answering open questions, and altered words to make audience think of more helpful information. Yet I was still too greedy and asked too much. And I only got less answers. The trial was a failure.

Then I further simplified the form, and saved some engagement.🙈

Here is the lesson I have learnt along my try-error journey.

When designing a survey:

Be clear about the goal of the survey, so you won't ask unhelpful questions;
Target to single group of people, and only ask related questions;
Keep the survey as short as possible, so you may get more answers.
Choose words carefully so you could get more helpful information. One example here is changing question like "What do you think xxx need to improve?" to "What made you want to leave xxx?".
Less open questions, more selections, and the questions become easier to answer.

When conducting a survey:

Choose the way to request answers. You may expect only few answers if you request for answers by volunteer, though the message could be received by more people. Or you may get more answers if you request each survey target.
Set deadline and keep sending reminders. People may miss your message or simply forgot the survey.
Analyze result and summarize to simple conclusions for an easier understanding.
Share and discuss the results with the related parties, plan for actionable.
Follow up the actionable regularly to ensure your effort is not wasted.

Gain and retain users for your project with user experimentation

Jennie — Thu, 12 May 2022 08:34:53 +0000

As a developer, my work is highly depending on tools, as if keep finding better tools to be more efficient is part of my job. When I cannot find a tool that works for me, I create one. And I share it as I hope my hard work could help others.

However, I found that sharing my creation to other people even other team members were not as simple as imagined. There are various reasons to refuse my kind offer:

They don't understand how this tool could help their work.
They don't have time to learn and try out new stuff that looks not promising because it is made by an infamous person.
They have no idea how to start.
They worry that the tool would not be maintained well.
They don't like its UI/UX etc.

Fair enough🤷🏻‍♀️. At this point, I require skills more than coding to "sell my product". Here are 2 ways that I found really helpful for resolving most of the refuse reasons mentioned above:

F2F user experimentation
User survey

The primary way: F2F user experimentation

Eyewitnesses can provide very compelling legal testimony, but rather than recording experiences flawlessly, their memories are susceptible to a variety of errors and biases.
from Eyewitness Testimony and Memory Biases

Observing how a user use your product by yourself can provide you a lot of unexpected information, even much more than interviewing in-person.

I got this conclusion from my first time user experimentation, and verified more times in the following experimentations I tried. I organized my wonderful first session for a tool that I would like to integrate, but it was not ready for quite some time.

What I did observe was that the project owner as well as the contributors wish to receive feedback passively from their target users. Yet they only received little feedback as the tool was not ready, and they were not that helpful as who provided those feedback were in fact not using the tool. As a result, the developer team figured out "the improvements" by their instincts and struggled in gaining users.

The session invited 2 major developers and 2 target users from a team who were eager to find such kind of solution to resolve their pain points.

The experimentation took around 1.5 hours and went on in the following flow:

Firstly, introduce the experimentation, why and how;
Then target user follow the instruction of doing specific things with the tool from scratch;
In the meantime, developer sit next to the target user, observe their behaviour, note down the issues user has met and provide guidance if necessary;
Lastly, 2 groups brainstorm together about the issues and possible improvements.

The tool developers were surprised at how buggy was the tool which they believed at least the main flow were well tested. And they also witnessed user could not easily understand how to even start from the UI/UX and the documents they provided.

If these were not seen by their own eyes, you may imagine how many rounds of communications required to make them understand what was going on and convince them to "waste" more effort on writing better documents.

As I expected, the developers found several points to improve, but still missed quite a few as they were busy on putting down the fire on the scene. In the end, me as the coordinator and a 3rd party helped noting down 50% more issues since I caught more details in my unique aspect.

It's effective to some point, while the downside is obvious as well - it takes precious coding time to design, prepare and follow up, yet each session only covers a small group. Since the sample size is small, the opinions they provide and the issues you found may not be the right thing to care about.

To obtain a larger size of sample, a well designed user survey and user behaviour tracking is more favourable, but we may not enjoy the effective observation and communication advantages from user experimentation.

Here I summarized some keys from my success and failure in the user experimentation.

When designing an experimentation:

Select a specific feature or behaviour. Focusing on depth not breadth.
Let user experience a complete process without disturbing if possible. And you may witness more.
Plan for following up. By another experimentation? An interview? Or a survey? To close the feedback loop.

When conducting the experimentation:

Involve the main devs, the decision makers. Witness is more convincing than being told.
Devs and users should be in pairs, just like peer programming do. No one can observe so many objects at the same time.
Note down observations immediately as much as possible. Or you will forgot at least half of them soon.
Ask users' thoughts, discuss the possible improvements. Engage and open your mind.

Introduction to workers and why we should use them

Jennie — Fri, 12 Nov 2021 03:21:21 +0000

As we know that browser runs all the Javascript of a web page in a single thread - the main thread, any excessive Javascript code may block the main thread and makes the page look laggy or even unresponsive.

We may improve this by off-loading some tasks to workers, and turns the single-threaded app into a multi-threaded, higher-performance, and potentially safer app.

Introduction of workers

There are 3 kinds of workers in the browser context:

Web worker - including dedicated worker, and shared worker
Service worker
Worklet- including PaintWorklet, AudioWorklet, AnimationWorklet, LayoutWorklet.

I will focus on the web worker and service worker in this post, as worklet is still experimental and designed for more specific usages.

Web worker

Web worker is designed as the background thread for running scripts. Web pages are allow to spawn multiple workers to do different tasks in an isolated context.

(Image from Workers Overview)

We may create such thread dedicated to a web page via the Worker API:

const myWorker = new Worker('worker.js');

Or sharing the thread among the same origin via the SharedWorker API.

const mySharedWorker = new SharedWorker('sharedWorker.js');

Their usage is almost the same but the connection between the web page and the shared worker is assigned ports. To communicate with the shared worker, the web page need to go through the port assigned.

In the main thread:

mySharedWorker.port.postMessage('Test message to worker.');
mySharedWorker.port.onmessage = (e) => {
  console.log('Receive data from shared worker:', e.data);
};

In the shared worker:

onconnect = (e) => {
  const port = e.ports[0];
  port.onmessage = (e) => {
    port.postMessage('Hi, I received your message "${e.data}"!`);
  };
}

While communicating with the dedicated worker is more straightforward.

In the main thread:

myWorker.postMessage('Test message to worker.');
myWorker.onMessage = (e) => {
  console.log('Receive data from dedicated worker:', e.data);
};

In the dedicated worker:

onmessage = (e) => {
  postMessage('Hi, I received your message "${e.data}"!`);
};

Another difference you may notice is that the dedicated worker is coupled with the web page, hence it will be terminated once user close the page.

Service worker

Service worker is designed as an extra layer working between the web page and the server.

(Image from Workers Overview)

Unlike the web worker, it has a more complicated lifecycle:

Register in main thread via navigator.serviceWorker.register(), and download the script for the service worker;
Install if the service worker is newly registered, or the script is updated;
Activate when no "old" service worker is controlling the clients under its scope;
Redundant when it is out-dated and being replaced.

It is given a set of APIs and events to achieve things like:

caching data
intercepting requests
managing browser push notifications

On top of these capabilities, we do a lot more interesting things. We will discuss these later.

Their limitations

Both web worker and service worker are quite powerful, but meanwhile they have various limitations. Including:

No access to DOM, window object and some other APIs like localStorage. This restriction keeps main thread and worker isolated and not disturbing each other. However, this also means if you would like off-load some heavy jobs like DOM operations, it's not possible with worker.

Messages between the main thread and the worker are cloned via the structured clone algorithm which does not work with various data types. And unlike JSON.stringify() simply strip off the data it does not support, passing a data type the structured clone does not support will simply throw an exception.

Experimental and limited support for ES modules. As mordern browsers support ES module better, we could lower down quite some overhead on compiling our scripts, and run ES module directly in the browser. But, that is not quite the case in the workers.

As of Mar 1, 2019, only Chrome 80+ supports this feature, while Firefox has an open feature request. No other browsers are known to have support for production usage of worker scripts written as modules.

In these browsers, you may spawn a worker with ES module via:

new Worker('worker.js', {
  type: 'module'
});

However, the ES module syntax import and export may not work well as you expected.

Spawning a worker from a worker almost cannot work, although some browser claims that we can spawn a sub worker from a dedicated worker. This could be a bit frastrating sometimes as some library (e.g. esbuild-wasm) has built-in worker logics, which prevent us from organizing the tasks properly according to our needs, and have to communicate between the worker in the library and our own worker through the main thread message channel.

Performance improvement with workers

Off-loading heavy tasks

As introduced previously in "Introduction of workers", we may assign some heavy tasks to workers to avoid blocking the main thread and affecting user experiences.

For example, when we have to group or process the raw data from BE for display, we may spawn a web worker to do this while continuing to render other things.

In the main thread, we spawn a web worker, then instruct the task and wait for the result:

function getData() {
  const myWorker = new Worker('processData.js');
  return new Promise((resolve, reject) => {
    myWorker.onMessage = ({ data }) => {
      resolve(data);
      myWorker.terminate();
    }
    myWorker.onError = (e) => {
      reject(e);
      myWorker.terminate();
    }
    myWorker.postMessage({ sortBy: 'update_time' });
  });
}

getData().then(data => console.log(data));

In the processData.js:

function process(rawData, options) {
  const processedData = rawData;
  // Do the processing
  return processedData;
}

onmessage = ({ data: options }) => {
  fetch('/get_raw_data')
  .then(res => res.json())
  .then(rawData => process(rawData, options))
  .then(data => postMessage(data));
};

Another classic use case is to zip/unzip files in the browser. The workers are often integrated within the library, like unzipit, js-untar, etc.

Prefetching data

Service worker provides APIs for caching data, and intercepting requests in the browser. This provide us an opportunity to get some data ready in advance.

In the main thread, we simply register the service worker script:

if ("serviceWorker" in navigator) {
  // do make sure the service worker script is under the right folder as it will only control the requests from the scripts under the same folder
  navigator.serviceWorker.register("/sw.js");
}

And fetch data as normal:

fetch('/get_data').then(res => {
  // If the service worker is successfully activated and the prefetched data is ready, you shall get the response immediately
  // ...
});

In the sw.js, we load data ahead and put into cache:

const preFetchUrls = [
  '/get_data'
];
caches.open('prefetchedData').then(prefetchedData => {
  preFetchUrls.forEach(requestUrl => {
    fetch(requestUrl)
      .then(res => prefetchedData.put(requestUrl,res));
  });
});

And intercept the request to replace the response with the cache if it exists:

addEventlistener('fetch', e => {
  const { method, url } = e.request;
  if (method !== 'GET' || !preFetchUrls.includes(url)) return;
  e.respondWith(async () => {
        const prefetchedData = await caches.open('prefetchedData');
    const cachedResponse = await cache.match(event.request);
    return cachedResponse || fetch(event.request);
  });
});

Caching for loading faster next time with Workbox

In the use case above, we made use of the caching and request intercepting ability of service worker to load some data ahead. We may also use these abilities to help our return user to load page faster by caching the necessary files.

Here I would like to recommend using Workbox, which will largely simplify the setups and apply different strategies easily.

In the main thread, we will need to register the service worker script as before:

if ("serviceWorker" in navigator) {
  navigator.serviceWorker.register("/sw.js");
}

While in the sw.js, we will import the Workbox libraries this time, which means you will require to bundle this file before use.

The example code from Workbox website is sufficient for most of the use cases:

import { registerRoute } from 'workbox-routing';
import {
  NetworkFirst,
  StaleWhileRevalidate,
  CacheFirst,
} from 'workbox-strategies';
import { CacheableResponsePlugin } from 'workbox-cacheable-response';
import { ExpirationPlugin } from 'workbox-expiration';

// Cache page navigations (html) with a Network First strategy
registerRoute(
  ({ request }) => request.mode === 'navigate',
  new NetworkFirst({
    cacheName: 'pages',
    plugins: [
      // Ensure that only requests that result in a 200 status are cached
      new CacheableResponsePlugin({
        statuses: [200],
      }),
    ],
  }),
);

// Cache CSS, JS, and Web Worker requests with a Stale While Revalidate strategy
registerRoute(
  ({ request }) =>
    request.destination === 'style' ||
    request.destination === 'script' ||
    request.destination === 'worker',
  new StaleWhileRevalidate({
    cacheName: 'assets',
    plugins: [
      new CacheableResponsePlugin({
        statuses: [200],
      }),
    ],
  }),
);

// Cache images with a Cache First strategy
registerRoute(
  ({ request }) => request.destination === 'image',
  new CacheFirst({
    cacheName: 'images',
    plugins: [
      new CacheableResponsePlugin({
        statuses: [200],
      }),
      // Don't cache more than 50 items, and expire them after 30 days
      new ExpirationPlugin({
        maxEntries: 50,
        maxAgeSeconds: 60 * 60 * 24 * 30, // 30 Days
      }),
    ],
  }),
);

If these are the only logics you needed in the service worker, you may also simply use a bundler plugin instead of copy paste and configure the bundler.

For Webpack users, check this doc: Workbox Webpack plugin.

For Rollup users, you may try the plugin rollup-plugin-workbox.

Other usages with workers

Mocking API without Node.js server

Once we have the ability to intercept a request, we may achieve some interesting things like mocking API. MSW is an perfect example.

Basically we could:

listen to the "fetch" event;
check the request matches our rules;
if it matches, replace with a cached response.

Just like what I demonstrated in the "Prefetching data" use case.

As a data processing layer

Another interesting use case is separating a data processing layer from our UI layer.

For example, when we are tracking the usage of an app, the app may throw all the information of a component to the worker. Then the worker selects the required data, and transforms the data into the right format before finally sending them to the server.

The layer isolation keeps the relatively non-critical but probably heavy tracking logics away from affecting the user experience, and crashing the app when there's something wrong.

Syncing in the background

Have you encounter any super slow data operations such as uploading multiple large files, or simply the BE server respond slowly? In these cases, the user is likely have to wait for the server response before exiting it.

Service worker does not have this limitation! It keeps running as long as the browser is still active. Therefore, we could cache these data somewhere in the browser first (like indexDB), and let the service worker to communicate with server instead to allow the user doing other things while waiting.

And more

There are a lot more possibilities with workers besides to the stuff mentioned above, like the well-known Progressive Web App, push notifications, code sandbox (without DOM access), etc.

Let's be creative and bring the web app to the next level!