Add a test case when your open-source project’s issue is fixed.

Building a stable project is hard and time consuming. When you open source that project, reliability and stability is important because you need to establish that trust and authority. As your open source project evolves and new features are introduced, regressions (bugs that reappear) can sneak back into the code. To prevent this, it is important to add test cases whenever an issue is fixed. These test cases act as a safety net, ensuring that future changes don’t reintroduce old bugs.

In this article, we’ll explore the importance of adding test cases for bug fixes by examining two issues from Zustand’s repository — Issue #84 and Issue #86. Zustand, a popular state management library for React, experienced these bugs in its subscription management. Both issues were subtle but critical, and writing test cases for them ensures they won’t surface again in future releases.

Mind you, these issues were reported on Zustand’s version — 2.2.1. Link to codesandbox: https://codesandbox.io/s/quirky-taussig-ng90m

The latest stable Zustand’s version is 4.5.5 but it’s good to learn what it means to add test cases to the bugs fixed after they were reported.

Why Should You Add a Test Case After Fixing a Bug?

When an issue is reported and fixed in a codebase, it’s tempting to move on after the fix. But without a dedicated test case that reproduces the problem, there is no guarantee that the issue won’t recur in future updates. Even if the code seems stable after the fix, new features or refactors can sometimes undo the correction.

Here’s why it’s crucial to add a test case after each bug fix:

1. Prevents Regressions: As the code evolves, automated tests ensure that past issues don’t creep back in.

2. Serves as Documentation: Test cases provide clear examples of what was fixed and how the software should behave under similar conditions.

3. Confidence in Future Development: Contributors can confidently make changes, knowing that tests will catch any breaks.

Now, let’s look at real-world examples from Zustand’s codebase and see how test cases were added after issues #84 and #86 were fixed.

Issue #84: Subscribers Can Be Corrupted During State Updates

Background

The bug reported in Issue #84 revolved around Zustand’s subscription system. Under certain conditions, subscribers could be “orphaned” — meaning components subscribing to the store would stop receiving updates.

When state updates happened during the component lifecycle (e.g., unmounts or re-renders), the array of subscribers could be corrupted. This caused a component that had subscribed to the store to never get updates, as its listener was overwritten by another subscriber.

Subscribers are a Set. Here’s a proof from Zustand’s source code related to createStore

Test Case for Fixing Issue #84

After fixing this issue, Zustand added a test case to ensure the correct subscriber is removed on unmount:

// https://github.com/pmndrs/zustand/issues/84
it('ensures the correct subscriber is removed on unmount', async () => {
  const useBoundStore = create(() => ({ count: 0 }))
  const api = useBoundStore

  function increment() {
    api.setState(({ count }) => ({ count: count + 1 }))
  }

  function Count() {
    const c = useBoundStore((s) => s.count)
    return <div>count: {c}</div>
  }

  function CountWithInitialIncrement() {
    useLayoutEffect(increment, [])
    return <Count />
  }

  function Component() {
    const [Counter, setCounter] = useState(() => CountWithInitialIncrement)
    useLayoutEffect(() => {
      setCounter(() => Count)
    }, [])
    return (
      <>
        <Counter />
        <Count />
      </>
    )
  }

  const { findAllByText } = render(
    <>
      <Component />
    </>,
  )

  expect((await findAllByText('count: 1')).length).toBe(2)

  act(increment)

  expect((await findAllByText('count: 2')).length).toBe(2)
})

This test checks that the correct subscriber is removed when components unmount and ensures the subscription system functions as expected. Here’s what happens in the test:

• A store is created with an initial count of 0.

• Components subscribe to the count state.

• The increment function is called when the component mounts, increasing the count.

• After unmounting and mounting components, the test ensures that the right subscribers are removed and that the state update propagates correctly to the components still mounted.

This ensures the original bug, where subscribers would be corrupted during the lifecycle, does not reoccur.

Issue #86: Subscribers Overwritten on Component Remount

Background

In Issue #86, an issue was reported where subscribers would stop receiving updates after components remounted. This happened because subscribers in Zustand were being overwritten with the same index when switching between mounted components.

The issue boiled down to a problem in managing multiple subscribers — each subscriber was assigned an index, and when a new component subscribed, the indices could conflict. This caused one subscriber to overwrite another, leading to components not receiving state updates after a remount.

Test Case for Fixing Issue #86

Here’s the test case added after fixing this issue:

// https://github.com/pmndrs/zustand/issues/86
it('ensures a subscriber is not mistakenly overwritten', async () => {
  const useBoundStore = create(() => ({ count: 0 }))
  const { setState } = useBoundStore

  function Count1() {
    const c = useBoundStore((s) => s.count)
    return <div>count1: {c}</div>
  }

  function Count2() {
    const c = useBoundStore((s) => s.count)
    return <div>count2: {c}</div>
  }

  // Add 1st subscriber.
  const { findAllByText, rerender } = render(
    <StrictMode>
      <Count1 />
    </StrictMode>,
  )

  // Replace 1st subscriber with another.
  rerender(
    <StrictMode>
      <Count2 />
    </StrictMode>,
  )

  // Add 2 additional subscribers.
  rerender(
    <StrictMode>
      <Count2 />
      <Count1 />
      <Count1 />
    </StrictMode>,
  )

  // Call all subscribers
  act(() => setState({ count: 1 }))

  expect((await findAllByText('count1: 1')).length).toBe(2)
  expect((await findAllByText('count2: 1')).length).toBe(1)
})

This test ensures that:

• Subscribers are not overwritten when components are mounted and unmounted.

• Multiple components can subscribe to the same store without conflict.

• When the state changes, all subscribed components update correctly.

By testing for the correct number of count1 and count2 components receiving updates, the test guarantees that the fix for this issue works and won't regress in future versions of Zustand.

Lesson here is — “Ensure Stability with Tests”

These two examples from Zustand illustrate why it is essential to add test cases after fixing bugs in an open-source project. They help ensure:

• The original bug is fixed.

• Future changes do not break the fix.

• Contributors can refactor, add features, or improve performance without worrying about breaking functionality that was previously addressed.

By writing test cases after bug fixes, you contribute to the long-term health of the project, helping both maintainers and users trust that the project is stable and well-maintained.

Conclusion

Adding test cases after fixing bugs is a crucial habit for maintaining the integrity of your open-source project. As demonstrated in Zustand’s handling of Issues #84 and #86, these test cases help catch potential regressions and ensure the system behaves as expected in real-world scenarios.

So, the next time you fix a bug in your open-source project, don’t stop at the fix. Add a test case to ensure the issue is gone for good, safeguarding your project’s future stability.

About me:

Hey, my name is Ramu Narasinga. I study large open-source projects and create content about their codebase architecture and best practices, sharing it through articles, videos.

I am open to work on an interesting project. Send me an email at [e-mail]

My Github - https://github.com/ramu-narasinga
My website - https://ramunarasinga.com
My Youtube channel - https://www.youtube.com/@thinkthroo
Learning platform - https://thinkthroo.com
Codebase Architecture - https://app.thinkthroo.com/architecture
Best practices - https://app.thinkthroo.com/best-practices
Production-grade projects - https://app.thinkthroo.com/production-grade-projects

References:

1. https://github.com/pmndrs/zustand/blob/v4.5.5/tests/basic.test.tsx#L628

2. https://github.com/pmndrs/zustand/issues/86

3. https://github.com/pmndrs/zustand/blob/v4.5.5/src/vanilla.ts#L89

4. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set