DEV Community: Dmitriy Kovalenko

Benchmark oriented development is a road to nowhere

Dmitriy Kovalenko — Wed, 25 Mar 2026 22:33:40 +0000

Cursor just released this article and a ton of people started worshiping Cursor like they just made a revolution in file search. They showed a beautiful graph saying that they are 1,300x faster than ripgrep, showing one specific query on the chromium codebase. You know, I happen to work a lot lately on the file search project of mine https://github.com/dmtrKovalenko/fff.nvim and I have a feeling that this is all one large manipulation a lot of people blindly believed.

We cannot say for sure because as always they do not open source the code and do not let you repeat the experiment, which makes this whole discussion absolutely useless, but I'll try to be constructive.

Here are my claims

1. The data is closed

I like to say myself that ripgrep is not the fastest code search especially on macos, but I specifically asked my colleague to give me his M2 macbook air and I spent quite some time experimenting but I never got the ripgrep on the same repo with the same query longer executing longer than 9 seconds, while usuallyit takes around 6 seconds when the cache is warmed up.

The main problem of all of those closed benchmarks and weird marketing articles that you can not verify anything yourself. You have neither the data nor the instrument to reproduce the success. You know, there is a reason why every research paper has to describe the experiment - so other people can reproduce it.

But let's try to figure out what they did

2. Indexing the code locally

As they mentioned in their blog, indexing is a very old technique and trigram search was first mentioned 80 years ago. So how did Cursor come up with such a beautiful solution only in 2026? Is everyone around dumb and never did anything like this before? No, actually all the theory in the blog post they made (only the one that makes sense) comes from this paper https://swtch.com/~rsc/regexp/regexp4.html that was behind the Google Code Search project.

So why didn't any of the editors before come up with the solution of using indexes? This is my main question to the Cursor team. Why didn't you mention that indexing the same repo on the same machine takes >3 minutes.

And it takes 1GB of space which tbh is not that much for 487,592 searchable non-binary files. Your implementation very likely takes MORE TIME but is likely more space efficient (maybe 2-4 times? we never know cause this is all just words)

So dear Cursor marketers, why didn't you consider this chart to be added to your blog post? Yes I'm using the infinite super power of marketing bar charts against you:

You know the biggest problem is not indexing the file, the biggest issue is maintaining the index, and this is the primary reason why every editor like VSCode is not using something like this. Inverted trigram indexes are actually a complicated thing. And you made it even more complicated. If we take ONLY the ASCII typeable characters subset you will have about 700k unique trigrams listed in the chromium codebase

A bit about the n-gram indexing

Tri-gram or bi-gram is just a unique pairs or trios of bytes within a string. Either the document or the query.

The trigram index is going to look like this in memory, there is a bunch of ways you can optimize it like using bitfields instead of arrays to reduce the space but the amount of data is massive.

(the amount of rows would be equal to every unique 3 characters in your codebase)

┌─────────┬───────────┐
│ trigram │ file_idx  │
├─────────┼───────────┤
│ "  M"   │ [1, 2, 3] │
│ " MA"   │ [1, 2]    │
│ "MAX"   │ [1, 2]    │
│ "AX_"   │ [1, 2]    │
│ "X_F"   │ [1]       │
│ "_FI"   │ [1, 3]    │
│ "FIL"   │ [1, 3]    │
│ "ILE"   │ [1, 3]    │
│ "LE_"   │ [1, 3]    │
│ "E_S"   │ [1, 3]    │
│ "_SI"   │ [1, 3]    │
│ "SIZ"   │ [1, 3]    │
│ "IZE"   │ [1, 3]    │
│ "ZE "   │ [1, 3]    │
└─────────┴───────────┘

The problem is that to keep it fast it absolutely has to use a flat allocated array of all the files somewhere to keep the lookup table fast. This brings 2 problems

a) you have to know and store the list of all the files in the file system
b) you have to update both file list and index every time something has changed

The blog post mentions that they use "layers" to keep the sync up-to-date. It is a smart move but the blog post never mentions ANYTHING about the way they do the layering. And this is actually the most important part because if they were able to figure out how to quickly and efficiently handle the "file deleted at index N" problem without hitting conflicts or growing another 1GB file next to the original index, this is actually something worth discussing. And they simply omitting it?!

Where do they do the updates? Do they use a persistent file watcher? Where index is stored, how much time do they spend on traversing the tree? What is going to happen when a user wants to search in node_modules? We don't know because they specifically omit this part entirely.

we control the state of the index by basing it off a commit in the underlying Git repository. User and agent changes are stored as a layer on top of it.

So you are saying that every single commit a user makes, you will spend 3 minutes indexing? Great job, but in fact they just do the indexing in the background, effectively falling back to regex search in the meantime. Because reindexing is a non-atomic operation that requires a lock.

This is the essential problem of indexing that IS NOT solved. You do the marketing like you solved this problem or it is not imporant while in fact this is exactly what is important.

My project uses in-memory indexing that looks pretty similar to what they describe and I'll tell you later how we bypass this exact limitation in fff.

3. n-gram indexing and its cost

So I actually read the blog post a few times because I am genuinely interested in how they do this and this is the only paragraph that actually describes something. Everything else is pure water. But the funniest part of this paragraph is that to me it looks like a complete LLM hallucination. In short: they are building an inverted index of the trigrams but adding a bloom filter to the next byte.

The way n-gram inverted index works is very simple

Here is the first suspicious part making me think they didn't really understand the point of trigrams

By having a mask that contains the characters following each trigram, our inverted index can be constructed using trigram keys, but we can query it using quadgrams! This already scopes down the potential documents much more than a simple trigram index could.

So I actually ran the n-gram indexing for the chromium project. And here is what I found for the exact "MAX_FILE_SIZE" query over the same indexed code using trigram and bi-gram search.

┌─────────────────┬───────────────────┬────────────────────┬───────┬───────┐
│      Query      │ Bigram candidates │ Trigram candidates │ Bi %  │ Tri % │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ MAX_FILE_SIZE   │ 13,241            │ 727                │ 3.0%  │ 0.2%  │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ mutex_lock      │ 13,934            │ 279                │ 3.2%  │ 0.1%  │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ return          │ 179,617           │ 140,367            │ 40.8% │ 31.9% │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ #include        │ 127,705           │ 125,227            │ 29.0% │ 28.5% │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ TODO            │ 191,186           │ 32,865             │ 43.4% │ 7.5%  │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ struct file     │ 92,419            │ 22,337             │ 21.0% │ 5.1%  │
├─────────────────┼───────────────────┼────────────────────┼───────┼───────┤
│ phylink_ethtool │ 3,069             │ 2                  │ 0.7%  │ 0.0%  │
└─────────────────┴───────────────────┴────────────────────┴───────┴───────┘

There is absolutely no reason to read the article further because for the exact query they showed, the trigram index actually returns <1k candidates, and there is absolutely no reason to optimize it even further. Unless of course you run your LLM in a loop asking to come up with a better approach (LLMs love to put bloom filters everywhere, just saying)

For any normal sized repository (e.g. 20k lines) quadgram indexing would essentially be equal to the exact search (every query will result in 1-2 candidates). This is simply an overkill even for a chromium 500k files repo.

But on top of adding an additional bloom filter imitating quadgrams, they decided to prefilter by frequency and hash every single trigram (there are 779,889 distinct trigrams in chromium), effectively making indexing way slower simply to reduce 727 candidates to 300. There are a ton of questions about how this works that we will never get answers to because this is a private proprietary thing. I have no idea why we are even discussing this.

In addition to that, with their idea of dynamic n-grams + a bloom filter, they now have the problem of storing the data because there is no primitive that can fit those as a key in the address space (and no, this is not because they support full Unicode indexing), so they have to hash every single key to simply store the index on the file system.

4. Ultimate performance problem

I once said that FFF is the fastest file search on the planet. While it was true, I never thought that this was the killer feature. Once in a while I get tweets like "fff feels slow in the 200GB Android root repo". I usually ignore those because I actually work with a former Android core dev now and I asked them how they do Android development and they told me that NO ONE EVER opens the Android root. The same thing happens with chromium, it literally mixes Android, iOS, web, V8, and other subprojects.

The indexing itself is a complicated operation which doesn't actually give any performance benefit on real repositories but anyway takes time to build and store. Without index fff is able to search the Linux kernel for a very rare string in under 6ms (the average repo you're gonna work with is way less than 100k files) which is absolutely fine, and under 1ms with the very simple index we have:

Index optimized for the actual users

for large repositories FFF is actually doing a very similar approach to what is described, and I originally thought they just stole my code (seems like not really 😭). We use the inverted bigram index approach but only over the most useful and rare ones. There is absolutely 0 reason to even index bigrams like "th" because the word "the" will exist in the absolute majority of files.

This is not what I even call a feature, but an optimization because n-gram indexing never works on the real Unicode files (because once you have many languages you will need to pick up the actual trigrams that exist in the words rather than all the combinations, and this simply doesn't work with source code files).

Here is how fff in-memory bi-gram prefiltering works:

If we take the 95 printable ASCII characters and lowercase them we get 69 distinct values (a-z, 0-9, punctuation, space). 69² = 4,761 possible unique bigram pairs - that's it. No banning, no frequency cutoffs. We index ALL of them because the key space is naturally bounded and tiny compared to trigrams (69³ = 328,509 keys). During the indexing phase we do the same reading of all the files to:

a) populate the virtual memory mapped cache
b) read the file content and extract bigrams for the inverted index

Because bigram extraction is just splitting bytes by 2 and lowercasing, modern CPUs chew through it because it is easy SIMD problem. Indexing the whole chromium repository takes around 2 seconds to list files and 7 seconds for the bigram index which runs completely asynchronously:

2026-03-24T19:43:11.331524Z INFO fff_search::file_picker: crates/fff-core/src/file_picker.rs:1115 Bigram index built in 7.59s - 4761 columns (3481 sparse, 1280 dense) for 487592 files

This is faster than gathering git status for those files using libgit2! The prefilter index is stored as a compressed immutable inverted index with a hybrid column storage - each bigram gets either a dense bitset or a sparse posting list, whichever is smaller.

Here's the full layout:

LOOKUP TABLE (178 KB fixed)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Maps every possible bigram key (u16) to its column index.
44,587 entries × 4 bytes. Most map to NO_COLUMN (bigram never seen).

┌──────────┬──────────┬──────────┬──────────┬───┬──────────┐
│ key 0    │ key 1    │ key 2    │ key 3    │...│ key 65535│
│ col: ──  │ col: ──  │ col: 0   │ col: ──  │   │ col: ──  │
└──────────┴──────────┴────┬─────┴──────────┴───┴──────────┘
                           │
                           ▼
COLUMNS (4,761 for chromium — one per unique lowercased ASCII bigram)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Each column stores "which files contain this bigram" in one of two formats:

BITSET (1,280 columns — common bigrams like "ma", "in", "re")
┌──────────────────────────────────────────────────────────────────────┐
│ One bit per file. For 487k files → 7,610 u64 words → ~60 KB/col      │
│                                                                      │
│ Column for bigram "fi":                                              │
│ ┌──────────────────┬──────────────────┬──────────────────┬─────┐     │
│ │ word 0           │ word 1           │ word 2           │ ... │     │
│ │ files 0–63       │ files 64–127     │ files 128–191    │     │     │
│ │ 0b..00101001     │ 0b..10000100     │ 0b..00000011     │     │     │
│ │    ↑  ↑  ↑       │    ↑     ↑       │ ↑ ↑              │     │     │
│ │    0  3  5       │   66    71       │128 129           │     │     │
│ └──────────────────┴──────────────────┴──────────────────┴─────┘     │
│ Query: result &= column  (word-by-word AND, SIMD-friendly)           │
└──────────────────────────────────────────────────────────────────────┘

And this is how the filtering process looks like. I do not have time and resources to build and interactive visualization but I think this is actually more clear:

QUERY: grep "MAX_FILE_SIZE"
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
1. Extract bigrams: "ma","ax","x_","_f","fi","il","le","e_","_s","si","iz","ze"
2. Look up each in the table → get column indices
3. AND all columns together:

   result = 1111...1111  (start: all files are candidates)
   result &= col["ma"]  →  ████░░░░████░░██░░░░...  (files with "ma")
   result &= col["ax"]  →  ██░░░░░░░██░░░██░░░░...  (intersect with "ax")
   result &= col["x_"]  →  █░░░░░░░░░█░░░█░░░░░...  (getting smaller)
   ...                      ↓ after all 12 bigrams:
   result = 13,241 bits set out of 487,592  →  97% of files eliminated

OVERLAY (live updates without reindexing)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
The base index is immutable. File changes go to a delta layer:

The crossover point between sparse and dense is elegant: when popcount × 4 bytes ≥ ⌈file_count/64⌉ × 8 bytes, the column switches from a posting list to a bitset. This happens at ~3.1% file occupancy. For chromium that means a bigram appearing in 15,000+ files gets a dense bitset, while rarer bigrams use compact posting lists. We store all 4,761 distinct case-insensitive ASCII bigrams - not losing anything on the prefiltering step.

Because fff is originally built with a background fs watcher in mind and an immutable file tree index, this is extremely easy for us to update the index every time a file is updated or deleted, though it might take ~10 seconds if the whole scan is invalidated. Here are real measurements on actual codebases (Apple M4 Max, all times measured from scan complete to index ready):

┌─────────────────┬─────────┬────────────┬──────────────────────┬──────────┬──────────┐
│ Repository      │  Files  │  Indexing  │ Cols (sparse/dense)  │  Memory  │  B/file  │
├─────────────────┼─────────┼────────────┼──────────────────────┼──────────┼──────────┤
│ whisper.cpp     │   1,150 │    0.10s   │ 4,320 (2,534/1,786)  │   747 KB │    665   │
│ ffmpeg          │   9,834 │    0.31s   │ 4,672 (3,078/1,594)  │   2.9 MB │    309   │
│ rust-skia       │  12,502 │    0.41s   │ 4,542 (3,081/1,461)  │   3.5 MB │    294   │
│ <Your B2B SAAS> │  21,462 │    0.52s   │ 4,314 (2,823/1,491)  │   3.7 MB │    269   │
│ linux kernel    │  98,462 │    1.94s   │ 4,489 (3,202/1,287)  │  23.7 MB │    287   │
│ chromium        │ 487,592 │    7.87s   │ 4,761 (3,481/1,280)  │ 113.1 MB │    243   │
└─────────────────┴─────────┴────────────┴──────────────────────┴──────────┴──────────┘

Okay and here is the real comparison, where you can literally see that the value coming from persistent indexing is visible only on the VERY LARGE repositories.

┌───────────────────────┬──────────────────┬──────────────────────┬──────────────────────┬──────────────────────┐
│ Repository            │ Tool             │   Time to Index      │   Index Size         │   Time to Query      │
├───────────────────────┼──────────────────┼──────────────────────┼──────────────────────┼──────────────────────┤
│ whisper.cpp (1,150)   │ Cursor*          │   ~14 s*             │   ~2 MB*             │   ?                  │
│                       │ zoekt            │   10s                │   4MB                │   ~3 ms              │
│                       │ fff              │   0.10 s (async)     │   747 KB             │   ~1 ms              │
│                       │ ripgrep          │   - (no index)       │   - (no index)       │   ~15 ms             │
├───────────────────────┼──────────────────┼──────────────────────┼──────────────────────┼──────────────────────┤
│ ffmpeg (9,834)        │ Cursor*          │   ~45 s*             │   ~20 MB*            │   ?                  │
│                       │ zoekt            │   28s                │   43MB               │   ~1 ms              │
│                       │ fff              │   0.31 s (async)     │   2.9 MB             │   ~1 ms              │
│                       │ ripgrep          │   - (no index)       │   - (no index)       │   ~120 ms            │
├───────────────────────┼──────────────────┼──────────────────────┼──────────────────────┼──────────────────────┤
│ chromium (487,592)    │ Cursor*          │   ~4 min*            │   ~1 GB*             │   ?                  │
│                       │ zoekt            │   2 min              │   1.4GB              │   ~3 ms              │
│                       │ fff              │   7.87 s (async)     │   113.1 MB           │   118 ms             │
│                       │ ripgrep          │   - (no index)       │   - (no index)       │   ~6 s               │
└───────────────────────┴──────────────────┴──────────────────────┴──────────────────────┴──────────────────────┘
* Cursor values are estimates - code is not open source, measurements cannot be reproduced

The question is: do you want to wait 3 minutes and burn 1GB of disk for a 35x improvement on the one repo nobody opens as a whole? Or do you want instant results on every real repository you actually work on?

Yes fff is 10 times slower than the "ephemeral code they showed" while effectively having no persistent index stored, and is still 34x faster than ripgrep (because ripgrep didn't take 16 seconds to find the files, it takes 6, let's be honest) but on the real repositories every query runs in sub-ms

and here is why even this doesn't really matter

5. I heard you are an AI agentic company, right?

All I want to say is that the main problem of file search is not the speed, it's accuracy.

It's like typing speed, I can type 200 words per minute with a 99% typo rate.

The main problem of file search that I am trying to solve is the way we suggest files for the user / AI agent. Yes I recently released fff-mcp that is replacing built-in file commands and actually delivers +10% of speed to your clanker run. Though it's not because of the search itself, it's because of the reduced round trips of the agent.

Let's try to do a search on chromium for actual code. You can literally see that LLMs LOVE to stack all kinds of guards to reduce the amount of incoming files on their own, similar to what is happening to the

Here is a beautiful example of 2 sessions, one with Claude's default tools and the one with fff

The one with fff mcp that is 3x less time and 4x less cost (lmao I should've used this in the announcement)

Most of the queries immediately look like "v8/ " which is parsed by the fff query parsing and does very fast SIMD-optimized prefiltering in the paths 4ns + bigram index 8ns, having the file search only touch 458 files on the real query generated by the agent. This query returned results in under 2ms

And the biggest value is NOT THE RAW SPEED of a file search, it's the reduced amount of round trips because fff has built-in fuzzy code matching that is pretty slow (it's impossible to prefilter) but it does show suggestions for your LLM, significantly reducing round trips. This is where the actual optimization for the coding agents is, and you folks are missing that.

In addition to that, fff knows about git status of every file, how often it's modified, it classifies which lines are definitions (function names, classes, variables) to suggest them first, keeps track of frecency and sorts the files by all kinds of heuristics (which is very slow) to prevent the user, whether a human or AI agent, from repeating the search which is exactly where the time waste is.

And it can still finish the prefiltering index on a 500k file codebase faster than an LLM will generate the first tool call.

Cheers.

How act() works inside react?

Dmitriy Kovalenko — Wed, 17 Feb 2021 22:30:49 +0000

Hola! Lazy dev here.

React testing is hard. Especially react testing outside the browser environment, like with Jest and JSdom. Let's try to reverse engineer react's act(), understand why do we need it, and think about UI testing overall.

History

Today I meet this tweet by @floydophone

// Detect dark theme var iframe = document.getElementById('tweet-1362123964412096513-153'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1362123964412096513&theme=dark" }

And was inspired to write about how your tests work inside your terminal when you are testing in node.js. Let's start from the question – why do we need this "magic" act() function.

What is act()

Here is a quote from react.js docs:

To prepare a component for assertions, wrap the code rendering it and performing updates inside an act() call. This makes your test run closer to how React works in the browser.

So the problem that act() is solving – It delays your tests until all of your updates were applied before proceeding to the next steps. When you are doing any kind of user interaction, like this



act(() => {
  button.dispatchEvent(new MouseEvent('click', {bubbles: true}));
});

React is not updating UI immediately, thanks to the Fiber architecture. It will update it asynchronously in some time after the click, so we need to wait for UI to be updated.

And here is a problem

The main problem here – act() is actually a crutch and you will probably agree that it is not a perfect solution. Tests that you (probably) are writing are synchronous. It means that commands and assertions that tests are doing are executed one-by-one without any waiting.

UI works differently – UI is async by nature.

Reverse engineer it

Let's look more closely at the implementation of this function, right from the react sources. We only need 2 files ReactTestUtilsPublicAct and ReactFiberWorkLoop.

I will skip not interesting parts, but the code is not so big so you can read it yourself 🙃 Let's start from the main point of the act function:



  let result;
  try {
    result = batchedUpdates(callback);
  } catch (error) {
    // on sync errors, we still want to 'cleanup' and decrement actingUpdatesScopeDepth
    onDone();
    throw error;
  }

And this magic batchedUpdates function has a pretty simple yet powerful implementation.



export function batchedUpdates<A, R>(fn: A => R, a: A): R {
  const prevExecutionContext = executionContext;
  executionContext |= BatchedContext;
  try {
    return fn(a);
  } finally {
    executionContext = prevExecutionContext;
    if (executionContext === NoContext) {
      // Flush the immediate callbacks that were scheduled during this batch
      resetRenderTimer();
      flushSyncCallbackQueue();
    }
  }
}

This particular function is called inside the react when during the render phase react exactly knows that all updates are done and we can render the dom. And it starts the reconciliation and synchronous dom updating after.

After batchedUpdates our code has 2 branches depends on how you used it. If you passed a synchronous function inside the act, like



 act(() => {
   ReactDOM.render(<Counter />, container);
 });

It will call the function flushWork which is nothing more than a sync while loop



const flushWork =
  Scheduler.unstable_flushAllWithoutAsserting ||
  function() {
    let didFlushWork = false;
    while (flushPassiveEffects()) {
      didFlushWork = true;
    }

    return didFlushWork;
  };

It looks like for concurrent mode a new specific hook implemented to stop all the effects together (unstable_flushAllWithoutAsserting)

But for now, it is just a sync while loop that stops the synchronous execution until all the DOM updating work is done. Pretty clumsy solution, don't you think?

Async execution

More interesting is coming when you are passing an async function as a callback. Lets go to another code branch:



if (
  result !== null &&
  typeof result === 'object' &&
  typeof result.then === 'function'
)

// ... not interesting

result.then(
  () => {
    if (
      actingUpdatesScopeDepth > 1 ||
      (isSchedulerMocked === true &&
        previousIsSomeRendererActing === true)
    ) {
      onDone();
      resolve();
      return;
    }
    // we're about to exit the act() scope,
    // now's the time to flush tasks/effects
    flushWorkAndMicroTasks((err: ?Error) => {
      onDone();
      if (err) {
        reject(err);
      } else {
        resolve();
      }
    });
  },
  err => {
    onDone();
    reject(err);
  },
);

Here we are waiting for our passed callback (the result is returned by batchedUpdates function) and if after we are going to more interesting function flushWorkAndMicroTasks. Probably the most interesting function here :)




function flushWorkAndMicroTasks(onDone: (err: ?Error) => void) {
  try {
    flushWork();
    enqueueTask(() => {
      if (flushWork()) {
        flushWorkAndMicroTasks(onDone);
      } else {
        onDone();
      }
    });
  } catch (err) {
    onDone(err);
  }
}

It is doing the same as the sync version (that only calling flushWork()). But it wraps the call enqueueTask, which is a hack only to avoid setTimeout(fn, 0).

an enqueueTask function



export default function enqueueTask(task: () => void) {
  if (enqueueTaskImpl === null) {
    try {
      // read require off the module object to get around the bundlers.
      // we don't want them to detect a require and bundle a Node polyfill.
      const requireString = ('require' + Math.random()).slice(0, 7);
      const nodeRequire = module && module[requireString];
      // assuming we're in node, let's try to get node's
      // version of setImmediate, bypassing fake timers if any.
      enqueueTaskImpl = nodeRequire.call(module, 'timers').setImmediate;
    } catch (_err) {
      // we're in a browser
      // we can't use regular timers because they may still be faked
      // so we try MessageChannel+postMessage instead
      enqueueTaskImpl = function(callback: () => void) {
        if (__DEV__) {
          if (didWarnAboutMessageChannel === false) {
            didWarnAboutMessageChannel = true;
            if (typeof MessageChannel === 'undefined') {
              console.error(
                'This browser does not have a MessageChannel implementation, ' +
                  'so enqueuing tasks via await act(async () => ...) will fail. ' +
                  'Please file an issue at https://github.com/facebook/react/issues ' +
                  'if you encounter this warning.',
              );
            }
          }
        }
        const channel = new MessageChannel();
        channel.port1.onmessage = callback;
        channel.port2.postMessage(undefined);
      };
    }
  }
  return enqueueTaskImpl(task);
}

The main goal of this function is only to execute a callback in the next tick of the event loop. That's probably why react is not the best in terms of bundle size :)

Why async?

It is a pretty new feature, probably needed more for concurrent mode, but it allows you to immediately run stuff like Promise.resolve aka microtasks for example when mocking API and changing real promise using Promise.resolve with fake data.



import * as ReactDOM from "react-dom";
import { act } from "react-dom/test-utils";

const AsyncApp = () => {
  const [data, setData] = React.useState("idle value");

  const simulatedFetch = async () => {
    const fetchedValue = await Promise.resolve("fetched value");
    setData(fetchedValue);
  };

  React.useEffect(() => {
    simulatedFetch();
  }, []);

  return <h1>{data}</h1>;
};


let el = null;
beforeEach(() => {
  // setup a DOM element as a render target
  el = document.createElement("div");
  // container *must* be attached to document so events work correctly.
  document.body.appendChild(el);
});

it("should render with the correct text with sync act", async () => {
  act(() => {
    ReactDOM.render(<AsyncApp />, el);
  });
  expect(el.innerHTML).toContain("idle value");
});

it("should render with the correct text with async act", async () => {
  await act(async () => {
    ReactDOM.render(<AsyncApp />, el);
  });

  expect(el.innerHTML).toContain("fetched value");
});

Both tests passing 😌. Here is a live example (you can open sandbox and run tests inside using "Tests" tab):

It is fun that it works, but if you will change Promise.resolve to literally anything like this:



const fetchedValue = await new Promise((res, rej) => {
  setTimeout(() => res("fetched value"), 0);
});

// it doesn't work ¯\_(ツ)_/¯

Replace

It is pretty easy to replace any act() call by using simple setTimeout(fn, 0):



button.dispatchEvent(new MouseEvent('click', {bubbles: true}));
await new Promise((res, rej) => {
 setTimeout(res, 0);
});

will work in most cases :) some sources

But why

The main question – why do we need it? So much ~not good~ code that confuses everybody? The answer – our tests that are running inside node.js and trying to be "sync" while the UI as async.

And that's why you will never need any kind of act() if you are rendering React components in the real browser and using async test-runner, like Cypress for component testing

Thank you

Thank you for reading, hope it is more clear why do we need act() for most plain react unit testing.

And no act() was not harmed in the making of this article :D

Why our visual regression is so slow?

Dmitriy Kovalenko — Fri, 13 Nov 2020 21:11:52 +0000

Hola! Lazy dev here, and today we are going to discuss computer eyes 👁👁 that help us to test how our products look like.

The visual regression tools that we are using every day are extremely slow. That is the fact. At least because the task to compare 2 images is hard. And I tried to fix this 👀.

But first of all, let's take a look at how visual regression tools work under the hood and what they are actually doing.

Image comparison

Image comparison itself is pretty hard. We can not just compare 2 image buffers using "===" and even we can not take all the pixels from the image and compare them one by one. Why? Because human eyes can not see the difference between all the colors. For example, how do you think – do these colors are the same or not?

If yes – 👍 you have a really good vision! But 95% of people will not spot the difference between rgba(200, 100, 100, 250) and rgba(200, 100, 95, 250) colors.

That's why in order to visually compare images – we need to take all the pixels in the image and it is a lot (for a full-HD screenshot (1920x1080) we have 1920 * 1080 = 2 090 880 pixels) – iterate them one by one and calculate the color difference.

This is a hard task for computers. So the algorithm is:

1) Read and decode the image
2) Make a loop with millions iterations, make some calculations and save the different pixels
3) Make an image of different pixels
4) Encode and save the image diff

But the result is extremely helpful when you are testing the user interface:

The slowness

But unfortunately, tools that we are commonly using to compare screenshots are extremely slow. And they are slow not because they had a bad code inside. The main problem is that they are often written in the wrong language* or doing some useless job under the hood.

For example, the most popular image comparison tool in the javascript community – pixelmatch is really slow in Node.js environment (but blazing fast in the browser otherwise).

Using pixelmatch to comparing 2 screenshots of https://cypress.io home page will take around 7-8 seconds.

Impact on your CI

This really affects our CI time. Let's calculate the CI time for visual regression if we are running 25000 screenshot tests per month. And this number is not something overwhelming. It is a very basic plan of https://percy.io, which is usually not enough for huge projects.

So, if we are running 25000 visual tests and each screenshot test running for 7 seconds we are spending 48,6 hours on CI!

25000 * 7 / 3600 = 48,611111111

That is a lot! This can take even more time than all the other UI tests and that's why performance for this sort of task really matters.

Cause if we can save at least 3 seconds per each snapshot we will save 20 hours per month.

25000 * 3 / 3600 = 20,83333

Solution

That was a pain point for me, so I decided to fix this – and wrote that fastest in the world image comparison tool.

And I am happy to introduce you odiff! 👀🥳🎉 It was designed to handle the "huge" images, be fast, memory-efficient, and save your CI time.

This tool is running the same comparison 2 times faster than analogs! Yes, it can save you those 3 seconds per snapshot :)

Here are some benchmarks:

Also, here are the results of comparing the same cypress.io home page screenshot:

Command	Mean [s]	Min [s]	Max [s]	Relative
`pixelmatch www.cypress.io-1.png www.cypress.io.png www.cypress-diff.png`	7.712 ± 0.069	7.664	7.896	1.82 ± 0.03
ImageMagick `compare www.cypress.io-1.png www.cypress.io.png -compose src diff-magick.png`	8.881 ± 0.121	8.692	9.066	2.09 ± 0.04
`odiff www.cypress.io-1.png www.cypress.io.png www.cypress-diff.png`	4.247 ± 0.053	4.178	4.344	1.00

How?

Why it is so fast? The answer is simple:

It is written in OCaml and compiled to the native binary executable. OCaml compiler is extremely fast and predictable so it is easy to profile and optimize performance-sensitive code. And also we have direct node.js bindings!
It is not doing a useless job under the hood. It is working directly with the low-level bytes buffer and avoids unnecessary memory allocations.
It is optimized by profiling produced assembly output 👯‍♀️

Check it out

Try it out right now! Give us your feedback, and do not forget about ⭐️ the project if you are interested!

dmtrKovalenko / odiff

The fastest pixel-by-pixel image visual difference tool in the world.

The fastest* (one-thread) pixel-by-pixel image difference tool in the world.

Why Odiff?

ODiff is a blazing fast native image comparison tool. Check benchmarks for the results, but it compares the visual difference between 2 images in milliseconds. It was originally designed to handle the "big" images. Thanks to OCaml and its speedy and predictable compiler we can significantly speed up your CI pipeline.

Demo

base	comparison	diff

Features

✅ Cross-format comparison - Yes .jpg vs .png comparison without any problems.
✅ Support for .png, .jpeg, .jpg, and .tiff
✅ Supports comparison of images with different layouts.
✅ Anti-aliasing detection
✅ Ignoring regions
✅ Using YIQ NTSC transmission algorithm to determine visual difference.

Coming in the nearest future:

⏹ Reading image from memory buffer
⏹ Reading images from url

Usage

Basic comparison

Run the simple comparison. Image paths can be one of supported formats, diff output can…

View on GitHub

Thank you for your time! Optimize your CI pipeline and in order to follow the tradition – no pixels were harmed in the making of this article 🙈

You might not need date-fns

Dmitriy Kovalenko — Tue, 29 Sep 2020 22:14:58 +0000

Hola! Lazy dev here and today we are going to discuss date-fns. People often choose date libraries before they really need it. "How we will format the date?", "Are there any alternatives?"

But really, are there?

Am I a hater?

Sorry, this question was required. No, I'm not. Moreover, I was a super-active user and evangelist of date-fns. I am the creator of date-io and @material-ui/pickers which have been proposing to choose date-fns over the other date libraries.

But one day I said that date-fns is not a panacea
After the twitter thread, that was transformed in this blog post, date-fns' maintainer blocked me everywhere because it says that you might not need it. So probably this thread contains some useful information for people that choosing date lib – so I decided to share it in the blog as well! Hope you will have some fun reading it :)

That's it for prerequisites so let's start the discussion

You might not need a date library at all

First of all, when you need to only show date values in the user-readable format – you can avoid date libraries at all. Today all the modern browsers (even IE11) and node.js perfectly support Intl.DateTimeFormat!

This means that if your task is only to show the date/time value in a user-readable format you can do the following:



const date = new Date(Date.UTC(2012, 11, 20, 3, 0, 0))
// Ouput will depend on user locale and timezone 
console.log(new Intl.DateTimeFormat().format(date));

It perfectly supports native IANA timezone and locale formatting. It means that you can not include locales in the bundle at all.



const date = new Date(Date.UTC(2012, 11, 20, 3, 0, 0));
// Results below assume UTC timezone - your results may vary

console.log(new Intl.DateTimeFormat('en-US').format(date));
// expected output: "12/20/2012"

console.log(new Intl.DateTimeFormat('fr').format(date));
// expected output: "20/12/2012"

If you need more

But there is a problem. When you need more than formatting – for example, parsing or you are working with dates too often so the native (not really best) Date API is not enough. You probably will start looking for some helpful date-management library.

🌈 One wonderful day we will probably be able to get rid of all libraries in favor of native API. Here is the proposal for Temporal, which provides a nice functional API for working with Date.

But today you probably will pick up date-fns. Nothing personal – only statistics. Date-fns is the most popular date library as of now. What about moment.js? It is dead.

Today date-fns is much more often used for new projects. Here are downloads stats from date-io.

Statistically, you will pick date-fns. But do you really need it?

Let's discuss some criterias that are commonly used to choose the date library, and see does date-fns is the best one or not?

Bundlesize

Date-fns is solving only 1 problem much better than any other date library. And that's not a bundlesize. 🎉 Surprise 🎉 date-fns takes mostly 18kb gzip without locales. Dayjs takes 6 (yes six kb).

But if compare not gzipped, but parsed size – date-fns is the biggest one

Ok. Ok. The problem date-fns solving REALLY nice is tree-shaking. Because each function has its own entry point and exported as esm, unused code will be removed from the bundle, right?

Treeshaking

Let's create a more "real-world" example, that uses the hardest to implement manually functions:

Formatting
Parsing
Display time from X to Y
3 locales

Result:

As you can see date-fns takes 13.88kb gzip when only importing the most important functionality. It is a lot.

Here is a pretty fun example of a react datepicker that has a peer dependency on date-fns. Date-fns takes 3 times more space than the datepicker itself and mostly 1/3 size of react. And it's only to make a single date-picker work.

Also as you saw in the bundlesize stats luxon did not change its size at all. This because luxon npm package provides only commonjs output which is not tree shakeable. So maybe one day it will become smaller.

But do not forget the most wonderful thing about Luxon – it is built over native Intl – so it doesn't bundle locales at all. You can support even 50 locales without any additional bundlesize for date formatting!

P.S. All date-fns' 75 locales bundle takes 80kb gzip

Bundlesize conclusion

Date-fns is not lightweight library for date/time management. There are underrated alternatives – e.g. Dayjs is much smaller when using around the same functionality.

API

The next criteria for choosing a library would be the API. API must be clear, well-typed, and comprehensive. And here the most unclear for me personally – why everybody is choosing date-fns?

Date-fns design is pretty straightforward – you have a separate function for everything. And this is totally perfect, but unfortunately, not for all javascript developers. The problem is that javascript doesn't have native function composition utils.

I mean that some complex code with date-fns is completely unreadable:



function checkIsBeforeDateFns(time: Date, maxTime: Date) {
  return isBefore(
    setMilliseconds(setSeconds(setMinutes(time, 0), 0), 0),
    maxTime
  );
}

Function executions need to be read like from the inside out. The first function call will be setMinutes and the last will be isBefore.

Let's compare the same functions in dayjs and luxon. They are using the old good chaining API. Most developers/editors/linters/static analyzers work like a charm with such APIs.



function checkIsBeforeDayjs(time: Dayjs, maxTime: Dayjs) {
  return time.minute(0).second(0).millisecond(0).isBefore(maxTime);
}

function checkIsBeforeLuxon(time: DateTime, maxTime: DateTime) {
  return time.set({ second: 0, minute: 0, millisecond: 0 }) < maxTime;
}

Much readable right? This is actually overall a common problem in functional programming. And it can be easily fixed by using some of the function composition techniques. For example here are the same functions with date-fns/fp submodule and ReasonML (now Rescript) – native functional language compiling to javascript. And this is awesome 💜



let checkIsBeforeDateFns = (time, maxTime) =>
  time
  |> DateFns.setMilliseconds(0)
  |> DateFns.setSeconds(0.)
  |> DateFns.setMinutes(0.)
  |> DateFns.isBefore(maxTime);

This is still just 4 function calls. But much much more readable. Beautiful!

By the way I am the maintainer of date-fns binding for ReasonML. Bring us a ⭐️

But ask yourself – do you and more important your team are ready for functional programming? And do you have all the required tools for it like pipe or compose?

If yes – take date-fns and be happy with functional programming 👨‍💻👩‍💻.

Performance

You should not think about performance before the problem was encouraged.

Premature optimization is the root of all evil © Donald Knuth

The performance difference will be visible only on the thousand function calls per second. But if you still interesting in performance differences between date-fns and the other libraries:

Short results from our date-io benchmark:

Date-fns is the fastest for date calculations (add, subtract, before, etc)
Date-fns is the fastest for date parsing
Moment is fastest for formatting (ha-ha didn't expect moment here)

Yes, date-fns is really fast because it works directly with a native date without creating any additional wrapper. Dayjs focused on size instead of speed and Luxon is using Intl which is super slow 🐌.

So yes date-fns is the best option if you have performance issues with other libs. But do you really have?

Conclusion

Make sure that the author of this post is incompetent, subjective, stupid, awful, and lazy. So you must reach your own conclusions for your particular project and team based on many factors.

BTW here is the repo with all date-fns comparison stuff from this post, you can check it out, play with bundlesize and API.

I will really be happy if you will think about date/time libraries in javascript and the requirement of date-fns after this reading 🤓

In the author's humble opinion there are no reasons to choose date-fns if you are not cooking functional programming. And, unfortunately, as far I can see literally nobody using their really good functional approach 😿.

So, in conclusion: if this lazy author one day will start a new project in javascript and will need some kind of date/time manipulations he will probably do the following:

Try to start with native Intl formatting
When lib will become really needed choose dayjs – because its
- a) ~Harder, Better, Faster, Stronger~
- a) smaller
- b) tree shakable by plugins
- c) have a nice API

Thank you

For this loooong read and by the tradition:

No date-fns maintainers were harmed in the making of this article 😉

The easiest way to connect cypress and react devtools 🌟

Dmitriy Kovalenko — Wed, 23 Sep 2020 13:35:49 +0000

Hola! Lazy dev here and today we will talk about cypress development & testing experience for react applications.

Wondering about how to use the awesome react dev tools while testing Cypress.io? Let's start.

Requirements

Together with the v16 react team provided a completely new devtools package. You probably ❤️ love it :)

Install

1) In order to run dev tools for your application running by cypress browser you need to install the standalone version of react-devtools package



yarn add --dev react-devtools

Tip: you can also install this package globally in order to run with different projects. It is ok because we will use this only for development.

Configure

2) Then add this as the very first <script> tag in the <head> of your index.html (or _document.js for next.js) when developing. This script actually connects react-devtools backend to the application.



<script src="http://localhost:8097"></script>

Alternatively, you can import this using javascript



import 'react-devtools'

⚠️ Important that this import comes before any other imports in your app (especially before react-dom). Make sure to remove the import before deploying to production, as it carries a large DevTools client with it.

If you use Webpack and have control over its configuration, you could alternatively add react-devtools as the first item in the entry array of the development-only configuration, and then you wouldn’t need to deal either with <script> tags or import statements.

Running

Now simply run your application, cypress tests, and react-devtools:



# app
yarn start

# cypress 
yarn cypress open

# devtools app
yarn react-devtools

(Yes, you need 3 console tabs). Running react-devtools will open an electron application which will wait for the react app connection

And now open the cypress app, run any spec, and observe that react-devtools connected! 👀🌟

Result

Fully working react devtools running over your tests in 2 minutes!

Limitations

The biggest limitation of react-devtools and cypress is that they won't work with cypress' time-traveling snapshots. This happens because separate steps of time travel are rendered as dom snapshots and are not the real react applications.

In order to debug a specific test step – stop the cypress execution when you need to debug:

Alternatives

There is another (probably more complex) way to load devtools using chrome --load-extension=${extensionFolder}. This way is described in the blog post by Gleb Bahmutov https://www.cypress.io/blog/2020/01/07/how-to-load-the-react-devtools-extension-in-cypress/

The end 🎉

Devtools works like a charm together with cypress tests. Have fun 🔥! Just don't get lost in open electron windows 😉.

P.S. No devtools were harmed in the making of this article

The neatest way to handle alert dialogs in React 🥰

Dmitriy Kovalenko — Fri, 26 Jul 2019 18:55:38 +0000

Time to read — 5 mins ☕️

Hola! Lazy dev here and we will talk about handling dialog alerts in react without tears 😢. If you are tired of tons of copy-pastes just to create new freaking «one question» modal dialog — prepare your coffee we are starting.

The goal

We want to make the neatest solution for displaying an alert. Pretty similar to what we have in a browser with a native alert function.



const isConfirmed = alert("Are you sure you want to remove this burrito?");

if (isConfirmed) {
  await api.deleteThisAwfulBurrito();
}

Sneak peek

Finally we will get to something like this.



const YourAwesomeComponent = () => {
  const confirm = useConfirmation()

  confirm({
    variant: "danger",
    title: "Are you sure you want to remove this burrito?",
    description: "If you will remove this burrito you will regret it 😡!!"
  }).then(() => {
    api.deleteThisAwfulBurrito();
  });
}

Interested? Let's write some code.

First of all, we need to start with creating actually the modal dialog. This is just a simple alert dialog built with ❤️ and material-ui



import {
  Button,
  Dialog,
  DialogTitle,
  DialogContent,
  DialogContentText,
  DialogActions,
} from "@material-ui/core";

export const ConfirmationDialog = ({
  open,
  title,
  variant,
  description,
  onSubmit,
  onClose
}) => {
  return (
    <Dialog open={open}>
      <DialogTitle>{title}</DialogTitle>
      <DialogContent>
        <DialogContentText>{description}</DialogContentText>
      </DialogContent>
      <DialogActions>
        <Button color="primary" onClick={onSubmit}>
          YES, I AGREE
        </Button>
        <Button color="primary" onClick={onClose} autoFocus>
          CANCEL
        </Button>
      </DialogActions>
    </Dialog>
  );
};

OK, but how we will adopt it to be working dynamically? That's an interesting thing to consider. Why do we need a lot of dialogs for each component if the user can see only one alert dialog simultaneously?

If you are showing an alert dialog over the other alert dialog...you probably need to reconsider the UX part of your application.

So here we go. Everything we need is to render only 1 top-level modal at the root of our application and show it when we need to. We'll use the power of react hooks to make it looks gracefully.

Wrap the context

Let's create a new context instance and wrap our component tree with it. Also, create a simple state that will save the currently displaying options for the alert (like title, description and everything you need).



interface ConfirmationOptions {
  title: string;
  description: string;
}

const ConfirmationServiceContext = React.createContext<
  // we will pass the openning dialog function directly to consumers
  (options: ConfirmationOptions) => Promise<void>
>(Promise.reject);

export const ConfirmationServiceProvider= ({ children }) => {
  const [
    confirmationState,
    setConfirmationState
  ] = React.useState<ConfirmationOptions | null>(null);

  const openConfirmation = (options: ConfirmationOptions) => {
    setConfirmationState(options);
    return Promise.resolve()
  };

  return (
    <>
      <ConfirmationServiceContext.Provider
        value={openConfirmation}
        children={children}
      />

      <Dialog open={Boolean(confirmationState)} {...confirmationState} />
    </>
  );
};

Now our dialog will be opened once we connect any consumer and call the provided function.

Resolve confirmation

And now we need to somehow deal with closing dialog and getting a callback from the consumers. Here was used Promise based API, but it is possible to make it works using a callback style. In this example, once the user accepted or canceled the alert, your awaiting promise will be resolved or rejected.

To do so we need to save Promise's resolving functions and call them on appropriate user action. React's ref is the best place for that.



  const awaitingPromiseRef = React.useRef<{
    resolve: () => void;
    reject: () => void;
  }>();

  const openConfirmation = (options: ConfirmationOptions) => {
    setConfirmationState(options);
    return new Promise((resolve, reject) => {
      // save the promise result to the ref
      awaitingPromiseRef.current = { resolve, reject };
    });
  };

  const handleClose = () => {
    // Mostly always you don't need to handle canceling of alert dialog
    // So shutting up the unhandledPromiseRejection errors 
    if (confirmationState.catchOnCancel && awaitingPromiseRef.current) {
      awaitingPromiseRef.current.reject();
    }

    setConfirmationState(null);
  };

  const handleSubmit = () => {
    if (awaitingPromiseRef.current) {
      awaitingPromiseRef.current.resolve();
    }

    setConfirmationState(null);
  };

That's it! Our dialog machine is almost ready! One thing is left — create a custom hook for better readability



export const useConfirmationService = () =>
  React.useContext(ConfirmationServiceContext);

Customization

You can easily customize dialog content by passing additional variant prop. Just add it to the ConfirmationOptions



export interface ConfirmationOptions {
  variant: "danger" | "info";
  title: string;
  description: string;
}

And render different dialog content as you wish.



<DialogActions>

    {variant === "danger" && (

      <>

        <Button color="primary" onClick={onSubmit}>

          Yes, I agree

        </Button>

        <Button color="primary" onClick={onClose} autoFocus>

          CANCEL

        </Button>

      </>

    )}

<span class="si">{</span><span class="nx">variant</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">info</span><span class="dl">"</span> <span class="o">&amp;&amp;</span> <span class="p">(</span>
  <span class="p">&lt;</span><span class="nc">Button</span> <span class="na">color</span><span class="p">=</span><span class="s">"primary"</span> <span class="na">onClick</span><span class="p">=</span><span class="si">{</span><span class="nx">onSubmit</span><span class="si">}</span><span class="p">&gt;</span>
    OK
  <span class="p">&lt;/</span><span class="nc">Button</span><span class="p">&gt;</span>
<span class="p">)</span><span class="si">}</span>



</DialogActions>

Are you ready?!

Here is the final working example. Feel free to just steal the implementation of ConfirmationService.tsx file if you want to. This is pretty standalone and isolated logic of what we were talking about.

P.S. No burritos were harmed in the making of this article