Rust is SLOW actually? - AOC 2023 day 1 performance comparison

UPDATE: I've stuck a comment below with updated benchmarks after the community helped me to improve my Rust code. Spoiler alert: Rust is wicked fast. WAY faster than the equivalent JavaScript. I've pasted that comment here at the end of the post, in case the comment gets buried someday.

---

I just got through with the Advent of Code 2023 Day 1 challenge, and I wrote about my trials and tribulations in great, long-winded detail. I'm a frontend guy who's used to working in TypeScript, a near-total Rust newb, so I thought it might be interesting to do a little comparison write-up. I ran through the challenge in Rust, then circled back and ported it to JavaScript (running in Node.js). After the lengthy account in my initial post, I figured the comparison would be short and sweet, just some general impressions about what I like about each language, with a fun little benchmark at the end that showed just how much faster Rust was.

Then I ran the benchmarks, and what I saw was shocking enough that I figured I'd better yank the benchmarks out of my comparison article and make a dedicated post in an attempt to get some answers.

Full disclosure: I'm not a benchmarker. When my apps run slow, I do performance testing, but beyond that, I'm usually happy just knowing something is "fast" or "slow." And in my mind, Rust is fast and JavaScript is slow.

But that's not what I'm seeing in these numbers, and I think I need a sanity check.

My test

To test execution speed, I used PowerShell's measure-command (I use Windows, btw -- why are you laughing?). measure-command is a rough equivalent to the time command in Unix.

I compiled the Rust code using cargo build --release, then ran each implementation 5 times and took the average execution speed.

The initial results

Running each, we end up with: JavaScript, avg 39.9254ms; Rust: 18.3171ms. Rust is faster by ~54%.

That makes sense. I don't doubt this result at all.

Let's see if that scales

I then ran each implementation on on a loop (internal to the code, not via the command line) 1000 times and see how it handles prolonged activity. That's an fs::read_to_string() (Rust) or an fs.readFileSync() (JS) followed by processing 1000 strings, 1000 times.

This is where the results get a little sus. JavaScript: 3260.6001ms; Rust: 6564.07532ms. Oof... Rust is more than twice as slow. That doesn't feel right...

I thought perhaps that Rust's file reader implementation just wasn't up to snuff, assuming Node's readFileSync implements some battle-tested C or C++ function, but no... removing file reads seemed to have little affect on either implementation.

I'm not exactly dissatisfied with Rust's performance: it actually scaled pretty well, as far as I understand it. With 1000x the work, Both programs scaled well; with 1000x the work, the Rust code was only ~358x slower. The prolonged activity didn't scale linearly, instead finishing significantly faster that one would expect. This, to me, suggests one of two things:

• There is a hefty invocation cost (this doesn't seem likely - running the program once only cost ~18ms)
• There is a "warm-up" period, with the code executing faster over time

I suppose a third option could be that invocation is slow (say, 17ms); the code executes initially very fast (1ms), and then slows down over the duration of the program's runtime. That seems less likely to me.

Is Node just really fast?

In any case, the JavaScript code was only ~82x slower at 1000x the load. So another possible explanation would be that the Node is doing some kind of magic like caching the file read or certain execution patterns... In which case, highly repetitive tasks could be a rare performance boon for scripting languages like JS.

It's also possible (likely) that, as a Rust newb, there is some kind of inefficiency in my code that is exacerbated by prolonged run time / repetition. That code, in addition to being mostly included in my previous article, can be found on GitHub.

It's also possible that I'm a JavaScript prodigy and have the unique ability to make it run faster than compiled code. /s

So I'm curious, Rustaceans and Node... Node-heads? What could be the cause of this discrepancy between a single execution and execution over time?

---

Getting the Rust right

UPDATE: I've got my answer. Rust is, in fact, WAY faster, ESPECIALLY in the 1000x test. Many thanks to all who commented and helped me get my Rust up to snuff. 😅

I updated my Rust code to use iterators directly (thanks Eduard!), globalize the HashMap instead of recreating it each function call (thanks Mike Stemle!), and generally be smarter about searching for numbers (not doing full string replaces, but stopping my loops as soon as a valid match was found). Where those changes could be ported to the JS version, I did so. Full code on GitHub.

Here are the new results:

Rust single iteration is down to 10.161ms (almost cut in half from the previous 18.317!).

JavaScript single iteration is up to 71.723. It looks like the previous version was more efficient at ~39ms, which just goes to show you that porting code one-to-one between languages as different as Rust and JavaScript isn't always the best practice. While it would be interesting to investigate further, I think it's most fair to compare the fastest working JS code to the fastest working Rust code here, and Rust ends up at ~25% of the JS code's total the execution time.

Rust 1000x iterations is down to an amazing 385.875 ms, 17x faster than before (at ~6 seconds) and only ~38x slower for 1000x the work. 🤯

JavaScript 1000x iterations comes out to 3691.43. This is, not surprisingly, MUCH slower than the Rust 1000x test (by ~100x) and slower than the previous JS version's 1000x test. What IS surprising is that it's not off the previous version by much, with the old code coming in at 3260.6. So despite the difference in single-iteration performance of the JS code from old to new nearly doubling, the 1000x iteration test only ends up being ~400ms slower.

That said, in that 400ms difference, the Rust code could have run it's full 1000 iterations and had a spot of tea.

Comments

[Mike Stemle]

You’re doing a lot more type conversion in Rust than you need to, too. You’re also not checking your inputs in JavaScript like you are in Rust.

I think the biggest thing, though, is that you’re recreating the HashMap<String, &str> for the words with every call of check_for_words(), but you’re using a global in the Node program. Since you’re calling the str.to_string() it isn’t optimizing it out in the compiler.

[Jacob W Runge]

Oh man, good catch. I bet that has a pretty significant impact. I'll give it a go!

I think the check-for_words func is going to end up being unnecessary, and I'll just end up grabbing the first number I see whether a word or a digit. I'll have to update the js for that to be a fair comparison, though! Either way, you're right... that hashmap needs re-scoped.

[Eduard]

First thing I see in the rust code is this:

fn first_or_last_number(input: &str, reverse: bool) -> Option<char> {
    let new_str = match reverse {
        true => input.chars().rev().collect::<String>(),
        false => input.to_string()
    };

    println!("New string: {}", new_str);

    for c in new_str.chars() {
        if c.is_digit(10) {
            return Some(c);
        }
    }

    None
}

There is no need to actually .collect the reversed string, if the last character was a digit, you are still building a possibly thousands of characters long string just to ignore all of them.

You should try using the iterators directly, make the function take a character iterator, then for the last digit just send a reversed iterator.

[Jacob W Runge]

OK, trying this out now. It makes perfect sense that it is more efficient to use the iterators, but I'm running into the problem of mismatched types (which is why I originally wrote the code like this): input.chars() returns a Chars iterator, but input.chars().rev() returns a Rev<Chars>> iterator.

So it seems like the tradeoff is duplicating code (if reverse, for c in input.chars() and again else for c in input.chars().rev()) to get optimal performance. That's what I'll go with, but is there no way to make it a little DRYer? (I'm guessing the answer is a custom macro? Not sure I'm ready to tackle that!)

[Eduard]

I believe you could make your function generic over a char iterator, which Rev implements: doc.rust-lang.org/std/iter/struct....

[Jacob W Runge]

Ah, thanks! I couldn't find it for some reason.

[Jacob W Runge]

Thanks! I'm going back through this code tonight, so I'll give this a go.

[Yeom suyun]

No matter how optimized the code is, if Rust was much faster than JS in a single execution, the noteworthy aspect is likely that JS performed exceptionally well in repeated executions.
This phenomenon could be attributed to hot code optimization and garbage collection.
Though not an expert, I believe JS has features to optimize frequently executed functions, and if GC doesn't run, continuously allocating memory might result in faster measurements than actual performance.

[Jacob W Runge]

Thanks, Yeom! This was really at the heart of why I was so confused. Node or JS itself has to be doing something like this to achieve the result it does. That also explains (at least partially) why the JS code is so much faster for 1000x the load than a single execution.

EDIT: The commenter above pointing out the HashMap being re-created each loop has probably identified the key issue.

Still really curious why the Rust code doesn't scale linearly. It must be doing something similar, but without relying on the garbage collector?

[Valeria]

Interesting 🧐 The first thing that comes into my mind is that Rust shines in cases where garbage collector creates a bottleneck: for example when you store a large amount of data in memory and change it rapidly. In this case you probably have one constant variable and no garbage to collect.

JavaScript indeed has a “cache” for operations: I’ve seen similar results with WASM vs JavaScript comparison.

Once again, it doesn’t mean that JavaScript is faster than Rust, it simply means that in this specific scenario one tool is suited better than the other.

Some years ago I’ve read an article on switching from Go to Rust by Discord and I think this is a great use case where Rust would outperform anything else.

[Jacob W Runge]

Cool -- I never knew about JS's operations cache. I'll need to look into that more!

A comment above pointed out that I was using a global map in the JS, but recreating the map each time in Rust. I'm updating the code and re-running benchmarks, so we'll see how it shakes out! I'll update the post with results.

[Jacob W Runge]

UPDATE: I've got my answer. Rust is, in fact, WAY faster, ESPECIALLY in the 1000x test. Many thanks to all who commented and helped me get my Rust up to snuff. 😅

I updated my Rust code to use iterators directly (thanks Eduard!), globalize the HashMap instead of recreating it each function call (thanks Mike Stemle!), and generally be smarter about searching for numbers (not doing full string replaces, but stopping my loops as soon as a valid match was found). Where those changes could be ported to the JS version, I did so. Full code on GitHub.

Here are the new results:

Rust single iteration is down to 10.161ms (almost cut in half from the previous 18.317!).

JavaScript single iteration is up to 71.723. It looks like the previous version was more efficient at ~39ms, which just goes to show you that porting code one-to-one between languages as different as Rust and JavaScript isn't always the best practice. While it would be interesting to investigate further, I think it's most fair to compare the fastest working JS code to the fastest working Rust code here, and Rust ends up at ~25% of the JS code's total the execution time.

Rust 1000x iterations is down to an amazing 385.875 ms, 17x faster than before (at ~6 seconds) and only ~38x slower for 1000x the work. 🤯

JavaScript 1000x iterations comes out to 3691.43. This is, not surprisingly, MUCH slower than the Rust 1000x test (by ~100x) and slower than the previous JS version's 1000x test. What IS surprising is that it's not off the previous version by much, with the old code coming in at 3260.6. So despite the difference in single-iteration performance of the JS code from old to new nearly doubling, the 1000x iteration test only ends up being ~400ms slower.

That said, in that 400ms difference, the Rust code could have run it's full 1000 iterations and had a spot of tea.

[Mike Stemle]

It’s marvelous to watch you dig in here and challenge yourself to understand with the help of the community. Solidarity.

[Jacob W Runge]

Thanks so much! Appreciate the help, and your time!