DEV Community

Alin Climente
Alin Climente

Posted on

Loading a 600mb+ CSV file is faster in Python than in Go?

Ever wondered how much time it takes for Go to load a CSV file in comparison with the slower Python language?

I downloaded a relatively big csv file from the web (658 MB) to see which one wins.

Let's start with Python and the well known pandas package:
Pandas6sec
Pretty bad isn't it? 6+ seconds! Tried with engine="c" flag, but got same result. It's still good enough, now we have available a lot of pandas goodness.

Let's see in Go:

Go1.5

Nice! Finished in 1.5 seconds -> 21% faster! But, remember pandas offers a lot out of the box, with Go you need to write a lot of code to benefit from those extra goodies.

Is Python slow? Yes, but if the packages implementation you are mostly using are in C/Rust you won't feel it as much.

Now, let's switch pandas with a newer alternative polars:
polars700ms
It's even faster than Go!(yes, because of Rust). We now have a lot of functionalities out of the box (not as much as pandas, but will get there).

Time to learn Rust I guess...

Hostinger image

Get n8n VPS hosting 3x cheaper than a cloud solution

Get fast, easy, secure n8n VPS hosting from $4.99/mo at Hostinger. Automate any workflow using a pre-installed n8n application and no-code customization.

Start now

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay