DEV Community


Discussion on: Comparing the same web scraper in Haskell, Python, Go

yujiri8 profile image
Ryan Westlund Author

Python seems to average about 1.6 seconds. The first run was 3 seconds which is probs because of filesystem caching or TLS resumption. Go is averaging about 1.25 and Haskell about 1.35.

I don't think performance really means much here though, because on such a short program, factors like the time to start the interpreter and parse source code, write to the console, etc, are much more significant than they should be. The Haskell binary dynamically loads 11 system libraries while the Go binary only loads 2 dynamically, and that might account for the speed difference there. I've heard dynamic linking increases startup costs.

kunde21 profile image
Chad Kunde

Compiling the go binary with cgo disabled will reduce startup time even more (zero dynamic libraries), but you're right in that it's a terrible measure of the general strength of a programming language/runtime.

As a FaaS function, the startup/first request time is important. As a long-running service, that time doesn't matter.

LOC is an even worse measurement (worst? possibly) since I think we've all seen some of the abominable obfuscated C or golfed python solutions.

Putting all of the language battles aside, I'd say the python/bs4 and go/colly solutions are the most easily understandable and implementation-ready options. That says way more about the libraries and their API designs than the programming languages in which they're implemented.

Another point for API design over implementation language.

Thread Thread
yujiri8 profile image
Ryan Westlund Author

I know LOC as a blind measurement is problematic but I didn't golf these implementations. I believe LOC of an idiomatic implementation is a reasonable measure. Still flawed of course, but so is every metric.