DEV Community

DEV Community is a community of 552,441 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create new account Log in
loading...

Discussion on: Comparing the same web scraper in Haskell, Python, Go

View post Full discussion

Replies for: I would recommend Colly (github.com/gocolly/colly) to get a better comparison since you are using BeautifulSoup for Python. Both scraper libraries ...

Collapse
yujiri8 profile image
Ryan Westlund Author •

Wow! I didn't know about that library. That does much more for me here than even BeautifulSoup! New&Improved Go version:

package main

import (
    "fmt"

    "github.com/gocolly/colly"
)

func main() {
    var col = colly.NewCollector()
    col.OnHTML("#recentcomments > *:first-child", func(e *colly.HTMLElement) {
        fmt.Println(e.Text)
    })
    col.Visit("https://fakenous.net")
}

That gets it down to about same number of "meaningful" lines as Python. Technically can drop 2 more lines by putting the function inline, but I wouldn't do that IRL.