I Migrated from Python Scrapers to Go and Search APIs - Complete Guide published: true

•Observation :
Examples using public third-party APIs. Not affiliated with or endorsed by any API provider. All trademarks belong to their respective owners.

I'm Pires Aurélio , a Go/Python developer from Angola. I build scrapers in production and document everything in public.

My work:

• Main scraper (Python/AsyncIO): https://github.com/Aureliopires186/py-websites-scraper
• Go SerpApi Examples: https://github.com/Aureliopires186/go-serpapi-examples
• Jobs Scraper Tracker: https://github.com/Aureliopires186/-serpApi-go-jobs-scraper/issues/3
• Blog - How to build REST API in Go: https://blogspotangola.blogspot.com/2026/09/how-to-create-simple-rest-api-with.html
• LinkedIn: https://www.linkedin.com/in/pires-aur%C3%A9lio-511330385

For 2 years I scraped sites with pure Python/Go. Then I hit Google. Everything broke.

This tutorial is how I solved it with Go + SerpApi.

1. Why My Python Scraper Failed on Google

My repo py-websites-scraper uses asyncio + aiohttp + readability-lxml. Great for blogs.

But for Google Jobs / Shopping:

• CAPTCHA in 10 requests
• 429 blocks
• Selectors change weekly

I was spending 80% time fighting anti-bot, 20% building features. Wrong.

2. Why SerpApi + Go Is The Stack

SerpApi handles proxies, CAPTCHAs, parsing. You get clean JSON.
Go handles concurrency perfectly.

My stack now: Go + SerpApi + Gin REST API (as I teach in my blog)

3. Setup (2 minutes)

bash
go mod init go-serpapi-demo
go get github.com/serpapi/google-search-results-golang
go get github.com/gin-gonic/gin
export SERPAPI_KEY=your_free_key_from_serpapi.com

#. Example 1: Google Jobs - The Code I Use

package main

import (
  "fmt"
  "log"
  "os"

  g "github.com/serpapi/google-search-results-golang"
)

// Só pegamos o que vamos usar - tipo seguro
type Job struct {
  Title string `json:"title"`
  CompanyName string `json:"company_name"`
  Via string `json:"via"`
}

func main() {
  apiKey := os.Getenv("SERPAPI_KEY")
  if apiKey == "" {
    log.Fatal("SERPAPI_KEY not set")
  }

  params := map[string]string{
    "engine": "google_jobs",
    "q": "Go Developer",
    "location": "Remote",
    "hl": "en",
  }

  search := g.NewGoogleSearch(params, apiKey)
  results, err := search.GetJSON()
  if err!= nil {
    log.Fatalf("SerpApi request failed: %v", err)
  }

  // FORMA SEGURA que o Mike pediu
  jobsRaw, ok := results["jobs_results"].([]interface{})
  if!ok {
    log.Printf("No jobs_results in response: %+v", results)
    return
  }

  if len(jobsRaw) == 0 {
    fmt.Println("No jobs found")
    return
  }

  for _, j := range jobsRaw {
    jobMap, ok := j.(map[string]interface{})
    if!ok {
      continue // pula item estranho sem quebrar
    }
    // print seguro
    fmt.Printf("[%v] %v - %v\n", jobMap["via"], jobMap["title"], jobMap["company_name"])
  }
}

#. Example 2: Google Shopping Price Tracker

package main

import (
  "fmt"
  "log"
  "os"

  g "github.com/serpapi/google-search-results-golang"
)

func main() {
  apiKey := os.Getenv("SERPAPI_KEY")
  if apiKey == "" {
    log.Fatal("SERPAPI_KEY not set")
  }

  params := map[string]string{
    "engine": "google_shopping",
    "q": "golang book",
  }

  search := g.NewGoogleSearch(params, apiKey)

  // NUNCA ignore erro - correção do Mike
  results, err := search.GetJSON()
  if err!= nil {
    log.Fatalf("failed to get shopping results: %v", err)
  }

  shoppingResults, ok := results["shopping_results"].([]interface{})
  if!ok {
    log.Printf("No shopping_results found: %+v", results)
    return
  }

  fmt.Printf("Found %d products\n", len(shoppingResults))
  for _, item := range shoppingResults {
    if m, ok := item.(map[string]interface{}); ok {
      fmt.Printf("- %v: %v\n", m["title"], m["price"])
    }
  }
}


By:Pires Aurélio 
Contact: [e-mail] 
https://blogspotangola.blogspot.com/?m=1
https://www.linkedin.com/in/pires-aur%C3%A9lio-511330385