I Added Prometheus to My Go Job Scraper and It Changed How I Think About Production

I Added Prometheus to My Go Job Scraper and It Changed How I Think About Production

I was building a Go scraper for Google Jobs (https://github.com/Aureliopires186/go-jobs-tracker) and I realized: if it runs in production and nobody can observe it, it doesn't exist.

So I added Prometheus. Here's the exact minimal setup that made my project look senior-level.

1. Why Metrics Matter

Without metrics, you are blind. You don't know:

• How many jobs you scraped?
• Is it failing silently?
• How long does it take?

Prometheus solves that.

1. The Full Implementation

Step A: Create metrics/metrics.go
This is my live file: https://github.com/Aureliopires186/go-jobs-tracker/tree/main/metrics

go
package metrics

import (
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promauto"
)

var (
    JobsScraped = promauto.NewCounterVec(
        prometheus.CounterOpts{
            Name: "jobs_scraped_total",
            Help: "Total number of jobs scraped per engine and status",
        },
        []string{"engine", "status"},
    )

    ScrapeDuration = promauto.NewHistogramVec(
        prometheus.HistogramOpts{
            Name:    "scrape_duration_seconds",
            Help:    "Duration of scrape jobs in seconds",
            Buckets: prometheus.DefBuckets,
        },
        []string{"engine"},
    )

    ErrorsTotal = promauto.NewCounterVec(
        prometheus.CounterOpts{
            Name: "scraper_errors_total",
            Help: "Total scraper errors by type",
        },
        []string{"type"},
    )
)
*Step B: Expose /metrics and /healthz in `main.go`*
package main

import (
    "net/http"
    "time"
    "github.com/prometheus/client_golang/prometheus/promhttp"
    "github.com/Aureliopires186/go-jobs-tracker/metrics"
)

func main() {
    // Start metrics server
    go func() {
        http.Handle("/metrics", promhttp.Handler())
        http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
            w.WriteHeader(200)
            w.Write([]byte("ok"))
        })
        http.ListenAndServe(":2112", nil)
    }()

    // Your scraping logic
    start := time.Now()
    // ... scrape ...
    duration := time.Since(start)

    metrics.JobsScraped.WithLabelValues("google_jobs", "success").Inc()
    metrics.ScrapeDuration.WithLabelValues("google_jobs").Observe(duration.Seconds())
}
*Step C: Install*
go get github.com/prometheus/client_golang@v1.19.0
go mod tidy
go run main.go
curl http://localhost:2112/metrics
You should see jobs_scraped_total
3. Docker + Prometheus Ready

My Dockerfile already exposes 2112.

`prometheus.yml`:
scrape_configs:
    - job_name: 'go-jobs-tracker'
    static_configs:
            - targets: ['localhost:2112']
Now you can create Grafana dashboards with `jobs_scraped_total` and `scrape_duration_seconds`.

4. What I Learned

Senior code is not about more features. It's about observability, health checks, and knowing when your system fails before the client does.

Full code: https://github.com/Aureliopires186/go-jobs-tracker

Follow me:
- LinkedIn: https://www.linkedin.com/in/pires-aur%C3%A9lio-511330385
- Blog: https://blogspotangola.blogspot.com/
- CoderLegion: https://coderlegion.com/user/Pires+Aurlio

If you are building scrapers in Go, add Prometheus on day 1. Your future self will thank you.

github.com/Aureliopires186/go-jobs-tracker/metrics