DEV Community

Big Man
Big Man

Posted on

Build a filtered image dataset from Google Images with Python or Node

Google Images is a good place to collect reference pictures, product shots or training images, but downloading results one by one doesn't scale, and the full-size image URL and dimensions aren't on the results page. This tutorial builds a small, filtered image dataset from Google Images in a few lines of Python or Node. You get a CSV of large landscape photos, each with its full-size URL, width, height and source page.

It uses Google Images Scraper, an Apify actor I built. (An earlier post covers the basics; this one is about filtering.) I ran both scripts below end to end before publishing.

The idea

  1. Ask Google Images for large photos only, using the actor's imageSize and imageType filters (the same filters as Google's Tools menu).
  2. Each result already includes imageWidth and imageHeight, so you can filter by exact size or orientation without downloading anything.
  3. Save what's left to CSV or JSON, with the source page for attribution.

Input

{
  "queries": [
    "lighthouse watercolor"
  ],
  "maxResultsPerQuery": 20,
  "imageSize": "large",
  "imageType": "photo",
  "country": "us"
}
Enter fullscreen mode Exit fullscreen mode

maxResultsPerQuery is the cost control: you pay per image returned ($0.0021 each), so 20 images cost about $0.04.

Python

"""Build a small, filtered image dataset from Google Images.

    pip install "apify-client>=3"
    export APIFY_TOKEN=...   # Apify Console -> Settings -> Integrations
    python build_dataset.py
"""
import csv
import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("daedal_quintuplet/google-images-scraper").call(
    run_input={
        "queries": ["lighthouse watercolor"],
        "maxResultsPerQuery": 20,
        "imageSize": "large",
        "imageType": "photo",
        "country": "us",
    }
)

rows = []
for image in client.dataset(run.default_dataset_id).iterate_items():
    # Keep landscape images at least 1000 px wide.
    if image["imageWidth"] >= 1000 and image["imageWidth"] > image["imageHeight"]:
        rows.append(image)

with open("images.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(
        f, fieldnames=["position", "title", "imageUrl", "imageWidth", "imageHeight", "contentUrl", "source"],
        extrasaction="ignore",
    )
    writer.writeheader()
    writer.writerows(rows)

print(f"Saved {len(rows)} landscape images to images.csv")
Enter fullscreen mode Exit fullscreen mode

In my run, Google returned 20 large photos for "lighthouse watercolor", and 9 were landscape and at least 1,000 px wide. The first rows of images.csv:

position title imageWidth imageHeight source
1 LIGHTHOUSE Watercolor Landscape Print… 2100 1680 Etsy
3 Easy Lighthouse Tutorial Watercolor and Pen Time-lapse 1280 720 YouTube
4 Portland Head Lighthouse Watercolor Painting Tutorial 1200 854 Jennifer Branch Painting Gallery

Results change between runs, as they do on Google itself.

Node.js

// Build a small, filtered image dataset from Google Images.
//   npm install apify-client
//   APIFY_TOKEN=... node build_dataset.mjs
import { writeFileSync } from 'node:fs';
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('daedal_quintuplet/google-images-scraper').call({
    queries: ['lighthouse watercolor'],
    maxResultsPerQuery: 20,
    imageSize: 'large',
    imageType: 'photo',
    country: 'us',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
// Keep landscape images at least 1000 px wide.
const rows = items.filter((i) => i.imageWidth >= 1000 && i.imageWidth > i.imageHeight);

writeFileSync('images.json', JSON.stringify(rows, null, 2));
console.log(`Saved ${rows.length} landscape images to images.json`);
Enter fullscreen mode Exit fullscreen mode

No code: n8n

  1. Add the Apify node with your Apify API token, choose the operation that runs an Actor and waits for it, and set the Actor to daedal_quintuplet/google-images-scraper.
  2. Paste the input JSON above.
  3. If the node returns only the run, add a second Apify node, "Get dataset items", with {{ $json.defaultDatasetId }}.
  4. Add a Filter node: imageWidth ≥ 1000 and imageWidth > imageHeight.
  5. Send the rows to Google Sheets, Airtable or a database.

Fields you can filter on

imageUrl (full size), imageWidth, imageHeight, thumbnailUrl, title, contentUrl (the page the image is on), origin (domain), source (site name), fileSize (when Google shows it), position and query.

A note on reuse

Images belong to their owners. The dataset gives you the source page for every image, so check its licence before you reuse one.

Price

$0.0021 per image ($2.10 per 1,000), paid per result, with no monthly fee. Apify's free plan includes $5 of usage a month.

Links: Store page · GitHub examples

Top comments (0)