DEV Community

Cover image for Bypassing Unsplash Search Limits with Topic and Collection Scraping
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Bypassing Unsplash Search Limits with Topic and Collection Scraping

Building image ingestion pipelines for computer vision datasets, UI placeholder generators, or digital asset catalogs usually starts with a search for royalty-free photography. The official Unsplash API imposes strict developer review processes and tight hourly rate limits. When you need to harvest structured photo metadata at scale—including direct asset URLs, native pixel dimensions, and explicit photographer attribution—registering a formal developer application introduces unnecessary administrative friction.

The Unsplash Photo Scraper provides programmatic extraction directly from Unsplash's public web catalog. It operates without an API key, session cookies, or user login, outputting clean structured records while handling pagination and challenge pages.

Understanding Unsplash's 60-Photo Search Cap

When querying Unsplash anonymously via web search, the platform enforces a hard ceiling: public search endpoints drop their pagination controls after roughly 60 photos. If you configure a pipeline strictly around search terms, you will hit this boundary regardless of your requested item cap.

To extract hundreds of related images in a single run, you need to route requests through different ingestion paths:

  • mode: "byTopic": Queries Unsplash's curated editorial topics (such as wallpapers, nature, or architecture).
  • mode: "byCollection": Scrapes user-curated galleries via their numeric collectionId (for example, 1065396).
  • mode: "byUser": Pulls a specific photographer's full portfolio using their username.

Unlike public keyword searches, topic, collection, and user endpoints are not restricted to the 60-result limit. They paginate continuously until the collection is fully extracted or your configured item limit is reached.

{
  "mode": "byTopic",
  "topicSlug": "nature",
  "orientation": "landscape",
  "freeOnly": true,
  "maxItems": 150
}
Enter fullscreen mode Exit fullscreen mode

Extracting Dimension-Accurate Image Metadata

Scraping image references often yields incomplete records where dimensions must be calculated after downloading the asset. This scraper inspects the native image metadata directly, emitting pre-calculated fields with every record.

Every returned item includes:

  • id: The unique 11-character photo identifier.
  • slug: The descriptive URL slug.
  • imageUrl and thumbnailUrl: Direct endpoints to full-resolution and preview files.
  • width and height: Native pixel dimensions.
  • orientation: Normalized to landscape, portrait, or squarish computed directly from width and height.
  • photographerUsername, photographerName, and photographerProfileUrl: Required metadata for license attribution compliance.

The actor omits empty fields rather than returning null values. For instance, if a photo belongs to Unsplash+ (marked with isPremium: true), the downloadUrl field is omitted because free direct asset downloads are disabled for paid catalog items.

{
  "id": "fWBZ9r4vO9M",
  "slug": "a-lush-green-forest-filled-with-lots-of-trees-fWBZ9r4vO9M",
  "title": "A lush green forest filled with lots of trees",
  "imageUrl": "https://images.unsplash.com/photo-...",
  "thumbnailUrl": "https://images.unsplash.com/photo-...&w=200",
  "width": 6000,
  "height": 4000,
  "orientation": "landscape",
  "isPremium": false,
  "photographerUsername": "v2osk",
  "photographerName": "v2osk",
  "photographerProfileUrl": "https://unsplash.com/@v2osk",
  "license": "https://unsplash.com/license",
  "downloadUrl": "https://unsplash.com/photos/fWBZ9r4vO9M/download",
  "recordType": "photo",
  "scrapedAt": "2025-02-15T12:00:00.000Z"
}
Enter fullscreen mode Exit fullscreen mode

Configuring Deep Detail Enrichment

By default, the actor scrapes gallery listing views to maximize speed. If your downstream pipeline requires geographic shoot metadata, publication timestamps, content safety flags, or editorial tags, enable the fetchPhotoDetails input parameter.

When fetchPhotoDetails: true is passed, the actor visits the individual page for each photo, appending:

  • locationName: Shoot location name if geotagged by the creator.
  • datePublished: The ISO 8601 creation timestamp.
  • isFamilyFriendly: The platform safety categorization flag.
  • tags: An array of associated editorial keywords.

Enabling deep enrichment adds a separate HTTP request per item, which noticeably extends total execution time. Reserve this setting for pipelines where semantic tagging and publication dates are mandatory.

Execution Walkthrough via API

You can trigger extractions using Apify's REST API or standard HTTP clients.

1. Define the Run Payload

Select your target mode and supply the required identifier. If you need royalty-free assets without commercial stock locks, set freeOnly: true.

{
  "mode": "byCollection",
  "collectionId": "1065396",
  "orientation": "landscape",
  "freeOnly": true,
  "fetchPhotoDetails": false,
  "maxItems": 50
}
Enter fullscreen mode Exit fullscreen mode

2. Run the Actor and Fetch the Dataset

Submit the payload to the actor execution endpoint. Once finished, retrieve the default dataset records directly into your data processing pipeline:

curl -X POST "https://api.apify.com/v2/acts/crawlerbros~unsplash-scraper/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "byUser",
    "username": "v2osk",
    "orientation": "landscape",
    "maxItems": 25
  }'
Enter fullscreen mode Exit fullscreen mode

3. Parse Records in Python

Process the streaming JSON output into pandas or your object storage workflow:

import requests

dataset_id = "YOUR_DATASET_ID"
url = f"https://api.apify.com/v2/datasets/{dataset_id}/items?format=json"

response = requests.get(url)
photos = response.json()

for photo in photos:
    print(f"ID: {photo['id']} | Dimensions: {photo['width']}x{photo['height']}")
    print(f"Download: {photo.get('downloadUrl', 'Locked (Premium)')}")
    print(f"Attribution: {photo.get('photographerName')} ({photo.get('photographerProfileUrl')})")
Enter fullscreen mode Exit fullscreen mode

Execution Pricing

The actor operates on a pay-per-event pricing model:

  • Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run, charged once when the run initializes (minimum 1 event).
  • Result (apify-default-dataset-item): $0.005 per emitted item at the FREE volume tier.

Higher monthly usage tiers decrease the per-result event rate:

  • BRONZE: $0.00433 per item
  • SILVER: $0.00367 per item
  • GOLD / PLATINUM / DIAMOND: $0.003 per item

Extracting 100 free photos on a 1 GB run costs $0.005 for the actor start event plus $0.50 for the 100 dataset result events at the default rate.

Operational Constraints

This scraper does not expose historical performance engagement metrics such as total views, like counts, or download counters, because Unsplash removed these statistics from public listing pages. If your pipeline relies on historical popularity metrics to rank photo quality, scraping the public web interface will not supply that data.


Unsplash Photo Scraper is what these steps drive. The README covers the inputs this article skipped, including the ones that change how much a run costs.

Prices quoted above are this Actor's published pay-per-event rates on the Apify Store, read from the Apify platform API on 2026-09-21. Check the Actor page for the current rates.

Top comments (0)