DEV Community

Cover image for Two Scraper Modes That Separate Listing Feeds from Client History
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Two Scraper Modes That Separate Listing Feeds from Client History

Tracking client demand and project budgets across freelance marketplaces requires dealing with distinct data depths. On Guru.com, public listing pages provide high-level summaries of open contracts, but critical risk signals—such as whether an employer has verified payments, their historical total spend, and their outstanding invoice count—are only rendered on specific job detail pages.

Attempting to scrape every detail page upfront slows down ingest pipelines and inflates dataset volume. The Guru Freelance Jobs Scraper addresses this by splitting data collection into two execution modes: broad listing extraction (browse) and targeted detail inspection (byJobId).

The Difference Between Browse and Detail Data Schemas

Guru.com organizes public job listings across seven top-level categories: Programming & Development, Design & Art, Writing & Translation, Sales & Marketing, Administrative & Secretarial, Education & Training, and Engineering & Architecture.

When scraping via mode: "browse", the scraper parses the HTML search and category pages (which default to 20 jobs per page). The returned payload includes surface metadata:

  • jobId, title, and jobUrl
  • category, subcategory, and parsed skills
  • description snippet
  • budgetType (such as Fixed Price or Hourly), budgetRange, hoursPerWeek, and duration
  • locationPreference, postedText, deadline, and quotesReceived
  • employerName and employerCountry
  • scrapedAt

This browse-level data is sufficient for monitoring aggregate job volumes or keyword frequencies. However, lead qualification workflows often require employer payment reliability metrics.

When invoked with mode: "byJobId", the scraper navigates directly to the target URLs provided in the jobUrls array. In addition to the basic fields above, detail mode parses the full description and extracts employer trust attributes:

  • employerPaymentVerified
  • employerFeedback
  • employerTotalSpend
  • employerJobsPosted
  • employerJobsPaid
  • employerPaidInvoices
  • employerOutstandingInvoices

Because empty fields are omitted from the output, downstream consumers can safely parse optional properties without handling null placeholder strings.

Server-Side Categories vs. Client-Side Keyword Filtering

Guru.com serves job listings as server-rendered HTML without requiring sessions, login credentials, or cookies. However, its search architecture handles parameters differently:

  1. Category and Subcategory filtering are true server-side filters. Passing category: "programming-development" and subcategory: "Web Development & Design" instructs the scraper to request only those specific directory endpoints.
  2. Keyword filtering is evaluated on the client side in Guru's native web interface. Within the scraper, the keyword input acts as a best-effort client-side post-filter against the fetched pages, checking against the title, description, and skills.

If a run applies a narrow keyword filter to a small page sample, it may return zero items even if matching jobs exist deeper in the marketplace. To collect higher volumes of keyword-specific records, broaden the category scope or increase maxPages (up to 50 listing pages) so the post-filter evaluates a wider pool of raw items.

This approach does not scrape private client contact information or direct proposals, as it only extracts public marketplace data.

Setting Up a Two-Stage Extraction Workflow

A common design pattern is running a broad category scan on a scheduled cadence, filtering the results for high-value leads, and dispatching a secondary run to pull full employer profiles for that subset.

Step 1: Run Browse Mode for High-Budget Fixed Price Jobs

Define an input configuration targeting fixed-price programming jobs:

{
  "mode": "browse",
  "category": "programming-development",
  "budgetType": "Fixed Price",
  "maxPages": 5,
  "maxItems": 50
}
Enter fullscreen mode Exit fullscreen mode

This request collects up to 50 matching items from the first 5 pages of the category index.

Step 2: Filter URLs and Fetch Client Histories

From the browse mode output, extract the jobUrl values that match specific criteria (for example, jobs with low quotesReceived or specific keywords in skills). Pass those URLs into a second run to retrieve the employer audit metrics:

{
  "mode": "byJobId",
  "jobUrls": [
    "https://www.guru.com/jobs/dentist-website-in-wordpress/2119938"
  ]
}
Enter fullscreen mode Exit fullscreen mode
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

# Execute detail crawl on target job URLs
run_input = {
    "mode": "byJobId",
    "jobUrls": [
        "https://www.guru.com/jobs/dentist-website-in-wordpress/2119938"
    ]
}

run = client.actor("crawlerbros/guru-jobs-scraper").call(run_input=run_input)

# Fetch dataset items containing full employer stats
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"Job: {item.get('title')}")
    print(f"Verified: {item.get('employerPaymentVerified')}")
    print(f"Total Spend: {item.get('employerTotalSpend')}")
    print(f"Paid Invoices: {item.get('employerPaidInvoices')}")
Enter fullscreen mode Exit fullscreen mode

If Guru.com temporarily rate-limits direct requests during high-frequency runs, the actor automatically switches to the free Apify AUTO datacenter proxy configuration to maintain connection reliability.

Cost Structure and Billing Events

The Guru Freelance Jobs Scraper operates on a PAY_PER_EVENT pricing model. Runs are billed based on two specific event types:

  1. Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run, charged once when the run begins.
  2. Result (apify-default-dataset-item): $0.005 per emitted item written to the default dataset at the FREE tier.

For higher volumes, the per-result event price scales across volume tiers:

  • FREE: $0.005 per result
  • BRONZE: $0.00433 per result
  • SILVER: $0.00367 per result
  • GOLD, PLATINUM, DIAMOND: $0.003 per result

There are no additional subscription-tier fees or separate compute calculations outside these specific per-event rates. A batch run of 200 listing results at 1 GB memory on the FREE tier incurs one $0.005 start event plus 200 result events at $0.005 ($1.000), totaling $1.005.

Evaluating whether to ingest browse items directly or chain them into byJobId requests depends on whether your downstream pipeline requires employer spend validation before acting on a lead.


Runs in this article used Guru Freelance Jobs Scraper. Its README is the reference for input fields and output structure; this post is only one path through them.

Prices quoted above are this Actor's published pay-per-event rates on the Apify Store, read from the Apify platform API on 2026-09-19. Check the Actor page for the current rates.

Top comments (0)