DEV Community

Cover image for Tracking Municipal Equipment Auctions Past GovDeals Search Limits
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Tracking Municipal Equipment Auctions Past GovDeals Search Limits

Government agencies, municipalities, and school districts offload retired assets through GovDeals. For heavy machinery distributors, fleet managers, and pricing analysts, these listings contain valuable commercial signals: auction clearing prices, equipment model years, and geographic asset distributions.

Querying this inventory manually creates blind spots. Filtering across thousands of distinct municipal sellers yields unstructured web interfaces, unstandardized item categories, and disparate bid tracking. Attempting to ingest listings via raw HTTP requests frequently runs into edge rate-limiting and session validation layers.

The GovDeals Scraper actor solves this by executing direct queries against the marketplace listings API to extract structured asset attributes, seller identifiers, and live bid states.

Structuring Unstructured Municipal Surplus Data

GovDeals lists everything from utility bucket trucks to office electronics. Manually parsing these public sales requires normalizing disparate metadata fields across different selling agencies.

The scraper maps GovDeals records directly to standardized properties, returning key attributes in every emitted listing:

  • Core identifiers: assetId, accountId, auctionId, eventId, and displayEventId
  • Item specifications: title, description, lotNumber, makeBrand, model, and modelYear
  • Category and format: categoryId, categoryName, auctionTypeId, and auctionTypeName
  • Financial values: currentBid, buyNowPrice, bidIncrement, currencyCode, and reserve state (hasReservePrice, isReserveNotMet)
  • Agency location: sellerName, locationCity, locationState, locationStateName, locationZip, and country
  • Timeline and assets: auctionStartDate, auctionEndDate, timeRemaining, auctionStatus, and imageUrl

Empty fields are omitted directly from the returned output, preventing your ingestion pipelines from choking on arbitrary null objects for categories that lack vehicle-specific fields like modelYear.

Query Modes and Targeted Filtering

The scraper provides three execution modes defined by the mode parameter: search, byCategory, and byState.

1. Keyword-Based Monitoring with Price Ceilings

For equipment procurement, broad searches return excessive unrelated inventory. Combining searchText with numerical bid boundaries keeps downstream datasets relevant.

{
  "mode": "search",
  "searchText": "excavator",
  "minCurrentBid": 5000,
  "maxCurrentBid": 50000,
  "auctionType": "3",
  "auctionStatus": "open",
  "maxItems": 100
}
Enter fullscreen mode Exit fullscreen mode

This configuration drops listings outside the target capital range before emitting records, avoiding the cost of ingesting low-value scrap or high-value multi-unit lots.

2. Monitoring Fleet Refresh Cycles by State and Category

When evaluating fleet lifecycle patterns across specific jurisdictions, the byCategory and byState modes browse GovDeals' native taxonomies directly.

{
  "mode": "byCategory",
  "category": "6",
  "vehicleMake": "Ford",
  "vehicleModelYear": 2017,
  "sortBy": "endingSoonest",
  "maxItems": 50
}
Enter fullscreen mode Exit fullscreen mode

Filtering by vehicleMake and vehicleModelYear extracts structured brand records directly, matching values like Caterpillar, Peterbilt, or Ford without relying on keyword matching inside free-text descriptions.

3. Collecting Historical Baseline Pricing

Setting auctionStatus to closed retrieves finished sales instead of live inventory. This allows data teams to build historical clearing-price distributions across specific regions:

{
  "mode": "byState",
  "state": "TX",
  "auctionStatus": "closed",
  "sortBy": "priceHighToLow",
  "maxItems": 200
}
Enter fullscreen mode Exit fullscreen mode

Running the Scraper with Python

You can trigger a GovDeals scraping run directly in an automated data pipeline using the Apify Python client.

Step 1: Install the client library

pip install apify-client
Enter fullscreen mode Exit fullscreen mode

Step 2: Configure and trigger the run

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run_input = {
    "mode": "search",
    "searchText": "dump truck",
    "auctionStatus": "open",
    "sortBy": "endingSoonest",
    "maxItems": 150
}

# Start the actor and wait for execution to complete
run = client.actor("crawlerbros/govdeals-scraper").call(run_input=run_input)

# Fetch results from the run's default dataset
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items

for item in dataset_items:
    asset_id = item.get("assetId")
    seller = item.get("sellerName")
    current_bid = item.get("currentBid")
    end_date = item.get("auctionEndDateDisplay")
    print(f"[{asset_id}] {seller}: ${current_bid} (Ends: {end_date})")
Enter fullscreen mode Exit fullscreen mode

Step 3: Stream items directly to a database

For continuous monitoring, fetch only newly scraped items by checking scrapedAt timestamps or filtering by assetId before writing to your warehouse.

Execution Pricing

The actor operates under a PAY_PER_EVENT pricing model. Runs are billed per concrete action rather than arbitrary duration units:

  • Actor Start: $0.005 per GB of memory allocated to the run (charged once per execution).
  • Result: $0.005 per listing emitted to the dataset under the FREE tier.

Volume discounts reduce per-result charges on higher tiers:

  • BRONZE: $0.00433 per result
  • SILVER: $0.00367 per result
  • GOLD: $0.003 per result
  • PLATINUM: $0.003 per result
  • DIAMOND: $0.003 per result

Extracting 1,000 open listings on a default run costs $0.005 for the start event plus $5.00 for the results at the standard rate.

System Boundaries

This tool does not submit bids, authenticate accounts, or interact with GovDeals payment gateways; it operates strictly as a read-only extractor for public marketplace listings.

For pipelines monitoring high-frequency price updates in the final minutes of an auction, polling intervals must account for the scraper's hard cap of 2,000 items per single execution.


GovDeals Scraper is what these steps drive. The README covers the inputs this article skipped, including the ones that change how much a run costs.

Prices quoted above are this Actor's published pay-per-event rates on the Apify Store, read from the Apify platform API on 2026-09-17. Check the Actor page for the current rates.

Top comments (0)