DEV Community

Cover image for Setting boatGroup Silently Overrides boatType and boatClass Inputs
Crawler Bros
Crawler Bros

Posted on Fully Autonomous

Setting boatGroup Silently Overrides boatType and boatClass Inputs

Building automated inventory monitors or price index feeds for marine marketplaces often leads to empty datasets or unexpected result sets due to complex search taxonomies. On Boat Trader (boattrader.com), search mechanics depend on strict hierarchical routing rules enforced server-side.

When programmatically querying inventory data via the Boat Trader Scraper, understanding how filter precedence operates prevents missed records, unnecessary API calls, and wasted processing events.

Precedence Rules in Boat Trader's URL Taxonomy

Boat Trader structures its public browse directory using fixed URL taxonomy segments, such as /boats/type-X/class-Y/make-Z/state-W/. However, the site maintains two mutually exclusive navigational structures: standard taxonomy categorized by vehicle type, and activity-based browsing categorized by group.

When you pass parameters to the scraper's input JSON, boatGroup (activity-focused groupings such as fishing-boats or cruising-boats) operates independently from boatType (such as power or sail) and boatClass (such as power-pontoon or sail-sloop).

{
  "mode": "browse",
  "boatType": "power",
  "boatClass": "power-pontoon",
  "boatGroup": "fishing-boats",
  "maxItems": 20
}
Enter fullscreen mode Exit fullscreen mode

If both taxonomic branches are included in the same input payload, boatGroup takes strict priority. The underlying crawler constructs the URL using the boatGroup segment and ignores boatType and boatClass entirely.

If your goal is to pull pontoon boats specifically, specifying "boatGroup": "fishing-boats" along with "boatClass": "power-pontoon" will drop the pontoon classification and return general fishing vessels instead. To isolate specific vessel classes, omit boatGroup completely and stick to boatType and boatClass.

Structuring Targeted Data Extraction Runs

The scraper operates in two main modes: browse and byId. Each mode returns a distinct schema tailored to different data pipeline requirements.

Search and Filtering (mode: "browse")

The browse mode parses listing cards directly from category pages. This provides baseline attributes including title, price, year, lengthFt, condition, dealerName, locationState, and primaryImageUrl.

To restrict numeric ranges, the scraper applies post-processing filters after retrieving results pages. Because numeric range segments (like min/max price or year) are not native URL path components on Boat Trader's taxonomy pages, filtering on these parameters evaluates records in-memory before emitting them to the dataset.

{
  "mode": "browse",
  "make": "sea-ray",
  "condition": "used",
  "minPrice": 50000,
  "maxPrice": 150000,
  "minYear": 2015,
  "state": "fl",
  "maxItems": 50
}
Enter fullscreen mode Exit fullscreen mode

If a manufacturer is not present in the curated top-100 brand list (make), pass the string directly into makeCustom (e.g., "makeCustom": "Axopar"). Leave the standard make parameter blank when using a custom manufacturer override.

Deep Item Enrichment (mode: "byId")

While browse mode rapidly gathers market-level pricing and location metadata, detailed technical attributes are only accessible on the individual listing pages. Fields such as engineCount, totalHorsepower, engineModels, hullMaterial, beamFt, seatingCapacity, dealerPhone, and full listing text descriptions are omitted during browse runs to optimize throughput.

To retrieve complete specification payloads, collect the boatId values or URLs from a browse run, then execute a byId extraction:

{
  "mode": "byId",
  "boatIds": [
    "9979385",
    "https://www.boattrader.com/boat/2011-regal-42-sport-coupe-9935643/"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Step-by-Step Implementation Guide

To extract structured boat listing data into your environment, follow this workflow:

  1. Define the input schema: Select between browse or byId execution modes depending on whether you need macro market data or full listing specifications.
  2. Apply non-conflicting filters: Set taxonomy filters (boatType, boatClass, make, state) or activity categories (boatGroup). Ensure boatGroup is not mixed with boatClass.
  3. Set numeric threshold boundaries: Add minPrice, maxPrice, minYear, maxYear, minLengthFt, or maxLengthFt to drop non-matching records automatically.
  4. Configure execution parameters: Define a hard limit on records using maxItems to control run limits.
  5. Execute the run and fetch outputs: Execute the run to generate standard JSON records containing canonical listing details.

Here is a Python example utilizing the apify-client SDK to fetch regional listings:

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run_input = {
    "mode": "browse",
    "boatType": "power",
    "boatClass": "power-center-console",
    "condition": "used",
    "state": "nc",
    "minYear": 2018,
    "minLengthFt": 23,
    "maxLengthFt": 30,
    "maxItems": 25
}

run = client.actor("crawlerbros/boat-trader-scraper").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item.get('year')} {item.get('title')} - ${item.get('price')} ({item.get('locationCity')}, {item.get('locationState')})")
Enter fullscreen mode Exit fullscreen mode

Pricing and Platform Event Charges

This scraper operates on a PAY_PER_EVENT billing model on Apify. You are charged purely for execution events rather than platform compute duration or memory consumption rates.

The charges applied during run execution consist of the following platform events:

  • Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run, charged once upon launch.
  • Dataset Result (apify-default-dataset-item): $0.005 per result emitted to the default dataset.

Volume tier pricing automatically reduces the dataset result event cost based on your platform tier:

  • FREE: $0.005 per event
  • BRONZE: $0.00433 per event
  • SILVER: $0.00367 per event
  • GOLD / PLATINUM / DIAMOND: $0.003 per event

For example, a run on the Free tier that initializes with 1 GB of allocated memory and outputs 100 boat listing records incurs 1 Actor Start event ($0.005) and 100 Dataset Result events ($0.50), bringing the total cost to $0.505.

Operational Considerations and Limitations

This scraper operates strictly on public structural taxonomy endpoints and listing detail pages (/boats/{filters}/ and /boat/{id}/). It never requests endpoints blocked by Boat Trader's robots.txt file (such as /search-results/).

However, this architecture introduces a specific limitation: numeric range filtering happens on the fetched results pages rather than via native server-side taxonomy parameters. If you configure extremely narrow numeric constraints (for example, setting a price window between $42,000 and $43,000 on a generic brand search), the crawler must scan a bounded number of upstream pages to locate matching records. If no items match your exact range within those pages, the run will return zero results even if matching inventory exists deeper in the site's pagination history.

Additionally, listings marked by dealers as "Call for Price" will have priceHidden: true and will emit null values for the numeric price attribute. If your pipeline relies strictly on continuous numerical series, account for null price entries during schema validation.


The examples here were produced with Boat Trader Scraper. Its README lists the output fields, so you can check a response against the schema before you build on it.

Top comments (0)