Bypassing Omio Cloudflare 403s with Virail Structured Data

Data pipelines monitoring intercity transport fares across European routes frequently hit severe anti-bot restrictions. Building a direct collector for Omio (omio.com) often results in immediate platform-level blocking: route queries submitted from cloud infrastructure return Cloudflare 403 challenges across standard IP ranges, regardless of proxy configuration.

When scraping consumer travel platforms directly becomes unviable due to aggressive edge security, target substitution provides a stable alternative. Virail (virail.com) is a multi-modal travel search engine that aggregates train and bus routes across the same corridors. Rather than loading route details via heavy client-side JavaScript applications, Virail embeds complete trip options directly within server-rendered HTML pages using structured schema.org data.

The Virail Train & Bus Trip Scraper extracts these structured route options without needing user authentication, browser session cookies, or specialized residential proxies.

Bypassing Aggressive Route Blocking via Server-Rendered Schemas

Direct requests to Omio often fail because the application relies on heavily guarded API endpoints and client-side challenge solvers. Virail serves pre-rendered schema data directly from its route pages. Because the trip data exists inside the raw markup, a collector can fetch complete options using standard HTTP requests.

The scraper uses Apify's free datacenter proxy group (AUTO) rather than expensive residential networks. Because no authentication or session initialization is required, runs avoid the common failure modes of session expiration and login rate-limiting.

When executing a run, the system extracts the following specific fields for each discovered option:

• transportType: Identifies whether the carrier operates a train or bus.
• operator: The specific transport company (e.g., "Eurostar", "BlaBlaCar Bus").
• departureStation and departureTime: Origin terminal and scheduled departure timestamp.
• arrivalStation and arrivalTime: Destination terminal and scheduled arrival timestamp.
• price and currency: Numeric fare value and the source listing currency.
• sourceUrl: The exact Virail route page URL parsed for the record.
• scrapedAt: ISO timestamp indicating when the data point was captured.

Fields that lack data for a given carrier or route are omitted from the resulting dataset item rather than populated with null values.

Configuring Route and Modal Filters

The scraper accepts JSON inputs to scope the search by location, transport mode, and cost threshold.

To run a multi-modal search across both rail and coach options while ordering results by price, pass the origin, destination, transport, and sortBy parameters in your payload.

{
  "origin": "Berlin",
  "destination": "Amsterdam",
  "transport": "both",
  "sortBy": "priceLowToHigh",
  "maxItems": 30
}

If your pipeline only requires options beneath a specific budget constraint, set the maxPrice property. The filter removes options exceeding this numerical threshold directly during processing.

{
  "origin": "Madrid",
  "destination": "Barcelona",
  "transport": "train",
  "maxPrice": 50,
  "maxItems": 20
}

The transport field accepts train, bus, or both. The sortBy field supports recommended (default site sorting), priceLowToHigh, or departureTime. The maxItems integer acts as a hard ceiling between 1 and 200 items.

Step-by-Step Execution via Apify Platform

Running the actor to capture live fare data involves four execution steps:

1. Define the route parameters: Identify the exact city names for origin and destination. Use standard well-known city names (e.g., "Paris" rather than a localized suburb name) to ensure Virail's internal location resolver matches the request.
2. Set execution payload: Configure the JSON payload with your target constraints, such as transport selection and maxItems.
3. Execute the Actor run: Trigger the actor. The task starts a compute instance that sends requests through the standard AUTO proxy group to fetch the matching Virail route page.
4. Export dataset output: Fetch the parsed records from the default Apify dataset. Each item contains recordType: "trip" alongside the trip details.

To run this pipeline headlessly using Python, invoke the Actor via the Apify API client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_KEY")

run_input = {
    "origin": "London",
    "destination": "Paris",
    "transport": "train",
    "sortBy": "priceLowToHigh",
    "maxItems": 20,
}

run = client.actor("crawlerbros/omio-scraper").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['operator']} - {item['price']} {item['currency']} at {item['departureTime']}")

Calculating Execution Costs for High-Volume Ingestion

Financial predictability is critical when running recurring data ingestion jobs. This actor operates strictly under Apify's Pay-Per-Event billing model. You are charged only for explicit platform events during a job execution:

• Actor Start (apify-actor-start): Billed at $0.005 per GB of memory allocated to the run upon initialization.
• Result Dataset Items (apify-default-dataset-item): Billed per returned record at the base event rate of $0.005 per result.

For standard usage on the base event rate, an ingestion run configured with 1 GB of allocated memory that extracts 20 trip items costs $0.005 for the actor start, plus 20 × $0.005 ($0.10) for the returned results, totaling $0.105 for the job.

Volume tiers automatically lower the per-result event price for high-throughput workloads:

• FREE: $0.005 per result
• BRONZE: $0.00433 per result
• SILVER: $0.00367 per result
• GOLD: $0.003 per result
• PLATINUM: $0.003 per result
• DIAMOND: $0.003 per result

Because you are charged per returned dataset item, setting a strict maxItems integer directly bounds the maximum event cost of any single execution.

Limitations and Operational Constraints

This approach is optimized for route-level discovery and fare tracking, but it does have specific functional boundaries.

The scraper reads aggregated summaries directly from Virail search result pages; it does not navigate into third-party operator checkout flows or interact with downstream booking APIs. Consequently, it cannot complete ticket purchases, verify seat maps, or capture live real-time inventory changes that occur after Virail's aggregate cache updates. If a query returns zero items, the city string either failed to resolve on Virail's route engine or no direct train/bus connections exist between those two points.

---

If you want to reproduce this, the Actor is Virail Train & Bus Trip Scraper. Read its input schema before the first run -- most failed runs are a missing required field, not a block.