Why `channelUrls` Input Silently Drops Invalid YouTube Handles

As data engineers, our primary goal isn't just to make data flow, but to ensure it flows reliably, predictably, and detectably. When integrating with external tools, especially those that abstract away complex web interactions like scraping, understanding the failure modes is paramount. It's not enough to know what a tool does; we need to understand what it won't do, and how its implied constraints will break our pipelines.

Today, we're diving into the YouTube Email Scraper, an Actor on the Apify platform that promises to extract creator contact emails from YouTube channels, including those hidden on linked social profiles. My focus here isn't a feature tour – you can find that on its Store page. Instead, we'll dissect the unspoken contracts in its input schema, output shape, and underlying platform behaviors to anticipate and defensively handle common failure states in your code.

How does channelUrls handle malformed inputs?

The channelUrls input field, despite its flexible description, can lead to silent data loss if fed incorrectly formatted handles or URLs. Instead of explicitly failing the run, the Actor will simply omit results for unparseable entries, making defensive validation of your input list crucial to prevent unexpected gaps in your dataset.

The youtube-email-scraper Actor states it accepts "YouTube channel URLs in any supported shape," including @handle, /channel/UC..., /c/name, /user/name formats, and plain @handle shortcuts. This flexibility is great, but it doesn't mean any string will parse. Consider an input list with a typo in a handle, like "@MKBHD_typo" instead of "@MKBHD". The Actor won't halt the run or return a global error. Instead, it will simply process the valid channelUrls and produce no output for the malformed one. Your code needs to detect this absence of a result.

Here's an example input, including a deliberate typo:

{
  "channelUrls": [
    "https://www.youtube.com/@Apify",
    "https://www.youtube.com/@MrBeast",
    "@MKBHD_typo",
    "@MKBHD"
  ],
  "followExternalProfiles": true,
  "maxExternalPerChannel": 3,
  "autoProxyFallback": true
}

When this run completes, you would expect three results in your dataset, corresponding to Apify, MrBeast, and MKBHD. The entry for @MKBHD_typo would be silently ignored. To counteract this, your application code should always compare the count of successful results against the count of inputs you provided. If they don't match, you have an unhandled input.

A simple Python check after fetching results might look like this:

from apify_client import ApifyClient

# Initialize the ApifyClient with your API token
client = ApifyClient("YOUR_APIFY_TOKEN")

# Prepare actor input with a known bad handle
actor_input = {
    "channelUrls": [
        "https://www.youtube.com/@Apify",
        "https://www.youtube.com/@MrBeast",
        "@MKBHD_typo", # Intentionally bad
        "@MKBHD"
    ],
    "followExternalProfiles": True,
    "maxExternalPerChannel": 3,
    "autoProxyFallback": True
}

# Run the Actor
print("Starting YouTube Email Scraper run...")
run = client.actor("crawlerbros/youtube-email-scraper").call(run_input=actor_input)

# Fetch results from the default dataset
results = client.dataset(run["defaultDatasetId"]).list_items().items
print(f"Run completed. Processed {len(actor_input['channelUrls'])} inputs, got {len(results)} results.")

# Defensive check: identify missing results
processed_channel_urls = {item.get("channelUrl") for item in results}
original_channel_urls_set = set(actor_input["channelUrls"]) # Simplify for comparison, canonical form not known ahead

# This simplified check doesn't account for canonicalization, but catches outright omissions
# A more robust check would require pre-canonicalizing inputs or matching on channel ID
for url in original_channel_urls_set:
    # A robust check would involve mapping input URLs to canonical channel URLs/IDs
    # For now, we'll just check if *any* result contains part of the input URL, or assume canonical form for now
    if not any(url in res.get("channelUrl", "") or url in res.get("channelHandle", "") for res in results):
        print(f"WARNING: No result found for input: {url}")

This simple check alerts you to discrepancies, prompting further investigation into why certain inputs didn't yield results.

When does maxExternalPerChannel introduce partial results?

The maxExternalPerChannel input field controls the depth of scraping, directly impacting how complete your email discovery might be for a given channel. If a channel links to more external profiles than this cap, you will receive a partial set of emails, and your code needs to anticipate that emails and sources might not contain every possible address.

The youtube-email-scraper follows Instagram, TikTok, and Linktree profiles to find additional emails. The maxExternalPerChannel field, with a default of 3, caps how many of these external profiles are actually fetched. If a YouTube channel prominently displays links to, say, ten different Instagram accounts (perhaps for different brands or projects), setting maxExternalPerChannel to 3 means you're explicitly telling the scraper to ignore seven of those potential sources.

The Actor's output, by design, omits fields with no value. This means if a channel has no emails, the emails and sources fields will simply be absent. However, when maxExternalPerChannel leads to some emails being found but not all possible emails, the emails and sources fields will be present, but they won't be exhaustive.

Consider a scenario where a channel lists six Instagram profiles. If you set maxExternalPerChannel: 2, the output will contain emails from only two of those profiles, if any are found. The externalLinks field in the output will still list all six original links discovered on the YouTube About page. This is your sentinel value: the externalLinks field can contain more items than were actually followed for email extraction, indicating a partial scrape for external profiles.

To detect this, you can compare the length of the externalLinks array in the output with your maxExternalPerChannel input. If len(externalLinks) > maxExternalPerChannel, you know that the scraper deliberately chose not to follow all potential email sources.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

actor_input = {
    "channelUrls": [
        "https://www.youtube.com/@Apify" # Assuming Apify links to > 3 external profiles for demonstration
    ],
    "followExternalProfiles": True,
    "maxExternalPerChannel": 3, # Matches default to highlight the cap
    "autoProxyFallback": True
}

run = client.actor("crawlerbros/youtube-email-scraper").call(run_input=actor_input)

results = client.dataset(run["defaultDatasetId"]).list_items().items

for item in results:
    channel_url = item.get("channelUrl")
    external_links = item.get("externalLinks", [])
    num_external_links_found = len(external_links)
    max_external_per_channel_input = actor_input["maxExternalPerChannel"]

    print(f"Channel: {channel_url}")
    print(f"  External links found on YouTube page: {num_external_links_found}")
    print(f"  maxExternalPerChannel set to: {max_external_per_channel_input}")

    if num_external_links_found > max_external_per_channel_input:
        print(f"  WARNING: Only {max_external_per_channel_input} external profiles were processed, "
              f"but {num_external_links_found} were available. Email results might be partial.")
    else:
        print("  All available external profiles within the cap were processed.")

    emails = item.get("emails", [])
    if emails:
        print(f"  Emails found: {', '.join(emails)}")
    else:
        print("  No emails found for this channel (or none within the scrape scope).")

This code provides insight into how deeply the scraper has searched, allowing you to flag channels where a more comprehensive email search might be warranted, perhaps by rerunning the Actor with a higher maxExternalPerChannel value.

Why does a synchronous run return HTTP 408 after 300 seconds?

Apify's synchronous run endpoint has a hard timeout of 300 seconds. If a youtube-email-scraper run takes longer than this, your client will receive an HTTP 408 "Request Timeout" response, meaning you must switch to asynchronous execution (polling or webhooks) for longer-running tasks.

While the youtube-email-scraper is generally fast ("Each channel typically takes under a second"), scraping many channels or encountering slow-loading external profiles can push a run over the 300-second synchronous limit. This 300-second (5-minute) cap is a platform-wide constraint for synchronous API calls, not specific to this Actor.

When this timeout occurs, your client connection is severed, and you get an HTTP 408. The Actor itself doesn't stop running immediately; it continues processing on the Apify platform until it finishes or hits other limits. However, your client will no longer be "waiting" for it.

To handle runs that might exceed this limit, especially when processing large batches of channelUrls, you must initiate the run asynchronously and then either poll for its status or configure a webhook.

Here's how you might initiate an asynchronous run and poll for completion in Python:

from apify_client import ApifyClient
import time

client = ApifyClient("YOUR_APIFY_TOKEN")

actor_input = {
    "channelUrls": [
        "https://www.youtube.com/@Apify",
        # ... many more URLs that might push runtime over 300s
    ],
    "followExternalProfiles": True,
    "maxExternalPerChannel": 3,
    "autoProxyFallback": True
}

# Start the Actor asynchronously
print("Starting YouTube Email Scraper run asynchronously...")
# The .call() method with `wait_for_finish=False` starts it async
run = client.actor("crawlerbros/youtube-email-scraper").call(run_input=actor_input, wait_for_finish=False)
run_id = run["id"]
print(f"Run {run_id} started. Polling for completion...")

# Poll for run status
status = run["status"]
while status not in ["SUCCEEDED", "FAILED", "ABORTED"]:
    print(f"Run {run_id} is {status}. Waiting 10 seconds...")
    time.sleep(10)
    run = client.run(run_id).get()
    status = run["status"]

print(f"Run {run_id} finished with status: {status}.")

if status == "SUCCEEDED":
    # Fetch results from the default dataset
    results = client.dataset(run["defaultDatasetId"]).list_items().items
    print(f"Fetched {len(results)} results.")
else:
    print(f"Run {run_id} did not succeed. Check logs for details.")

For even more robust, event-driven integration, especially in production systems, consider using webhooks. Apify webhooks allow you to POST to a specified URL when a run completes (or fails), removing the need for client-side polling altogether. This is often preferable for long-running batch jobs.

What happens when autoProxyFallback fails or isn't enough?

Even with autoProxyFallback enabled, repeated blocking or persistent network issues will eventually lead to fetch_failed error records in the output, rather than indefinite retries or a full run failure. Your code must process these explicit error records.

The youtube-email-scraper includes autoProxyFallback, which "transparently retries a fetch via Apify residential proxy when the direct request looks like a block page." This is a powerful feature for reliability. However, it's not a silver bullet. Persistent blocking, issues with the proxy network itself, or genuinely missing pages will still result in channels that cannot be processed.

When a channel's About page or a linked external profile cannot be fetched even after proxy fallback, the Actor emits a specific error record into the dataset instead of a successful channel record. This error record has a type: "youtube_email_scraper_error" and includes a reason and message, such as reason: "fetch_failed" with message: "Could not fetch About page (blocked / offline / not found).".

It's critical to parse your dataset results not just for the channelUrl objects but also for these type: "youtube_email_scraper_error" entries. This allows you to differentiate between channels that had no emails (where emails is simply absent) and channels that couldn't be scraped at all.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

actor_input = {
    "channelUrls": [
        "https://www.youtube.com/@Apify",
        "https://www.youtube.com/@SomeClosedChannel" # Assuming this URL will consistently fail or be blocked
    ],
    "followExternalProfiles": True,
    "maxExternalPerChannel": 3,
    "autoProxyFallback": True
}

run = client.actor("crawlerbros/youtube-email-scraper").call(run_input=actor_input)

results = client.dataset(run["defaultDatasetId"]).list_items().items

successful_results = []
error_records = []

for item in results:
    if item.get("type") == "youtube_email_scraper_error":
        error_records.append(item)
    else:
        successful_results.append(item)

print(f"Total items in dataset: {len(results)}")
print(f"Successful channel records: {len(successful_results)}")
print(f"Error records: {len(error_records)}")

if error_records:
    print("\nDetails of error records:")
    for error_item in error_records:
        print(f"  Channel URL: {error_item.get('channelUrl')}")
        print(f"  Reason: {error_item.get('reason')}")
        print(f"  Message: {error_item.get('message')}")
        print(f"  Scraped At: {error_item.get('scrapedAt')}")

This explicit handling of error records allows you to implement retry logic for fetch_failed channels (perhaps after a delay or with different proxy settings if available), report on scrape failures, or simply filter them out for clean output.

How do proxy session limits impact data consistency?

Residential proxy sessions, used by autoProxyFallback, typically persist for only about 30 minutes. This means that if you're chaining multiple youtube-email-scraper runs or other Actor runs that rely on proxy sessions, you cannot assume session continuity across long-duration or sequential jobs, impacting data consistency and potentially requiring fresh IPs.

The autoProxyFallback mechanism is useful for bypassing temporary blocks. However, it relies on residential proxy sessions. These sessions are relatively short-lived, with Apify residential proxies typically holding an IP address for around 30 minutes. In contrast, datacenter proxies can persist for 26 hours.

Why does this matter? If your workflow involves processing a very large channelUrls list that's broken into multiple sequential youtube-email-scraper runs, or if you're running multiple Actors in parallel from a single client without managing proxy groups, you cannot rely on consistent IP addresses or session stickiness across runs that are spaced apart by more than 30 minutes.

This primarily affects advanced use cases where you might be trying to maintain consistent IP routing for an extended period, or if you're trying to circumvent rate limits that span longer than 30 minutes. For typical single-run usage, the autoProxyFallback handles session rotation transparently. But for complex orchestrations, be aware that each new request, or a request after a session timeout, might use a different residential IP, and you should not build assumptions of IP stickiness into your application logic.

What about data freshness and storage retention?

If your data freshness requirements demand very recent data, and you're running this Actor on a schedule, remember that schedules are created DISABLED by default and the Actor must have run at least once before it can be scheduled. Always enable your schedules and verify the first run.

Unnamed storage objects (datasets, key-value stores, request queues) also expire, so if you rely on historical output, always name your storages or retrieve results within the retention period (e.g., free plan retains only 10 most recent runs for 4 months). This also highlights that the prefill option in input schema is only for the Console UI and not applied to API calls. For API usage, always explicitly pass a complete input dictionary.

How does maxTotalChargeUsd prevent unexpected costs?

The maxTotalChargeUsd query parameter (or ACTOR_MAX_TOTAL_CHARGE_USD environment variable within the Actor) provides a hard financial circuit breaker for your runs. When the accumulated event charges reach this limit, the run terminates, saving you from runaway costs, though it continues to consume platform resources briefly as it winds down.

Understanding costs is crucial for any data pipeline. The youtube-email-scraper is billed on a PAY_PER_EVENT model, meaning its event costs scale with the number of discrete actions it performs. The charged events are:

• result (apify-default-dataset-item): $0.002 per event. This is charged for each successful channel record or error record emitted to the default dataset. Discount tiers apply: FREE $0.002, BRONZE $0.00167, SILVER $0.00133, GOLD $0.001, PLATINUM $0.001, DIAMOND $0.001. The number of result events is directly proportional to the number of input channelUrls you provide, plus any error records generated.
• Actor Start (apify-actor-start): $0.005 per GB of memory allocated to the run. This is a one-time charge per run, scaled by memory.

On top of these event charges, you also pay for Apify platform usage, which includes resources like proxy traffic, storage, and computation time, billed separately at your Apify plan's rates.

The maxTotalChargeUsd parameter is your safety net. You can set it as a query parameter in your API call to client.actor("crawlerbros/youtube-email-scraper").call():

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

actor_input = {
    "channelUrls": [
        "https://www.youtube.com/@Apify",
        # ... many more URLs
    ],
    "followExternalProfiles": True,
    "maxExternalPerChannel": 3,
    "autoProxyFallback": True
}

# Example of setting a maximum charge limit
run = client.actor("crawlerbros/youtube-email-scraper").call(
    run_input=actor_input,
    max_total_charge_usd=0.05 # Example cap, replace with your desired limit
)

# Remember to handle asynchronous runs if your expected runtime exceeds 300s
# ... (polling or webhook logic as shown previously)

If the combined event charges reach your specified limit, the run will terminate gracefully. While it doesn't instantly kill the process, it will stop further expensive operations, preventing unexpected cost overruns. This is particularly important when experimenting with large datasets or when the behavior of external sites is unpredictable, leading to many proxy retries. Always consider setting a reasonable maxTotalChargeUsd for production workflows.

What are the youtube-email-scraper's concrete limitations?

The youtube-email-scraper has several specific limitations that prevent it from finding all possible emails or processing all channels: it cannot extract YouTube's click-to-reveal emails, some Instagram or TikTok profiles may be inaccessible due to login walls or regional restrictions, and certain specialized YouTube channel types may lack a public About page.

These are not general scraping limitations, but specific boundaries of this Actor:

1. Click-to-reveal emails: The Actor cannot extract emails hidden behind YouTube's "View email address" button. This feature requires a logged-in YouTube session, which the Actor explicitly avoids (it's login-free by design). Any emails behind this gate will be missed.
2. Instagram/TikTok login walls or region restrictions: External profile scraping for Instagram and TikTok can occasionally hit login walls or regional access restrictions. When this happens, the scraper will skip that specific bio and proceed, potentially missing emails from those sources. The externalLinks field will still show the link, but sources won't contain instagram_bio or tiktok_bio entries for that URL.
3. Missing About pages: Some custom-branded or music-artist YouTube channels do not have a standard public "About" page. For these, the Actor will emit a parse_failed error record, indicating it couldn't even start extracting channel metadata.
4. Subscriber count granularity: The subscriberCount field reflects YouTube's publicly displayed rounded value (e.g., "12K" becomes 12000). Exact subscriber counts below YouTube's display threshold are not available. This is a representation limitation, not a data extraction failure.
5. Limited external link following: The Actor only follows Instagram, TikTok, and Linktree links for email discovery. Other external links found on the YouTube About page (e.g., Twitter, personal websites) are listed in externalLinks but are not crawled for emails.

Understanding these explicit limitations helps set realistic expectations for the data you can obtain and informs where manual intervention or a multi-tool approach might be necessary.

Checked against the Actor's input schema and Apify docs on 2026-09-27.

The Actor's README is the source of truth for its inputs, outputs and limits. Need a hand wiring this into your stack? Email [e-mail]