Find Social Media Profiles by Username with Python

The Account Takeover and Lead Enrichment Identity Problem

Engineering teams building lead scoring, brand verification, or fraud prevention services frequently need to map a single handle across dozens of third-party networks. Given a username from an onboarding form or a security audit list, the requirement is straightforward: discover where that exact handle is actively registered, filter out false positive matches, and normalize the output into an internal relational store.

Writing and maintaining 400 distinct scrapers internally is an operational sinkhole. Every platform updates its DOM, modifies response headers, and implements aggressive rate limiting against routine profile discovery. The Social Media Finder Actor on Apify addresses this by exposing a uniform interface: you submit a list of target usernames, choose optional platform filters, and collect normalized discovery records.

The real engineering challenge begins once you move beyond manual console testing. When integrating this lookup into an automated pipeline, you have to account for synchronous execution timeouts, API schema nuances, wildcard variations that inflate output records, and dataset pagination. This guide provides a complete, production-grade implementation that triggers execution, polls for completion, pages the output dataset, filters invalid statuses, and writes verified profiles directly to SQLite.

Checked against the Actor's input schema and Apify docs on 2026-09-09.

Exact Input Configuration for Targeted Platform Checks

The Actor expects an input JSON object with three fields: queries, platforms, and maxResults. The queries property is required and accepts an array of usernames. The schema supports a wildcard token {?}, which automatically expands into three common separator characters: underscore, hyphen, and period. For instance, passing alex{?}smith instructs the crawler to check alex_smith, alex-smith, and alex.smith.

The platforms array is optional. Leaving it empty forces a check against the entire database of 400+ platforms. In production pipelines, doing this for large batches of usernames is inefficient if your application only cares about developer and professional networks. Pass an explicit array of platform names matching the supported list (such as GitHub, GitLab, Twitter, Reddit, LinkedIn, and YouTube) to keep execution times fast.

When invoking Actors via API, never rely on console defaults. In the Apify platform, fields defined with prefill in the schema populate the Console UI during manual runs, but they are completely ignored when initiating a run through raw API endpoints. Only fields with an explicit default apply automatically. You must always submit your exact configuration payload explicitly in the request body.

{
  "queries": [
    "johndoe",
    "jane{?}smith"
  ],
  "platforms": [
    "GitHub",
    "Twitter",
    "Reddit",
    "LinkedIn",
    "YouTube"
  ],
  "maxResults": 100
}

The output dataset items conform to the Actor's output schema, which includes four core fields per inspected platform:

[
  {
    "username": "johndoe",
    "platform": "GitHub",
    "url": "https://github.com/johndoe",
    "status": "Found"
  },
  {
    "username": "johndoe",
    "platform": "Twitter",
    "url": "https://twitter.com/johndoe",
    "status": "Not Found"
  }
]

Notice that the Actor outputs items with a status of "Not Found" when a profile does not exist. Your ingest pipeline must account for these rows rather than assuming every returned record represents an active public profile.

Why Does the Synchronous Apify Run Endpoint Return HTTP 408?

The synchronous run endpoint returns HTTP 408 because execution exceeded the 300-second hard gateway timeout. Checking hundreds of platforms per username requires multiple sequential or batched network calls, which routinely pushes total runtime beyond five minutes. To avoid dropped connections, you must trigger an asynchronous run via POST and poll the status endpoint until the run completes.

Calling the synchronous endpoint POST /v2/acts/crawlerbros~social-media-finder/run-sync-get-dataset-items works reliably only when searching one or two usernames across a tiny platform subset. As soon as you scale your queries array or leave platforms unrestricted, the HTTP socket stays open until the platform gateway kills it at five minutes.

To build an ingestion pipeline that does not collapse under network drops or long execution queues, you must decouple run initiation from data extraction:

1. Post the input configuration to /v2/acts/crawlerbros~social-media-finder/runs.
2. Capture the run ID and default dataset ID from the returned JSON response.
3. Poll /v2/actor-runs/{runId} until status transitions to SUCCEEDED.
4. Stream dataset items via the dataset API using pagination query parameters.

To prevent runaway costs on orphaned jobs, supply the maxTotalChargeUsd query parameter when triggering the run. When this cap trips, the Apify platform terminates container execution. Keep in mind that container termination is not instantaneous; resources consume a small amount of compute during shutdown, but the cap guarantees a hard ceiling on credit consumption.

import os
import time
import requests

APIFY_TOKEN = os.environ["APIFY_TOKEN"]
ACTOR_ID = "crawlerbros~social-media-finder"

headers = {
    "Authorization": f"Bearer {APIFY_TOKEN}",
    "Content-Type": "application/json"
}

run_payload = {
    "queries": ["octocat", "torvalds{?}linux"],
    "platforms": ["GitHub", "GitLab", "Reddit", "Medium", "YouTube"],
    "maxResults": 50
}

# Initiate asynchronous run with a protective spend cap
run_url = f"https://api.apify.com/v2/acts/{ACTOR_ID}/runs?maxTotalChargeUsd=2.0"
response = requests.post(run_url, json=run_payload, headers=headers)
response.raise_for_status()

run_data = response.json()["data"]
run_id = run_data["id"]
default_dataset_id = run_data["defaultDatasetId"]
print(f"Run initiated: {run_id}, writing to dataset: {default_dataset_id}")

Polling the Run and Paging Dataset Records in Python

Once the run is dispatched, poll the run endpoint at reasonable intervals. Checking every five seconds keeps API traffic well within platform rate limits while preventing stale pipeline delays.

Once the run finishes with SUCCEEDED, you fetch the records. Do not attempt to download the entire dataset in a single HTTP GET request if you are running multi-thousand record batches. Apify imposes storage rate limits of 60 requests per second per storage object and returns large payloads in memory-heavy JSON arrays. Instead, page through the dataset using the offset and limit query parameters.

def wait_for_run_completion(run_id: str, poll_interval: int = 5) -> None:
    poll_url = f"https://api.apify.com/v2/actor-runs/{run_id}"
    while True:
        res = requests.get(poll_url, headers=headers)
        res.raise_for_status()
        status = res.json()["data"]["status"]
        print(f"Run status: {status}")

        if status in ("SUCCEEDED", "FAILED", "ABORTED", "TIMED-OUT"):
            if status != "SUCCEEDED":
                raise RuntimeError(f"Actor run ended with status: {status}")
            break
        time.sleep(poll_interval)

def fetch_dataset_page(dataset_id: str, offset: int, limit: int) -> list:
    dataset_url = f"https://api.apify.com/v2/datasets/{dataset_id}/items"
    params = {
        "offset": offset,
        "limit": limit,
        "clean": 1
    }
    res = requests.get(dataset_url, headers=headers, params=params)
    res.raise_for_status()
    return res.json()

The clean=1 parameter strips empty or hidden debug records, returning only valid rows generated by the crawler.

How Do You Filter Out Missing Accounts and False Positives?

You filter out missing profiles by checking that the status field equals Found rather than Not Found. The Actor inspects public HTTP response headers across platform targets and includes unassigned handles in the output dataset. Inspecting the status string and discarding unverified endpoints prevents populating your internal database with dead links or unclaimed profile pages.

In addition to filtering out "Not Found" statuses, network-level anomalies can occasionally return malformed URLs or unexpected platform strings. Every production ETL step should validate that the URL scheme is valid and that the discovered entity contains non-empty strings for both username and platform.

Here is a resilient parsing and validation filter:

from urllib.parse import urlparse
from typing import Dict, Any, Optional

def clean_and_validate_record(item: Dict[str, Any]) -> Optional[Dict[str, str]]:
    status = item.get("status", "").strip()
    if status != "Found":
        return None

    username = item.get("username", "").strip()
    platform = item.get("platform", "").strip()
    url = item.get("url", "").strip()

    if not username or not platform or not url:
        return None

    parsed = urlparse(url)
    if parsed.scheme not in ("http", "https") or not parsed.netloc:
        return None

    return {
        "username": username,
        "platform": platform,
        "url": url
    }

This simple transformation guarantees that failed checks and empty profile detections are discarded before any write operation touches storage.

Landing Validated Profiles into SQLite

To finalize the end-to-end task, we write our clean, validated records into an SQLite database. In an internal microservice, you will typically ingest these rows into PostgreSQL or Snowflake, but SQLite provides an identical transactional contract with zero infrastructure dependencies.

We define a table named social_profiles with a compound unique constraint on (username, platform). This ensures repeated pipeline executions are idempotent: running the pipeline daily or re-running a failed batch updates existing profile URLs without throwing primary key errors or generating duplicate rows.

import sqlite3
from typing import List, Dict

def init_database(db_path: str = "social_identities.db") -> sqlite3.Connection:
    conn = sqlite3.connect(db_path)
    with conn:
        conn.execute("""
            CREATE TABLE IF NOT EXISTS social_profiles (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                username TEXT NOT NULL,
                platform TEXT NOT NULL,
                profile_url TEXT NOT NULL,
                discovered_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
                UNIQUE(username, platform)
            );
        """)
        conn.execute("""
            CREATE INDEX IF NOT EXISTS idx_social_profiles_username 
            ON social_profiles(username);
        """)
    return conn

def persist_profiles(conn: sqlite3.Connection, profiles: List[Dict[str, str]]) -> int:
    query = """
        INSERT INTO social_profiles (username, platform, profile_url)
        VALUES (:username, :platform, :url)
        ON CONFLICT(username, platform) DO UPDATE SET
            profile_url = excluded.profile_url,
            discovered_at = CURRENT_TIMESTAMP;
    """
    with conn:
        cursor = conn.executemany(query, profiles)
        return cursor.rowcount

Now combine the pagination loop, the validation logic, and the database persistence function into an operational execution script:

def sync_actor_results_to_db(dataset_id: str, db_conn: sqlite3.Connection) -> None:
    offset = 0
    limit = 50
    total_saved = 0

    while True:
        items = fetch_dataset_page(dataset_id, offset, limit)
        if not items:
            break

        valid_records = []
        for item in items:
            cleaned = clean_and_validate_record(item)
            if cleaned:
                valid_records.append(cleaned)

        if valid_records:
            rows_written = persist_profiles(db_conn, valid_records)
            total_saved += rows_written
            print(f"Saved {rows_written} verified profiles from page offset {offset}")

        offset += len(items)

    print(f"Ingestion complete. Total verified profiles stored: {total_saved}")

if __name__ == "__main__":
    wait_for_run_completion(run_id)
    db = init_database()
    sync_actor_results_to_db(default_dataset_id, db)
    db.close()

How Does Wildcard Expansion Impact the Max Results Cap?

Wildcard expansion triples evaluated usernames across every selected platform, causing rapid exhaustion of the configured maxResults limit. Because john{?}doe generates three distinct query strings, searching across four platforms yields twelve potential records instead of four. Setting maxResults without factoring in this expansion multiplier truncates output before evaluating subsequent usernames in your input array.

The Actor processes usernames by expanding {?} into _, -, and .. If you pass five usernames into the queries list and two of them include {?}, your total query count is not five, but nine:

3 static queries = 3 evaluations
2 wildcard queries = 2 * 3 = 6 evaluations
Total = 9 username targets

If you configured platforms with 10 target networks, those 9 evaluations yield 90 total potential checks. If you set "maxResults": 50, the Actor terminates after emitting 50 records. Because the Actor processes queries sequentially, the final usernames in your input array will be completely omitted from the run.

If your objective is to search an entire list of names without clipping:

1. Omit the maxResults key entirely from your payload, or set it strictly higher than the total expected candidate checks.
2. Pre-calculate the expansion factor in Python before submitting the run payload to prevent accidental truncation.

def calculate_expected_record_ceiling(queries: list, platforms: list) -> int:
    platform_count = len(platforms) if platforms else 400
    expanded_query_count = 0

    for q in queries:
        if "{?}" in q:
            expanded_query_count += 3
        else:
            expanded_query_count += 1

    return expanded_query_count * platform_count

input_queries = ["johndoe", "sarah{?}connor", "dev_guru"]
target_platforms = ["GitHub", "Reddit", "Twitter"]

safe_max_results = calculate_expected_record_ceiling(input_queries, target_platforms)
print(f"Recommended minimum maxResults ceiling: {safe_max_results}")
# Evaluates to (1 + 3 + 1) * 3 = 15 records

Edge Cases and Limitations of Username Lookups

When deploying this integration to production, several constraints inherent to social platform scraping must be accounted for:

1. Authentication Gates (Facebook): The social-media-finder Actor explicitly does not support Facebook. Facebook prevents unauthenticated public profile queries by routing standard profile paths through login barriers. Do not include Facebook in your platforms filter; the Actor will either ignore it or return no matches.
2. Strict Exact Matching: Standard username searches are exact. Searching for janedoe will not discover profiles registered under jane_doe, janedoe1, or jane-doe unless you explicitly use the {?} wildcard or submit each variant as a distinct item in the queries array.
3. No Email or Phone Matching: The tool accepts only handles. Passing emails, phone numbers, or real names with spaces will result in malformed URL lookups that return "Not Found" across platforms.
4. Platform Rate Limiting: Certain platforms periodically block scraping nodes using IP-level rate limits. While the Actor verifies links via public HTTP requests, a platform experiencing temporary network mitigation may return a 429 or 403 status code, causing a false negative where a public account exists but is marked "Not Found".
5. Request Queue Concurrency Constraints: If you intend to scale this lookup by wrapping it in custom Apify Actors, remember that an Apify Request Queue can only be processed by one Actor or task run at a time. Multiple runs can write into a shared request queue, but you cannot fan out parallel worker runs across a single queue instance.

Estimating Compute Units and Platform Run Costs

Execution cost on Apify is calculated through Compute Units (CUs), governed by memory allocation and running time:

CU = (memory_mb / 1024) * duration_hours

A task allocated 1024 MB of RAM running for exactly one hour consumes 1 CU. Doubling memory is CU-neutral only for autoscaling runs, which apply strictly to solutions processing multiple tasks or URLs where each task executes for at least 30 seconds.

Pricing tiers determine your effective hourly rate per Compute Unit:

• Free Tier: $0/mo, with CU rates at $0.20 per CU.
• Starter Tier: $19/mo, with CU rates at $0.20 per CU.
• Scale Tier: $199/mo, with CU rates at $0.16 per CU.
• Business Tier: $999/mo, with CU rates at $0.13 per CU.

If your run uses proxy groups, bandwidth charges apply according to your tier:

• Free and Starter: $8/GB for residential proxies.
• Scale: $7.50 for residential proxies.
• Business: $7 for residential proxies.

Datacenter proxy sessions persist for up to 26 hours, whereas residential proxy sessions rotate or die after approximately 30 minutes.

Because the social-media-finder Actor inspects public HTTP endpoints without rendering heavy headless browsers on every check, compute consumption scales primarily with query volume and platform count. Restricting the platforms array directly reduces the number of outgoing HTTP round trips, lowering the execution duration and keeping overall CU burn low.

Production Scheduling and Dataset Retention Rules

Running this enrichment pipeline on an automated cadence requires careful handling of schedule settings and dataset life cycles.

Apify schedules use standard 6-field cron expressions with a minimum execution interval of 10 seconds. However, two platform rules catch teams off guard:

• Prior Run Requirement: An Actor or Actor task must have run successfully at least once before the Apify system permits creating an active schedule for it.
• Default State: Any newly created schedule is initialized in a DISABLED state by default. After provisioning a schedule programmatically or via templates, your automation must explicitly patch the schedule state to enabled: true.

Another critical architectural consideration is dataset retention. Datasets generated by routine runs are unnamed by default. On the Free tier, unnamed storages expire quickly: Apify retains only your 10 most recent runs, keeping them for a maximum of 4 months. On paid tiers, retention expands, but unnamed storages are still purged according to standard tier schedules.

If your database synchronization pipeline crashes midway through extraction, relying on the unnamed dataset to exist days later is risky. To ensure records survive until your downstream workers safely consume them, pass a custom named dataset during processing or copy records to a named dataset immediately after run completion. Named storages are permanently exempt from automatic deletion across all plan tiers:

def create_named_backup(source_dataset_id: str, backup_name: str) -> None:
    # Named datasets are exempt from automatic expiration policies
    named_dataset_url = f"https://api.apify.com/v2/datasets/{backup_name}?token={APIFY_TOKEN}"
    requests.post(named_dataset_url)

    transfer_url = (
        f"https://api.apify.com/v2/datasets/{source_dataset_id}/items"
        f"?token={APIFY_TOKEN}&clean=1"
    )
    items = requests.get(transfer_url).json()

    # Bulk push records to permanent named storage
    target_push_url = f"https://api.apify.com/v2/datasets/{backup_name}/items?token={APIFY_TOKEN}"
    push_res = requests.post(target_push_url, json=items)
    push_res.raise_for_status()
    print(f"Successfully backed up {len(items)} items to named storage: {backup_name}")

By decoupling execution from ingestion, polling asynchronously, applying strict schema validation, and storing verified profiles in an indexed SQLite database, you can build a resilient, scalable identity resolution service without writing hundreds of custom platform parsers.

The Actor's README is the source of truth for its inputs, outputs and limits. Need a hand wiring this into your stack? Email [e-mail]