TikTok Video Scraper GitHub: How to Extract Public Video Metadata with Python

The fastest way to turn a single public TikTok video URL into a stable, JSON-shaped record you can archive, feed into a database, or hand to an AI agent is to combine an open-source TikTok video scraper repository with a small Python normalization layer that flattens fields and appends each result to a JSONL file. This article is for research engineers, social analytics teams, content moderation architects, dataset curators, and AI-agent builders who need a per-video primitive they can call repeatedly without re-writing parsing logic.

The data-scrape/tiktok-video-scraper repository ships a Python reference for extracting a single public video's metadata. Pair it with data-scrape/tiktok-data-scraper-api when your workflow needs to walk many videos behind a hashtag, sound, or creator handle.

TL;DR

• Use data-scrape/tiktok-video-scraper to fetch a single public TikTok video's metadata into a JSON file.
• Define a narrow input up front: one video URL per run. Batch jobs iterate the URL list externally so the scraper stays a one-record primitive.
• Normalize each record into a stable schema with video_url, video_id, creator_handle, caption, hashtags, music_title, music_author, view_count, like_count, comment_count, share_count, published_at, scraped_at, and source_repo.
• Append each normalized record to a JSONL file so the dataset grows by line and can be streamed into pandas, DuckDB, or a downstream queue.
• Combine the per-video primitive with data-scrape/tiktok-data-scraper-api when you also need a hashtag or creator pipeline that walks many videos.
• Capture one representative sample first, validate the field schema against your assumptions, and only then automate at scale.

Why per-video metadata is the right primitive

Most TikTok analysis eventually distills into one question: what does this specific video say, who posted it, and what is the engagement snapshot right now? A scraper that answers that question for a single URL becomes the building block for every higher-level workflow — a creator audit, a hashtag velocity study, a moderation evidence pack, or a context bundle for an AI agent.

A handful of problems recur once you start building:

• Schema drift. Caption delimiters, hashtag rendering, music attribution, and creator display names are formatted inconsistently across videos. Raw JSON rarely has the field names a downstream consumer expects.
• Mixed video types. In-feed videos, share-link variants, and search-result pages can carry the same video URL but different surrounding metadata.
• Numeric counters as strings. Like, view, comment, and share counts are returned as locale-formatted strings such as 15.4K or 1,200. Treat them as strings until you coerce, never as integers in raw form.
• Time zone confusion. published_at may be delivered as a local timestamp without a zone suffix. Always normalize to UTC in your output and document the conversion.
• Rate of layout change. TikTok ships front-end changes frequently. Anything that consumes raw HTML or relies on selector paths needs a re-validation job.
• Compliance scope. Public video URLs are in scope. Private accounts, follower-only live replays, and direct messages are not.

This article takes the middle path: start with an open-source per-video reference, then add a small, auditable Python layer for normalization and JSONL append.

What the verified repositories provide

data-scrape/tiktok-video-scraper is a Python reference built around extracting a single public video into a JSON document. It exposes a scraper.py entry point, a requirements.txt, an examples/ folder, and a README. Read the README for the current setup steps and supported arguments before running it — the names of CLI flags and the shape of the response may differ from the illustrative examples in this article.

data-scrape/tiktok-data-scraper-api is a sibling repository that targets hashtag and creator discovery rather than single-video resolution. When your analysis needs to iterate over hundreds or thousands of videos behind one creator handle or hashtag, you keep the same normalization layer and only swap the source adapter.

The data-scrape profile hosts the rest of the open-source repositories that share a similar pattern, including adjacent per-record → JSONL normalization pipelines for other domains.

Pipeline design

A maintainable per-video extraction pipeline has five stages: capture raw JSON for a single public video URL, normalize the response into a stable field schema, coerce numeric and date fields into typed columns, append the record to a JSONL file, and surface a small validation job that flags schema drift. Keep the raw JSON as an artifact alongside the normalized record so any future schema change can be replayed against the original payload.

Setup and configuration

Clone the repository and install dependencies inside a virtual environment so packages do not collide with system Python:

git clone https://github.com/data-scrape/tiktok-video-scraper.git
cd tiktok-video-scraper
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Set configuration through environment variables instead of hard-coding URLs or pacing. The example below forwards a single video URL, an output path, and a polite per-request delay to the scraper's CLI. Adjust variable names to match the current version of scraper.py; never commit a real target URL or proxy configuration to source control.

export TIKTOK_TARGET_URL="https://www.tiktok.com/@examplecreator/video/7280000000000000001"
export TIKTOK_OUTPUT_FILE="raw_video.json"
export TIKTOK_REQUEST_DELAY=2.0
python scraper.py \
  --url "$TIKTOK_TARGET_URL" \
  --output "$TIKTOK_OUTPUT_FILE" \
  --delay "$TIKTOK_REQUEST_DELAY"

Treat the first call as a smoke test. If the JSON contains every field your downstream code expects, the same command embedded in a loop is your batch driver. If a field is missing or renamed, update the normalization step rather than chasing one-off branches inside the consumer.

Normalization: From raw JSON to a stable JSONL record

The script below reads raw_video.json, flattens the response into a stable schema, coerces numeric and date fields, and appends one JSON line to videos.jsonl. Re-running it against a new URL grows the same file incrementally, which is the property you want for streaming analytics and audit trails.

import datetime as dt
import json
import os
import pathlib
import re

RAW_PATH = pathlib.Path(os.environ.get("TIKTOK_OUTPUT_FILE", "raw_video.json"))
JSONL_PATH = pathlib.Path(os.environ.get("TIKTOK_JSONL_FILE", "videos.jsonl"))
SOURCE_REPO = "tiktok-video-scraper"

SCHEMA = (
    "video_url", "video_id", "creator_handle", "creator_display_name",
    "caption", "hashtags", "music_title", "music_author",
    "view_count", "like_count", "comment_count", "share_count",
    "published_at", "scraped_at", "source_repo",
)


def normalize_tags(raw_tags):
    """Lowercase, strip leading '#', drop empty strings."""
    if not raw_tags:
        return []
    cleaned = []
    for tag in raw_tags:
        if not tag:
            continue
        cleaned.append(tag.lstrip("#").strip().lower())
    return [t for t in cleaned if t]


def parse_int(value):
    """Strip 'K', 'M', commas, and spaces; convert to int when possible."""
    if value is None:
        return None
    digits = re.sub(r"[^\d]", "", str(value))
    return int(digits) if digits else None


def normalize_record(raw):
    creator = raw.get("creator") or {}
    music = raw.get("music") or {}
    return {
        "video_url": raw.get("url") or "",
        "video_id": raw.get("video_id") or "",
        "creator_handle": creator.get("handle") or "",
        "creator_display_name": creator.get("display_name") or "",
        "caption": raw.get("caption") or "",
        "hashtags": normalize_tags(raw.get("hashtags")),
        "music_title": music.get("title") or "",
        "music_author": music.get("author") or "",
        "view_count": parse_int(raw.get("view_count")),
        "like_count": parse_int(raw.get("like_count")),
        "comment_count": parse_int(raw.get("comment_count")),
        "share_count": parse_int(raw.get("share_count")),
        "published_at": raw.get("published_at") or "",
        "scraped_at": dt.datetime.now(dt.timezone.utc).isoformat(timespec="seconds"),
        "source_repo": SOURCE_REPO,
    }


def main():
    raw = json.loads(RAW_PATH.read_text(encoding="utf-8"))
    record = normalize_record(raw)
    with JSONL_PATH.open("a", encoding="utf-8") as fh:
        fh.write(json.dumps(record, ensure_ascii=False) + "\n")
    print(f"Appended {record['video_url']} to {JSONL_PATH}")


if __name__ == "__main__":
    main()

Three details matter here. First, normalize_tags defends against hashtag drift by lowercasing and stripping the leading hash, so joins against a hashtag list stay stable across runs. Second, parse_int strips locale-formatted suffixes such as K and M, returning None for unknown values so downstream analytics can identify missing data instead of silently coercing it to 0. Third, scraped_at is recorded in UTC so joins across runs and time zones are reproducible.

Representative output

After running the smoke test and the normalization script, videos.jsonl contains one line that looks like the example below. Treat this as a representative shape, not real-time data; verify fields against the current scraper response before wiring the schema into production.

{
  "video_url": "https://www.tiktok.com/@examplecreator/video/7280000000000000001",
  "video_id": "7280000000000000001",
  "creator_handle": "examplecreator",
  "creator_display_name": "Example Creator",
  "caption": "A short demo video #python #data",
  "hashtags": ["python", "data"],
  "music_title": "Demo Track",
  "music_author": "Demo Artist",
  "view_count": 15400,
  "like_count": 920,
  "comment_count": 45,
  "share_count": 12,
  "published_at": "2026-09-10T08:30:00+00:00",
  "scraped_at": "2026-09-10T10:00:19+00:00",
  "source_repo": "tiktok-video-scraper"
}

A stable per-video JSONL schema means downstream tools such as pandas, DuckDB, or a CSV-to-Parquet job can read the file without per-record discovery logic, and any field-level diff between two captures of the same URL falls out naturally.

Use cases

A per-video metadata primitive supports several workflows:

• Hashtag velocity research. Walk a hashtag's recent posts through the per-video scraper and compute weekly deltas in view_count to rank hashtags by acceleration rather than absolute volume.
• Sound trending. Capture music_title and music_author per video and count co-occurrences across your sample to detect emerging sounds before they appear in official charts.
• Creator vetting. Sample a creator's recent uploads and compute per-video engagement rates, useful when comparing reach claims against the actual public record.
• Content moderation archive. Snapshot known unsafe URLs with their public fields preserved so an evidence pack remains reproducible if the platform takes action later.
• AI-agent context packs. Bundle a curated set of public video records as a versioned text asset that an agent can quote and re-read without refetching the live site each time.

In every case, the analysis stays inside the boundaries of public data: no private accounts, no direct messages, no credentialed views of follower-only live replays.

Build, buy, and tool comparison

Most teams pick one of three paths for TikTok video metadata:

• Self-hosted per-video scraper. Best for engineers who want full control over request shape, schema, and cadence. Maintenance burden falls on the team when the source page changes.
• Self-hosted hashtag or creator pipeline. Best when the analysis requires thousands of videos per run and the team is willing to maintain pagination and concurrency code. The data-scrape/tiktok-data-scraper-api repository sits in this lane.
• Managed scraper API. Best for teams that want a hosted endpoint, a normalized response, and a quota model. Cost and quota must be verified against the provider's current pricing page; do not assume numbers from older blog posts.

A pragmatic sequence is to use data-scrape/tiktok-video-scraper for the first dozen records, validate the per-video schema against your analytical assumptions, and only then evaluate a managed endpoint or a larger pipeline for ongoing refresh.

Checklist before production

• Schema is documented, including the units for the count fields.
• published_at is recorded in UTC with an explicit offset; scraped_at uses the same convention.
• Numeric fields are coerced — no embedded K, M, commas, or ranges remain in the JSONL.
• Raw JSON is archived alongside the normalized record so future schema changes can be replayed.
• videos.jsonl deduplication is handled by video_id; appending an already-captured record is a deliberate action.
• Refresh cadence matches the analytical question: weekly for trend snapshots, daily for active campaigns, hourly only when justified.
• Access to the JSONL is scoped to the team that needs it, and sensitive columns are stripped if the file is shared externally.

Compliance and maintenance

Extracting metadata from public TikTok video URLs is acceptable for personal research, internal trend analysis, and academic study, but you must respect the relevant terms of service, applicable privacy law, and any contractual limits in your jurisdiction. Do not collect follower-only posts, direct messages, or any field that is not visible on the public video page. Do not bypass authentication, CAPTCHA, rate limits, or any other access control. If a target page asks you to stop automated access, stop and remove the captured records.

For long-running use, schedule a monthly review of the scraper response, re-validate a sample of records against the live public page, and pin the scraper to a known-good commit. The schema-drift monitoring job is what tells you that a field disappeared or changed type before an analyst does.

FAQ

Is there an official TikTok video metadata API?
TikTok offers official APIs for authorized partners and certain research programs, but access is restricted and requires approval. The repository-based workflow described here is for public web data that does not rely on official API access.

What fields does per-video metadata include?
The exact fields depend on the current scraper response. The schema in this article covers the high-value public fields: video URL and ID, creator handle and display name, caption, hashtags, music title and author, view/like/comment/share counts, publish timestamp, and the scraper repository that produced the record.

Can this access private accounts or direct messages?
No. This workflow is for public videos and public creator profiles only. Private data, direct messages, and account-cracking techniques are out of scope and should not be attempted.

How often should I refresh per-video snapshots?
Match the cadence to the question. Weekly is reasonable for trend snapshots. Daily is reasonable for active campaigns. Hourly refresh is rarely justified and increases the risk of throttling.

What happens when TikTok changes its page layout?
The scraper may stop returning the expected fields. Your normalization layer will log missing fields, giving you a clear signal of what broke. Update the mapper or switch to a newer repository version when available.

Can I feed JSONL into a CRM, a notebook, or an AI agent?
Yes. JSONL is one of the cleanest formats for pandas, DuckDB, or any LLM tool that accepts file-based context. Add a short header comment that documents the capture time and source repository.

Next Steps

If you want a transparent, maintainable way to capture public TikTok video metadata, start with the verified repositories:

• data-scrape/tiktok-video-scraper for per-video metadata extraction.
• data-scrape/tiktok-data-scraper-api for hashtag and creator discovery pipelines.
• data-scrape for the full list of open-source public-web-data repositories.

For an adjacent public-web-data pattern that uses the same per-record → JSONL approach for a different domain — including a similar bulk-export pipeline for real-estate records — see data-scrape/zillow-data-scraper.

Build the normalization layer first, run it on a small sample, and only scale once the schema and compliance checks are solid.