Merge YouTube exports without losing where each value came from

One video appeared in two of my September 10 exports. YouTube search returned 7,493,319 views and a duration of 7,341 seconds for Programming with Mosh's Python Full Course for Beginners. A later metadata lookup returned the same title, but both numbers were null.

An ordinary dictionary update would have erased the useful values. Replacing those nulls with zero would have invented a collapse in views.

I maintain the six Apify Actors used below. This guide combines their existing successful cloud exports into a small local index, retaining the source of every observation. You can reproduce the example offline with Python 3.11 or later; it needs no account, token or installed package.

The actual overlap

Both observations refer to video K5KVEU3aaeQ. These are dated measurements, not current view counts.

Observation on September 10, 2026, UTC	View count	Duration, seconds	Source detail
Search, 17:53:13	7,493,319	7,341	Query: python tutorial
Metadata, 18:33:17	null	null	metadata_source: oembed

The metadata Actor's cloud response supplied basic oEmbed information. Detailed metadata had worked locally, but that did not establish cloud access to those fields. Keep the source marker in any downstream export that uses this Actor.

The index groups observations by YouTube video ID. It deliberately leaves both records intact, so a report can choose the value appropriate to its purpose without rewriting the evidence.

Choose the export for the question

These six existing runs produced 15 rows across 14 distinct videos. I read their datasets back and used selected metadata columns for the downloadable sample; it contains no caption text or full descriptions.

Source and tested example	Rows	Useful distinction
YouTube Search: Python tutorial queries	4	Retain the query and search_rank. The cap applies per query, with duplicate IDs and exclusions reducing output.
Playlist Extractor: Corey Schafer's Python playlist	3	Keep playlist_id and playlist_index; playlist position is different from search rank.
Channel RSS: recent Linus Tech Tips uploads	3	Use the source publication date for a recent-upload feed. RSS does not provide a complete channel archive.
Shorts Scraper: Apify's Shorts tab	3	This export calls its identifier short_id. The index maps that identifier to the same video-ID namespace while preserving the original fields.
Video Metadata: one known Mosh video	1	The cloud response used oEmbed and left detailed metrics null. It cannot fill every gap in discovery output.
Transcript Scraper: one known video with English captions	1	Caption availability, language and generation status describe a track. Translated-track delivery remains unverified.

The search sample ran on 1.0.3 and the caption sample on 1.0.12; their later 1.0.4 and 1.0.13 releases changed schema declarations without changing those runtimes. The other sample builds are Playlist/Shorts 1.0.4, Channel RSS 1.0.9 and Video Metadata 1.0.6. These independent examples illustrate the record contracts; they are not a single chained discovery-to-caption run.

The six recorded runs used about $0.00467 of platform resources, excluding builds. That is an owner-test resource measurement, not the amount a customer would pay. Each Actor's Pricing tab governs customer fees. No new cloud run is needed to reproduce the local example.

Run the local index

Download these three files from the existing workflow Gist:

• youtube_observation_index.py
• test_youtube_observation_index.py
• youtube-observations-sample.json

Keep them in one directory, then run:

python test_youtube_observation_index.py
python youtube_observation_index.py youtube-observations-sample.json > first-index.json
python youtube_observation_index.py youtube-observations-sample.json youtube-observations-sample.json > repeat-index.json

The check exercises the real search/oEmbed overlap, missing values, Shorts IDs and repeat imports. It also checks deliberately invalid inputs and a synthetic successful empty run; those checks are separate from the cloud examples.

Confirm the repeat output and inspect the overlap:

import json
from pathlib import Path

assert Path("first-index.json").read_bytes() == Path("repeat-index.json").read_bytes()
index = json.loads(Path("first-index.json").read_text())
assert len(index["runs"]) == 6
assert len(index["videos"]) == 14
assert sum(len(v["observations"]) for v in index["videos"]) == 15

video = next(v for v in index["videos"] if v["video_id"] == "K5KVEU3aaeQ")
for observation in video["observations"]:
    print(observation["source"], observation["fields"].get("view_count"))

The output includes 7493319 for search and None for metadata. It does not choose a single authoritative view count for you.

Import your own successful exports

Download a run's dataset as JSON. Wrap it in a list containing an object with these fields: source (one of the six Actor names above), run_id, observed_at, status and items. Set items to the downloaded array. Use the run's actual ID and its timezone-bearing finish time; observed_at is the collection checkpoint, not the video's publication date.

Only SUCCEEDED exports enter this index. Inspect failures and partial datasets separately before deciding whether they are usable. A successful export with zero rows remains in the runs list, because silence from a bounded search does not prove that videos were removed.

Supply all the export files you want to combine as arguments. The script builds a fresh index from those files; it does not read an existing index, start Actors, send alerts or maintain a scheduled monitor. Keep the source files as your archive and write each comparison to a new output filename. Shell redirection can truncate an existing file before Python validates the inputs.

The same source/run ID with identical contents is harmless to import twice. Conflicting contents under that identity cause an error, which catches accidentally relabelled or incomplete downloads. Rows within one export remain separate observations, including repeated appearances of a video. Their export_row is simply the one-based row number in that downloaded file.

Deciding which value to display

For a research reading list, a title and canonical video link may be enough. A trend chart needs compatible measurements over time. Mixing an old search view count with a new metadata timestamp would make a number look fresher than it is, even if you correctly ignored the null.

Keep that decision in the report: select an observation whose field is present, show its collection time and identify its source. Playlist position belongs to a particular playlist; search rank belongs to a particular query. Neither is a permanent property of the video.

The downloadable script retains all fields you supply, so choose what you store. For captions, this sample keeps availability and language metadata without redistributing the text. Use content you have permission to process, and respect source restrictions. The in-memory index suits small research exports; a larger archive needs durable storage and an explicit retention policy.