YouTube System Design: From Video Upload to Global Playback
A viewer presses Play. A few seconds later, video is streaming smoothly, even on a changing mobile network. That looks simple until you consider what happened before the first frame: a creator uploaded a large file over an unreliable connection; the platform validated it, transcoded it into compatible formats, published a consistent set of segments, and distributed those bytes across a global caching network.
Designing a YouTube-like video-sharing platform is really two different system-design problems: safely turning uploads into playable assets, and delivering those assets to enormous numbers of viewers at low latency and reasonable cost.
This article walks through both, including the failure cases that often get omitted from architecture diagrams. It describes a proposed architecture, not YouTube's proprietary internal implementation.
For an interactive explanation with diagrams and step-by-step flows, see Video Streaming Platform System Design on SeeItFlow.
1. Scope and requirements
We'll design video-on-demand (VOD): creators upload videos, and viewers discover and play them later. Live streaming, DRM internals, recommendation algorithms, ads, and moderation internals are separate systems and are outside this design.
The functional requirements are:
- Upload large videos, with pause/resume and per-part retries.
- Process originals into playable representations; generate thumbnails and metadata.
- Publish only validated, complete media outputs.
- Stream at adaptive quality, with seeking and playback recovery.
- Support public and restricted visibility, search, metadata edits, and deletion.
- Record view events and show an eventually consistent display count.
The non-functional requirements drive the architecture: durable finalized uploads, safe retries, bounded processing delays, fast startup, low rebuffering, global delivery, origin protection, and explicit privacy/deletion semantics.
A useful separation is ingest/processing, control/metadata, and delivery/playback. A video being uploaded, processed, discoverable, and playable are distinct states; one flag cannot safely represent all four.
2. Estimate the workload before choosing infrastructure
In a video platform, playback demand will often dwarf upload count. Rather than inventing real-world YouTube statistics, use clearly stated assumptions.
Suppose, purely as a capacity-planning example, we have:
- 100,000 uploads/day, averaging 500 MB each.
- 1 million simultaneous viewers at peak.
- An average delivered bitrate of 3 Mbps per active viewer.
Then:
Upload ingress: 100,000 × 500 MB ≈ 50 TB/day of original uploads, before replication and derived media.
Peak playback bandwidth: 1,000,000 × 3 Mbps ≈ 3 Tbps delivered to viewers. That is an aggregate delivery-network figure, not a claim that a single origin must serve it.
Processing work: depends on media duration, resolution, codec, and how many representations are encoded. A two-hour 4K upload and a 20-second clip should not count as equal-sized processing jobs.
Storage: originals + transcoded representations + audio + thumbnails + retention/replication overhead. Segmentation mostly divides a rendition into smaller objects; additional renditions and codecs are what significantly multiply encoded-media storage.
These illustrative numbers explain why the upload service, processing workers, and playback delivery tier need to scale independently.
3. High-level architecture
CREATOR
|
| 1. Create upload session
v
Upload API / Metadata DB
| short-lived, scoped upload credentials
v
Object Storage (original media) <---- Creator uploads parts directly
|
| 2. Verified, idempotent finalize
v
Metadata transaction + durable processing job / outbox
|
v
Dispatcher -> Processing Queue -> Transcoding Workers
|
v
Probe / Encode / Package
|
v
Validate versioned outputs
|
v
Atomic publish pointer
|
v
Object Storage (origin)
|
v
VIEWER -> Playback API / Authorization -> CDN Edge -> Shield -> Origin
| |
v v
Metadata DB HLS/DASH segments
|
+--> Search indexing / async view aggregation
This is a logical diagram. Actual implementations may combine services or use different managed products. The important properties are the durability boundaries and which dependencies sit on the critical playback path.
4. Resumable uploads: don't make the app server carry every byte
Sending a multi-gigabyte file through the application tier consumes network bandwidth and connections for work that object storage can handle more efficiently.
A safer upload flow is:
- The creator authenticates and requests an upload session. The server creates a stable
upload_id, records ownership, size limits, and expiry. - The server issues short-lived, tightly scoped upload permissions for the intended object/session.
- The client uploads independently identifiable parts directly to object storage, retrying failed parts without restarting the entire file.
- The client asks to finalize. The backend checks authoritative storage-side part/completion state and applicable integrity constraints. It does not trust a client-side “all chunks uploaded” assertion.
- Once the object is finalized and metadata/job state is committed, the client receives a stable video identity and a processing status—not a promise that playback is ready.
Finalization must be idempotent. If the server commits and its response is lost, retrying the same upload_id must return the existing result rather than create a second video or an unrelated processing workflow.
There is an important ordering nuance: object-storage multipart completion and a metadata database transaction are not automatically one atomic operation. The implementation needs a recoverable state machine and a reconciliation sweep for cases where storage finalization succeeds but metadata completion fails, or vice versa. A durable source object must not be silently orphaned or incorrectly treated as ready.
Expired sessions and abandoned multipart uploads need automated cleanup. Upload credentials must never grant broad bucket access.
5. Durable processing: the database-to-queue gap
Video processing is CPU/GPU intensive and highly variable in duration. It belongs outside the upload request.
A tempting implementation is:
UPDATE videos SET state = 'PROCESSING';
publish_to_queue(video_id);
But these are two separate systems. The database commit can succeed while publishing to the queue fails, leaving a creator waiting forever.
A transactional outbox or equivalent durable job record closes that gap:
One database transaction:
- Mark the upload finalized / processing pending
- Insert processing_job(job_id, source_version, state='PENDING')
Commit
Separate dispatcher:
- Read undispatched jobs
- Publish to processing queue
- Record dispatch progress after broker acknowledgement
A crash after queue publish but before recording success may cause duplicate delivery. That's acceptable only because processing is designed to be idempotent at its publication boundary. An outbox gives durable at-least-once handoff, not exactly-once execution.
Each job should identify its source version, processing profile, and attempt. Retry transient failures with backoff and limits; quarantine deterministic failures such as unsupported codecs or irreparably corrupt sources. Monitor oldest job age, queued media minutes, and processing SLO attainment, not just raw queue depth.
6. Transcoding: a rendition is more than a resolution
A creator might upload a high-bitrate file in a format that some viewers cannot decode or stream efficiently. The platform produces multiple compatible representations.
A representation can vary in resolution, bitrate, codec, frame rate, audio format, and HDR profile. An illustrative ladder might include 1080p, 720p, 480p, and 360p, but there is no universal ladder. Choose representations based on source characteristics, device compatibility, network diversity, encoding cost, and expected demand. Avoid upscaling merely to populate a preset rung.
A practical processing workflow is a dependency graph:
Probe and validate source
|
+--> Video encode: representation A --+
+--> Video encode: representation B --+--> Package/segment
+--> Audio encode --------------------+ |
+--> Thumbnail extraction v
Validate
|
Publish
Independent representations can process concurrently. Track progress and failure per stage, so a failed optional rendition can be retried without recomputing everything. Whether a reduced ladder is publishable depends on an explicit minimum-playable-set policy; it is not always all-or-nothing.
Encoding is expensive. Precomputing every possible quality improves first-play readiness but wastes work on rarely viewed videos. Selective or on-demand encoding can reduce cost, at the price of more complex coordination and potentially higher first-view latency.
7. The most important correctness boundary: safe publication
Imagine a transcoding worker crashes after writing half the segments. Its job is redelivered while the original worker is still running. If both attempts overwrite the same public object keys, a viewer may see a manifest referencing incomplete or mixed output.
The solution is attempt-scoped staging and immutable published versions:
- Each processing attempt writes into a private, unique staging namespace.
- The system verifies required representations, segments, manifest references, and basic media integrity.
- All objects referenced by the candidate manifest must already be durably readable before publication.
- A conditional/transactional metadata update changes the video's active published version only if the candidate is valid and the publishing worker still has authority.
- Older published versions remain intact for in-flight readers and rollback until a safe retention/cleanup policy removes them.
The atomic operation is the metadata pointer change, not an imaginary transaction spanning thousands of object-storage writes. Duplicate or late workers may waste compute, but they cannot publish over the active version unless they pass the required fencing/conditional checks.
Versioned URLs are also CDN-friendly: a segment at video/v7/720p/segment-42.m4s never changes its bytes. Reprocessing creates v8 instead of silently mutating v7.
8. HLS, MPEG-DASH, and adaptive bitrate streaming
The platform packages media into short, time-aligned segments and manifests/playlists that tell the player which representations and segments exist.
Two common adaptive streaming families are HLS and MPEG-DASH. Their exact manifest structures differ: for example, HLS commonly uses a multivariant playlist and media playlists, whereas DASH uses an MPD describing adaptation sets/representations and segment addressing. They should not be described as byte-for-byte identical formats.
For a typical segmented VOD session:
- The player gets authorized playback information and a manifest URL.
- It selects a supported representation using device capability, estimated throughput, and current buffer health.
- It requests media segments over HTTP and buffers ahead.
- When throughput drops, it requests subsequent segments at a lower sustainable bitrate; when conditions improve, it may step up cautiously.
- On seeking, it locates the segment covering the requested playback time rather than downloading everything from the beginning.
Rendition switching works best when segments and suitable random-access/keyframe boundaries are aligned. A startup algorithm does not always begin at the lowest resolution. Nor should it blindly chase the highest bitrate: rebuffering usually hurts the viewing experience more than a temporary reduction in picture quality.
Segment duration is a trade-off: shorter segments allow finer-grained adaptation but increase request and packaging overhead. It is a design choice, not a fixed number that every streaming service must use.
9. CDN design: hot videos are a caching problem—with exceptions
A million viewers cannot all fetch every segment from one storage origin without enormous egress and request pressure. A CDN places popular segments closer to viewers.
Viewer -> CDN edge (hit: serve locally)
|
+-- miss -> Regional/shield cache
|
+-- miss -> Object-storage origin
Immutable segments can use long-lived, versioned cache keys. Manifests and authorization-related responses may require different caching policies.
An origin shield reduces the number of separate edge misses that reach origin. Request coalescing helps prevent multiple simultaneous misses for the same object from triggering duplicate upstream fetches. Neither makes origin load literally zero: cold content, cache-key fragmentation, regional traffic skew, and purge events still matter.
One subtle bug can destroy the cache hit ratio: including a viewer-specific signed token in the cache key when the underlying segment bytes are identical. Normalize the cache key only if authorization is still independently enforced before cached bytes are served. Removing tokens from the key without enforcing access at the CDN or trusted delivery layer would create a security vulnerability.
A broad CDN outage also needs a bounded fallback strategy. Sending unlimited global traffic directly to origin can turn a CDN incident into an origin outage. Consider alternate healthy edges or providers where configured, controlled origin fallback, and load shedding.
10. Metadata, visibility, and playback authorization
A video record might track:
video_id, creator_id, title, description, duration,
source_object, processing_state, active_media_version,
visibility, moderation_state, created_at, updated_at
Keep processing readiness, discoverability, and authorization separate. A video can have a valid encoded output but still be private, blocked, or intentionally unpublished.
For restricted content, the playback API verifies entitlement and issues a short-lived credential that the delivery path actually checks. A metadata flag alone does not protect segments if anyone with a cached URL can fetch them.
Unlisted is not the same as private. In a common product model, unlisted means omitted from normal discovery but accessible to people with the link; private means access is restricted to authorized viewers. The exact policy should be stated, not assumed.
A public-to-private change requires more than updating one database row:
- Stop issuing new access credentials and make the authoritative authorization decision reflect the restriction.
- Invalidate or bypass stale authorization caches, including remote-region replicas where necessary.
- Remove the video from search/discovery asynchronously, with priority over ordinary metadata edits.
- Bound or revoke previously issued delivery credentials and purge cached media when the policy requires it.
Short-lived signed URLs do not provide instantaneous revocation by themselves. Already-issued credentials may remain valid until expiry unless the delivery architecture supports a revocation check or equivalent enforcement. CDN purge propagation is also not necessarily instantaneous. A correct design states the permitted exposure window and chooses enforcement accordingly.
11. Search and view counts are separate workloads
Metadata lives in a transactional store; video binaries live in object storage. As catalog size and ranking needs grow, a dedicated search index supports text matching and ranking without forcing search traffic through the metadata database.
Index updates are usually asynchronous. Search may briefly show stale results, but playback authorization remains the final access gate. A search outage should not break playback for a viewer who already has a valid video link.
View counting has its own scale trap. Incrementing one database row synchronously for every view creates a hot write key for viral content. Instead:
Player event -> Event ingestion -> Stream/queue
|
v
Dedupe + aggregation
|
v
Periodic counter updates
First define what counts as a view; there is no universal watch-time threshold. Events need stable identities for bounded deduplication, and a viral video may require sharded or hierarchical aggregation to avoid a single hot reducer.
The user-visible count can be approximate and eventually consistent if the product permits it. Bounded dedupe does not guarantee an exact lifetime count, and a dropped event may never be recovered. Financially consequential metrics—such as ad billing or creator payouts—require a separate, stricter measurement and audit design.
12. Multi-region delivery is not the same as multi-region ownership
A CDN can serve cached segments worldwide, but it does not automatically make upload processing, metadata writes, or source objects resilient to a regional failure.
At a large enough scale, regional ingest and origin infrastructure can improve locality and availability. That adds harder questions:
- Which region owns writes for a given video or metadata partition?
- Where are originals and derived objects replicated?
- Can a region serving a cache miss actually reach a copy of that segment?
- How do we prevent two regions from accepting conflicting authoritative writes during a partition?
One workable model uses a single authoritative write owner per video/partition, asynchronous read replicas, selective object replication, and fencing before write ownership moves. An in-progress multipart upload tied to a failed region may have to restart; seamless continuation requires explicit design, not a routing rule.
Selective replication saves storage and transfer cost but means some content may become unavailable when its only reachable origin fails. The availability promise must match the actual replication policy.
13. Failure scenarios that test whether the design is real
| Failure | What can go wrong | Safe response |
|---|---|---|
| Upload finalize response lost | Client retries and creates duplicate work | Stable upload_id, idempotent finalize, recoverable state |
| Metadata commit succeeds, queue publish fails | Video remains processing forever | Durable outbox/job record and retrying dispatcher |
| Worker lease expires while worker still runs | Two attempts race to publish | Attempt-specific staging and fenced, conditional publication |
| One rendition fails | Partial ladder exists | Retry failed stage or apply an explicit minimum-rendition policy |
| Manifest published before segments | Early viewers get 404s | Validate object availability before switching active version |
| Viral video's first few requests | Cold-edge misses overwhelm origin | Coalescing, origin shield, rate limiting |
| CDN cache-key fragmentation | Cache hit ratio collapses | Normalize safe keys; enforce authorization independently |
| Network bandwidth collapses | Buffer drains, playback stalls | ABR downshift, buffer-aware retries |
| Privacy change meets stale cache | Restricted video remains accessible | Authoritative access checks, revocation strategy, cache invalidation |
| Region fails | Some origins or metadata owners disappear | Replication-aware routing and fenced failover |
| Telemetry pipeline fails | View count stops updating | Keep playback independent; buffer or accept bounded event loss |
| Video deletion stalls | Artifacts remain at CDN/storage | Durable, observable deletion workflow with retries |
A few details are worth emphasizing.
A failed transcode should not destroy an already-playable video. Reprocessing writes a new version and leaves the current version active until validation succeeds. Rollback should restore a known-good version pointer, while respecting current privacy/deletion state.
A missing segment is not always a transient network error. Repeated 404s may signal a publication race, incorrect manifest, or missing origin replica. Player retries can help temporarily, but they do not replace fixing the source inconsistency.
A deletion is a workflow, not one SQL statement. Stop new authorization promptly, update discovery, revoke delivery as policy requires, and track eventual cleanup of CDN and stored artifacts. Do not promise immediate worldwide physical deletion without an enforcement mechanism that actually provides it.
14. What to measure in production
A successful architecture needs observability for both creator and viewer experiences.
For uploads, track session success, part retries, finalize latency, abandoned sessions, orphaned objects, and finalization reconciliation failures.
For processing, track oldest-job age, queued media minutes, stage/rendition failure rate, retry/DLQ growth, processing SLO attainment, and cost per encoded media minute.
For playback, track startup time to first frame, rebuffer ratio, fatal playback errors, bitrate/quality switches, segment 4xx/5xx, CDN hit ratio, shield misses, and origin egress.
For security and lifecycle, monitor authorization denials, privacy/deletion propagation lag, revoked-credential behavior, stuck deletion jobs, and unexpected access to deleted/private media.
Keep operational dashboards separated by region, video/version, codec, and device class where practical; otherwise a small but severe compatibility or regional problem may disappear inside global averages.
15. Architectural trade-offs: no universal answer
| Decision | Baseline choice | When to reconsider |
|---|---|---|
| Upload bytes | Direct-to-object-storage multipart upload | Small prototype or inline inspection requirement |
| Processing | Asynchronous jobs with durable handoff | Tiny files and truly negligible processing work |
| Encoding | Precompute a useful baseline ladder | Large long tail makes selective/lazy encoding worthwhile |
| Media publication | Immutable versions + atomic active pointer | Generally retain; this is a correctness mechanism |
| Delivery | CDN + shield + protected origin | Single-region, tiny viewer population |
| Search | Dedicated index at sufficient scale | Database full-text search may suffice early |
| Display views | Async, approximate aggregation | Audited financial counters need stronger guarantees |
| Regions | Start simple; add regional origins when justified | Strict global availability/upload locality requires them |
| CDN providers | Single provider initially | Provider-level availability or routing requirements justify multi-CDN |
The point of a system-design interview is not to name every possible component. It is to explain which requirement forces each component to exist, and how the design behaves when that component fails.
16. How to present this in a system-design interview
Start by clarifying VOD versus live, upload versus playback scope, privacy, target scale, and SLOs. Estimate ingest, processing work, storage, and peak playback egress. Then draw the ingest/processing and delivery paths separately.
Spend your deepest technical time on four boundaries:
- Upload finalization: what does the server actually know, and how do retries behave?
- Durable job handoff: what happens if the database commits but queue publication fails?
- Safe media publication: can duplicate workers or incomplete segments ever become viewer-visible?
- Playback and privacy: how do CDN caching, ABR, and access revocation coexist?
Then test the design with a viral-video spike, a processing backlog, a CDN outage, and a public-to-private transition. These scenarios reveal much more than adding more boxes to the architecture diagram.
Final takeaway
A robust video platform doesn't become reliable just because it has object storage, Kafka, transcoding workers, and a CDN. It becomes reliable when the transitions between those systems are explicit: an upload is verified before being finalized, a processing job cannot be silently lost, incomplete media cannot be published, cached bytes cannot bypass authorization, and retries cannot corrupt a previously good version.
That is the difference between a diagram that looks scalable and a video-streaming architecture that can survive production failures.
Explore the full video streaming system design on SeeItFlow
The article is a condensed, standalone walkthrough. For the full visual and technical treatment:
- Complete Design — requirements, architecture, and end-to-end flow
- Visual Design Journey — follow the architecture as it evolves
- Architecture Deep Dive — processing, media packaging, CDN, and lifecycle mechanics
- Trade-offs & Decisions — alternatives and why each choice fits
- Failure & Scale Scenarios — stress-test the design
- Interview Perspective — questions, estimates, and follow-ups
SeeItFlow is a free visual learning platform for software engineering and system design: seeitflow.com.
Top comments (0)