On November 18, AWS added GPU-accelerated Event Functions to Lambda Managed Instances. That makes an event-driven media worker easier to imagine. Before increasing its concurrency, count the model copies: AWS documents that each worker process initializes independently by default, so three concurrent workers can put three copies of the model on the same GPU.
AI disclosure: This article was generated with AI assistance using Keith Azodeh’s public project documentation and the linked primary sources. It has not been reviewed by Keith before publication.
The distinction is in the scaling and concurrency guide. Multiple worker processes share a physical GPU, but each can hold its own CUDA context and model. Sharing hardware does not automatically share model weights.
A local worker has a useful constraint
The public Stereo Depth Lab case study describes a working local prototype built with Python, PyTorch, OpenCV, and FFmpeg. It converts images and short clips into stereo output. Its worker processes one job at a time, with GPU inference where available and a CPU fallback.
That one-job rule makes resource ownership easier to reason about. There is one active conversion, with its own input, settings, intermediate data, and output.
The Gaussian Lite rendering path also carries information between frames: optical flow moves persistent depth-seeded points, with confidence, decay, and resets at hard scene cuts. That is useful video history, and it belongs to a particular clip.
The documented tool remains local and under research. The cloud worker discussed here is a proposed adaptation, not an announced deployment or a performance result.
Moving that kind of application into a managed GPU service creates two separate questions: how many model copies fit, and which state each job owns.
“Loaded once” needs a process boundary
A developer might place model initialization outside the request handler to avoid loading weights for every request. That can be sensible within a process. It does not mean every process sees the same initialized object.
In AWS’s documented default arrangement, a new worker runs its own initialization. A successful single-job trial therefore tells you little about the memory needed when several workers initialize and process inputs together.
Consider an intentionally fictional planning example. Suppose the usable GPU-memory budget is 20 GiB, the model weights require 8 GiB, and each active job needs another 2 GiB. These numbers are arbitrary; they are not measurements of Stereo Depth Lab or its depth models.
With one model per worker:
- One active worker needs roughly 10 GiB.
- Two need roughly 20 GiB, leaving no allowance in this simplified budget.
- Three need roughly 30 GiB, exceeding it.
A shared model could change the arithmetic to 8 GiB plus three 2 GiB working sets: roughly 14 GiB before other overhead.
That is an architectural difference, not evidence of a threefold throughput improvement. Neither estimate includes every allocation, and actual working sets can vary with input dimensions, batching, and processing stages. Measure peak memory with the intended workload.
A model server changes the design
AWS offers a second pattern: one inference server owns the model, and worker processes forward requests over local HTTP. Its vLLM and SGLang images use a shared server; the guide also describes starting a server for another framework.
For a custom depth-processing pipeline, that is a design to implement and validate. The existence of those base images does not establish that they can run the prototype’s models or renderer unchanged.
A proposed split would keep model inference behind a bounded service while the conversion worker owns the clip’s temporal state and export work. Each inference response must return to the correct job and frame.
This is where reducing model duplication can introduce a different mistake. If the shared service also holds one mutable “previous frame” or one current point set for every caller, two clips can overwrite each other’s history.
Separate worker processes do not automatically share Python globals. The risk arises if the new shared server is designed around a single mutable session.
A model can be reusable infrastructure while frame history remains job-specific. Treat those lifetimes explicitly.
Tune for completed conversions
The GPU function documentation exposes per-environment concurrency through PerExecutionEnvironmentMaxConcurrency and describes scaling on GPU utilization, CPU utilization, and concurrency. Managed scaling helps distribute work; the application still needs a safe admission limit for its own memory footprint.
For the proposed media worker, compare a serial baseline with concurrent runs using synthetic clips at representative resolutions. Record peak GPU memory, queue time, processing time, failures, and whether each output retains the correct clip’s settings and history.
Include overlapping initialization and a failed job followed by another job. A stable warm run can hide a larger startup peak or leftover application state.
These are proposed acceptance checks, not tests already run against Lambda or Stereo Depth Lab.
The industry consequence is practical. Managed GPUs reduce infrastructure work, making small media and vision services easier to attempt. Capacity planning moves closer to application design: model placement, bounded work, and state ownership determine whether additional concurrency produces more finished outputs.
Before raising the number of workers, draw the process boundaries and count what each process loads. The GPU count alone will not answer that question.
Top comments (0)