Bootstrapping Vast.ai GPU Instances Without Repeating the Setup Work

TL;DR: We rent GPUs on Vast.ai for AI video work. For short batches, setup can take a large share of the session: one LatentSync self-test took 15.9 minutes, with about 11 minutes spent on setup. We now put reusable software in a public Docker image built by GitHub Actions, use onstart.sh to prepare each instance, and select the weights to download with DL_GROUPS. We batch jobs, copy the results out, and destroy the instance when we are done.

Why boot time deserves attention

A GPU hourly rate tells only part of the story. The meter runs while an instance starts, dependencies install, weights download, and we check that the environment works.

Our LatentSync 1.6 setup on a rented RTX 3090 took about nine minutes. In a queued self-test, the whole session lasted 15.9 minutes, of which setup accounted for about 11 minutes. The total was $0.07, so this was an inexpensive test, but it made the workflow problem clear: repeating setup for every clip would waste time even when the GPU itself is cheap.

Our MiniMax H3 image-to-video session had a different shape. We downloaded 72 GB of weights in about eight minutes, then generated a batch of 82 hero clips in one session. The session cost $3.59 and lasted about three hours including setup. Each five-second clip took about 86 seconds to generate. Once the weights were present, keeping the instance busy made much better use of that initial download.

The approach below is the small bootstrap pattern we use for this kind of work. The image carries software and scripts. The instance downloads the weights needed for its assigned batch.

Put stable dependencies in the image

We built a public image on GHCR using GitHub Actions. That build took about 11 minutes. Build time belongs in CI, where we can do it once, rather than in every rented GPU session.

The image is a useful home for system packages, Python dependencies, and bootstrap scripts. Here is a Dockerfile skeleton based on the PyTorch runtime image we used for LatentSync:

FROM pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime

RUN apt-get update \
    && apt-get install -y --no-install-recommends build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN python -m pip install --no-cache-dir huggingface_hub

WORKDIR /workspace/bootstrap
COPY onstart.sh download_models.py model-groups.json ./
RUN chmod +x /workspace/bootstrap/onstart.sh

Add the inference repositories and their required Python packages to your image build. The example concentrates on the bootstrap mechanism; huggingface_hub supplies the downloader below. We needed build-essential for the LatentSync environment because insightface needed g++ during installation. We also installed the repository requirements and downloaded the LatentSync 1.6 weights from ByteDance/LatentSync-1.6, including whisper/tiny.pt and latentsync_unet.pt.

Keep a model-groups.json file alongside the scripts. Map each group to the actual repositories your runner needs. For example, the latentsync group can include ByteDance/LatentSync-1.6; fill in the repositories required by your H3 setup for the h3 group. This manifest is project configuration: a placeholder repo name in it will produce a failed download, not a working model.

That distinction matters for large models. Our H3 run pulled 72 GB of weights. Baking every possible weight into every image would make the base image needlessly large and tie unrelated workloads together. A group lets the instance fetch the weights for the batch it is about to run.

Let DL_GROUPS choose the downloads

Our on-start entry point is deliberately short. It checks the requested groups, runs the downloader, and fails immediately if preparation fails:

#!/usr/bin/env bash
set -euo pipefail

: "${DL_GROUPS:?Set DL_GROUPS to a group in model-groups.json}"

mkdir -p /workspace/models
python /workspace/bootstrap/download_models.py

The downloader reads the manifest from the image. It accepts comma-separated group names, rejects unknown or empty groups, and keeps each repository under /workspace/models:

import json
import os
from pathlib import Path

from huggingface_hub import snapshot_download

manifest_path = Path("/workspace/bootstrap/model-groups.json")
manifest = json.loads(manifest_path.read_text())
groups = [name.strip() for name in os.environ["DL_GROUPS"].split(",")]

for group in groups:
    if not group or group not in manifest or not manifest[group]:
        raise SystemExit(f"Missing model repository configuration: {group!r}")

    for repo_id in manifest[group]:
        destination = Path("/workspace/models") / group / repo_id
        destination.mkdir(parents=True, exist_ok=True)
        snapshot_download(repo_id=repo_id, local_dir=destination)

The manifest format is simply a JSON object whose keys are group names and whose values are lists of repository IDs. Before publishing the image, check that each group lists every weight repository your runner expects. If some files need a different download method, put that handling in the bootstrap scripts and keep the same group boundary.

We prefer an early, visible failure to discovering missing weights after a queue has started. set -euo pipefail stops the shell script on an error; the Python check catches an unset group in the manifest. Neither replaces a quick inference check after startup, but both catch configuration mistakes before we submit a batch.

Launch the instance with one workload in mind

After building and publishing the image, we pass the group through Vast’s environment option. This example selects the H3 weights:

vastai create instance OFFER_ID \
  --image ghcr.io/YOUR_USER/YOUR_PUBLIC_IMAGE:TAG \
  --disk 150 \
  --label h3-batch \
  --onstart 'bash /workspace/bootstrap/onstart.sh' \
  --env '-e DL_GROUPS=h3' \
  --ssh --direct

Replace the offer ID and image name with your own. The on-start command runs the script already present in the image. Setting DL_GROUPS=latentsync would select that manifest group instead.

Before creating an instance, we check the offer’s reliability, driver compatibility, disk, and inbound traffic cost. We filter for reliability of at least 0.97. The driver and CUDA version must fit the image: an A100 offer we tested in Sweden had driver version 535, which did not fit our setup. We provision at least 150 GB of disk when pulling large weights. A 40 GB disk can be enough for smaller work, but it is a poor default for a large model download.

If you are setting up a Vast account, this is our referral link. The more important choice for a batch is still the specific offer and its billing fields.

Check before launch	Why we check it
inet_down_cost	Weight downloads may add a separate inbound traffic charge. We look for 0.
driver_version and cuda_max_good	The host must support the CUDA image we intend to run.
disk_space	We need room for the image, weights, inputs, and outputs.
reliability	We filter for at least 0.97.

The inbound charge surprised us once. In an earlier test on an RTX PRO 6000 Blackwell, downloading 245 GB of model weights incurred $0.64 in inbound traffic billing. The reported 56-minute session total was $2.18, with $1.47 attributed to the GPU. We now check the traffic field before choosing an offer, especially when the bootstrap will pull large weights.

Batch the work, then move the output

Selective downloads reduce unnecessary setup, but they do not erase the setup for the group you chose. For short clips, the practical next step is to queue enough work to use the prepared instance. Our H3 session produced 82 hero clips after the initial weight download. In the LatentSync self-test, setup occupied about 11 minutes of a 15.9-minute run; a longer queued batch would spread that time across more outputs.

We also treat the output transfer as part of the batch. Before destroying an instance, we copy results back and back them up. In one transfer, we verified that the cloud copy matched local output by file count and size: 5,342 files and 1.379 GiB. Only after verifying the files do we destroy the rented instance and confirm the instance list is empty.

A stopped instance can still bill for storage. We left one stopped for days; its 90 GB disk cost about $0.0167 per hour, or about $0.40 per day. The balance went negative, the stopped instance could not be restarted with that balance, and we had never downloaded the trained result. That mistake changed our shutdown order: copy, verify, destroy. We also use a cost cap and a detached deadline watchdog for batches, so a failed controller is less likely to leave the meter running.

What it cost us

The H3 A100 SXM4 80GB offer in Czechia was $1.19 per GPU hour, or about $1.25 per hour with disk. A five-second image-to-video clip took about 86 seconds and cost about $0.03; the 82-clip session cost $3.59 including roughly three hours of setup and generation. Our LatentSync RTX 3090 offer was $0.27 per hour with free inbound traffic. An eight-second talk clip took 238 seconds and cost about $0.09; the queued self-test cost $0.07 in 15.9 minutes. These are our measured rental runs, not fixed prices for future offers.

The public image makes the software setup repeatable; onstart.sh makes the remaining preparation explicit; DL_GROUPS keeps each instance focused on its batch. We still check the offer, run an inference test, and verify the output before destroying the instance. That routine has been more useful to us than optimizing the generation command while leaving setup and storage time unattended.