A machine learning platform runs inference workloads on ECS (Elastic Container Service — managed container orchestration) with EC2 launch type. The service auto-scales based on CPU utilization. The container image is 2.8 GB — it includes a PyTorch model baked directly into the image. The image size is fixed: the model cannot be externalized.
During off-peak hours, the service runs on 4 instances. At 9 AM every weekday, a traffic spike triggers auto scaling. Eight new EC2 instances (virtual machines — pull container images from ECR before the first container can start) are launched. Each one pulls the 2.8 GB image from ECR (Elastic Container Registry — managed Docker image registry) before the first task can start.
Time from instance launch to first healthy ALB (Application Load Balancer) target: 6 minutes 40 seconds. The SLA for scale-out completion is 3 minutes. Every morning, the platform breaches it.
The team accepts that the image size cannot change. A senior engineer says there are two distinct solutions here — and they solve the problem differently depending on whether the instance has seen the image before.
Which answer correctly describes both mechanisms and when each applies?
A) Set ECSIMAGEPULL_BEHAVIOR=prefer-cached (skips ECR pull if the image is already on the instance disk) on all instances — this skips the ECR pull entirely for new instances that have never seen the image, solving the cold scale-out problem; but prefer-cached only helps if the image is already cached locally — new instances have nothing cached
B) Use SOCI indexing (Seekable OCI — lazy-loads only needed layers at startup, works on Fargate) in ECR for Fargate — Fargate detects the index and starts the container without downloading the full image; apply prefer-cached on EC2 for warm instances; but this scenario is EC2 launch type, not Fargate — and prefer-cached only helps warm instances that already have the image cached
C) Use SOCI indexing (Seekable OCI — lazy-loads only needed layers at startup; managed on Fargate, available on EC2 by running the soci-snapshotter with containerd on your instances) in ECR for the cold scale-out problem — new instances start containers without downloading the full 2.8 GB; use ECSIMAGEPULL_BEHAVIOR=prefer-cached on EC2 for warm instances to skip re-pulling an image already on disk
D) Pre-warm all auto-scaling instances by running a scheduled Lambda (serverless compute) that pulls the image to each new EC2 instance before it joins the ASG (Auto Scaling Group); this adds custom pre-warm scripts, version drift risk, and ASG lifecycle hook complexity that SOCI solves natively
Answer in the comments.
Top comments (4)
D — Pre-warming via Lambda (serverless compute) describes a real pattern (AMI pre-baking or user data scripts that pre-pull images). But it adds operational complexity — pre-warm scripts, image version drift, lifecycle hook management — when SOCI solves the cold-pull problem natively without any of that.
The mistake in option B: SOCI is not Fargate-only — it works on the EC2 launch type as well, provided the instances run the soci-snapshotter. And on Fargate (managed container service, fresh instance per task — no persistent local cache), there is no local disk cache at all, so prefer-cached is meaningless on Fargate.
The answer is C.
The two mechanisms solve different problems, and conflating them is the trap.
SOCI (Seekable OCI indexing — lazy-loads only needed layers at startup) solves the cold pull problem. A new EC2 instance that has never seen the 2.8 GB image has nothing cached. Normally, it must download the entire image before the container can start. With SOCI, you push a SOCI index alongside your image to ECR (managed Docker image registry). When the ECS (managed container orchestration) agent pulls the image, it detects the index and uses lazy loading — the container starts immediately, fetching only the layers it actually needs during execution. For a large ML image, this cuts cold scale-out from 6+ minutes to under 60 seconds in practice. One honest caveat for this scenario: on the EC2 launch type, lazy loading is not a flip-a-flag feature the way it is on Fargate — your instances must run containerd with the soci-snapshotter installed, typically baked into the AMI. The mechanism is identical; the setup is yours to own.
prefer-cached solves the warm re-pull problem. On EC2 (virtual machine with local disk), the ECS agent caches image layers locally. By default, it still pulls remotely on each new task to check for updates. Setting prefer-cached tells the agent: if the image is already on disk, use it — skip the remote pull entirely. But it only helps instances that have already pulled the image at least once.
The mistake in option A: prefer-cached does nothing for new instances — they have no cache to serve from.