DEV Community: Nasim

Why Your Terraform Cloud Runs Are Stuck

Nasim — Mon, 12 Oct 2026 23:30:37 +0000

Workspaces, queues, locks, and the concurrency ceiling nobody documents.

Our Terraform Cloud organization has 270 workspaces. One morning, 20 of them had a run waiting on someone to approve it, and 46 more runs queued behind those. The dashboard's "needs attention" count said twenty; the real number of things wrong was one. A change merged two days earlier, never approved, sat at the head of fifteen of those queues with two days of ordinary merges piled up behind it. Clearing it was straightforward. What cost the time was finding out the next morning that seventeen more workspaces held another 67 runs in the same state, going back nine days — and had never shown up in any sweep we ran, because the tooling searched four component names, which matched 54 workspaces. The other 216 were invisible to it.

If you manage more than a handful of Terraform Cloud (TFC) or Terraform Enterprise (TFE) workspaces, you know the moment: the dashboard shows a wall of queued, errored, and "needs attention" runs, and the honest answer to "why is this stuck" is that nobody on the team knows the model.

This is that model. Six layers — workspace, run, queue, lock, concurrency ceiling, trigger — and a triage script at the end that turns a wall of runs into a five-minute answer.

The Workspace Is the Unit of Serialization

A workspace isn't a folder of Terraform code. It's the unit of state isolation (one state file, always), variable scope, execution mode, and — the part people miss — run concurrency.

A workspace processes one apply-track run at a time. Not one apply: one run, plan through apply. Every other run that could eventually apply queues behind it, strictly FIFO, until the current one reaches a terminal status.

There's one exemption. Plan-only runs bypass both the queue and the workspace lock entirely. That covers every speculative plan — PR-triggered plans, CLI speculative plans, any run against a configuration version marked speculative — plus the planning stage of saved-plan runs. They can never apply, so they can never corrupt state, so TFC starts them immediately even while an apply-track run holds the workspace's one slot. They don't become the current run and they don't block anything behind them.

Exclude plan-only runs before you compute queue depth. On an active repo they're the majority of what you see — in the organization these examples come from, 72 of the 100 most recent runs were plan-only — and counting them turns a healthy workspace into a phantom incident.

Two structural notes before you scale:

Group workspaces by volatility. HashiCorp's workspace guidance recommends sizing state files to minimize blast radius and separating long-lived resources (databases, networking) from frequently-changing ones (compute). The side effect is queue hygiene — volatile workspaces churn through their FIFO queue fast, and shouldn't be stuck behind something that changes twice a year.
The anchor workspace is a serialization point. If you're assuming roles into multiple AWS accounts from a single anchor workspace — the standard Control Tower and AFT pattern — every downstream workspace consuming its outputs is rate-limited by how fast that one queue drains. One anchor is one slot.

Past a few dozen workspaces, the answer isn't more workspaces, it's Projects — a grouping layer below the org with per-project team permissions.

The Run Is a State Machine

Yellow is where a run waits on a person. Red is where it is writing, or has failed. Grey and green are terminal.

pending → plan_queued → planning → planned
   → [cost_estimated] → [policy_checked | policy_override]
   → confirmed → apply_queued → applying → applied

A common misconception: a plan with no changes never waits for you. It terminates as planned_and_finished, a final state that exists for plan-only runs and runs whose plan contains nothing to apply. TFC closes it out itself. There's nothing to confirm and nothing to discard, so no-op plans are not a source of dashboard noise. (The allow_empty_apply option overrides this if you deliberately want a no-op apply — useful after a Terraform version bump or a batch of moved blocks.)

"Queued," "errored," and "needs attention" aren't different failure modes. They're different resting points in one state machine, and the resting point is the diagnosis.

PLANNED — real changes, awaiting confirmation or auto-apply.
POLICY_OVERRIDE — a soft-mandatory policy failed. Needs a human decision, not a retry.
ERRORED — root-cause the failure class first. IAM/auth, state lock, provider crash, and resource conflict need four different fixes, and blind retry only helps one of them.
APPLYING — leave it alone. It's mid-write to real infrastructure and real state.

One resting point catches people because it looks like progress: cost_estimated. If cost estimation is enabled org-wide, a run awaiting approval sits here rather than at planned. Filter on planned alone and you miss every run that is waiting for you.

Queues Are Per-Workspace, Not Per-Org

There is no global run queue. There's one queue per workspace, and one stuck run at the head of it can accumulate an arbitrary number of runs behind it — every one of which looks like an independent problem in a dashboard count.

Across workspaces, runs execute in parallel, up to the org ceiling.

That gives you the first fork in triage: do the stuck runs cluster in a few workspaces (queue-head problem — fix the run at the front and the rest drain by themselves), or are they spread thin, one per workspace across dozens (systemic — auto-apply disabled broadly, or a new Sentinel policy shipped without a grace period)?

Locks Are State-Level, Not Run-Level

Terraform holds a state lock during plan and apply so two processes can't write the same state file. Inside TFC the run queue already prevents concurrent apply-track runs, so locks matter in two places:

Local CLI applies against a TFC-backed state, running alongside CI-driven runs. The queue doesn't govern locally-initiated applies; the lock is the only thing standing between you and a corrupted state file. If anything besides TFC can reach the state, workspaces alone are not sufficient isolation.
A run that crashed hard enough to leave the lock held without transitioning to a terminal status. This is the classic "state locked" error, it needs a deliberate manual unlock, and it is the last thing you should ever script an automatic recovery for. Confirm nothing is mid-write first.

Depth and Ceiling Are Two Different Numbers

Conflating these is where most "why won't my runs start" confusion lives.

Per-workspace depth is 1, always, on every tier — apply-track runs only.

The org concurrency ceiling caps how many runs are actively planning or applying across the entire organization. On SaaS, this is set by your plan tier and nothing else — adding agents does not raise it. HCP Terraform now has five tiers (Free, Essentials, Standard, Premium, Enterprise); the legacy free plan reached end-of-life on March 31, 2026 and was replaced by a 500-managed-resource free tier. HashiCorp's support KB publishes concurrency figures for exactly two of them: 1 concurrent run on Free, 3 on Standard. Essentials and Premium integers aren't published anywhere — check your org's plan page rather than trusting a blog post, including this one. Standard and Premium orgs can also purchase additional remote and agent runs in increments of five, up to fifty additional runs.

The concurrency number is not exposed through the API. GET /organizations/:org?include=entitlement-set returns operations, agents, policy limits and much else, but no run-concurrency field. The plan page in the UI is the only place it appears.

Self-hosted TFE is the opposite situation: the ceiling is a dial you own. On Kubernetes, TFE_CAPACITY_CONCURRENCY defaults to 10 concurrent jobs per agent pod and tunes up to 50, so it becomes capacity planning against cluster autoscaling.

This is the limit that quietly bites teams as workspace count grows without a plan or agent-pool upgrade. Zero conflicts, zero locks, zero errors — and runs still sitting in PENDING, because the org is at capacity. It presents identically to "something is stuck," but the fix isn't debugging a workspace.

What Actually Creates Runs

VCS webhooks, CLI plans and applies, direct API calls, and run triggers — a downstream workspace firing automatically after an upstream workspace applies successfully. That last one is the anchor-workspace fan-out, and it's capped at 20 source workspaces per workspace. HashiCorp warns about cascading failures and recommends short trigger chains with failure notifications. One anchor apply spawning fifteen downstream runs is exactly when you hit the org ceiling while every individual workspace queue looks perfectly healthy.

The sneakier backlog source: a superseded commit doesn't kill its predecessor's run. Push commit A, then commit B thirty seconds later, and you get two runs. The first isn't auto-discarded for being stale — it sits in PLANNED awaiting confirmation until something clears it.

Half of this has a built-in fix: "Automatically cancel speculative plans for outdated commits" (Settings → Version Control → General) cancels in-flight and pending plan-only runs when new commits land on the same branch. Turn it on. It does nothing for apply-track runs, which is the gap you have to close yourself.

And that gap is worse than "some wasted compute." A queued run is pinned to the configuration version it was created from, and that never updates. Approving a nine-day-old run applies nine-day-old code — which can be the opposite of what your current main says, with nothing in the plan's summary counts to warn you.

Policy Enforcement, Briefly

Sentinel has exactly three enforcement levels: advisory (logs a warning, blocks nothing), soft-mandatory (blocks, overridable by permitted users), and hard-mandatory (blocks unconditionally; the only way past it is removing the policy from the set).

If you're staring at a pile of POLICY_OVERRIDE runs, don't override them N times. Find the shared policy and stage it properly — advisory, then soft-mandatory, then hard-mandatory — so teams get a grace period instead of a wall.

Confirm the policy is attached before you trust it. A policy set committed to a repo enforces nothing until it's wired to the org; GET /organizations/:org/policy-sets returning an empty array is the whole answer, and it's a five-second check.

Triage, Automated

Three checks replace clicking into runs one at a time. All are read-only, and all go through the organization runs endpoint — GET /organizations/:org/runs — rather than iterating workspaces. Iterating workspaces is one request each; on a few hundred workspaces that's minutes, against seconds for the org endpoint.

Three things about this endpoint that will silently break naive code:

Its pagination metadata omits total-count and total-pages. You get current-page, next-page, prev-page and page-size. Follow next-page until it's empty; don't compute page counts.
Run objects carry a workspace ID but not a name. include=workspace solves it — the workspaces come back in .included on each page, so you can attach names as you stream rather than building a separate lookup.
plan_only runs are not excluded by default. On an active org they'll be most of what you get back — filter them out explicitly for anything queue-related.

On rate limits: the general API limit is 30 requests/second — X-RateLimit-Limit: 30, with a sub-second X-Ratelimit-Reset. Some endpoints carry tighter limits depending on tier, so check the headers rather than assuming. Sequential pagination needs no sleeps; concurrent fan-out needs 429 retry.

Shared setup:

#!/usr/bin/env bash
set -euo pipefail
: "${ORG:?set ORG to your Terraform Cloud organization name}"
TF_TOKEN="${TF_TOKEN:-$(jq -r '.credentials["app.terraform.io"].token' \
  ~/.terraform.d/credentials.tfrc.json)}"
API="https://app.terraform.io/api/v2"

# Emits one run object per line, with .ws_name attached.
# include=workspace returns the workspace in .included, so there's no second
# lookup pass and no temp file. $1 is an extra query string.
tfc_runs() {
  local qs="${1:-}" page=1 resp
  while :; do
    resp=$(curl -sf -H "Authorization: Bearer $TF_TOKEN" \
      "$API/organizations/$ORG/runs?include=workspace&page%5Bsize%5D=100&page%5Bnumber%5D=$page&$qs")
    jq -c '
      (reduce (.included[]? | select(.type == "workspaces")) as $w
        ({}; .[$w.id] = $w.attributes.name)) as $ws
      | .data[]
      | .ws_name = ($ws[.relationships.workspace.data.id]
                    // .relationships.workspace.data.id)
    ' <<<"$resp"
    page=$(jq -r '.meta.pagination["next-page"] // empty' <<<"$resp")
    [[ -z "$page" ]] && break
  done
}

1. Queue-head detection. filter[status_group]=non_final returns what's running, queued, or awaiting confirmation. Filter plan-only out explicitly — they skip the queue, so counting them inflates every depth number.

tfc_runs 'filter%5Bstatus_group%5D=non_final' \
| jq -s -r '
    map(select(.attributes["plan-only"] != true))
    | group_by(.relationships.workspace.data.id)
    | map(sort_by(.attributes["created-at"]))
    | map({
        ws:       .[0].ws_name,
        head:     .[0].id,
        status:   .[0].attributes.status,
        since:    .[0].attributes["created-at"],
        blocking: (length - 1)
      })
    | sort_by(-.blocking)[]
    | "\(.blocking)\t\(.ws)\thead=\(.head)\tstatus=\(.status)\tsince=\(.since)"'

2   aft-prod-apps   head=run-a1  status=errored  since=2026-09-01T10:00:00.000Z
2   prod-network          head=run-b1  status=planned  since=2026-09-02T09:00:00.000Z
0   prod-eks              head=run-c2  status=applying since=2026-09-03T08:10:00.000Z

Sorting ascending by created-at and taking .[0] picks the oldest non-terminal run — the one holding the lock. Leading column is how many runs are stacked behind it. One workspace in double digits is a queue-head problem: fix that run and the rest drain. Every workspace showing 0 is systemic.

2. Concurrency-ceiling check. Here you want plan-only runs, because speculative plans skip the workspace queue but still consume org concurrency. Name every operation explicitly rather than relying on defaults.

OPS='plan_only,plan_and_apply,save_plan,refresh_only,destroy,empty_apply'

tfc_runs "filter%5Bstatus_group%5D=non_final&filter%5Boperation%5D=$OPS" \
| tee /tmp/nonfinal.ndjson \
| jq -r '.attributes.status' | sort | uniq -c | sort -rn

echo -n 'live concurrency: '
jq -s 'map(select(.attributes.status | IN(
    "planning","applying","plan_queued","apply_queued","cost_estimating",
    "policy_checking","post_plan_running","pre_plan_running","queuing","fetching"
  ))) | length' /tmp/nonfinal.ndjson

IN() needs jq 1.6 or later. If live concurrency sits at your tier's cap while runs pile up in pending, stop debugging — you're waiting on capacity. The histogram costs nothing on the way past: a large errored count and a large pending count are different mornings.

3. Supersession sweep. filter[status_group]=discardable is the built-in definition of "paused and safe to clear." Within each workspace, only the newest reflects current config.

tfc_runs 'filter%5Bstatus_group%5D=discardable' \
| jq -s -r '
    map(select(.attributes.actions["is-discardable"]))
    | group_by(.relationships.workspace.data.id)
    | map(sort_by(.attributes["created-at"]) | reverse | .[1:])
    | flatten | .[]
    | "\(.id)\t\(.ws_name)\t\(.attributes["created-at"])"'

Discard with POST /runs/:id/actions/discard, which accepts an optional comment — use it, so the audit trail says why.

Two verbs, and using the wrong one fails: a run that has finished planning is discardable; a run that never started is only cancelable, and discard answers 409 on it. Route on the run's own actions.is-discardable / actions.is-cancelable flags rather than matching status names, and you can't get it wrong.

Two caveats on this sweep. Newest-per-workspace is a proxy for "newest configuration version," and it's wrong for a manually-queued run pinned to an older config, so review the list before wiring it to a cron. And half this problem has a native fix already — turn on "Automatically cancel speculative plans for outdated commits" and this sweep only has to handle apply-track runs.

One ordering detail if you clear a deep queue: work newest-first, lock-holder last. Discard the head first and TFC immediately starts the next queued run, so you end up fighting a plan that's building while you're still clearing.

Run all three and the flat scary number resolves into three buckets: fix the head of the queue, wait on or scale capacity, clear the superseded noise. The dashboard gives you counts. The state machine gives you causes.

Further reading

Remote operations: ordering and timing — the plan-only and saved-plan queue exemption
Run states and stages — every status and what leaves it
Manage your HCP Terraform plan — concurrency by tier, buying additional runs
/runs API reference — attributes and actions used above

Distributed Tracing and W3C Trace Context

Nasim — Mon, 13 Apr 2026 17:45:01 +0000

When a user makes a single request to an application or clicks “checkout” on an e-commerce site, it involves multiple services, API gateway → auth service → cart service → Redis cache → inventory database → payment → notification. Traditional logs show what happened within a single service. Distributed tracing shows the entire story of a request as it traveled across all the services in the application.

The core data model has three concepts:

Trace - journey of request. Each trace has unique trace-id
Span - unit of work within that trace defined with a start time and duration. Could be calling another service, doing a database query. Child spans are also created
Context - the metadata that goes with the request to tell each service that this request is part of another request. This the W3C Trace Context

The Gantt chart (known as a waterfall) immediately shows you the critical path for the application.
Auth → Cart → Inventory → Payment
Reading the Gantt Chart
Each horizontal bar represents one span of work within one service. The left edge of each bar is the start time of that work, the right edge is the finish time, and the width is the length of that span of work. The vertical axis represents the different services, and the horizontal axis is the wall-clock time between 0ms and 600ms.
The vertical lines that are indented within each service represent the parent-child relationships between those services. Each service’s work starts directly when these vertical lines start: it was called directly by its parent service.

Look at the time at when each service starts:

Auth service starts at ~20ms and finishes at ~100ms. API Gateway must authenticate the user before continuing with any other work; Cart cannot start until Auth is complete.
Cart service starts at ~110ms (after Auth) and makes an inline call to Redis (a child service). Cart cannot return successfully to its caller until Redis returns.
Inventory service starts at ~270ms (after Cart) and makes an inline call to Postgres (a child service). Postgres is called inline so that the Inventory service can return to Cart with the requested inventory information.
The Payment service starts at ~410ms (after Inventory is called) and takes 140ms to complete (the longest task after the API Gateway’s root span).
The Notification service starts at ~555ms and is very short at ~25ms in length. This is likely a fire-and-forget service that is called asynchronously when the Purchase operation is complete.

The process strictly follows a sequence: Auth → Cart (with Redis) → Inventory (with Postgres) → Payment → Notification. None of these steps run simultaneously. That’s the main issue—the design enforces a linear chain when, in theory, some parts could operate in parallel.

Now, about why Payment is the bottleneck. A bottleneck isn’t just the slowest step; it’s the part on the critical path that, if improved, would speed up the entire workflow. Here, the longest sequence—the critical path—takes 580ms, and Payment’s 140ms sits toward the end. Cutting time in Payment directly reduces the overall response time. For example, even if you eliminated the 70ms Redis cache retrieval in Cart, Payment would still take 140ms afterward, leaving the total duration almost unchanged.

There’s also another factor: Payment involves calling an external third-party API (Stripe), which adds extra dependency and latency.
You can’t control latency here—it can jump unpredictably. Network round-trips add delays on top of Stripe’s processing time, and any outage or rate limit from Stripe stalls your whole checkout flow.

The amber highlight in the diagram shows it’s slower compared to other parts, but the real problem is where it sits—in a late spot in a chain with no parallel work and relying on an external system you can't tweak.

So, now what? Two main handles: parallelism and externalization.
First, run Inventory and Auth checks at the same time. Starting both simultaneously after the request could cut down about 200ms of sequential waiting to around 130ms, limited by whichever is slower.

Second, take Payment out of the critical path. Accept the order right away, then process payment asynchronously using a queue. This lets users see a quick “order received” response while payment happens behind the scenes—just like most large e-commerce platforms do.

Also, add a timeout and retry strategy. Since Stripe is external, its slowest 1% delays become yours too. Setting a strict timeout—say 200ms—and handling errors gracefully means a Stripe glitch won’t wreck the checkout experience.

Without this trace, you might blame the database or cache, but the waterfall view clears that up instantly.

Now, lets dive into W3C Trace Context

Before W3C settled on a standard in 2021, each had their own way of passing trace context around. Zipkin used X-B3-TraceId headers. AWS X-Ray used X-Amzn-Trace-Id. Jaeger had its own format. When requests moved between these systems, the trace often got lost. W3C Trace Context introduced two HTTP headers that all compliant systems should support

Header 1: traceparent

This is the core carrier. It encodes four fields in a single, compact header:

version — always 00 currently. Future versions can add fields.
trace-id — 128-bit identifier for the entire distributed trace. All spans across all services share this value.
parent-id — 64-bit identifier of the calling span. The receiving service uses this as its parent span when creating its own span.
flags — currently 1 byte. The lowest bit is the sampling flag (01 = sampled, 00 = not sampled).

Header 2: tracestate

Vendor extension list, an ordered comma-separated set of key=value pairs.

Each tracing vendor gets a unique key (e.g. dd for Datadog, b3 for Zipkin B3). This lets vendor-specific metadata (sampling priority, origin, session IDs) ride alongside the standard traceparent without polluting it. When a service forwards a request, it must preserve the full tracestate and may prepend its own entry at the front. Entries beyond 32 are dropped from the tail.

Headers propagation across service boundaries

The sampled flag, which shows as 01 or 00, might seem like a small detail but it actually matters a lot. Tracking every single request in a busy production system is usually too costly—for example, a service handling 100k requests per second. Sampling helps by recording just a slice of traces, yet still keeping full detail for those captured.

The sampled bit (flags 01 vs 00) plays a crucial role in tracing. Tracing every request in a high-traffic system, like one handling 100k requests per second, quickly becomes impractical. Sampling helps by recording only a subset of traces while maintaining detail for those captured.

There are three main sampling methods:
Head-based sampling decides at the root span before downstream calls. It sets the sampled flag to 01 or 00 and propagates it. This approach is straightforward and efficient but can't prioritize capturing interesting events like errors since they haven’t occurred yet.

Tail-based sampling collects all spans from every service. The collector delays the sampling decision until the full trace is received, allowing it to keep all error traces and a small fraction of normal ones. This requires a more complex, stateful collector, such as the OpenTelemetry Collector with tail sampling.

Parent-based sampling simply follows the parent’s sampling decision via the sampled flag. It’s a safe default that ensures the whole trace is either fully recorded or entirely dropped, preventing disconnected spans without a root.

A span isn’t just about its name and how long it lasts. OpenTelemetry, which leads as the open-source implementation of W3C Trace Context, sets semantic conventions—standard attribute names. This helps tracing tools interpret your data consistently, without needing extra setup.

Spans can include events, which are specific log entries tied to a moment within the span, like noting a “cache miss occurred”. They also hold links, which reference other traces—useful in async or queue situations where the producer and consumer traces are related but happen separately.

Async and Queue Propagation
Handling async and queue propagation differs from straightforward HTTP propagation. Since producer and consumer don’t operate simultaneously, you can’t rely on passing a traceparent in a request header.
Instead, the traceparent value is embedded within the message metadata—such as Kafka headers, SQS attributes, or AMQP headers. The consumer then reads this metadata and treats it as a span link, not a parent. This means the consumer’s trace connects to the producer’s but remains a separate trace-id. This approach keeps the causal link clear without letting trace information grow uncontrollably deep.

How does this fit into the OpenTelemetry stack?

As the work-horse OpenTelemetry SDK auto-instruments common frameworks like Express, Django, Spring, and net/http, automatically creating spans and managing traceparent injection and extraction. Each service uses the OTLP exporter to send span data to the OTel Collector, which processes the data—sampling, redacting sensitive info, batching, and distributing it to multiple backends at once. This vendor-neutral setup lets you switch between tools like Jaeger and Grafana Tempo without modifying your application code.

Key implementation considerations

Clock skew — spans from different machines have timestamps from different clocks. A child span with a start time before its parent is a common artifact of NTP drift. Tracing SDKs work around this by using monotonic clocks for durations and accepting some display weirdness on absolute timestamps.
Baggage — a sibling to tracestate, the W3C Baggage header (baggage: userId=alice, env=prod) lets you propagate arbitrary key-value pairs across service boundaries. It's useful for passing tenant IDs or feature flag states without touching application code, but beware: every hop re-serializes and forwards it, so large baggage values add latency.
Instrumentation gaps — if any service in the chain drops the traceparent header (an unmigrated internal proxy, a legacy service, a third-party vendor), the trace breaks. The downstream spans become "orphans" with no parent. Good tracing platforms surface these as disconnected sub-traces.
Security — traceparent is safe to expose externally. tracestate may contain internal vendor metadata you don't want leaking. Strip tracestate at your public ingress, then re-inject a fresh one for internal propagation.
The W3C standard deliberately kept the spec minimal — it defines the wire format, not how to collect, store, or visualize. That separation of concerns is what made it possible for the whole ecosystem to converge on it.

Diagrams created with the assistance of AI.

Ray: Building Distributed Systems—Just Like Running a Restaurant

Nasim — Tue, 14 Oct 2025 16:37:06 +0000

In our fast-paced digital world, data keeps piling up at a staggering rate. To make sense of it all, we need some supercharged computing tools that can handle what we throw at them. That is where Ray comes in. It is an open-source framework designed to help you build distributed applications without all the headaches. In the following sections, we’ll break down each Ray component using our restaurant analogy, translating abstract concepts into familiar, real-world operations. Let’s get cooking!

Ray Data: Ingredient Prep Station
Role: Prepares every ingredient (chopping, washing, stalking) so when the kitchen manager is ready, everything is good to go with the cooking process.
Ray Version: Handles all the preparations for massive data transformations—loading and mapping to transform data sets pre-work to avoid unnecessary time before training.
Key Analogy: If you have a hundred tomatoes to chop or a hundred bags of flour, it is much easier to get it done on Ray Data before it is time to train than during actual cooking.

Ray Train: The Chefs Cooking the Main Dishes
Role: Cooks distribute what they're preparing and they can follow instructions, manage temperature and time, and check in along the way.
Ray Version: Distributes CPU/GPUs for model training, distributes/checks in along the way.
Key Analogy: You want different cooks (hardware) trained on different dishes (models) at the same time as possible so you can kill two birds with one stone and work efficiently.

Ray Tune: The Research Team Experimenting with Recipes
Role: The restaurant's R&D team is constantly exploring new tastes—iterating on sauces, thickness of pasta/ice cream/whatever other courses there are.
Ray Version: Runs many trials in parallel for different parameters to assess best hyperparameters for a model.
Key Analogy: Tune is like a lab that tries dozens or hundreds or iterations on a single recipe to find the best version—most palatable tasting notes—and best version for this request.

Ray Serve: Waitstaff and Quick Service Station
Role: The system for delivering courses to diners in a timely manner at the speed these clients want, especially if there is a rush and volume/inventory can help accommodate.
Ray Version: Deploys and manages models/APIs, auto-scaling for client request routing.
Key Analogy: Serve does not train models because Serve is a waitstaff who does not cook; it takes what is made in the kitchen (Run) to clients and can take many requests at once without sacrificing turnaround time.

Ray RLlib: The Robot Chef Experimenting with Reinforcement Learning
Role: A chef using learning mechanisms over time—finds the best way to create various dishes with notes/feedback along the way.
Ray Version: Builds, distributes/runs reinforcement learning algorithms from agents that can get feedback to learn over time.
Key Analogy: RLlib is a robot chef who can learn through trial and error, figuring out which seasonings people like after adjusting and trying repeatedly.

Ray Core: Kitchen Manager and Scheduling System
Role: Schedules all kitchen staff, knows what each cook is doing, who is available to support what task, ensuring order fulfillment does not bottleneck.
Ray Version: The core component to handle fundamental parallel computing—starting/stopping processes for workers, knowing what's going on, who's got what tasks, and generating reports about assigned resources.
Key Analogy: If one cook is busy, he'll have to wait for the next and it will take longer to line up what's being prepared next but Ray Core will essentially keep everyone in line.

Ray Cluster = The Kitchen: Just like a restaurant has a kitchen where all the cooking happens, a Ray cluster is a collection of computers that work together. They pool their resources, whether it is memory or processing power, to get things done efficiently.

Ray Workers = The Kitchen Staff: In any kitchen, you have distinct roles. There are chefs whipping up meals, sous chefs prepping ingredients, and dishwashers keeping things clean. Ray workers are like that staff—they are computers or components of computers that tackle their tasks independently.

Ray Scheduler = The Head Chef: Think of the head chef as the one in charge of getting things done. They assign tasks to the staff, making sure everyone knows what to work on and when. The Ray scheduler does the same thing by distributing tasks to the workers and keeping everything on track.

Ray Tasks = Recipes: Just like a well-structured recipe guides a chef, Ray tasks are steps that need to be executed. Whether it is crunching data or performing calculations, these tasks are straightforward units of work.

Ray Jobs = Customer Orders: When a customer puts in an order, the kitchen team jumps into action to make it happen. A Ray job is like that order—it is made up of a group of tasks that need to come together to complete a bigger goal, such as training a machine learning model or processing a big dataset.

Ray Actors = Specialized Chefs/Stations: Some chefs have specialties; for instance, a pastry chef only focuses on desserts. In Ray, actors are specialized workers that keep their state between task executions, just like those chefs focus on their craft.

Ray Tasks with Dependencies = Recipe Steps: In cooking, some steps cannot happen until others are finished—like you need to caramelize onions before adding them to the dish. Ray takes care of ordering these tasks properly, ensuring everything gets done at the right time.

Ray Libraries = Specialized Cookbooks: Just as a chef might have cookbooks for various cuisines, Ray comes with built-in libraries that help you deal with common tasks. For instance, it has RLlib for reinforcement learning and Tune for optimizing hyperparameters.

Ray Client = Restaurant Manager/Front of House: Finally, much like a restaurant manager bridges the kitchen and the dining area, the Ray Client makes it easy for users outside the cluster to interact with it.

Ray's Object Store is like a restaurant's central pantry since it streamlines everything all Ray components (tasks/actors/workers) do by facilitating communication and possession of what is needed.

The Central Pantry (Object Store):
The Object Store integrates everything from processed tasks and necessary tasks so there is one place for everyone to come together without issue. In a vast restaurant setting, it is the only way to operate, and a modern-day restaurant must become Ray.
The Ingredients (Objects):
Everything that gets served eventually is in the pantry. It is all there as individual items or groups of items, if necessary, with separate and respective characteristics. In Ray, everything is stored and compartmentalized as an "object".
Access and Sharing:
Since it is a shared pantry, one ingredient can be used by many chefs or dishes and there is no overlap or redundancy. This means that it is easily accessible and reapplied without offense. In Ray, Object Store allows multiple workers to easily access or repurpose what someone else has done so distributed processing is easier and quicker without wasting energy.
Efficiency:
It saves time from searching everywhere and cuts down on ordering redundancy, excess shipping or transporting from other places. Everything they need is in-house. The same with Ray—if the Object Store can pre-exist certain uses, Ray saves on sending over certain pieces of data/transferring network access/computational time of selecting what is used.

Connecting it All — Ray as a Restaurant (single end-to-end flow)

Guest places order (user request / Job starts): someone orders a meal — a job is sent to Ray.
Restaurant manager (Ray Core / Scheduler) parses the order and breaks it down into tasks: Ray Core breaks the job down into tasks and schedules them to workers.
Pantry & prep station (Ray Data) prepare and stage ingredients: data pipelines partition, preprocess, cache, and share datasets for efficient access and effective workers.
Head chef assigns & staff coordinate (Scheduler _+ _Workers / Ray Train): the scheduler gives tasks to the cooks; workers perform distributed compute and training jobs (Ray Train), coordinating efforts for cooking and baking.
Line cooks & sous-chefs execute dishes (Workers / Ray Train): workers perform the compute-heavy steps (model training, batch jobs, data transforms) on the ingredients.
Tasting lab runs experiments (Ray Tune): parallel experiments evaluate various recipe configurations (hyperparameters), feeding better versions back into training.
Robot chef explores improvements continuously (Ray RLlib): reinforcement-learning agents autonomously learn better policy/strategies that improve recipes/models.
Waitstaff & hosts deliver dishes (Ray Serve): Ray Serve hosts the trained model, manages inference requests, scales instances, adapts to traffic — bringing results to the user.
Meal served & feedback loop (Job done → iterate): the order is fulfilled; metrics, feedback fed into Ray Tune, RLlib, retraining, get into action for iterative development.

Ray is like a restaurant that can easily ramp up for a busy night; it provides a means of scaling for computers to work collaboratively with ease on otherwise complicated tasks.

Scaling the Kitchen
Easy Scaling-Ray can add more machines to the cluster, just like a restaurant easily adds in more chefs for a rush, without any issues bringing in one, two, or three more trained professionals to manage a larger situation.
Execution Efficiency-Ray effectively works across many workers creating effective compilation of many hands and minds to cut down on complicated processes—like a kitchen where each team member takes part in a unique, critical role.
Complicated Execution-Ray can manage processes of advanced complexities adding in support to establish an all-encompassing effort—like a restaurant that can put out lots of meals with disparate processes, techniques and timing.

Real-World Examples
Recommendation Systems
Developers implement Ray by working on thousands of computations that require a quick turnaround—from movie recommendation engines that need instantaneous audience analytics to compile the best results.
Scientific Inquiry
Ray engages in many pilot studies, from raw data assessment to model training, in research inquiries seeking simple predictive analytics or extensive exploration into existing data sets.
Conclusion
Ray streamlines distributed computing for developers. When a once-strained restaurant manager can easily expand and organize and optimize service during a rush it creates better opportunities for quality work and better ideas. Similarly, Ray makes scaling and establishing powerful parallel projects to be simple and worthwhile endeavors, transforming developer thinking into reality.

Connect with me on LinkedIn

Is Linux dead or awful

Nasim — Thu, 29 Feb 2024 14:14:04 +0000

The market share of Linux desktops is significantly lower compared to Windows and macOS, which dominate the desktop market. Some technologists have long predicted the downfall of Linux. However, developers, system administrators, and enthusiasts continue to use Linux due to its flexibility, customization options, security features, and the availability of free and open-source software.
Microsoft is still the sultan of OS, their Windows Mobile platform failed to generate interest compared to the pioneering iPhone and Android smartphones, resulting in a lackluster ecosystem with minimal interest. In contrast, Linux is gaining ground in both the mobile and server markets.
Although Linux offers many benefits, such as flexibility, security, and open-source nature, it can be challenging for some users. Different Linux distributions have user interfaces that differ from Windows or macOS, which can be unfamiliar. Software compatibility may require workarounds, and the command-line interface can be daunting for users accustomed to graphical user interfaces. Additionally, while customization is a strength of Linux, it can be overwhelming for new users.
Despite these challenges, the large and active Linux community provides support and documentation. However, finding the right resources and solutions for specific issues can be challenging for beginners. Nonetheless, with the right customization, Linux can be a superior tool if you have nerves, it offers increased privacy, a seamless desktop interface, and enhanced control. The anti-Linux cult has loyal disciples but to be fair all OS's have faults until it fits your use case.