The Self-Calling Logic App: Processing Big Files One Chunk at a Time

A user drops a 5,000-row CSV into Blob Storage. Every row needs to hit a SaaS API that rate-limits aggressively (HTTP 429, "Remaining: 12s").

The textbook Logic App — Blob trigger → Parse CSV → For each → HTTP PUT — works fine for 50 rows in dev. At 5,000 rows in production, it falls apart:

• One massive run. Finding a single failed row across thousands of loop iterations in Run History is miserable.
• All-or-nothing recovery. If the workflow dies at row 2,731, there is no clean way to resume from row 2,731.
• Compounding 429 waits. Every rate-limit sleep stretches the same run, pushing it toward action and workflow timeouts.

The standard advice is to batch with take() and skip() inside an Until loop — which still traps everything inside one run. We took a different approach: process one chunk, then have the workflow call itself for the next one.

Each run processes 100 rows and passes the cursor forward, turning a 5,000-row file into 50 small, independent runs. Hop 1 (receiver → processor) uses the built-in Workflow action; every self-call after that uses a raw Http POST (§5 explains why).

1. Pass a cursor, not state

The processor's HTTP trigger accepts the blob path and a Skip/Take cursor:

"triggers": {
  "ReqChunk": {
    "type": "Request",
    "kind": "Http",
    "inputs": {
      "schema": {
        "type": "object",
        "properties": {
          "ContainerName": { "type": "string" },
          "BlobName":      { "type": "string" },
          "Skip":          { "type": "integer" },
          "Take":          { "type": "integer" },
          "Depth":         { "type": "integer" }
        }
      }
    }
  }
}

That payload is the entire state between runs — no tracking table, no external queue. (Depth is a recursion circuit-breaker, covered in §6.)

2. Fire-and-forget the first hop

The receiver (the blob-triggered workflow) starts chunk 0 using the built-in Workflow action with operationOptions: "DisableAsyncPattern":

"TriggerSync": {
  "type": "Workflow",
  "inputs": {
    "host": { "workflow": { "id": "wf_chunk_processor" } },
    "body": { "ContainerName": "...", "BlobName": "...", "Skip": 0, "Take": 100 }
  },
  "operationOptions": "DisableAsyncPattern"
}

By default, the Workflow action waits for the child run to finish. DisableAsyncPattern turns it into fire-and-forget: the receiver gets the trigger's immediate 202 Accepted response and cleans up the source blob without waiting on the chunk chain.

That decouples the receiver from chunk 1. How chunk N triggers chunk N+1 is a different story (§5).

3. Keep the full file out of the Logic App

Reading the whole blob in the Logic App and slicing with take(skip(...)) forces chunk 1 and chunk 50 alike to download and parse the entire file — and dumps the full payload into Run History on every single run.

Instead, the processor passes the cursor to a small Azure Function that streams the blob from storage and returns only the [Skip, Skip+Take) slice plus totalRowCount:

"ReadChunk": {
  "type": "Http",
  "inputs": {
    "method": "POST",
    "uri": "@parameters('readChunkFunctionUrl')",
    "headers": { "Content-Type": "application/json" },
    "body": {
      "ContainerName": "@triggerBody()?['ContainerName']",
      "BlobName": "@triggerBody()?['BlobName']",
      "Skip": "@coalesce(triggerBody()?['Skip'], 0)",
      "Take": "@coalesce(triggerBody()?['Take'], parameters('chunkSize'))"
    }
  }
}

The response is { rows, headers, totalRowCount }. Because rows only holds the current slice, the full CSV never touches the Logic App's run history. And because chunkSize is a workflow parameter (5 in our demo, 100+ in production), you can tune batch size per environment without touching the workflow definition.

Inside the function, we stream the blob character by character, materializing row dictionaries only for rows inside [Skip, Skip+Take). It still scans to the end of the stream to compute totalRowCount, keeping per-chunk memory flat regardless of file size — at the cost of every chunk re-scanning the whole file just for that count (50 chunks, 50 full scans of a 5,000-row file). Passing totalRowCount forward as part of the cursor once chunk 0 computes it would trade that for a two-pass first chunk.

While Logic Apps Standard can colocate .NET functions inside the same project, we deploy ReadChunk as a standalone Function App. Our Logic App hosts several other workflows, and isolating the function ensures a bad build or worker mismatch can't take the whole host down.

4. Respect the API's exact 429 backoff

ForEachRow runs at low concurrency (repetitions: 2), wrapping each call in an Until loop. Rather than guessing with exponential backoff, we parse the API's "Remaining: 12s" response and wait that exact duration:

"PutWithRateLimitRetry": {
  "type": "Until",
  "expression": "@or(less(outputs('Put')['statusCode'], 300), not(equals(outputs('Put')?['body']?['_errorCode'], 'RATE_LIMIT')), greaterOrEquals(variables('attempt'), parameters('maxRetries')))",
  "limit": { "count": 6, "timeout": "PT30M" },
  "actions": {
    "Put": { "type": "Http", "retryPolicy": { "type": "none" }, "...": "..." },
    "IsRateLimited": {
      "type": "If",
      "expression": { "equals": [ "@outputs('Put')?['body']?['_errorCode']", "RATE_LIMIT" ] },
      "actions": {
        "IncrementAttempt": { "type": "IncrementVariable", "inputs": { "name": "attempt", "value": 1 } },
        "WaitAsTold": {
          "type": "Wait",
          "inputs": { "interval": { "count": "@int(substring(outputs('Put')?['body']?['remaining'], 0, sub(length(outputs('Put')?['body']?['remaining']), 1)))", "unit": "Second" } }
        }
      },
      "runAfter": { "Put": [ "SUCCEEDED", "FAILED" ] }
    }
  }
}

Two gotchas to watch for:

• Set retryPolicy: none on the HTTP action. Otherwise the built-in retry and your Until loop multiply each other.
• Initialize attempt at the top level and reset it per row. Logic Apps doesn't allow InitializeVariable inside a loop.

A 429 isn't always a 429 by the time your workflow sees it. retryPolicy: none disables retries for ordinary failures, but Logic Apps still enforces its own non-configurable retry on 429 Too Many Requests. Most 429s are silently retried by the platform and report Succeeded without ever reaching IsRateLimited. Our Until loop is a second line of defense once the platform's own retries run out — in one captured run, that took 4 platform retries (~1 minute) before Put finally surfaced Failed and our custom wait took over.

5. Call yourself for the next chunk

When the chunk finishes, if Skip + rowsRead < totalRowCount, the workflow POSTs the next cursor to its own HTTP trigger:

"HasMoreRows": {
  "type": "If",
  "expression": {
    "less": [
      "@add(coalesce(triggerBody()?['Skip'], 0), length(body('ReadChunk')?['rows']))",
      "@body('ReadChunk')?['totalRowCount']"
    ]
  },
  "actions": {
    "TriggerNextChunk": {
      "type": "Http",
      "inputs": {
        "method": "POST",
        "uri": "@parameters('chunkTriggerUrl')",
        "headers": { "Content-Type": "application/json" },
        "body": {
          "ContainerName": "@triggerBody()?['ContainerName']",
          "BlobName":      "@triggerBody()?['BlobName']",
          "Skip": "@add(coalesce(triggerBody()?['Skip'], 0), length(body('ReadChunk')?['rows']))",
          "Take": "@coalesce(triggerBody()?['Take'], parameters('chunkSize'))",
          "Depth": "@add(coalesce(triggerBody()?['Depth'], 0), 1)"
        }
      },
      "retryPolicy": { "type": "none" }
    }
  },
  "runAfter": { "LogSummary": [ "SUCCEEDED", "FAILED" ] }
}

HasMoreRows runs on both SUCCEEDED and FAILED so one bad row doesn't strand the rest of the file; failures are logged per chunk while the chain continues.

Two ways the chain can die silently. First: runAfter only watches SUCCEEDED/FAILED — if ReadChunk itself fails, everything downstream (ForEachRow, LogSummary, HasMoreRows) reports Skipped, not Failed, and the chain just stops with nothing surfaced beyond that one run's log. Second: length(body('ReadChunk')?['rows']) on a missing rows key evaluates to 0, so if ReadChunk returns an error body instead of failing outright, Skip never advances and HasMoreRows keeps firing with the same cursor until DepthGuard eventually trips. And TriggerNextChunk itself has retryPolicy: none with nothing watching its failure branch — if that one POST fails, the chain ends one chunk short, silently. None of these throw an error a human will see without going and looking.

Why use a raw Http POST instead of the built-in Workflow action? On paper, Workflow is cleaner — no URL parameter to manage, and runs appear linked in history. In practice, a workflow cannot invoke itself with the built-in action because listCallbackUrl() isn't valid inside an action definition (InvalidTemplate: "not expected at this location").

Using Http introduces a deploy-time chicken-and-egg problem: chunkTriggerUrl only exists after the workflow is deployed, but parameters.json is packaged before deployment. Our CI/CD pipeline resolves this by deploying first, querying the management API for listCallbackUrl, patching parameters.json on the live app, and restarting. If that patch step fails partway, the live app is left with a stale or missing chunkTriggerUrl until the next successful deploy — worth a post-deploy health check rather than trusting it blindly.

6. Put a hard brake on recursion

Every recursive workflow needs a guaranteed exit. Normally HasMoreRows (§5) stops the chain once Skip reaches totalRowCount.

But that trusts ReadChunk to never return 0 rows while Skip < totalRowCount. If a parsing bug ever caused an empty slice mid-file, Skip would stop advancing and the workflow would call itself infinitely.

To guarantee termination, we increment Depth on every self-call and check it before processing any rows:

"DepthGuard": {
  "type": "If",
  "expression": {
    "greaterOrEquals": [ "@coalesce(triggerBody()?['Depth'], 0)", "@parameters('maxDepth')" ]
  },
  "actions": {
    "Terminate_DepthExceeded": {
      "type": "Terminate",
      "inputs": {
        "runStatus": "Failed",
        "runError": { "code": "MaxDepthExceeded", "message": "Chunk chain exceeded maxDepth without finishing." }
      }
    }
  }
}

Size maxDepth generously above the largest file you expect, and build it in from day one — retrofitting it later means updating every call site that constructs the trigger payload.

7. Claim the blob before starting

To prevent duplicate trigger fires or re-uploads from kicking off parallel chains on the same file, the receiver workflow:

1. Claims the blob by copying it to an in-progress container.
2. Triggers chunk 0 via the DisableAsyncPattern Workflow action (§2) with { Skip: 0, Take: chunkSize, Depth: 0 }.
3. Deletes the original blob only after verifying the processor trigger returned 2xx (202 Accepted).

Proof: what an actual run looks like

Here is a real trace on a 12-row CSV with chunkSize = 5 — first the receiver's claim-and-trigger run, then a single chunk run in the processor:

That single upload produced three chained runs. Skip advances by the rows read in each pass until it hits Total rows:

Run	Depth	Skip	Take	Rows read	Processed through	Total rows	Status
1	0	0	5	5	5	12	Succeeded
2	1	5	5	5	10	12	Succeeded
3	2	10	5	2	12	12	Succeeded

And here is the §4 retry loop in action when a row exhausted the platform's built-in 429 retries and fell through to our Until loop:

Attempt 1 ran for 1m 7s while the platform's internal retries played out invisibly before Put finally returned Failed (TooManyRequests). IsRateLimited then took the True branch, waited as instructed, and succeeded on attempt 2.

What you get

	Single run + For each	Self-calling chunks
Run history	1 run, thousands of iterations	N runs, ~100 rows each
Restart after failure	Re-run everything	POST {Skip: 2700, Take: 100} yourself
429 waits	Accumulate in one long run	Spread across short runs
Tuning	Change the definition	Change one parameter
Monitoring	One giant blob of output	Per-chunk summary logs

Targeted restart is the biggest win in practice: when a chunk fails due to a mapping bug, we fix the map, deploy, and re-POST that single chunk's cursor instead of reprocessing the entire file.

When not to do this

• Strict ordering or exactly-once across the whole file. Chunks run independently, so per-row operations must be idempotent (our target API uses upsert), or you should use an ordered queue.
• High-frequency event streams. For tens of thousands of messages per hour, use Service Bus with a queue-triggered workflow. This pattern is built for batch file uploads.
• Linked parent/child run history. Because the self-call uses a plain Http action (§5), each chunk appears as an independent top-level run in Monitoring.

Appendix: real recursion is possible, but you don't want it here

Out of curiosity, we tried to make the self-call actually recursive — each run blocking on its child, the way Factorial(n) blocks on Factorial(n-1).

Genuine recursion turned out to need neither the built-in Workflow action nor its InvalidTemplate problem from §5 — just a plain Http self-call (identical to §5) plus one addition: a Response action with operationOptions: "Asynchronous".

"Resp": {
  "type": "Response",
  "kind": "Http",
  "inputs": { "statusCode": 200 },
  "operationOptions": "Asynchronous"
}

That gives the trigger's response a Location header — and Logic Apps' Http action automatically polls any 202 + Location response to completion. Compare the two triggers directly:

# wf_chunk_processor (§5, no Response action) — final answer, nothing to poll
HTTP/1.1 202 Accepted
Content-Length: 0

# wf_chunk_processor_recursive (Response action added) — more coming
HTTP/1.1 202 Accepted
Transfer-Encoding: chunked
Location: https://.../runs/{runId}/operations/{opId}?...
Retry-After: 10

A plain Http action calling the second one won't just read that 202 and move on — it follows Location until the real result lands. So the self-call in §5 stops being fire-and-forget: it now blocks until the whole chain beneath it finishes, same as a real recursive call waiting on its return value.

And since it blocks, each run can read its child's response and fold it into its own — same as Factorial(n) combining n with Factorial(n-1)'s result. We used it to bubble every chunk's run ID back up to the original caller:

"runIds": "@union(createArray(workflow()?['run']?['name']), if(equals(actions('TriggerNextChunkHttp')?['status'], 'Succeeded'), coalesce(body('TriggerNextChunkHttp')?['runIds'], json('[]')), json('[]')))"

Here's chunk 0's final Resp output on a real run — note HasMoreRows took 32 seconds (it's blocked on the entire rest of the chain, not just its own chunk), and the response body carries all three run IDs, collected from the bottom of the chain back up:

Don't use this for chunked file processing, though — it reintroduces the exact problem the whole pattern exists to avoid. The first chunk's run stays open for the entire file's processing time, not just its own slice; a 50-chunk file goes back to being one long-lived run instead of 50 short ones. It's a genuinely interesting result, just the wrong tool for this job.

---

Have you built something similar, or hit a limit I haven't? I'd like to hear about it.