DataWeave partition() for Bulk API Responses: Stop Returning Silent 200 OKs

Our bulk import API returned 200 OK on every request. Nobody noticed that 40% of records were failing silently for 3 days.

TL;DR

• partition() from dw::core::Arrays splits an array into success/failure groups in one pass
• Build summary (total, succeeded, failed, successRate) + per-record results + error extraction
• Empty batch trap: successCount / total * 100 divides by zero on empty arrays
• The caller needs to know EXACTLY what failed and why — not just "200 OK"

The Problem: Silent Bulk Failures

We had a bulk import API. Clients sent 5,000-10,000 records per batch. The API processed each record individually — some succeeded, some failed (duplicate keys, invalid formats, missing fields).

The old response:

{"status": "OK", "message": "Batch processed"}

That's it. No per-record status. No failure count. No error details. The client had no idea which records failed. They found out 3 days later when reconciliation showed missing data.

The Solution: partition() + Summary Builder

%dw 2.0
import partition from dw::core::Arrays
output application/json
var parts = payload partition (item) -> item.status == "SUCCESS"
var successCount = sizeOf(parts.success)
var total = sizeOf(payload)
---
{
  summary: {
    total: total,
    successful: successCount,
    failed: total - successCount,
    successRate: (successCount / total * 100) as String ++ "%"
  },
  results: payload map {
    id: $.id,
    status: $.status,
    (error: $.error) if $.status == "FAILED"
  }
}

---

100 production-ready DataWeave patterns with tests: mulesoft-cookbook on GitHub

---

How partition() Works

partition() takes an array and a predicate. Returns an object with two keys:

• success: all items where the predicate returned true
• failure: all items where it returned false

import partition from dw::core::Arrays
---
[1, 2, 3, 4, 5] partition (n) -> n > 3
// {success: [4, 5], failure: [1, 2, 3]}

One pass. No need to filter twice. Cleaner than separate filter calls for success and failure.

The Conditional Key Trick

results: payload map {
  id: $.id,
  status: $.status,
  (error: $.error) if $.status == "FAILED"
}

The (error: $.error) if $.status == "FAILED" syntax conditionally includes the error field. Successful records have no error key — it's absent, not null. Clean API response.

Trap: Divide by Zero on Empty Batch

A health check sent an empty array to our bulk endpoint. total = 0. The successRate calculation: 0 / 0 * 100. Runtime error: "Cannot divide by zero."

CloudHub showed a 500 error. The health check saw "API is down." The API was fine — it just crashed on empty input.

The fix:

successRate: if (total > 0) ((successCount / total * 100) as String ++ "%") else "N/A"

One guard. I now add this to every bulk response builder. Empty batches are valid input — your API should handle them gracefully.

What the Caller Gets

Before (useless):

{"status": "OK", "message": "Batch processed"}

After (actionable):

{
  "summary": {
    "total": 5,
    "successful": 3,
    "failed": 2,
    "successRate": "60.0%"
  },
  "results": [
    {"id": "R1", "status": "SUCCESS"},
    {"id": "R2", "status": "FAILED", "error": "Duplicate key"},
    {"id": "R3", "status": "SUCCESS"},
    {"id": "R4", "status": "SUCCESS"},
    {"id": "R5", "status": "FAILED", "error": "Invalid format"}
  ]
}

The caller knows: 60% success rate. R2 failed on duplicate key. R5 failed on format. They can retry just the 2 failures instead of resending all 5.

What I Do Now

Every bulk API I build includes:

1. partition() for success/failure split
2. Summary with counts + percentage
3. Per-record results with conditional error field
4. Empty-batch guard on all division operations
5. The errors array as a convenience extraction for clients that only want failures

---

100 patterns with MUnit tests: github.com/shakarbisetty/mulesoft-cookbook

60-second video walkthroughs: youtube.com/@SanThaParv