5 DataWeave Array Traps That Silently Destroy Your Data

Last October I reviewed a Mule integration that had been running for 3 months. The flow pulled employee records from Workday, filtered active ones, and pushed them to an HR portal. Everything looked clean. No errors in CloudHub. No alerts.

The downstream portal had 2,400 employees. Workday had 4,200.

Forty percent of the records had been silently dropped. The filter was the problem, and the fix was one character.

I found 4 more traps like this in the same project. All silent. All invisible until someone compared the numbers manually. Here are all 5, with the exact code that breaks and the exact code that fixes it.

TL;DR

• filter silently drops records when you compare String to Number (no error, no warning)
• reduce with wrong accumulator shape gives you wrong totals for weeks
• groupBy returns an Object, not an Array — chaining map after it throws a cryptic error
• distinctBy keeps the first match and silently drops duplicates from other systems
• zip truncates to the shorter array — extra records vanish without warning

Trap 1: filter Silently Eats Records

This is the most dangerous one. No error. No warning. Records just disappear.

The cookbook pattern for filtering active employees:

%dw 2.0
output application/json
---
payload filter (employee) -> employee.active == true

Input (5 employees, 3 active):

[
  {"name": "Alice Chen", "age": 30, "department": "Engineering", "active": true},
  {"name": "Bob Martinez", "age": 25, "department": "Marketing", "active": false},
  {"name": "Carol Nguyen", "age": 35, "department": "Engineering", "active": true},
  {"name": "David Kim", "age": 28, "department": "Sales", "active": false},
  {"name": "Elena Rossi", "age": 32, "department": "Engineering", "active": true}
]

Output: 3 records. Works perfectly.

Now the trap. A different source system sends the active field as the String "true" instead of Boolean true. Same data, different type.

payload filter (employee) -> employee.active == true

Output: empty array. Zero records. No error.

DataWeave's == operator checks type AND value. String "true" is not equal to Boolean true. The comparison returns false for every record. filter removes all of them.

The fix is one operator:

payload filter (employee) -> employee.active ~= true

The ~= operator coerces types before comparing. String "true" ~= Boolean true returns true. Records stop vanishing.

I've seen this bite 3 different teams in the past year. One ran with empty downstream data for 11 weeks before a headcount audit caught it. Another had a status field that came as the Number 1 from a REST API and the String "1" from a database query. Same field name, different type, depending on which system sent the message. Their filter caught records from one source and silently dropped the other.

The DataWeave Playground won't warn you either. It runs the filter, gets an empty array, and shows [] with a green checkmark. Everything looks normal.

Rule: never use == in a filter unless you control both sides of the comparison. Use ~= when the source type isn't guaranteed.

Trap 2: reduce With Wrong Accumulator Shape

The cookbook pattern for computing invoice totals:

%dw 2.0
output application/json
var taxRate = 0.075
var totals = payload.lineItems reduce (item, acc = {hours: 0, amount: 0}) ->
  ({
    hours: acc.hours + item.quantity,
    amount: acc.amount + (item.quantity * item.unitPrice)
  })
---
{
  invoiceId: payload.invoiceId,
  customer: payload.customer,
  totalHours: totals.hours,
  subtotal: totals.amount,
  tax: totals.amount * taxRate,
  total: totals.amount + (totals.amount * taxRate)
}

This works because the accumulator acc = {hours: 0, amount: 0} matches the shape of what reduce returns on each iteration.

The trap: you write acc = 0 because you think you're just summing a number.

var totals = payload.lineItems reduce (item, acc = 0) ->
  acc + (item.quantity * item.unitPrice)

This gives you a single number. But if you later try totals.hours, you get null. No error. Your invoice shows totalHours: null and subtotal: 23500.00. The finance team doesn't notice the missing hours for 2 weeks.

The shape of the accumulator must match the shape of what you return inside the lambda. acc = 0 means you return a Number. acc = {hours: 0, amount: 0} means you return an Object with both fields.

Rule: always declare your accumulator with the exact shape you need in the output. If you need 2 fields, initialize both.

---

100 production-ready DataWeave patterns with tests: mulesoft-cookbook on GitHub

---

Trap 3: groupBy Returns an Object, Not an Array

The cookbook pattern for grouping orders by category:

%dw 2.0
output application/json
---
payload groupBy (order) -> order.category

Input:

[
  {"orderId": "ORD-1001", "product": "Laptop", "category": "Electronics", "amount": 1299.99},
  {"orderId": "ORD-1002", "product": "Headphones", "category": "Electronics", "amount": 199.95},
  {"orderId": "ORD-1003", "product": "Desk Chair", "category": "Furniture", "amount": 449.00}
]

Output is an Object keyed by category:

{
  "Electronics": [{"orderId": "ORD-1001", ...}, {"orderId": "ORD-1002", ...}],
  "Furniture": [{"orderId": "ORD-1003", ...}]
}

The trap: you chain map after groupBy.

payload groupBy $.category map (group) -> { count: sizeOf(group) }

This throws: Cannot coerce :object to :array. Because groupBy returned an Object and map expects an Array.

The fix: use mapObject instead of map. Or use pluck to convert back to an array.

(payload groupBy $.category) mapObject (items, category) -> {
  (category): {
    count: sizeOf(items),
    total: items reduce (item, acc = 0) -> acc + item.amount
  }
}

I spent 40 minutes staring at that coercion error the first time. The error message says "Cannot coerce :object to :array" but it doesn't tell you which function returned the Object. You have to know that groupBy always returns an Object — it's not in the error.

The other approach that works is pluck, which converts the Object back into an Array of key-value pairs:

(payload groupBy $.category) pluck (items, category) ->
  {category: category as String, count: sizeOf(items)}

Use mapObject when you want to keep the grouped Object structure. Use pluck when you need an Array back.

Rule: after groupBy, use mapObject or pluck. Never map.

Trap 4: distinctBy Keeps the First Match and Drops Everything Else

The cookbook pattern for deduplicating records:

%dw 2.0
output application/json
---
payload distinctBy (customer) -> customer.customerId

Input (same customer from 3 systems):

[
  {"customerId": "C-100", "name": "Alice Chen", "email": "[e-mail]", "source": "Salesforce"},
  {"customerId": "C-101", "name": "Bob Martinez", "email": "[e-mail]", "source": "Salesforce"},
  {"customerId": "C-100", "name": "Alice Chen", "email": "[e-mail]", "source": "SAP"},
  {"customerId": "C-102", "name": "Carol Nguyen", "email": "[e-mail]", "source": "Salesforce"},
  {"customerId": "C-101", "name": "Robert Martinez", "email": "[e-mail]", "source": "HubSpot"}
]

Output: 3 records. The Salesforce versions of C-100 and C-101.

The trap: Alice's SAP record has [e-mail]. Her Salesforce record has [e-mail]. distinctBy kept the Salesforce record and dropped the SAP record. Silently.

If SAP had the more current email, you just lost it. Bob's HubSpot record says "Robert Martinez" (his legal name), but distinctBy kept "Bob Martinez" from Salesforce. That could break a compliance check.

distinctBy keeps the first record in the array that matches the key. Every subsequent duplicate is thrown away regardless of which source is more authoritative.

The fix is to sort by source priority before deduplicating:

var priorityOrder = {"SAP": 1, "HubSpot": 2, "Salesforce": 3}
---
(payload orderBy priorityOrder[$.source])
  distinctBy (customer) -> customer.customerId

Now SAP records win over HubSpot, and HubSpot wins over Salesforce. The authoritative source comes first, and distinctBy keeps it.

Rule: if deduplicating across sources, sort by source priority BEFORE distinctBy. Put the authoritative source first in the array.

Trap 5: zip Truncates to the Shorter Array

The cookbook pattern for combining parallel arrays:

%dw 2.0
output application/json
var pairs = payload.names zip payload.scores
---
pairs map (pair) -> ({name: pair[0], score: pair[1]})

Input:

{
  "names": ["Alice", "Bob", "Carol"],
  "scores": [92, 87, 95]
}

Output: 3 pairs. Clean.

The trap: the arrays have different lengths.

{
  "names": ["Alice", "Bob", "Carol", "David", "Elena"],
  "scores": [92, 87, 95]
}

zip produces 3 pairs. David and Elena are silently dropped. No error. No warning that 2 names had no matching scores.

This happens in production when one API returns 50 records and the other returns 47 because of a pagination bug upstream. You lose 3 records and nobody knows until the reconciliation report at month-end. I've seen this with a payroll integration where one system had 312 employee IDs and the other had 309 after a terminated-employee purge ran on one side but not the other. Three paychecks went missing.

The core problem: zip assumes both arrays are the same length. It doesn't check. It doesn't warn. It just stops at the end of the shorter one.

Rule: before zip, check sizeOf on both arrays. If they differ, log it and decide — pad the shorter one, truncate the longer one, or fail explicitly.

var names = payload.names
var scores = payload.scores
---
if (sizeOf(names) != sizeOf(scores))
  error("Array length mismatch: $(sizeOf(names)) names vs $(sizeOf(scores)) scores")
else
  (names zip scores) map (pair) -> ({name: pair[0], score: pair[1]})

What All 5 Traps Have in Common

They're silent. No exception. No log line. No CloudHub alert. The data comes out wrong and you don't know it until someone counts the rows manually or an auditor asks why the numbers don't add up.

Every one of these is in the mulesoft-cookbook with the correct pattern, input data, and expected output you can test in the DataWeave Playground.

I run through these 5 checks on every integration I review now. Takes 10 minutes. Has caught production bugs 3 times in the past 6 months. The filter trap alone was responsible for 2 of those catches.

---

100 patterns with MUnit tests: github.com/shakarbisetty/mulesoft-cookbook

60-second video walkthroughs: youtube.com/@SanThaParv