Partition bulk records into accepted and rejected lists

#partition #dataweave #arrays #validation

Problem

My DataWeave script threw an unresolved reference error when calling partition without an explicit import. The Arrays module must be imported to your DataWeave code by adding the line import * from dw::core::Arrays to the header of your DataWeave script docs.

Input

{
  "records": [
    { "id": "ORD-1001", "email": "ana@example.com" },
    { "id": "ORD-1002", "email": "bad-address" },
    { "id": "ORD-1003", "email": "raj@example.org" },
    { "id": "ORD-1004" },
    { "id": "ORD-1005", "email": "mei@example.net" }
  ]
}

Working configuration

%dw 2.0
// partition() lives in dw::core::Arrays — not imported by default
import * from dw::core::Arrays
output application/json
// keys are ALWAYS success/failure — re-map them to domain names
var split = payload.records partition (r) ->
    (r.email default "") matches /.+@.+\..+/
---
{
  accepted: split.success map (r) -> r.id,
  rejected: split.failure map (r) -> {
    id: r.id,
    reason: "missing or invalid email"
  },
  retryCount: sizeOf(split.failure)
}

Output

{
  "accepted": [
    "ORD-1001",
    "ORD-1003",
    "ORD-1005"
  ],
  "rejected": [
    {
      "id": "ORD-1002",
      "reason": "missing or invalid email"
    },
    {
      "id": "ORD-1004",
      "reason": "missing or invalid email"
    }
  ],
  "retryCount": 2
}

The accepted array contains IDs from records with valid emails, while the rejected array holds objects for records missing or invalidating email addresses. The retryCount field reports the size of the failure partition as two.

The trap

The trap is an unresolved reference error when calling partition without importing the Arrays module.

What I do now

I add import * from dw::core::Arrays to the DataWeave header before using partition. I map the success and failure keys to domain names like accepted and rejected. I calculate retry counts using sizeOf on the failure partition.

Runs shown: DataWeave CLI 2.12.2

104 DataWeave patterns + 8 Exchange modules with 208 MUnit tests: github.com/shakarbisetty/mulesoft-cookbook | 60-second walkthroughs: youtube.com/@SanThaParv

DEV Community