DEV Community

Cover image for CRM integration. Shall we talk about it?
DimiDan
DimiDan

Posted on

CRM integration. Shall we talk about it?

You've got one CRM integration in production and it's been fine for months. Now there's a second one to wire up. Same contract, same schema, same node shape. Should be a couple of days.

You pull the first payload from CRM B and put it next to CRM A out of habit.

CRM A, a monetary field:

{ "key": "estimated_cost", "value": 1275.43, "spec": { "type": "number", "format": "decimal" } }
Enter fullscreen mode Exit fullscreen mode

CRM B, same kind of field:

{ "key": "amount", "value": "6624", "spec": { "type": "number" } }
Enter fullscreen mode Exit fullscreen mode

Huh.

One's a JSON number. One's a string. One tags the format, one doesn't. Both say "type": "number". And every validator between those adapters and your UI is perfectly happy with both.

So what's the issue

This is the part that takes a minute to sit with. You go looking for who got it wrong, and nobody did.

Go read the contract. It says there's a field called value. That's it. It never said what could go in value. So the person building CRM A looked at a number and sent a number. The person building CRM B looked at the same thing and sent a string, probably because that CRM's own API hands back strings. Both reasonable. Both shipped. Both correct against the spec as written.

The contract wasn't violated. It just didn't have an opinion, and two people filled the silence differently.

That's the whole category of problem I want to talk about, because it doesn't show up in code review and it doesn't show up in CI. It shows up eight months later when someone asks why one CRM's totals are off by a cent.

The good idea underneath

Before piling on, credit where it's due, because the foundation here is right.

Every source produces the same node. Consumers never branch on where the data came from.

type FieldNode = {
  key: string
  label: string
  value: ???
  spec: ???
  treePath: string
  children: FieldNode[]
}
Enter fullscreen mode Exit fullscreen mode

One shape, one tree, one set of consumers. Adding a CRM is adapter work and nothing above the boundary moves. This genuinely scales, and if you're designing this today, start here.

But notice what it does to your risk. Two fields carry all the meaning — value and spec. The rest is plumbing. Leave those two vague and the uniformity is a paint job.

Rule 1: name the field, then actually say what goes in it

Here's the type that caused everything above:

value: string | number | null
Enter fullscreen mode Exit fullscreen mode

Looks like a decision. Isn't one. It's a union that permits both answers, which means you'll get both answers the moment two adapters get written in parallel by people who never talk.

So say the thing:

  • value is a decimal string, or null. Never a JSON number.
  • Containers (array, object) carry null.
  • Numeric nodes carry a format.
  • format: 'money' also carries currency, ISO 4217.

That last one isn't bureaucracy. format: 'decimal' can't tell a cost from a tax rate. They're both decimals. Only the source system knows which one is money, and that fact turns out to constrain your whole architecture. Hold that thought for Rule 3.

And if you're about to argue you don't need strings because you're not doing currency math: value: number still eats integers past 2^53, which is where record IDs live. It mangles anything past 15 significant digits. It'll hand you 1e+21 when you least want it. A decimal string has none of those problems and costs you nothing.

Rule 2: shut the door on undeclared fields

Here's the sequel to the story, and it's more common than the first part.

CRM A's adapter stamps format: "decimal" on every numeric field. Handy. Consumers start reading it. Someone builds a formatting rule on top of it.

Except format was never declared in the schema. It's just riding along in the payload — invisible to validation, missing from your generated client types, guaranteed by nobody. And CRM B doesn't send it at all, so the formatting rule works for half your customers.

Whatever flavor you get, it's one bug: data present, schema silent.

One flag closes the whole category. additionalProperties: false in OpenAPI, .strict() in zod, whatever yours is called. Undeclared fields stop being possible. Everything else you do about this is whack-a-mole.

Fair warning: turning it on will break things that currently work, because at least one of those undeclared fields is load-bearing somewhere. That's not a reason to skip it. The dependency was already there — you just couldn't see it.

Rule 3: the normalization layer can't save you

Okay, so the obvious fix. Build a normalization layer. One place that takes whatever each adapter produces and forces it into canonical form. Money becomes a decimal string, numerics get a format tag, everyone goes home.

You do need that layer. It also can't do the job on its own, and this is the bit I'd most want to hand to past-me.

Picture a money field whose true value is 450.20.

  • Adapter one does JSON.parse on the CRM response. The value lands in a float64, comes back out as "450.2". Normalization checks it: it's a string, it matches a decimal literal, format's present. Passes.
  • Adapter two reads the response as text and never touches a float. Emits "450.20". Normalization checks it. Passes, identically.

Your layer cannot tell these apart. Both are well-formed strings. One of them quietly threw information away, and no amount of schema work will catch it, because a validator can only confirm the shape of what you handed it. It can't reconstruct what got dropped before it ever saw the data.

So this isn't one boundary. It's two, doing different jobs:

Boundary Job Why it can't fold into the other
Read-time — inside each adapter Decode source numerics as text. json.Number in Go, a reviver in JS, never bare JSON.parse. Tag format and currency from provider field metadata — HubSpot's Properties API, Salesforce describe(). It's the only layer that touches the raw HTTP response. Precision lives or dies here and nowhere else. It's also the only place that knows Amount is money and source_user_id isn't.
Enforcement — the shared service Strict schema. Validate your own output. Fail closed. Derive format when an adapter didn't send one. Own the spec. It's the only layer that sees every adapter side by side. One stamping format while another skips it is completely invisible from inside either adapter.

Read-time on its own is just a promise everyone makes and nobody keeps, because "be careful with floats" isn't a mechanism. Enforcement on its own validates already-damaged data and hands you a green checkmark.

If you keep one line from this post: the guarantee gets created upstream of where it gets checked. Any design that stuffs all the normalization into one shared service has a hole in it that more schema won't fill.

This is also, incidentally, why "which service owns this fix?" can sit open in a doc for weeks. Every answer is partly right and none of them is enough. When that happens, the question is usually the problem. This one quietly assumed there was one owner.

Rule 4: the mapper that quietly eats your fields

Different failure, same silence.

Early on, your wire type and your domain type are the same type. The only mapper that compiles is a spread:

fields.map(f => ({ ...f, label: f.display_label }))
Enter fullscreen mode Exit fullscreen mode

Nothing can go missing. Not because anyone's being careful — because there's no other way to write it.

Then the domain type picks up something the wire doesn't have. A render hint, say, or a UI kind. Perfectly reasonable in isolation. But now the two types have forked, so you need a translation function, and a translation function is a list of fields:

const map = (f: ApiField): FieldNode => ({
  key: f.key,
  label: f.display_label,
  type: f.children?.length ? 'group' : 'leaf',
  value: String(f.value),
  treePath: f.path,
  children: f.children?.map(map) ?? [],
})
Enter fullscreen mode Exit fullscreen mode

Whatever isn't on that list is gone. Including every field anyone adds to the contract next quarter. And you won't catch it in review, because that mapper is correct for the types it was handed.

Derive the domain type from the wire type instead, and forgetting a field becomes a compile error:

type FieldNode = Omit<WireField, 'display_label'> & { label: string }
Enter fullscreen mode Exit fullscreen mode

Worth saying the other half too: the spread version preserved fields by being transparent, not by being right. It happily carried undeclared junk along with everything else, which is Rule 2's problem wearing a different hat. You want both halves. Parse strictly, then pass through.

Rule 5: don't add a second way to say what something is

Tempting move: put a declared "kind" field next to your structural type. spec.type plus tree position already tells you everything behavioral — expandable, container, nested — but a kind field feels more explicit, so it goes in.

Two things go wrong. It drifts from the structure it's supposed to describe, which is the boring failure. And it tends to land on a property name the wire already uses for something else. Look at type in that mapper above: it's the UI kind in the domain type and the source record type on the wire. That collision is exactly what forced the field-enumerating translation in Rule 4.

Cheap thing to try on your own code right now: for every enum on a core domain type, grep for reads of each member. The ones that get written and never read are telling you the type is carrying a distinction nothing actually needs.

The schema - what it should enforce

value never gets coerced. z.coerce.string() would quietly accept a raw number and enforce absolutely nothing. You want a bad value to fail, not get tidied up behind your back.

The decimal-literal regex is your precision tripwire. Anything that went through a float and back emits exponent notation at large magnitudes, and 1e+21 doesn't match. It only catches the loud cases. The quiet ones are Rule 3's job.

spec is required. Optional-chaining a discriminator is itself the bug. spec?.type makes "field missing" and "value unknown" look identical, and both slide into the same default branch.

currency sits next to format, not inside it. Rule 1's point: only the source system knows which decimals are money, so that tag has to be born at read time.

Where to actually enforce this

Writing the schema is the easy part. Where you run it decides whether it does anything.

The producer validates its own output. Your shared service checks its response before sending and fails closed. Skip this and consumer-side validation just turns a silent bug into a loud one, later, in somebody else's service, at 2am.

The consumer parses at the boundary, per node. A recursive strict schema blows up the entire tree over one malformed grandchild, which means a whole panel disappears because of one bad leaf. Walk the tree, validate node by node, keep the bad ones and mark them unavailable, and report them. Silently dropping them is its own kind of lie — the user just sees a shorter list and no explanation.

And know what none of this covers. A perfectly-shaped value that lost precision upstream passes every check on that list. The read-time boundary needs a shared decode helper and human review, not a test. Fixture-based tests have a cousin of the same blind spot: they'll stay green while the same code fails on the live route, because your fixtures carry fields the real projection drops. Make at least one test exercise the projection itself.

The short version

  • For every field in your contract: is the content specified, or just the name? A union that permits two encodings is an unspecified field wearing a type annotation.
  • Is additionalProperties: false on? If not, you have undeclared fields in production right now, and one of them matters.
  • Where does precision get created versus checked? Different layers means your validator can't verify the thing it looks like it's verifying.
  • Does any mapper enumerate fields? It's dropping something today and it'll drop everything you add tomorrow.
  • Does the producer validate its own output, or only the consumer?
  • Which members of your core enums actually get read?

A contract isn't the fields you named. It's the behaviors you made impossible.

Top comments (0)