Structured Matching, Patching, and Diffing Done Right

If you work with YAML or JSON configuration---Kubernetes manifests, Helm charts, IaC definitions, anything really in software ---you've probably hit the same wall. You start with a nice declarative format. Then you need to diff two versions, so you reach for diff. You need to patch a field, so you reach for yq or jq. You need to match documents in a stream, so you reach for grep. Before long, your "declarative" pipeline is held together by text munging.

The higher-level tools don't escape this either. Helm templates YAML as strings. You can't diff two YAML documents as YAML---you convert to JSON first. Config-as-data systems like kpt and porch target narrow use cases--- package management, WYSIWYG authoring---because they're built on top of a hodgepodge data format model. They hide the mess rather than fix it. And the mess leaks: sooner or later you're back to grep, sed, and yq to glue things together.

But this article isn't about building a better Helm or kpt. It's about getting the underlying data model right. What happens when matching, patching, and diffing all operate on the same typed tree representation, with the same extension mechanism, producing output that feeds back into each other?

That's what Tony format does. It operates directly on the intermediate representation (IR) of structured data---a typed tree of objects, arrays, strings, numbers, booleans, and nulls. Matching, patching, and diffing are all defined as operations on this tree, not on its serialized text. And because the three operations share the same IR and the same tag-based extension mechanism, they compose in ways that text-based tools never could, and they generalize in ways config management tools never could either.

The IR: One Tree to Rule Them All

At the core of Tony is ir.Node, a uniform representation for structured data:

type Node struct {
    Type    Type      // Null, Bool, Number, String, Object, Array
    Fields  []*Node   // object keys
    Values  []*Node   // object values or array elements
    Tag     string    // operation tag, e.g. "!or", "!key(name)"
    String  string
    Bool    bool
    Int64   *int64
    Float64 *float64
}

This IR can be parsed from YAML, JSON, or Tony's own format, and encoded back to any of them. Every operation---match, patch, diff---works on *ir.Node. No serialization boundaries, no format-specific quirks. A patch written against YAML works identically against JSON.

The Tag field is what makes things interesting. Tags are YAML-compatible annotations (e.g. !or, !dive, !key(name)) that extend nodes with operational semantics. They compose via dot-separation: !all.field.glob chains three operations into one. The same tag mechanism drives matching, patching, and diffing, which means the three operations speak the same language.

Every tag resolves to a registered Symbol that implements the Op interface:

type Op interface {
    Match(doc *ir.Node, ctx *OpContext, f MatchFunc) (bool, error)
    Patch(doc *ir.Node, ctx *OpContext, mf MatchFunc, pf PatchFunc, df DiffFunc) (*ir.Node, error)
}

Both methods are on the same interface. A single operation can define both matching and patching behavior---!if tests a condition and applies a transformation in one step. The MatchFunc and PatchFunc callbacks enable recursive delegation back to the engine, threading context through the entire tree. New operations can be added without modifying the core engine: just register a new symbol.

The Three Operations

Matching

Tony's Match(doc, pattern) answers a simple question: does this document satisfy this pattern? But "pattern" means something much richer than string equality.

A plain match is structural. For objects, every field in the pattern must exist in the document with a matching value. For arrays, elements match positionally. For scalars, values must be equal. Crucially, the pattern is a subset---the document can have fields the pattern doesn't mention.

# Document
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: production

# Pattern - matches because all specified fields are present and equal
kind: Deployment
metadata:
  namespace: production

This is already more useful than grep 'kind: Deployment' because it understands nesting. But tags unlock the real power:

# Match any Deployment or StatefulSet in the monitoring namespace
kind: !or [Deployment, StatefulSet]
metadata:
  namespace: !glob 'monitor*'

!or matches if any alternative matches. !glob matches against a glob pattern. !and, !not, !has-path, !type, !field, and !subtree round out the vocabulary. !subtree searches the entire document depth-first. !let binds variables before matching. Each callback delegates sub-matching back to the engine, so compound operations compose naturally.

Patching

Tony's Patch(doc, patch) transforms a document by merging a patch into it. Without tags, this works like a structural merge patch:

• Objects merge field-by-field. Patch fields override document fields; document fields not in the patch are preserved.
• Arrays merge positionally up to the shorter length, then the patch's remaining elements are appended.
• Scalars replace.

# Document
metadata:
  name: frontend
  labels:
    app: web
spec:
  replicas: 1

# Patch
spec:
  replicas: 3
  strategy:
    type: RollingUpdate

# Result
metadata:
  name: frontend
  labels:
    app: web
spec:
  replicas: 3
  strategy:
    type: RollingUpdate

The patch only mentions what changes. Everything else is left alone. This is the same mental model as JSON merge patch (RFC 7396), but extended to arbitrary structured data with tag support.

Tags turn simple patches into a transformation language:

!delete removes a field:

metadata:
  annotations:
    old-annotation: !delete null

!if conditionally applies patches:

!if
  if:
    kind: Deployment
  then:
    spec:
      replicas: 3
  else:
    !pass null

!dive recursively searches the document tree, applying conditional patches to every subtree. This is how you express transformations over arbitrarily nested structures without knowing the depth ahead of time.

A real example: Kubernetes CRDs embed OpenAPI schemas that can be thousands of lines deep. Stripping description fields from podTemplate subtrees to reduce CRD size used to require a shell script. With !dive, it's a patch:

# Match any CRD, then dive into its version schemas
- match:
    kind: CustomResourceDefinition
  patch:
    spec:
      versions: !dive
      - match:
          podTemplate: null
        patch:
          podTemplate:
            description: !delete null
            properties: !dive
            - match: !irtype {}
              patch:
                description: !delete null

!dive walks the tree bottom-up, applying each match/patch pair at every node. The outer dive finds any podTemplate subtree anywhere in the version schema. The inner dive then strips description from every object inside it. No matter how deeply nested the schema is, the patch handles it. The result: you can ship CRDs with pod templates inside without repeating megabytes of repeated descriptions.

!all applies a patch to every element of an array or every value of an object. !pipe shells out to an external command (marked unsafe, can opt out).

Keyed Lists

One of the most practical features is !key(field), which tells Tony to treat an array as a map keyed by a field value. This solves the classic problem of patching Kubernetes arrays like containers or volumes:

# Document
spec:
  containers: !key(name)
  - name: app
    image: myapp:v1
  - name: sidecar
    image: proxy:v1

# Patch
spec:
  containers: !key(name)
  - name: app
    image: myapp:v2

Without !key, array patching is positional---swap the order and your patch breaks. With !key(name), Tony matches elements by their name field and merges them structurally. The sidecar container is untouched because the patch doesn't mention it.

Diffing

Tony's Diff(from, to) computes minimal/small structural difference between two documents. The output is itself a valid Tony document that moreover contains the common ancestors of a fine grained diff, making it easier to read than jsondiff which uses path references.

Diff uses tags to annotate what changed:

• !replace when types differ: !replace { from: "old", to: "new" }
• !delete for removed fields or elements
• !insert for added fields or elements
• !strdiff for character-level string changes
• !arraydiff for array changes with key matching
• !addtag / !rmtag / !retag for tag-only changes

Fields that are identical are omitted entirely. The diff is minimal.

For strings, The Go Tony o tool computes character-level diffs using the diff-match-patch algorithm. But it's pragmatic: if the diff is larger than half the size of the smaller string, it falls back to a simple !replace. No point in a character-level diff that's harder to read than the replacement.

When arrays are tagged with !key(field), diffing matches elements by key value instead of position. Reordering doesn't produce noise---only actual additions, removals, and modifications are reported:

# from
containers: !key(name)
- name: app
  image: v1
- name: sidecar
  image: proxy:v1

# to
containers: !key(name)
- name: sidecar
  image: proxy:v2
- name: app
  image: v1

# diff - only the sidecar image changed, reordering is ignored
- name: sidecar
  image: !replace
    from: proxy:v1
    to: proxy:v2

For plain arrays (without !key), !arraydiff uses an abstracted longest-common-subsequence to produce a positional diff with !insert and !delete entries---and both the result and the per-item differences are valid patches:

# from
items:
- a
- b
- c

# to
items:
- a
- x
- c
- d

# diff
items: !arraydiff
  1: !replace
    from: b
    to: x
  3: !insert d

Diffs are reversible. libdiff.Reverse(diff) produces a diff that, when applied to to, yields from:

diff := Diff(a, b)
reversed, _ := libdiff.Reverse(diff)
// Patch(b, reversed) == a

The reversal logic is straightforward because the tags carry complete information: !delete becomes !insert and vice versa, !replace swaps its from and to fields, !retag(x,y) becomes !retag(y,x).

Composition

The real payoff is that these three operations form a closed loop.

A diff is a valid patch. The same tags that annotate changes (!delete, !insert, !replace) are recognized by the patch engine. Diff(a, b) produces output that you can pass directly to Patch(a, diff) to get b. Reverse the diff and apply it to b to get a.

Matching composes with patching. !if uses match semantics in its condition and patch semantics in its body. !dive matches to find subtrees and patches to transform them. The Op interface makes this explicit---every operation is both a matcher and a patcher, and the engine threads recursive callbacks for both through the entire tree.

Diffs compose with matching. Because a diff is structured Tony data, you can match against it, filter it, or patch it before applying it. A diff isn't an opaque blob you feed to patch---it's a document you can inspect and transform with the same tools you use on any other document.

This isn't three systems bolted together. It's one system and one structure-preserving format with three entry points. Match, patch, and diff are perspectives on the same tree-walking, tag-dispatching engine. The Op interface, the IR, and the tag registry are shared all the way down.

Conclusion

Three operations. One IR. One tag system. Matching, patching, and diffing are the same operation viewed from different angles, and the model makes that explicit. That's the whole idea.