vCluster custom resource sync, one plugin for every kind | Kupe Cloud

vCluster runs a full Kubernetes control plane as a pod inside a namespace of a host cluster. We use it to run our tenants' clusters: each one gets cluster-admin inside a real API boundary, without the cost of a dedicated cluster underneath. Anything self-contained inside the virtual cluster works out of the box: Deployments, Services, the operators a tenant installs. The trouble starts when a tenant writes an Argo CD Application that only the central Argo CD on the host can reconcile, an HTTPRoute that needs to attach to a shared Gateway, or an alert rule that belongs on the host ruler. Every multi-tenant platform grows these crossings: objects written on one side of a boundary that a controller on the other side has to act on.

The options for syncing resources across that boundary are thinner than you'd expect. vCluster's open-source syncer handles the core resources well: Pods, Services, Secrets and friends. But the resources we most needed to sync were custom resources, and some of them needed references translated on the way across. vCluster does offer custom-resource syncing and sync patches, but they ship in the Pro image, behind a licence validated against vCluster Platform. That's a reasonable product boundary; it just wasn't a dependency we wanted in the boot and data path of every tenant cluster.

The open-source routes weren't in better shape. Loft's own generic CRD sync plugin pointed in the right direction but has been archived, which leaves the vCluster plugin SDK and a controller of your own.

So that's what we built: an open vCluster generic sync plugin, Apache-2.0, a single plugin process that syncs any resource kind you declare in its configuration. It carries the pieces a real platform needs: a direction and mode per kind, reference translation, label and namespace selectors, and status sync. Every crossing is explicit, reviewable and independent of a licence service.

Resources cross the boundary for different reasons

It's worth naming the actual objects rather than filing them all under “multi-tenant integration”. Each crossing has an owner, a direction and a reason:

• Some objects leave the tenant cluster for a host controller: an Application for the central Argo CD, a PrometheusRule for the host Mimir ruler.
• Some go the same way but need their status back: an HTTPRoute attaches to the host Gateway, and the tenant needs to see whether it was accepted.
• Some only cross on a condition: a dashboard ConfigMap leaves only when it carries the dashboard label.
• Some travel the other way: platform-managed Secret objects come back into the tenant cluster.
• Some are read-only imports: GatewayClass objects are mirrored so tenants can discover the shared gateway classes.

Those aren't interchangeable operations. A route needs its service references translated; a gateway class must remain host-owned; a repository credential should be admitted only by a narrow label selector. A single broad “sync CRDs” switch would hide all of that policy in code and RBAC.

Our plugin makes the policy a list of syncResources entries. Each entry names an apiVersion, kind, a direction (toHost or fromHost) and an optional selector, plus a mode: sync reconciles the source object to its target, while mirror is a read-only import. Patches translate references and nothing else: rewriteName, rewriteNamespace, rewriteRef, rewriteHostRef, rewriteLabelSelector and none are the complete vocabulary.

That restriction is deliberate. A sync configuration should say which reference is being translated and why. It shouldn't become a second, unreviewed templating language.

The plugin protects the host copy

A resource crossing a cluster boundary isn't a blind copy. The plugin has a few rules that keep source-of-truth decisions visible.

statusSync only operates for a resource with a real status subresource, and status always flows from host to virtual cluster. The plugin strips tenant-written status before it creates the host object. A tenant cannot write an Accepted=True condition into a route or an application and have the platform treat it as host state.

Selectors support labels plus namespace include and exclude rules. Exclusions win. Deletes are provenance-gated: the plugin removes only an object it created and stamped, never an unrelated object that happens to share a name. Trace logging redacts Secret data, and the plugin exports Prometheus metrics on its own endpoint.

The configuration parser is strict as well. An unknown field is an error, so a misspelt key in the sort of YAML that lives for months in a tenant chart fails at load time.

The Argo field we had to leave alone

One lesson came the hard way, from a real failure in our own Argo CD integration. Argo stores a pending sync in the top-level operation field of an Application. Our syncer treated the virtual copy as complete desired state and could remove that field after Argo wrote it but before Argo's worker read it. Auto-sync then never ran.

The fix became hostOwnedFields. These are top-level fields owned by a host controller: the syncer neither copies them from the virtual object nor deletes them when the virtual object lacks them, and host-side changes do not trigger reconciliation. Application.operation is now host-owned by default.

That rule sounds narrow until you run a controller on both sides of a boundary. The object may be shared; every field does not have to be.

A small custom-resource sync

The smallest useful configuration syncs one custom resource from a tenant vCluster to its host cluster. For this walkthrough we'll borrow the test-only Widget CRD from the plugin's own end-to-end suite.

First install the CRD on both sides:

kubectl apply -f https://raw.githubusercontent.com/kupecloud/vcluster-generic-sync-plugin/main/test/testdata/widget-crd.yaml
vcluster connect my-vcluster -- kubectl apply -f https://raw.githubusercontent.com/kupecloud/vcluster-generic-sync-plugin/main/test/testdata/widget-crd.yaml

Then add the plugin and the RBAC required to discover the CRD and reconcile Widget objects:

plugin:
  generic-sync:
    version: v2
    image: ghcr.io/kupecloud/vcluster-generic-sync-plugin:latest # pin a release in production
    imagePullPolicy: IfNotPresent
    rbac:
      role:
        extraRules:
          - apiGroups: ['example.com']
            resources: ['widgets']
            verbs: ['create', 'delete', 'patch', 'update', 'get', 'list', 'watch']
      clusterRole:
        extraRules:
          - apiGroups: ['apiextensions.k8s.io']
            resources: ['customresourcedefinitions']
            verbs: ['get', 'list', 'watch']
    config:
      version: v1
      syncResources:
        - apiVersion: example.com/v1
          kind: Widget
          direction: toHost

rbac:
  clusterRole:
    enabled: true

The two version fields are unrelated: v2 is vCluster's plugin format, while config.version: v1 is the plugin's own configuration schema. The rbac blocks are plain vCluster chart values; the chart folds extraRules into the Role and ClusterRole it creates for the syncer pod.

Create the matching test object through the vCluster connection. The plugin's base configuration guide explains the RBAC shape in more detail. In a real installation, scope those rules to each kind and namespace you intend to sync; * is easier to write and much harder to explain later.

Shared Gateway, tenant routes

Gateway API shows why a generic syncer needs patches. The tenant's HTTPRoute has two kinds of references:

• parentRefs point at a Gateway that already exists on the host. Keep those values unchanged.
• backendRefs point at Services from the tenant cluster. Rewrite those references to the host-side names created by vCluster.

The plugin's Gateway API example installs the Gateway API CRDs on both clusters and supplies the complete RBAC. The sync configuration does the real work:

plugin:
  generic-sync:
    version: v2
    image: ghcr.io/kupecloud/vcluster-generic-sync-plugin:latest # pin a release in production
    config:
      version: v1
      syncResources:
        - apiVersion: gateway.networking.k8s.io/v1
          kind: HTTPRoute
          direction: toHost
          statusSync: true
          patches:
            - path: spec.parentRefs[*]
              type: none
            - path: spec.rules[*].backendRefs[*]
              type: rewriteRef

type: none is a sharp tool. It lets a route target a shared host Gateway directly, so the host must enforce which vCluster namespaces may attach, which hostnames they may claim, and where the listener sends traffic. The plugin can preserve a reference; it cannot decide whether that reference is safe for your gateway policy.

Seven crossings, scoped permissions

Our production configuration contains seven resource kinds built from the same pieces: a read-only GatewayClass mirror; host-side HTTPRoute, Application, PrometheusRule and dashboard sync; repository-secret sync guarded by Argo labels; and platform-managed Secret import.

The RBAC follows the resource boundary. Application and repository-secret permissions live in namespace Roles rather than a broad ClusterRole, so the plugin in one tenant vCluster cannot list another tenant's credentials. hostNamespace is useful for controllers such as central Argo CD, but it's not a reason to make every host namespace visible.

There's a limit to what “generic” means here. The plugin can sync arbitrary declared kinds, yet it stamps kupe.cloud/* labels and annotations as its provenance contract. That's how it knows which objects it owns, which copies it may delete, and which changes are worth reconciling. The generic part is the configured resource type; the ownership contract stays specific.

Start with one crossing

If you're adding the plugin to your own vCluster setup, start with a single kind: the one that already has an obvious source of truth. Write one syncResources entry for it, grant the narrowest RBAC that lets it work, and watch what lands on the host: the object, its translated references, its status. Once that first crossing behaves, the rest are the same pattern with different nouns.

If something goes wrong, look at ownership before configuration. In our experience the first failure is a field or a reference that both sides think they own, and hostOwnedFields or a patch is usually the fix.

The plugin lives at github.com/kupecloud/vcluster-generic-sync-plugin, with the configuration reference, worked examples and the end-to-end suite the Widget walkthrough came from. If you hit a bug or something that doesn't behave as documented, open an issue and we'll happily help.