DEV Community: DarkEdges

VEX demo update: adding Docker Scout attestations (and three new gotchas)

DarkEdges — Sat, 13 Jun 2026 03:58:54 +0000

This is a follow-up to
From 70 CVEs to 0: a hands-on VEX suppression workflow with Trivy.
The original demo covered Trivy's VEX repository and the filesystem-embed
approach. This update adds Docker Scout attestations as a first-class
distribution channel and surfaces three new gotchas that aren't in the docs.

The demo repo is the same:
github.com/darkedges/trivy-vex-demo

pull the latest and the new steps are already in run.sh / run.ps1.

What changed

The demo now has three distribution channels for the same OpenVEX statements:

channel	mechanism	consumed by
1	`docker scout attestation add`	Docker Scout (from registry)
2	VEX repository (spec v0.1)	Trivy via `--vex repo`
3	Wiz registry attestation	Wiz (unchanged from original)

Channels 1 and 2 are independent; updating one does not update the other.
They use different product PURLs and different packaging, but start from
the same vexctl create output.

Gotcha 4: Docker Scout won't match `pkg:oci/`

The original demo used a pkg:oci purl pinned to the image digest:

pkg:oci/pingaccess@sha256:51689e8c…

Docker Scout ignores it. Scout expects the pkg:docker/ form with the
full org/name path and the tag as the version component, separated by @:

pkg:docker/pingidentity/pingaccess@8.3.4-edge

So the generate-vex.sh script now runs two loops: one producing
.openvex.json files with a pkg:oci digest purl for Trivy/Wiz, and a
second producing .vex.json files with the pkg:docker tag purl for Scout.
Same CVE set, different product identifiers, different output directories.

# Trivy/Wiz channel - digest-pinned
vexctl create \
  --product="pkg:oci/pingaccess@sha256:51689e8ccf1ec6bef28c855a2f2fafdd3556f753609adad2e258580e3bc9397c" \
  --vuln="CVE-2022-46337" \
  --file="vex/statements/CVE-2022-46337.openvex.json" ...

# Scout channel - tag-based
vexctl create \
  --product="pkg:docker/pingidentity/pingaccess@8.3.4-edge" \
  --vuln="CVE-2022-46337" \
  --file="vex/statements-scout/CVE-2022-46337.vex.json" ...

The dry-run check before attaching anything is:

docker scout cves pingidentity/pingaccess:8.3.4-edge \
  --vex-location ./vex

If not_affected CVEs still appear, the first thing to check is the PURL
prefix - pkg:oci/ in a Scout context is a silent no-match.

Gotcha 5: attestations and filesystem VEX are mutually exclusive (Scout)

In the original post I mentioned that Trivy doesn't auto-discover VEX
embedded in the image filesystem. Scout has a related but different rule that
is much easier to trigger accidentally:

If an image has any attestation, Scout reads only attestations and ignores
all filesystem-embedded VEX documents entirely.

"Any attestation" includes the provenance and SBOM attestations that modern
BuildKit adds automatically when you docker build without explicitly opting
out. So this build:

docker build -f embed/Dockerfile -t myimage:latest .

…will silently have provenance attached (BuildKit default since Docker Desktop
4.11), and Scout will never read the *.vex.json files you COPYd into
/usr/share/vex/. You won't get an error; Scout just returns unfiltered
results.

To actually use the filesystem path you must suppress both:

docker build --provenance=false --sbom=false \
  -f embed/Dockerfile -t myimage:latest .

In practice: prefer attestations. They travel with the image in the
registry, require no extraction, and don't silently vanish because BuildKit
added something. The filesystem path is useful when you can't push to a
registry at all.

Gotcha 6: Scout attestations need the containerd image store

Running docker scout attestation add against a local registry:2 requires
the containerd image store to be active in your Docker daemon. With the
default overlay2 store the command may succeed but Scout won't resolve the
attestation manifest correctly when reading it back.

Check with:

docker info | grep 'containerd-snapshotter'
# containerd-snapshotter: true  ← you're good
# (nothing)                     ← not enabled

To enable it, add to /etc/docker/daemon.json and restart Docker:

{
  "features": { "containerd-snapshotter": true }
}

⚠️ Enabling the containerd image store can affect existing local images - they
won't be visible in the new store until re-pulled. Enable it in a dev
environment before hitting production.

The run.sh / run.ps1 scripts now detect this automatically and skip
steps 9–11 (the Scout attestation flow) rather than blocking, so the rest
of the demo - Trivy VEX repo, filesystem fallback, suppression proofs - runs
uninterrupted regardless of your daemon config.

The attestation flow end-to-end

Once the containerd store is active:

# 1. Verify suppression locally before attaching anything
docker scout cves pingidentity/pingaccess:8.3.4-edge \
  --vex-location ./vex

# 2. Push to a local registry:2 (no public push)
docker run -d -p 5000:5000 --name vex-registry registry:2
docker tag pingidentity/pingaccess:8.3.4-edge \
  localhost:5000/pingidentity/pingaccess:8.3.4-edge
docker push localhost:5000/pingidentity/pingaccess:8.3.4-edge

# 3. Attach each VEX document as an in-toto attestation
for f in vex/statements-scout/*.vex.json; do
  docker scout attestation add \
    --file "$f" \
    --predicate-type https://openvex.dev/ns/v0.2.0 \
    localhost:5000/pingidentity/pingaccess:8.3.4-edge
done

# 4. Re-scan to prove suppression
docker scout cves localhost:5000/pingidentity/pingaccess:8.3.4-edge

A few operational notes:

Attestations cannot be removed once attached; re-attaching a new document for the same CVE overwrites it.
docker scout attestation add works without a rebuild - the image is already in the registry.
A plain registry:2 supports OCI artifact manifests (used by attestations) from Docker Registry API v2.9 onward; if yours is older you'll need to upgrade or fall back to the filesystem path.

Updated toolchain image

The toolchain/Dockerfile now includes docker-cli and the Docker Scout
plugin installed via the official install.sh:

RUN apk add --no-cache ca-certificates curl jq bash docker-cli

# Docker Scout CLI - verify latest at https://docs.docker.com/scout/install/
RUN curl -fsSL https://raw.githubusercontent.com/docker/scout-cli/main/install.sh | bash

This lets the toolchain container run Scout commands when the Docker socket is
mounted, as a self-contained alternative to installing Scout on the host.

Summary: which mechanism to use when

you want to…	use
Suppress in Trivy, no registry push	`--vex <file>` or `--vex repo`
Suppress in Scout, image already in registry	`docker scout attestation add`
Suppress in Wiz	registry attestation (same Scout command, public registry)
Embed for distribution (consumers extract + `--vex`)	`COPY` into image, build with `--provenance=false --sbom=false`

The full demo still runs in one command and nothing leaves the host (local
registry:2 stands in for a real registry):

./run.sh        # pauses after each step
./run.sh -y     # unattended

Questions on PURL matching, CycloneDX VEX, or Grype's behaviour with any of
this? Drop them in the comments.

From 70 CVEs to 0: a hands-on VEX suppression workflow with Trivy (and a path to Wiz)

DarkEdges — Sat, 13 Jun 2026 00:30:16 +0000

Run Trivy against almost any vendor container image and you'll get a wall of
findings. Most of them don't matter, the vulnerable code path is never
executed in your deployment. The problem is that knowing that has
traditionally lived in spreadsheets and Jira comments, where no scanner can
see it.

VEX (Vulnerability Exploitability eXchange) turns that triage into
machine-readable statements that scanners apply automatically. This post
walks through a complete, runnable workflow, baseline scan → OpenVEX
generation → a hosted VEX repository → suppressed re-scan, and covers three
gotchas I hit that the docs don't mention.

Everything here is in a one-script demo repo:
github.com/darkedges/trivy-vex-demo

⚠️ The demo marks every CVE not_affected mechanically to exercise the
plumbing. In real life the assessment is the valuable part, don't ship
VEX statements you can't defend.

The setup

Target: pingidentity/pingaccess:8.3.4-edge (digest
sha256:51689e8c…). Tools, pinned and current at time of writing: Trivy
v0.71.0, vexctl v0.4.1, OpenVEX v0.2.0, and the
VEX Repository Specification v0.1.

A small Alpine-based toolchain image carries everything:

FROM alpine:latest
ARG TRIVY_VERSION=0.71.0
ARG VEXCTL_VERSION=0.4.1
RUN apk update && apk upgrade --no-cache && \
    apk add --no-cache ca-certificates curl jq
RUN curl -fsSL -o /tmp/trivy.tar.gz \
      "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz" && \
    tar -xzf /tmp/trivy.tar.gz -C /usr/local/bin trivy && rm /tmp/trivy.tar.gz
RUN curl -fsSL -o /usr/local/bin/vexctl \
      "https://github.com/openvex/vexctl/releases/download/v${VEXCTL_VERSION}/vexctl-linux-amd64" && \
    chmod +x /usr/local/bin/vexctl

Step 1, Baseline

trivy image --format json --output baseline-report.json \
  pingidentity/pingaccess:8.3.4-edge

Result: 70 findings (2 CRITICAL / 18 HIGH / 42 MEDIUM / 8 LOW) across 51
unique CVE/GHSA IDs, all in bundled Java jars and git-lfs. The Alpine OS
layer itself: zero.

Step 2, Generate OpenVEX statements

One statement per CVE with vexctl. The critical detail is the product
identifier: a pkg:oci purl pinned to the image digest, never the tag,
tags move, VEX assertions shouldn't.

vexctl create \
  --product="pkg:oci/pingaccess@sha256:51689e8ccf1ec6bef28c855a2f2fafdd3556f753609adad2e258580e3bc9397c" \
  --vuln="CVE-2022-46337" \
  --status="not_affected" \
  --justification="vulnerable_code_not_in_execute_path" \
  --status-note="DEMO assessment for a suppression-workflow POC." \
  --author="DarkEdges Security <nirving@darkedges.com>" \
  --file="CVE-2022-46337.openvex.json"

Which produces:

{
  "@context": "https://openvex.dev/ns/v0.2.0",
  "@id": "https://darkedges.com/vex/demo/pingaccess/CVE-2022-46337",
  "author": "DarkEdges Security <nirving@darkedges.com>",
  "statements": [
    {
      "vulnerability": { "name": "CVE-2022-46337" },
      "products": [
        { "@id": "pkg:oci/pingaccess@sha256:51689e8c…" }
      ],
      "status": "not_affected",
      "justification": "vulnerable_code_not_in_execute_path",
      "status_notes": "DEMO assessment for a suppression-workflow POC."
    }
  ]
}

Loop that over the 51 IDs from the baseline JSON, then consolidate:

vexctl merge statements/*.openvex.json > pingaccess-8.3.4-edge.openvex.json

Tip: leave the purl qualifiers off in statement products. Per Trivy's
matching rules, a purl without qualifiers matches regardless of arch or
repository_url; with qualifiers, you're signing up for exact matching.

Step 3, The instant win: `--vex` flag

trivy image --show-suppressed \
  --vex pingaccess-8.3.4-edge.openvex.json \
  pingidentity/pingaccess:8.3.4-edge

70 findings → 0 reported, 70 suppressed. Each suppression is recorded in
the JSON report under Results[].ExperimentalModifiedFindings with
Type: ignored and the justification, auditable, not deleted.

Step 4, Publish via a VEX repository

A VEX repository is just static files: a manifest at
/.well-known/vex-repository.json and a tar.gz archive containing an index
plus the documents.

vex-repo/                       # webroot
├── .well-known/vex-repository.json
└── v0.1/vex-data.tar.gz        # contains: index.json + pkg/oci/pingaccess/vex.json

The manifest:

{
  "name": "DarkEdges Demo VEX Repository",
  "versions": [{
    "spec_version": "0.1",
    "locations": [{ "url": "http://vex-server/v0.1/vex-data.tar.gz" }],
    "update_interval": "1h"
  }]
}

repositories:
  - name: darkedges-demo
    url: http://vex-server
    enabled: true

Then trivy vex repo download fetches it, and scanning with --vex repo
gives the same result: 0 findings, 70 suppressed, except now the
statements are centrally hosted, versioned, and every Trivy in your org picks
them up automatically on the manifest's update interval.

The three gotchas

1. The repository index needs `repository_url` for OCI purls

This one cost me an hour. The vex-repo-spec says index ids are purls
"without version and qualifiers", so pkg:oci/pingaccess, right?

No match. Silently. Trivy's index lookup (pkg/vex/repo.go) makes an
exception for OCI purls and keeps the repository_url qualifier as part
of the package identity. The index entry must be:

{
  "id": "pkg:oci/pingaccess?repository_url=index.docker.io%2Fpingidentity%2Fpingaccess",
  "location": "pkg/oci/pingaccess/vex.json",
  "format": "openvex"
}

If your repo downloads fine but suppresses nothing, check this first.

2. Embedding VEX in the image does nothing (for scanning)

Docker's Hardened Images article mentions copying VEX documents into the
image at build time:

FROM pingidentity/pingaccess:8.3.4-edge
COPY pingaccess-8.3.4-edge.openvex.json /usr/share/vex/

I tested it: Trivy does not auto-discover VEX from the image filesystem.
A plain scan of the derived image still reports all 70 findings. Embedding is
a distribution mechanism, the consumer must extract the file and pass it
via --vex. (What Trivy can auto-discover is VEX attached as a signed OCI
registry attestation, trivy image --vex oci, which requires pushing.)

3. VEX binds to the digest, derived images don't inherit it

The statements identify the product by the base image's digest. My derived
image (built locally, never pushed) has no repo digest at all, so neither
the local file nor the repository suppressed anything on it. Expected, but
worth internalizing: issue VEX for the digest you actually ship, and
regenerate when it changes.

mechanism	original image (digest-pinned)	derived image
`--vex <file>`	✅ 70/70 suppressed	❌ 0
`--vex repo`	✅ 70/70 suppressed	❌ 0
embedded file alone	n/a	❌ 0

What about Wiz?

Wiz ingests OpenVEX automatically, zero config, but from registry
attestations, not files or repositories. The hand-off from this workflow is
one command against the image in a Wiz-scanned registry:

docker scout attestation add \
  --file pingaccess-8.3.4-edge.openvex.json \
  --predicate-type https://openvex.dev/ns/v0.2.0 \
  <registry>/<org>/pingaccess:8.3.4-edge
# or: cosign attest --type openvex --predicate <file> <image@digest>

Same OpenVEX documents, two consumers: Trivy via the repository, Wiz via the
attestation. That's the point of a standard.

Try it

The repo runs the whole thing, toolchain build, baseline, 51 statements,
hosted repository, all the re-scans, with one command and colored output
(there's a GIF of the full run in the README):

git clone https://github.com/darkedges/trivy-vex-demo
cd trivy-vex-demo
./run.sh        # pauses after each step; ./run.sh -y to run unattended

Nothing is pushed anywhere; the only network traffic is Docker Hub pulls and
Trivy DB downloads. PowerShell users get an equivalent run.ps1.

If you've hit other VEX edge cases, CycloneDX VEX, CSAF, Grype's matching
behaviour, I'd love to hear about them in the comments.

Deploy Ping Identity Products on Kubernetes with a Single Operator

DarkEdges — Wed, 27 May 2026 12:42:32 +0000

Running Ping Identity's product suite on Kubernetes typically means wrestling with the ping-devops Helm chart directly, a large umbrella chart with hundreds of values, no shared defaults across products, and no native Kubernetes status model. The pingone-operator wraps that chart in a set of purpose-built custom resources so you declare what you want and the operator figures out the Helm details.

In this post I'll walk through how the operator works, how to install it, and how to get PingFederate (and the rest of the suite) running in a cluster.

The design in one paragraph

The operator introduces six custom resource definitions:

PingEnvironment shared configuration: tenant ID, tier (development / staging / production), base domain, and ingress defaults.
PingFederate, PingDirectory, PingAccess, PingAuthorize, PingAuthorizePAP, one CR per product, each referencing a PingEnvironment via spec.environmentRef.

Underneath, everything still maps to a single ping-devops Helm release per environment. Products are enabled or disabled in that release based on which product CRs exist. You get independent Kubernetes API objects (separate RBAC, separate status, deploy or remove products without touching shared config) without needing separate Helm releases.

Prerequisites

Requirement	Version
Kubernetes	1.26+
kubectl	matching cluster
Helm	3.x (CLI, for install only)
Ping Identity DevOps account	register here

You'll also need a devops-secret in whatever namespace products will be deployed to:

kubectl create secret generic devops-secret \
  --namespace pingone \
  --from-literal=PING_IDENTITY_ACCEPT_EULA=YES \
  --from-literal=PING_IDENTITY_DEVOPS_USER=<your-devops-user> \
  --from-literal=PING_IDENTITY_DEVOPS_KEY=<your-devops-key>

Installing the operator

Option A: from the OCI Helm chart

helm upgrade --install pingone-operator \
  oci://ghcr.io/darkedges/charts/pingone-operator \
  --version 0.1.0 \
  --namespace pingone-system \
  --create-namespace

Option B: from source (Docker Desktop)

git clone https://github.com/darkedges/pingone-operator
cd pingone-operator
make docker-desktop   # builds image, loads it into Docker Desktop, deploys

Either way, verify the operator is running:

kubectl get pods -n pingone-system
# NAME                                                READY   STATUS    RESTARTS   AGE
# pingone-operator-controller-manager-...             1/1     Running   0          30s

Then install the CRDs if they aren't already:

make install
# or: kubectl apply -f config/crd/bases/

Creating your first environment

A PingEnvironment holds the shared config that all product CRs in the same namespace will inherit.

# env-dev.yaml
apiVersion: pingone.io/v1alpha1
kind: PingEnvironment
metadata:
  name: env-dev
  namespace: pingone
spec:
  tenantId: myorg-dev           # Helm release name: myorg-dev-ping
  tier: development             # development | staging | production
  domain: dev.myorg.example.com # base domain for auto-derived hostnames

  # Shared ingress: all product CRs inherit this unless they override it
  ingress:
    enabled: true
    className: nginx
    annotations:
      nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      cert-manager.io/cluster-issuer: "letsencrypt-prod"

kubectl apply -f env-dev.yaml
kubectl get pingenvironments -n pingone
# NAME      PHASE   AGE
# env-dev   Ready   5s

No Helm release is created yet, the environment is a coordination point, not a trigger by itself.

Adding PingFederate

Create a PingFederate CR that references the environment:

# pf-dev.yaml
apiVersion: pingone.io/v1alpha1
kind: PingFederate
metadata:
  name: dev-pf
  namespace: pingone
spec:
  environmentRef: env-dev       # references the PingEnvironment above
  version: "13.0.2-edge"
  replicas: 1

  engineIngress:
    enabled: true
    # hostname auto-derived: pf.dev.myorg.example.com
    tlsSecretRef: pf-tls

  adminIngress:
    enabled: true
    # hostname auto-derived: pf-admin.dev.myorg.example.com
    tlsSecretRef: pf-admin-tls

  config:
    serverProfile:
      url: https://github.com/pingidentity/pingidentity-server-profiles.git
      path: getting-started/pingfederate

kubectl apply -f pf-dev.yaml

The PingFederate reconciler validates the CR and queues the PingEnvironment reconciler. That reconciler discovers all product CRs pointing at env-dev, builds the combined Helm values, and installs or upgrades the myorg-dev-ping release.

Watch it come up:

kubectl get pingfederates -n pingone
# NAME     ENVIRONMENT   PHASE   AGE
# dev-pf   env-dev       Ready   90s

kubectl get pods -n pingone
# NAME                                        READY   STATUS    AGE
# myorg-dev-ping-pingfederate-engine-0        1/1     Running   80s
# myorg-dev-ping-pingfederate-admin-0         1/1     Running   80s

Hostnames are derived automatically

When spec.domain is set on the PingEnvironment, you don't need to spell out every hostname. The operator derives them:

Product	Auto-derived hostname
PingFederate engine	`pf.<domain>`
PingFederate admin	`pf-admin.<domain>`
PingDataConsole	`pd-console.<domain>`
PingAccess admin	`pa-admin.<domain>`
PingAccess engine	`pa.<domain>`
PingAuthorize	`paz.<domain>`
PingAuthorizePAP	`paz-pap.<domain>`

Override any of them by setting hostname inside the component's ingress block:

engineIngress:
  enabled: true
  hostname: sso.myorg.example.com   # explicit override
  tlsSecretRef: sso-tls

Adding more products

Each product is its own CR. Add them in any order, the operator reconciles the full set each time any one of them changes.

PingDirectory

apiVersion: pingone.io/v1alpha1
kind: PingDirectory
metadata:
  name: dev-pd
  namespace: pingone
spec:
  environmentRef: env-dev
  replicas: 1
  storageSize: 8Gi
  config:
    serverProfile:
      url: https://github.com/pingidentity/pingidentity-server-profiles.git
      path: baseline/pingdirectory
    userBaseDN: "dc=myorg,dc=com"

PingAuthorize (with a startup dependency on PingDirectory)

apiVersion: pingone.io/v1alpha1
kind: PingAuthorize
metadata:
  name: dev-paz
  namespace: pingone
spec:
  environmentRef: env-dev
  replicas: 1
  ingress:
    enabled: true
    tlsSecretRef: paz-tls
  container:
    waitFor:
      - application: pingDirectory
        service: ldaps
        timeoutSeconds: 300
  config:
    serverProfile:
      url: https://github.com/pingidentity/pingidentity-server-profiles.git
      path: paz-pap-integration/pingauthorize
    serverProfileLayers:
      - name: paz
        url: https://github.com/pingidentity/pingidentity-server-profiles.git
        path: baseline/pingauthorize

PingAuthorizePAP

apiVersion: pingone.io/v1alpha1
kind: PingAuthorizePAP
metadata:
  name: dev-pap
  namespace: pingone
spec:
  environmentRef: env-dev
  ingress:
    enabled: true
    tlsSecretRef: paz-pap-tls
  config:
    serverProfile:
      url: https://github.com/pingidentity/pingidentity-server-profiles.git
      path: paz-pap-integration/pingauthorizepap

PingAccess (waits for PingFederate)

apiVersion: pingone.io/v1alpha1
kind: PingAccess
metadata:
  name: dev-pa
  namespace: pingone
spec:
  environmentRef: env-dev
  version: "8.1.0-edge"
  adminIngress:
    enabled: true
    tlsSecretRef: pa-admin-tls
  engineIngress:
    enabled: true
    tlsSecretRef: pa-tls
  container:
    waitFor:
      - application: pingFederate
        service: https
        timeoutSeconds: 300
  config:
    serverProfile:
      url: https://github.com/pingidentity/pingidentity-server-profiles.git
      path: getting-started/pingaccess

Startup ordering with `waitFor`

The container.waitFor block controls which services a container waits for before it starts. Under the hood this maps to the WAIT_FOR env var in the ping-devops Docker images.

container:
  waitFor:
    - application: pingDirectory
      service: ldaps
      timeoutSeconds: 300
    - application: pingFederate
      service: https
      timeoutSeconds: 300

Logical application names like pingDirectory, pingFederateAdmin, pingAccessEngine are resolved by the operator to the correct Helm sub-chart service names automatically.

Layered server profiles

The operator supports Ping Identity's layered profile pattern. Use serverProfileLayers to chain profiles:

config:
  serverProfile:
    url: https://github.com/myorg/ping-profiles.git
    path: extensions/pingfederate
    parent: baseline           # chains to the layer named "baseline"

  serverProfileLayers:
    - name: baseline
      url: https://github.com/pingidentity/pingidentity-server-profiles.git
      path: baseline/pingfederate

The operator translates these to the SERVER_PROFILE_* and SERVER_PROFILE_<LAYER>_* env var convention the Docker images expect.

Resource sizing with `tier`

Set tier once on the PingEnvironment and the operator applies appropriate CPU/memory requests and limits to every product:

Tier	PingFederate	PingDirectory	PingAccess	PingAuthorize
`development`	500m / 512Mi	500m / 1Gi	500m / 512Mi	500m / 1Gi
`staging`	1 / 1Gi	1 / 2Gi	1 / 1Gi	1 / 2Gi
`production`	2 / 2Gi	2 / 4Gi	2 / 2Gi	2 / 4Gi

Checking status

Each resource has its own Phase field and standard Kubernetes conditions:

kubectl get pingenvironments,pingfederates,pingdirectories -n pingone
# NAME                               PHASE   AGE
# pingenvironment.pingone.io/env-dev Ready   10m
#
# NAME                               ENVIRONMENT   PHASE   AGE
# pingfederate.pingone.io/dev-pf     env-dev       Ready   8m
# pingdirectory.pingone.io/dev-pd    env-dev       Ready   7m

Drill into conditions with:

kubectl describe pingfederate dev-pf -n pingone

Removing a product

Delete the product CR. The operator detects the deletion, rebuilds Helm values with that product disabled, and upgrades the release in place. The other products keep running.

kubectl delete pingaccess dev-pa -n pingone
# PingAccess pods terminate; PingFederate and PingDirectory are unaffected

Delete the PingEnvironment to tear down the entire Helm release:

kubectl delete pingenvironment env-dev -n pingone
# Finalizer triggers Helm uninstall; all product pods are removed

What's next

The source is at https://github.com/darkedges/pingone-kubernetes-provider
The examples/getting-started/ directory has a complete working example for Docker Desktop with nginx ingress and cert-manager self-signed TLS
Full field reference is in CONFIGURATION.md

From Laptop to Cluster: Running darkedges-entraid-tokenexchange with Docker and Kubernetes

DarkEdges — Wed, 29 Apr 2026 23:38:03 +0000

I wanted one clean deployment path for local testing and real cluster rollout, so I packaged the same app for both Docker Compose and Kubernetes (Helm).

This post is the exact runbook I now use for darkedges-entraid-tokenexchange.

Why this setup works

This app is a Next.js token-exchange broker with:

OIDC sign-in (single or multiple providers)
Redis-backed sessions
Entra token exchange support
Optional Vault integration

The key idea is simple: run the same container image everywhere, then swap environment values per environment.

Local path: Docker Compose

The repository already includes a ready-to-run docker-compose.yaml with:

app on port 3000
vault
vault-init and vault-data-init

Step 1: Create your environment file

cp .env.example .env

Populate at minimum:

NEXTAUTH_SECRET=<random-32-byte-secret>
NEXTAUTH_URL=http://localhost:3000

ENTRAID_TENANT_ID=<tenant-guid>
ENTRAID_CLIENT_ID=<app-client-id>
ENTRAID_CLIENT_SECRET=<app-client-secret>
ENTRAID_SCOPE=https://graph.microsoft.com/.default

REDIS_URL=redis://localhost:6379
VAULT_ADDR=http://localhost:18200
VAULT_TOKEN=dev-root-token

If you support multiple providers, also set OIDC_PROVIDERS as valid JSON.

OIDC_PROVIDERS: quick configuration pattern

Use a JSON array. Each provider entry needs the same core fields.

OIDC_PROVIDERS='[
  {
    "id": "pingfederate",
    "name": "PingFederate",
    "description": "Sign in with corporate credentials",
    "icon": "lock",
    "baseUrl": "https://id.example.com",
    "clientId": "your-client-id",
    "clientSecret": "your-client-secret",
    "scopes": ["openid", "profile", "email"]
  }
]'

For multiple providers, add more objects to the same array (for example Entra, PingFederate, Auth0).

Tips:

Keep it valid JSON inside the quotes: double quotes for keys/strings, no trailing commas.
Keep provider id values unique.
Make sure each baseUrl and client credentials match the IdP app registration.
Use each provider's correct callback URL in the IdP config.

If login page/provider rendering fails, the first thing to check is JSON validity in OIDC_PROVIDERS.

Entra app types you should provision

For this project, think in terms of three Entra app registrations (each with an Enterprise Application/service principal):

Web login app for OIDC sign-in (msentraid provider in OIDC_PROVIDERS)
API/resource app exposing your delegated scope (for example access_as_user)
Backend token-exchange app for OBO/Graph calls (ENTRAID_CLIENT_ID / ENTRAID_CLIENT_SECRET)

If you support External ID (ciamlogin.com issuer), add a fourth confidential app and map it to ENTRAID_CIAM_* variables.

1-minute env mapping check

App	Key env vars
Web login app	`OIDC_PROVIDERS[].baseUrl`, `OIDC_PROVIDERS[].clientId`, `OIDC_PROVIDERS[*].clientSecret`
API/resource app	Usually represented in `NEXT_PUBLIC_ENTRAID_SCOPE`
Backend token-exchange app	`ENTRAID_TENANT_ID`, `ENTRAID_CLIENT_ID`, `ENTRAID_CLIENT_SECRET`, `ENTRAID_SCOPE`
Optional CIAM app	`ENTRAID_CIAM_TENANT_ID`, `ENTRAID_CIAM_CLIENT_ID`, `ENTRAID_CIAM_CLIENT_SECRET`, `ENTRAID_CIAM_SCOPE`

Step 2: Build and start

docker compose up --build -d

Step 3: Verify runtime

docker compose ps
docker compose logs -f app

Open:

App: http://localhost:3000
Vault UI (optional): http://localhost:18200

Step 4: Stop or reset

docker compose down

Full reset including volumes:

docker compose down -v

Cluster path: Kubernetes + Helm

The chart is already in the repo at:

helm/darkedges-entraid-tokenexchange

There is also an environment-flavored values file:

helm/broker.yaml

Step 1: Build and push image

docker build -t <registry>/darkedges-entraid-tokenexchange:<tag> .
docker push <registry>/darkedges-entraid-tokenexchange:<tag>

Step 2: Create namespace

kubectl create namespace broker

Step 3: Add pull secret for private registry

kubectl create secret docker-registry darkedges-registry-credentials \
  --docker-server=<registry> \
  --docker-username=<username> \
  --docker-password=<password> \
  --namespace broker

Step 4: Update values

In helm/broker.yaml (or your own values file), set:

image.repository and image.tag
ingress.hosts and ingress.tls
env for non-sensitive config
secretEnv for secrets
REDIS_URL to a reachable Redis service

Step 5: Install or upgrade release

helm upgrade --install darkedges-entraid-tokenexchange \
  helm/darkedges-entraid-tokenexchange \
  -f helm/broker.yaml \
  --namespace broker \
  --create-namespace

Step 6: Verify rollout

kubectl get pods -n broker
kubectl get svc -n broker
kubectl get ingress -n broker
kubectl rollout status deploy/darkedges-entraid-tokenexchange -n broker
kubectl logs deploy/darkedges-entraid-tokenexchange -n broker --tail=200

Step 7: Access the app

Preferred:

Ingress host over HTTPS

Quick test from laptop:

kubectl port-forward svc/darkedges-entraid-tokenexchange 3000:3000 -n broker

Then open http://localhost:3000.

Lessons learned

Three things made this reliable:

Keep one container artifact and promote it across environments
Keep non-secret and secret config split (env vs secretEnv)
Validate rollout every time with both status and logs, not just pod readiness

Production checklist

Before go-live:

Move secrets from values files to a proper secret manager
Set NEXTAUTH_URL to your real HTTPS URL
Configure CPU and memory requests/limits
Use managed or HA Redis
Harden Vault TLS/auth and remove dev credentials

Handy commands

Upgrade:

helm upgrade darkedges-entraid-tokenexchange \
  helm/darkedges-entraid-tokenexchange \
  -f helm/broker.yaml \
  -n broker

Rollback:

helm rollback darkedges-entraid-tokenexchange 1 -n broker

Uninstall:

helm uninstall darkedges-entraid-tokenexchange -n broker

If you already have Docker Compose and Helm in this repo, this workflow should get you from local proof-of-concept to repeatable Kubernetes deployment with minimal friction.

PingFederate Token Exchange Processor Policy

DarkEdges — Mon, 20 Apr 2026 20:31:45 +0000

Overview
Complete Configuration Dependency Map
PROCESSORPOLICIES Policy
Processor Mapping 1 - PingFederate ↔ PingFederate
Processor Mapping 2 - Microsoft Entra ID ↔ PingFederate
Token Processor Comparison Matrix
Access Token Mapping - Token Exchange Context
OGNL Expression - Actor Claim Transformation
AccessTokenManagement Configuration
Real-World Example - HR Chatbot Token Exchange
Token Lifecycle Sequence
Configuration File Locations
Validation Checklist
Troubleshooting
References

Overview

PROCESSORPOLICIES is the default PingFederate Token Exchange Processor Policy implementing RFC 8693 delegation semantics. When a client submits a token exchange request, this policy:

Selects the correct processor mapping based on the token types presented
Validates both subject and actor tokens using the configured token processors
Extracts claims and maps them to a standardised attribute contract
Passes the fulfilled attributes to the Access Token Mapping which produces the final JWT

Supported exchange patterns:

Pattern	Subject Token Type	Actor Token Type
PingFederate ↔ PingFederate	`urn:ietf:params:oauth:token-type:access_token`	`urn:ietf:params:oauth:token-type:access_token`
Microsoft Entra ID ↔ PingFederate	`urn:ietf:params:oauth:token-type:access_token:msft`	`urn:ietf:params:oauth:token-type:access_token`

Semantic: RFC 8693 delegation - the actor (contact-hr-client) acts on behalf of the subject (user.1). The issued token contains an actor claim (RFC 8693 §4.1) recording this explicitly.

Complete Configuration Dependency Map

Token Exchange HTTP Request
POST /as/token.oauth2
grant_type=urn:ietf:params:oauth:grant-type:token-exchange
│
├── subject_token      (user's access token)
├── subject_token_type
├── actor_token        (chatbot's client credentials token)
├── actor_token_type
├── client_id          (contact-oauth-client)
├── client_secret
└── scope
         │
         ▼
┌─────────────────────────────────────────────────┐
│  /oauth/tokenExchange/processor/settings        │
│  defaultProcessorPolicyRef: PROCESSORPOLICIES   │
└───────────────────┬─────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────┐
│  /oauth/tokenExchange/processor/policies        │
│  PROCESSORPOLICIES                              │
│  actorTokenRequired: true                       │
│                                                 │
│  ┌─────────────────────────────────────────┐    │
│  │ Processor Mapping 1                     │    │
│  │ subjectTokenType: access_token (PF)     │    │
│  │ actorTokenType:   access_token (PF)     │    │
│  │ subjectProcessor: PFSubjectProcessor    │    │
│  │ actorProcessor:   PFActorSubject        │    │
│  └─────────────────────────────────────────┘    │
│                                                 │
│  ┌─────────────────────────────────────────┐    │
│  │ Processor Mapping 2                     │    │
│  │ subjectTokenType: access_token:msft     │    │
│  │ actorTokenType:   access_token (PF)     │    │
│  │ subjectProcessor: MSFTTOKENPROCESSOR    │    │
│  │ actorProcessor:   PFTOKENPROCESSOR      │    │
│  └─────────────────────────────────────────┘    │
└───────────────────┬─────────────────────────────┘
                    │
         ┌──────────┴──────────┐
         ▼                     ▼
  Token Processors         Token Processors
  (validate subject)       (validate actor)
  ┌───────────────┐        ┌────────────────┐
  │PFSubjectProc  │        │PFActorSubject  │
  │MSFTTOKENPROC  │        │PFTOKENPROCESSOR│
  └───────┬───────┘        └───────┬────────┘
          │                        │
          └──────────┬─────────────┘
                     │
                     ▼
         Attribute Contract Fulfillment
         (map claims from both tokens)
                     │
                     ▼
┌─────────────────────────────────────────────────┐
│  /oauth/accessTokenMappings                     │
│  Context: TOKEN_EXCHANGE_PROCESSOR_POLICY       │
│  Policy:  PROCESSORPOLICIES                     │
│  Manager: AccessTokenManagement                 │
│                                                 │
│  actor      ← OGNL expression (JSON object)     │
│  vaultloc.  ← TOKEN_EXCHANGE_PROCESSOR_POLICY   │
│  aud        ← CONTEXT (ClientId)                │
│  sub        ← TOKEN_EXCHANGE_PROCESSOR_POLICY   │
│  scope      ← TOKEN_EXCHANGE_PROCESSOR_POLICY   │
│  groups     ← TOKEN_EXCHANGE_PROCESSOR_POLICY   │
└───────────────────┬─────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────┐
│  /oauth/accessTokenManagers                     │
│  AccessTokenManagement                          │
│                                                 │
│  Algorithm:  RS256                              │
│  Lifetime:   120 seconds                        │
│  SigningKey:  5jqt7j8mxbwl2awtpc465yzx1         │
│  Issuer:     https://id.example.com             │
│  Adds:  iss, iat, exp, jti                      │
└───────────────────┬─────────────────────────────┘
                    │
                    ▼
           Issued JWT Access Token

PROCESSORPOLICIES Policy

Core Configuration

Property	Value
`id`	`PROCESSORPOLICIES`
`name`	`PROCESSORPOLICIES`
`actorTokenRequired`	`true` - actor token is mandatory in all requests
Default Policy	Yes - set as `defaultProcessorPolicyRef`

Attribute Contract

The policy defines the attributes available for downstream mapping:

Attribute	Type	Source (Mapping 1)	Required
`subject`	Core	SUBJECT_TOKEN → `sub`	Yes
`actor`	Extended	ACTOR_TOKEN → `client_id`	No
`vaultlocation`	Extended	SUBJECT_TOKEN → `vaultlocation`	No
`scope`	Extended	SUBJECT_TOKEN → `scope`	No
`groups`	Extended	SUBJECT_TOKEN → `groups`	No
`given_name`	Extended	SUBJECT_TOKEN → `given_name`	No
`family_name`	Extended	SUBJECT_TOKEN → `family_name`	No
`email`	Extended	SUBJECT_TOKEN → `email`	No

Processor Mapping Selection Logic

Request arrives with subject_token_type and actor_token_type
         │
         ├─ subject_token_type = urn:...:access_token   AND
         │  actor_token_type   = urn:...:access_token
         │        └─► Mapping 1 - PFSubjectProcessor + PFActorSubject
         │
         ├─ subject_token_type = urn:...:access_token:msft   AND
         │  actor_token_type   = urn:...:access_token
         │        └─► Mapping 2 - MSFTTOKENPROCESSOR + PFTOKENPROCESSOR
         │
         └─ No match
                  └─► HTTP 400 invalid_request

Issuance Criteria

Both mappings have empty conditionalCriteria:

"issuanceCriteria": { "conditionalCriteria": [] }

All token exchanges that pass processor validation are approved - no additional OGNL conditions are applied.

Processor Mapping 1 - PingFederate ↔ PingFederate

Overview

Used when both the subject and actor tokens were issued by this PingFederate instance (https://id.example.com). This is the primary path for the HR Chatbot integration.

subject_token  ──► PFSubjectProcessor  (validates user's access token)
actor_token    ──► PFActorSubject      (validates chatbot's client credentials token)

PFSubjectProcessor

Validates the user's access token (subject token).

Property	Value
ID	`PFSubjectProcessor`
Plugin Type	`com.pingidentity.pf.tokenprocessors.jwt.JwtTokenProcessor`
Allowed Issuer	`https://id.example.com`
JWKS URL	`https://id.example.com/pf/JWKS`
Require Audience	`true`
Required Audience	`contact-hr-client`
Require Expiration	`true`
Require Issued At	`true`
Clock Skew	0 seconds
JWKS Cache Duration	720 minutes

Attribute Contract Produced:

Claim	Type	Source
`sub`	Core	JWT `sub` claim
`vaultlocation`	Extended	JWT `vaultlocation` claim
`scope`	Extended	JWT `scope` claim
`groups`	Extended	JWT `groups` claim
`given_name`	Extended	JWT `given_name` claim
`family_name`	Extended	JWT `family_name` claim
`email`	Extended	JWT `email` claim

Example subject token claims validated by this processor:

{
  "scope": ["openid", "profile", "email"],
  "authorization_details": [],
  "client_id": "contact-hr-client",
  "iss": "https://id.example.com",
  "iat": 1776667958,
  "jti": "xExadYEtur8OlKf9NW3pVI",
  "vaultlocation": "referenceid/msentraid/0BUTfOKfKbCi2Rf4S-krNcFQUAJ2R4YDIqf8Xvl5nK4",
  "aud": "contact-hr-client",
  "sub": "user.1",
  "groups": ["Administrators"],
  "family_name": "Seawell",
  "email": "xxxx@hotmail.com",
  "exp": 1776675158
}

PFActorSubject

Validates the chatbot's client credentials token (actor token).

Property	Value
ID	`PFActorSubject`
Plugin Type	`com.pingidentity.pf.tokenprocessors.jwt.JwtTokenProcessor`
Allowed Issuer	`https://id.example.com`
JWKS URL	`https://id.example.com/pf/JWKS`
Require Audience	`false` - no audience check
Require Expiration	`true`
Require Issued At	`true`
Clock Skew	0 seconds
JWKS Cache Duration	720 minutes

Attribute Contract Produced:

Claim	Type	Source
`sub`	Core	JWT `sub` claim
`scope`	Extended	JWT `scope` claim
`client_id`	Extended	JWT `client_id` claim - used as `actor` in output

Example actor token claims validated by this processor:

{
  "scope": "",
  "authorization_details": [],
  "client_id": "contact-hr-client",
  "iss": "https://id.example.com",
  "iat": 1776667937,
  "jti": "ujPePxrAUOLlrj03ORmPe1",
  "exp": 1776675137
}

Note: The actor token has an empty scope because it was obtained via Client Credentials grant - the chatbot is authenticating as itself, not as a user.

Attribute Fulfillment - Mapping 1

Output Attribute	Source	Input Claim
`actor`	`ACTOR_TOKEN`	`client_id`
`vaultlocation`	`SUBJECT_TOKEN`	`vaultlocation`
`subject`	`SUBJECT_TOKEN`	`sub`
`scope`	`SUBJECT_TOKEN`	`scope`
`groups`	`SUBJECT_TOKEN`	`groups`
`given_name`	`SUBJECT_TOKEN`	`given_name`
`family_name`	`SUBJECT_TOKEN`	`family_name`
`email`	`SUBJECT_TOKEN`	`email`

Processor Mapping 2 - Microsoft Entra ID ↔ PingFederate

Overview

Used when the subject token originated from Microsoft Entra ID / Azure AD (identified by subject_token_type: urn:ietf:params:oauth:token-type:access_token:msft), while the actor token is still a PingFederate bearer token.

subject_token  ──► MSFTTOKENPROCESSOR  (validates Entra ID JWT)
actor_token    ──► PFTOKENPROCESSOR    (validates PF bearer token)

MSFTTOKENPROCESSOR

Validates tokens issued by Microsoft Azure AD / Entra ID.

Property	Value
ID	`MSFTTOKENPROCESSOR`
Plugin Type	`com.pingidentity.pf.tokenprocessors.jwt.JwtTokenProcessor`
Tenant ID	`4161be3f-bf2b-41d4-a02b-e6f82b529d53`

Allowed Issuers:

Issuer	JWKS URL	Protocol
`https://sts.windows.net/4161be3f-bf2b-41d4-a02b-e6f82b529d53/`	`https://login.microsoftonline.com/common/discovery/keys`	ADFS / v1
`https://login.microsoftonline.com/4161be3f-bf2b-41d4-a02b-e6f82b529d53/v2.0`	`https://login.microsoftonline.com/4161be3f-bf2b-41d4-a02b-e6f82b529d53/discovery/v2.0/keys`	OIDC v2

Allowed Audiences:

Audience
`https://fram.connectid.darkedges.com/openam/oauth2`
`e83c2af3-43d1-4f62-8bff-e619c29b5026`

Property	Value
Require Audience	`true`
Require Expiration	`true`
Require Issued At	`false`
Clock Skew	0 seconds

Attribute Contract Produced:

Claim	Type	Source
`sub`	Core	JWT `sub`
`email`	Extended	JWT `email`

PFTOKENPROCESSOR

Validates PingFederate bearer access tokens by introspecting against the AccessTokenManagement token manager.

Property	Value
ID	`PFTOKENPROCESSOR`
Plugin Type	`org.sourceid.wstrust.processor.oauth.BearerAccessTokenTokenProcessor`
Access Token Manager	`AccessTokenManagement`
Scope as single string	`false`

Attribute Contract Produced (from AccessTokenManagement introspection):

Claim	Type	Source
`aud`	Core	Token `aud` claim
`expires_at`	Core	Token expiry
`authorization_details`	Core	Token claim
`scope`	Core	Token scope
`iss`	Core	Token issuer
`client_id`	Core	Token client
`sub`	Extended	Token subject
`email`	Extended	Token email

Attribute Fulfillment - Mapping 2

Output Attribute	Source	Input Claim
`actor`	`NO_MAPPING`	(not included)
`vaultlocation`	`NO_MAPPING`	(not included)
`subject`	`SUBJECT_TOKEN`	`sub`
`scope`	`NO_MAPPING`	(not included)
`groups`	`NO_MAPPING`	(not included)
`given_name`	`NO_MAPPING`	(not included)
`family_name`	`NO_MAPPING`	(not included)
`email`	`SUBJECT_TOKEN`	`email`

Most attributes are NO_MAPPING because Microsoft tokens do not contain PingFederate-specific claims such as vaultlocation or groups.

Token Processor Comparison Matrix

Property	PFSubjectProcessor	PFActorSubject	MSFTTOKENPROCESSOR	PFTOKENPROCESSOR
Role	Subject validator	Actor validator	Subject validator	Actor validator
Token Format	JWT (PF-issued)	JWT (PF-issued)	JWT (Azure-issued)	Opaque Bearer
Issuer	`id.example.com`	`id.example.com`	Azure AD (v1 + v2)	(any PF-issued)
JWKS Source	`pf/JWKS`	`pf/JWKS`	Azure Discovery	AccessTokenManagement
Require Audience	✅ Yes	❌ No	✅ Yes	N/A
Required Audience	`contact-hr-client`	-	Azure app audiences	N/A
Require Expiration	✅ Yes	✅ Yes	✅ Yes	N/A
Require Issued At	✅ Yes	✅ Yes	❌ No	N/A
Clock Skew	0s	0s	0s	N/A
Key Claims Extracted	`sub`, `scope`, `groups`, `vaultlocation`, names, `email`	`client_id`	`sub`, `email`	`sub`, `scope`, `client_id`

Access Token Mapping - Token Exchange Context

Mapping Identity

ID:      urn:ietf:params:oauth:grant-type:token-exchange|PROCESSORPOLICIES|AccessTokenManagement
Context: TOKEN_EXCHANGE_PROCESSOR_POLICY → PROCESSORPOLICIES
Manager: AccessTokenManagement

Attribute Sources

Output Claim	Source Type	Value
`actor`	`EXPRESSION`	OGNL - builds JSON object `{ "sub": tepp.actor }`
`vaultlocation`	`TOKEN_EXCHANGE_PROCESSOR_POLICY`	`vaultlocation`
`aud`	`CONTEXT`	`ClientId` - the requesting client ID
`sub`	`TOKEN_EXCHANGE_PROCESSOR_POLICY`	`subject`
`scope`	`TOKEN_EXCHANGE_PROCESSOR_POLICY`	`scope`
`groups`	`TOKEN_EXCHANGE_PROCESSOR_POLICY`	`given_name` ⚠️
`given_name`	`NO_MAPPING`	-
`family_name`	`NO_MAPPING`	-
`email`	`NO_MAPPING`	-

⚠️ Note: In the current configuration, groups in the Access Token Mapping reads from given_name in the processor policy contract. Verify this is intentional if groups are required in the issued token.

OGNL Expression - Actor Claim Transformation

The Expression

#jsonObj = new org.json.simple.JSONObject(),
#jsonObj.put("sub", #this.get("tepp.actor")),
#jsonObj

Step-by-Step Breakdown

Step	Code	Action
1	`new org.json.simple.JSONObject()`	Create empty JSON object
2	`#this.get("tepp.actor")`	Read `actor` from processor policy output
3	`#jsonObj.put("sub", ...)`	Set the `sub` field of the JSON object
4	`#jsonObj`	Return the constructed object as the claim value

Input → Output

tepp.actor = "contact-hr-client"
                   │
                   ▼
      { "sub": "contact-hr-client" }
                   │
                   ▼
  Issued JWT contains:
  "actor": { "sub": "contact-hr-client" }

Why a JSON Object?

RFC 8693 §4.1 defines the act (actor) claim as a JSON object, not a string:

❌  "actor": "contact-hr-client"
✅  "actor": { "sub": "contact-hr-client" }

This enables:

Chain of delegation: nested act objects for multi-hop scenarios
Additional actor identity: can include iss, email, etc.
Standards compliance: downstream services can parse it uniformly

AccessTokenManagement Configuration

Overview

AccessTokenManagement is the JWT Access Token Manager that signs and issues the final token after all processor policy and attribute mapping work is complete.

Settings

Setting	Value	Notes
`id`	`AccessTokenManagement`
`name`	`AccessTokenManagement`
Plugin	`JwtBearerAccessTokenManagementPlugin`
Token Lifetime	120 seconds	~2 minutes
Use Centralized Signing Key	`true`	Uses PF global signing key
JWS Algorithm	`RS256`	RSA + SHA-256
Include Key ID (`kid`)	`true`	Enables key rotation discovery
Include X.509 Thumbprint	`false`
JWKS Cache Duration	720 minutes	12 hours
Enable Token Revocation	`false`	No revocation endpoint
JWT ID Length	22 characters	Unique per token
Include Issued At	`true`	`iat` always present
Issuer Claim Value	`https://id.example.com`
Client ID Claim Name	`client_id`
Scope Claim Name	`scope`
Space Delimit Scope Values	`true`
Authorization Details Claim	`authorization_details`

Signing Key

Property	Value
Key Pair ID	`5jqt7j8mxbwl2awtpc465yzx1`
Algorithm	RSA 2048-bit
Signature Algorithm	RS256
Usage	Token signing (all JWT tokens)
Public JWKS	`https://id.example.com/pf/JWKS`

Attribute Contract

Claims the manager can include in issued tokens:

Attribute	Multi-Valued	Notes
`vaultlocation`	No	Custom - credential vault reference
`actor`	No	RFC 8693 delegation claim (JSON object)
`sub`	No	Subject identifier
`aud`	No	Audience (requesting client)
`scope`	No	Granted scopes
`groups`	Yes	Multi-valued - user's group memberships
`given_name`	No	User's first name
`family_name`	No	User's surname
`email`	No	User's email address

Always-Added Standard Claims

Regardless of attribute mapping, these claims are always added automatically:

{
  "iss": "https://id.example.com",
  "iat": 1776667961,
  "exp": 1776675161,
  "jti": "XWcCCzPtj0OFR6HU8UA7Cw"
}

Default Access Token Manager

{
  "defaultAccessTokenManagerRef": {
    "id": "AccessTokenManagement"
  }
}

This is also the global default - used for all standard OAuth flows in addition to token exchange.

Real-World Example - HR Chatbot Token Exchange

Context

The darkedges-hr-chatbot application:

Obtains a client credentials token for itself (actor_token) at startup via initialize_agent_token()
Receives a user access token after OAuth callback (subject_token)
Performs a token exchange to obtain a delegated token that records both identities

Token Exchange Request

curl -s -X POST 'https://id.example.com/as/token.oauth2' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
  --data-urlencode 'subject_token=eyJhbGciOiJSUzI1NiIsImtpZCI6IlA1X1FfaDdqaGVpRkpWQnBVRlh6M2RPRmRGb19SUzI1NiIsInBpLmF0bSI6IjRld3AifQ.eyJzY29wZSI6WyJvcGVuaWQiLCJwcm9maWxlIiwiZW1haWwiXSwiYXV0aG9yaXphdGlvbl9kZXRhaWxzIjpbXSwiY2xpZW50X2lkIjoiY29udGFjdC1oci1jbGllbnQiLCJpc3MiOiJodHRwczovL2lkLnBpbmcuZGFya2VkZ2VzLmNvbSIsImlhdCI6MTc3NjY2Nzk1OCwianRpIjoieEV4YWRZRXR1cjhPbEtmOU5XM3BWSSIsInZhdWx0bG9jYXRpb24iOiJyZWZlcmVuY2VpZC9tc2VudHJhaWQvMEJVVGZPS2ZLYkNpMlJmNFMta3JOY0ZRVUFKMlI0WURJcWY4WHZsNW5LNCIsImF1ZCI6ImNvbnRhY3QtaHItY2xpZW50Iiwic3ViIjoidXNlci4xIiwiZ3JvdXBzIjpbIkFkbWluaXN0cmF0b3JzIl0sImZhbWlseV9uYW1lIjoiU2Vhd2VsbCIsImVtYWlsIjoibmlydmluZ3VrQGhvdG1haWwuY29tIiwiZXhwIjoxNzc2Njc1MTU4fQ.Cwk_hCqTocEZoE0yYOFnTjMd6UYBE5BToVOpj51GvNQcHAS76sw0p7pygqm5ze9kxntgyG6OQ8KjKxMUwRmCfC4wZimVRW32-1wTt7UNgKxZcCEAw23VO9XNVgCGdQBShWcqpla8-4cSxU0VIqZJQroVsP9L_hy8mUrRmN7dLWAt2f4KkgNuZmWK7xPbhRUQeIkOcjHhc9FQN4MB08O_DU0on6RbeW54pD0ndsviwMAV3MLLh898DkVSzy2_PpPNr8jgRWPBgcjmAuH2h5a_mcjr6Ei6c0tGOZchS05BwA2qjvWI8w9_C-7Ucn3_GIycIbPCh2ni9dAM9e_CjNfpdg' \
  --data-urlencode 'subject_token_type=urn:ietf:params:oauth:token-type:access_token' \
  --data-urlencode 'actor_token=eyJhbGciOiJSUzI1NiIsImtpZCI6IlA1X1FfaDdqaGVpRkpWQnBVRlh6M2RPRmRGb19SUzI1NiIsInBpLmF0bSI6IjRld3AifQ.eyJzY29wZSI6IiIsImF1dGhvcml6YXRpb25fZGV0YWlscyI6W10sImNsaWVudF9pZCI6ImNvbnRhY3QtaHItY2xpZW50IiwiaXNzIjoiaHR0cHM6Ly9pZC5waW5nLmRhcmtlZGdlcy5jb20iLCJpYXQiOjE3NzY2Njc5MzcsImp0aSI6InVqUGVQeHJBVU9MbHJqMDNPUm1QZTEiLCJleHAiOjE3NzY2NzUxMzd9.OEXjyYbBb4KmVMlBZJ8ucnn5_CacufyKL3-E_XsBcQWMhxhm_W9eCOpG3y_xmFGy9wSSNGpPzgVBzeHZ5xyYlSgt2fpBcA2UolQLNT0MKJrbqpJZicqmUh5HalGv6rXG4iuRjpFJ3_-N8zLUrk1t8puZYsSTPaYCrSb1K37_3moPzaNxIgrFplXftax5ez9kgu0QqtA3WyYNJUHAHdFv8cyBbOUy7MMdzTdMlZFaOoO7JHEdFpCzzTkuhtC1D95AADTApvJGsy6Lo4llnJoofnJmmXEjWaAY3hEm2kbXW1he2nR1fZtYQa-_-LxfwR6X5BAxrn96G8JWpc5Y2KKKFw' \
  --data-urlencode 'actor_token_type=urn:ietf:params:oauth:token-type:access_token' \
  --data-urlencode 'requested_token_type=urn:ietf:params:oauth:token-type:access_token' \
  --data-urlencode 'client_id=contact-oauth-client' \
  --data-urlencode 'client_secret=xxxxx' \
  --data-urlencode 'scope=openid profile email'

Actor Token Claims (Chatbot - Client Credentials)

{
  "scope": "",
  "authorization_details": [],
  "client_id": "contact-hr-client",
  "iss": "https://id.example.com",
  "iat": 1776667937,
  "jti": "ujPePxrAUOLlrj03ORmPe1",
  "exp": 1776675137
}

Actor token has empty scope - obtained via Client Credentials, representing the application identity, not a user.

Subject Token Claims (User - Authorization Code)

{
  "scope": ["openid", "profile", "email"],
  "authorization_details": [],
  "client_id": "contact-hr-client",
  "iss": "https://id.example.com",
  "iat": 1776667958,
  "jti": "xExadYEtur8OlKf9NW3pVI",
  "vaultlocation": "referenceid/msentraid/0BUTfOKfKbCi2Rf4S-krNcFQUAJ2R4YDIqf8Xvl5nK4",
  "aud": "contact-hr-client",
  "sub": "user.1",
  "groups": ["Administrators"],
  "family_name": "Seawell",
  "email": "xxxx@hotmail.com",
  "exp": 1776675158
}

Subject token has full user scopes and claims - obtained via Authorization Code flow, representing the user's identity.

Exchanged Token Claims (Issued by PingFederate)

{
  "scope": ["openid", "profile", "email"],
  "authorization_details": [],
  "client_id": "contact-oauth-client",
  "iss": "https://id.example.com",
  "iat": 1776667961,
  "jti": "XWcCCzPtj0OFR6HU8UA7Cw",
  "actor": {
    "sub": "contact-hr-client"
  },
  "vaultlocation": "referenceid/msentraid/0BUTfOKfKbCi2Rf4S-krNcFQUAJ2R4YDIqf8Xvl5nK4",
  "aud": "contact-oauth-client",
  "sub": "user.1",
  "exp": 1776675161
}

What Changed Between Input and Output

Property	Subject Token	Actor Token	Exchanged Token	Notes
`sub`	`user.1`	-	`user.1`	Preserved from subject
`client_id`	`contact-hr-client`	`contact-hr-client`	`contact-oauth-client`	Requesting client replaces original
`aud`	`contact-hr-client`	-	`contact-oauth-client`	Audience = requesting client
`actor`	-	-	`{ "sub": "contact-hr-client" }`	Added - records delegation
`scope`	`["openid","profile","email"]`	`""`	`["openid","profile","email"]`	Preserved from subject
`vaultlocation`	present	-	preserved	Passed through
`iat`	`1776667958`	`1776667937`	`1776667961`	New issuance time
`exp`	`1776675158`	`1776675137`	`1776675161`	New: iat + 120s
`jti`	unique	unique	unique	Fresh JWT ID

Chatbot Log Output

DEBUG: Initiating token exchange - actor: eyJhbGciOi..., subject: eyJhbGciOi...
✓ Token exchange successful for user
2026-04-20 16:52:41 - ✓ Token exchange completed for xxxx@hotmail.com

Token Lifecycle Sequence

T0   User authenticates via PingFederate (Authorization Code flow)
     ├─ client_id: contact-hr-client
     ├─ Response: subject_token (user's access token)
     ├─ Lifetime: 120 seconds
     └─ Contains: sub, scope, vaultlocation, groups, email, names

T1   HR Chatbot app starts - initialize_agent_token() called
     ├─ POST /as/token.oauth2
     ├─ grant_type: client_credentials
     ├─ client_id: contact-hr-client
     ├─ Response: actor_token (chatbot's application token)
     ├─ Lifetime: 120 seconds
     └─ Cached globally in app memory + Redis

T2   OAuth callback fires - user token received in session
     ├─ actor_token available (from T1)
     └─ subject_token available (from OAuth callback)

T3   Token exchange request sent to PingFederate
     ├─ /as/token.oauth2
     ├─ grant_type: urn:ietf:params:oauth:grant-type:token-exchange
     ├─ subject_token: user's token (from T0)
     ├─ subject_token_type: urn:ietf:params:oauth:token-type:access_token
     ├─ actor_token: chatbot's token (from T1)
     ├─ actor_token_type: urn:ietf:params:oauth:token-type:access_token
     ├─ client_id: contact-oauth-client
     └─ scope: openid profile email

T4   PingFederate - policy selection
     ├─ PROCESSORPOLICIES selected (default policy)
     └─ Mapping 1 selected (both token types = access_token)

T5   PingFederate - token validation
     ├─ PFSubjectProcessor validates subject_token
     │   ├─ Verify RS256 signature against pf/JWKS
     │   ├─ Check issuer = https://id.example.com ✓
     │   ├─ Check audience = contact-hr-client ✓
     │   ├─ Check exp not reached ✓
     │   └─ Extract: sub, scope, groups, vaultlocation, email, names
     └─ PFActorSubject validates actor_token
         ├─ Verify RS256 signature against pf/JWKS
         ├─ Check issuer = https://id.example.com ✓
         ├─ Check exp not reached ✓
         └─ Extract: client_id

T6   PingFederate - attribute fulfillment
     ├─ actor        ← actor_token.client_id    = "contact-hr-client"
     ├─ subject      ← subject_token.sub        = "user.1"
     ├─ scope        ← subject_token.scope      = ["openid","profile","email"]
     ├─ vaultlocation← subject_token.vaultlocation
     ├─ groups       ← subject_token.groups     = ["Administrators"]
     ├─ given_name   ← subject_token.given_name
     ├─ family_name  ← subject_token.family_name = "Seawell"
     └─ email        ← subject_token.email      = "xxxx@hotmail.com"

T7   Access Token Mapping - OGNL transformation
     └─ actor → { "sub": "contact-hr-client" }  (JSON object per RFC 8693)

T8   AccessTokenManagement - JWT issuance
     ├─ Sign with RS256, key 5jqt7j8mxbwl2awtpc465yzx1
     ├─ Add: iss, iat, exp (iat+120), jti
     └─ Issued token returned

T9   Chatbot receives exchanged token
     └─ Stored in session metadata as "access_token"

T9+120s  Exchanged token expires
          └─ Next user action triggers new token exchange

Configuration File Locations

Component	Resource Type in data.json
Token Exchange Policy	`/oauth/tokenExchange/processor/policies`
Default Policy Setting	`/oauth/tokenExchange/processor/settings`
All Token Processors	`/idp/tokenProcessors`
Access Token Managers	`/oauth/accessTokenManagers`
Default ATM Setting	`/oauth/accessTokenManagers/settings`
Access Token Mappings	`/oauth/accessTokenMappings`
Signing Key Pair	`/keyPairs/signing`
OIDC Policy	`/oauth/openIdConnect/policies`

Source file: profiles/pingfederate/bulk-export/shared/data.json

Validation Checklist

Pre-Exchange

PingFederate Configuration:
☐ PROCESSORPOLICIES exists and is set as default processor policy
☐ Both processor mappings configured (Mapping 1 + Mapping 2)
☐ actorTokenRequired = true
☐ AccessTokenManagement token manager exists
☐ Signing key 5jqt7j8mxbwl2awtpc465yzx1 is valid and not expired
☐ JWKS endpoint accessible: https://id.example.com/pf/JWKS

PFSubjectProcessor:
☐ Issuer: https://id.example.com
☐ Audience check enabled, value: contact-hr-client
☐ Expiration + Issued At checks enabled
☐ JWKS URL reachable

PFActorSubject:
☐ Issuer: https://id.example.com
☐ Audience check disabled
☐ Expiration + Issued At checks enabled
☐ client_id in attribute contract

MSFTTOKENPROCESSOR:
☐ Both Azure issuers configured
☐ Both JWKS URLs reachable
☐ Required audience values present

PFTOKENPROCESSOR:
☐ References AccessTokenManagement
☐ Scope handling configured

Per-Request

Subject Token:
☐ JWT (3 dot-separated parts)
☐ RS256 signature valid
☐ iss = https://id.example.com
☐ aud = contact-hr-client
☐ exp > now (not expired)
☐ iat present and reasonable
☐ sub claim present
☐ scope claim present

Actor Token:
☐ JWT (3 dot-separated parts)
☐ RS256 signature valid
☐ iss = https://id.example.com
☐ exp > now (not expired)
☐ client_id claim present

Request Parameters:
☐ grant_type = urn:ietf:params:oauth:grant-type:token-exchange
☐ subject_token_type = urn:ietf:params:oauth:token-type:access_token
☐ actor_token_type = urn:ietf:params:oauth:token-type:access_token
☐ actor_token present (required by policy)
☐ client_id = contact-oauth-client (or other authorised client)

Issued Token

☐ JWT signed RS256
☐ kid header = 5jqt7j8mxbwl2awtpc465yzx1
☐ iss = https://id.example.com
☐ sub preserved from subject_token
☐ actor present and is a JSON object: { "sub": "<client_id>" }
☐ aud = requesting client_id
☐ exp = iat + 120
☐ jti unique 22-character value
☐ vaultlocation preserved (if present in subject token)
☐ scope matches requested scopes

Troubleshooting

Symptom	Likely Cause	Resolution
`Agent access token not available, skipping token exchange`	`initialize_agent_token()` failed at startup	Check Redis connectivity; verify PF `/as/token.oauth2` is reachable; check `contact-hr-client` credentials
`invalid_request` on token exchange	Missing required parameter or wrong token type	Confirm `actor_token` and `actor_token_type` present; verify token type URNs
`invalid_token` on subject or actor	JWT validation failed	Check token not expired; verify issuer; confirm audience matches processor config
`actor` claim missing from issued token	OGNL expression failed	Check `org.json.simple` library available; verify `tepp.actor` is populated
`actor` claim is string not object	Wrong mapping - not using OGNL	Ensure Access Token Mapping uses EXPRESSION source for `actor`
`vaultlocation` missing from issued token	Attribute fulfillment issue	Confirm subject token contains `vaultlocation`; check Mapping 1 contract
Token exchange succeeds but groups empty	`groups` maps from `given_name` in current config	Review Access Token Mapping - `groups` attribute source currently points to `given_name` ⚠️
Timeout acquiring actor token	PingFederate slow or unreachable	10-second timeout in chatbot; check network/TLS; check PF health

References

RFC 8693 - OAuth 2.0 Token Exchange
- §1.1: Delegation vs. Impersonation Semantics
- §4.1: act (Actor) Claim
- §4.2: scope Claim
- §4.3: client_id Claim
PingFederate 12.3 Documentation
Project Files
- TOKENEXCHANGE.md - Token exchange curl examples
- data.json - Full PingFederate configuration export
- darkedges-hr-chatbot/app.py - Chatbot token exchange implementation
- darkedges-hr-chatbot/auth_handler.py - perform_token_exchange() method

Weekend Build Recap: Trust-Aware API Access with OpenID Federation

DarkEdges — Sun, 12 Apr 2026 20:51:16 +0000

This weekend we built and validated a trust-driven access control flow across our OpenID Federation demo stack.

Core outcome:

If app.idamaas.xyz is not an active trusted subordinate, API access is blocked.
If the required trust mark for oidfapi.verifymy.id is revoked or missing, API access is blocked.
When either trust condition is restored, access is restored.

What We Developed

Federation-aware app validation in app.idamaas.xyz.
Trust-anchor-backed trust mark enforcement (no self-asserted shortcut).
Admin trust mark lifecycle controls: issue, revoke, enable, cleanup.
Diagnostics endpoints to explain trust decisions.

Demo Story 1: `app.idamaas.xyz` Enabled/Disabled

Enabled (expected success)

When app.idamaas.xyz is active as a subordinate:

/demo/discover-and-call returns ok: true
apiResult.ok is true

Disabled (expected block)

When app.idamaas.xyz is deactivated/deleted as subordinate:

/demo/discover-and-call returns HTTP 403
error is client_not_trusted

Demo Story 2: Trust Mark Enabled/Disabled

Trust mark enabled (expected success)

Required trust mark:

issuer: https://trust-anchor.zkp.au
id: urn:darkedges:trustmark:identity-verification

When active:

trustMarkValidation.ok is true
/demo/discover-and-call succeeds

Trust mark revoked (expected block)

When required trust mark is revoked:

/demo/discover-and-call returns HTTP 403
error is required_trust_mark_missing

Key Endpoints Used

app demo flow: GET /demo/discover-and-call
app diagnostics: GET /demo/federation-details
trust mark list: GET /federation_trust_mark_list
trust mark fetch: GET /federation_trust_mark
subordinate admin: /admin/subordinates
trust mark admin: /admin/trust-marks

Operational Outcome

By the end of the weekend, trust state directly controlled access:

disable subordinate -> blocked
revoke trust mark -> blocked
re-enable either control -> restored

This gives us a practical, explainable trust control model for federation demos and production hardening.

OpenIDFederation #TrustAnchor #TrustMarks #ZeroTrust

Enriching Vault OIDC Tokens with SPIFFE Identity Metadata using Terraform

DarkEdges — Wed, 03 Dec 2025 19:39:08 +0000

In modern microservices architectures, machine identity is just as critical as human identity. When services communicate, they often need to prove not just who they are, but what they are—their environment, business unit, or SPIFFE ID.

HashiCorp Vault is excellent for this. Its Identity Secrets Engine can issue OIDC tokens that serve as verifiable credentials for your workloads. However, getting those tokens to contain rich, custom metadata requires connecting a few specific dots: AppRole, Identity Entities, and OIDC Templates.

In this article, I'll walk through how to implement a robust machine identity pipeline using Terraform, where an application authenticates via AppRole and receives an OIDC token enriched with custom metadata.

The Goal

We want an application (e.g., a "ChatBot") to:

Authenticate to Vault using the AppRole method.
Request an OIDC token.
Receive a token containing custom claims like spiffe_id, business_unit, and environment.

Step 1: Define the Identity Entity

First, we define the "who". In Vault, an Entity represents a unique identity. This is where we store the metadata we want to eventually see in our token.

# identities.tf

resource "vault_identity_entity" "application" {
  for_each = local.application_identities_map
  name     = each.key

  # This metadata is what we'll inject into the token later
  metadata = {
    environment   = each.value.identity.environment
    business_unit = each.value.identity.business_unit
    spiffe_id     = "spiffe://vault/application/${each.value.identity.environment}/${each.value.identity.business_unit}/${each.value.identity.name}"
  }
}

Step 2: Configure AppRole Authentication

Next, we configure the AppRole auth backend. This is the standard way for machines to authenticate.

# approle.tf

resource "vault_approle_auth_backend_role" "applications" {
  for_each       = local.application_identities_map
  backend        = vault_auth_backend.approle.path
  role_name      = each.key
  token_ttl      = 3600
  bind_secret_id = true
}

The "Secret Sauce": Binding AppRole to the Entity

This is the most critical step. By default, when you log in with AppRole, Vault creates a generic entity for that role. To use our custom metadata defined in Step 1, we must explicitly bind the AppRole to our specific Entity using an Entity Alias.

Important Gotcha: When creating an alias for AppRole, the name of the alias must be the Role ID, not the Role Name.

# approle.tf

resource "vault_identity_entity_alias" "approle_applications" {
  for_each       = local.application_identities_map

  # CRITICAL: Use role_id, not role_name
  name           = vault_approle_auth_backend_role.applications[each.key].role_id

  mount_accessor = vault_auth_backend.approle.accessor
  canonical_id   = vault_identity_entity.application[each.key].id
}

We also need to ensure the AppRole inherits the entity's properties. We can enforce this via a generic endpoint configuration:

resource "vault_generic_endpoint" "approle_entity_inherit" {
  for_each   = local.application_identities_map
  depends_on = [vault_approle_auth_backend_role.applications]
  path       = "auth/approle/role/${each.key}"

  data_json = jsonencode({
    entity_alias_sole_inherit = true
  })
}

Step 3: Configure the OIDC Template

Finally, we configure the OIDC provider and role. The template field is where the magic happens. We use Vault's template syntax to pull values dynamically from the authenticated entity's metadata.

# identity_tokens.tf

resource "vault_identity_oidc_role" "application_identity" {
  name      = "application_identity"
  key       = vault_identity_oidc_key.application_identity.name
  client_id = "spiffe://vault.darkedges.au/gateway"
  ttl       = 86400

  # The Template: Injecting metadata into the JSON payload
  template = <<EOT
{
  "azp": {{identity.entity.metadata.spiffe_id}},
  "nbf": {{time.now}},
  "groups": {{identity.entity.groups.names}},
  "appinfo": {
    "business_unit": {{identity.entity.metadata.business_unit}},
    "environment": {{identity.entity.metadata.environment}}
  }
}
EOT
}

Step 4: Testing the Flow

Now, let's see it in action. We'll use PowerShell to simulate the application workflow.

1. Get Credentials & Login

First, we retrieve the Role ID and Secret ID, then authenticate to get a Vault token.

# Get Role ID and Secret ID
$ROLE_ID = docker-compose exec vault vault read -field=role_id auth/approle/role/ChatBot/role-id
$SECRET_ID = docker-compose exec vault vault write -f -field=secret_id auth/approle/role/ChatBot/secret-id

# Login
$APPTOKEN = docker-compose exec vault vault write -field=token auth/approle/login role_id="$ROLE_ID" secret_id="$SECRET_ID"

2. Request the OIDC Token

Using the Vault token we just got, we request the OIDC token.

$OIDC_TOKEN = docker-compose exec -e VAULT_TOKEN="$APPTOKEN" vault vault read -field=token identity/oidc/token/application_identity

3. The Result

When we decode the JWT, we see our rich metadata is successfully populated!

{
  "appinfo": {
    "business_unit": "engineering",
    "environment": "production"
  },
  "aud": "spiffe://vault.darkedges.au/gateway",
  "azp": "spiffe://vault/application/production/engineering/ChatBot",
  "exp": 1764848993,
  "groups": [
    "ChatBot group"
  ],
  "iat": 1764762593,
  "iss": "https://vault.darkedges.au/v1/identity/oidc",
  "sub": "ea0e0006-d4f5-cbde-3cb6-c013d4dba5f2"
}

Conclusion

By combining Identity Entities, AppRole Aliases, and OIDC Templates, we've turned Vault into a powerful identity provider for our internal services. This setup allows downstream systems (like API gateways or service meshes) to make authorization decisions based on trusted, verifiable metadata like business_unit or environment, rather than just a raw IP address or generic token.

a complete example is available at https://github.com/darkedges/spiffe-vault-terraform

Inspiration

Auto-Detecting CSV Schemas for Lightning-Fast ClickHouse Ingestion with Parquet

DarkEdges — Fri, 07 Nov 2025 09:06:41 +0000

As a data engineer, one of the most repetitive tasks I face is ingesting data from CSV files. The problem isn't just loading the data; it's the ceremony that comes with it. Every time a new data source appears, I have to manually inspect the columns, define a table schema, and write a script to load it. What if the CSV has 100 columns? What if the data types are ambiguous? This process is tedious and error-prone.

I wanted a better way. My goal was to create a Node.js script that could:

Read any CSV file without prior knowledge of its structure.
Auto-detect the schema, including column names and data types.
Convert the CSV to Parquet, a highly efficient columnar storage format.
Prepare for ingestion into ClickHouse, which loves Parquet.

In this article, I'll walk you through the proof-of-concept I built to solve this exact problem.

Why Parquet and ClickHouse?

ClickHouse is an open-source, column-oriented database built for speed. It's a beast for analytical queries (OLAP).

Apache Parquet is a columnar storage format. Instead of storing data in rows, it stores it in columns. This is a perfect match for ClickHouse because it allows the database to read only the columns it needs for a query, dramatically reducing I/O and speeding up performance.

Combining the two means you get efficient storage and lightning-fast analytics.

The Concept: From CSV to Parquet

Our script will perform a three-step process:

Schema Detection: Read the CSV headers and a few sample rows to infer the data type of each column (e.g., string, number, boolean).
Data Transformation: Convert the raw string values from the CSV into their inferred types.
Parquet Conversion: Write the transformed data and the detected schema into a new .parquet file.

Let's start with a simple CSV file to demonstrate.

Our Example CSV: `sample-data.csv`

id,first_name,last_name,email,is_active,created_at,balance
1,John,Doe,john.doe@example.com,true,2023-01-15T10:00:00Z,150.75
2,Jane,Smith,jane.smith@example.com,false,2023-02-20T11:30:00Z,200.00
3,Peter,Jones,peter.jones@example.com,true,2023-03-10T09:05:00Z,50.25
4,Mary,Williams,mary.w@example.com,true,2023-04-01T15:45:00Z,300.50

This file has a nice mix of data types: integers, strings, booleans, timestamps, and floating-point numbers.

Step 1: Setting Up the Project

First, let's set up a simple Node.js project and install the necessary libraries. We'll use papaparse for robust CSV parsing and parquetjs for writing Parquet files.

npm init -y
npm install papaparse parquetjs

Step 2: Auto-Detecting the Schema

This is the core of our solution. We need a function that can look at the string data from the CSV and make an educated guess about its real type.

Here’s a simple type inference function. It checks if a value is a boolean, a number, or a date. If it's none of those, it defaults to a string.

index.js

// Function to infer data type from a string value
function inferType(value) {
    if (value === 'true' || value === 'false') return 'BOOLEAN';
    if (!isNaN(value) && value.trim() !== '') return 'DOUBLE'; // Use DOUBLE for all numbers for simplicity
    if (!isNaN(Date.parse(value))) return 'UTF8'; // Dates will be stored as strings (UTF8)
    return 'UTF8'; // Default to string
}

Now, we can use this to build a schema from the CSV file. We'll read the first data row to infer the types for each column.

index.js

const fs = require('fs');
const papa = require('papaparse');

// ... inferType function from above ...

async function detectSchema(filePath) {
    const fileContent = fs.readFileSync(filePath, 'utf8');
    const result = papa.parse(fileContent, { header: true, preview: 1 });

    if (result.errors.length > 0) {
        throw new Error('Error parsing CSV for schema detection.');
    }

    const firstRecord = result.data[0];
    const schema = {};

    for (const header in firstRecord) {
        const value = firstRecord[header];
        schema[header] = { type: inferType(value) };
    }

    return schema;
}

When we run detectSchema('sample-data.csv'), it will produce an object like this, which is exactly the format parquetjs needs:

{
  "id": { "type": "DOUBLE" },
  "first_name": { "type": "UTF8" },
  "last_name": { "type": "UTF8" },
  "email": { "type": "UTF8" },
  "is_active": { "type": "BOOLEAN" },
  "created_at": { "type": "UTF8" },
  "balance": { "type": "DOUBLE" }
}

Step 3: Converting CSV to Parquet

With the schema detected, we can now read the entire CSV and write it to a Parquet file. The process involves:

Creating a ParquetSchema and a ParquetWriter.
Iterating through the CSV rows.
Casting each value to its detected type (e.g., converting the string "true" to the boolean true).
Appending the transformed row to the writer.

Here's the code to bring it all together:

index.js

const { ParquetSchema, ParquetWriter } = require('parquetjs');

// ... detectSchema and inferType functions ...

async function convertCsvToParquet(csvPath, parquetPath) {
    console.log('Detecting schema...');
    const schemaDefinition = await detectSchema(csvPath);
    const schema = new ParquetSchema(schemaDefinition);

    const writer = await ParquetWriter.openFile(schema, parquetPath);

    const fileContent = fs.readFileSync(csvPath, 'utf8');
    const result = papa.parse(fileContent, { header: true, skipEmptyLines: true });

    console.log(`Found ${result.data.length} records. Converting to Parquet...`);

    for (const row of result.data) {
        const processedRow = {};
        for (const key in row) {
            const value = row[key];
            const type = schemaDefinition[key].type;

            // Cast values to their proper types
            if (type === 'BOOLEAN') {
                processedRow[key] = value === 'true';
            } else if (type === 'DOUBLE') {
                processedRow[key] = parseFloat(value);
            } else {
                processedRow[key] = value;
            }
        }
        await writer.appendRow(processedRow);
    }

    await writer.close();
    console.log(`Parquet file written to ${parquetPath}`);
}

// Run the conversion
convertCsvToParquet('sample-data.csv', 'output.parquet').catch(console.error);

After running this script (node index.js), you'll have an output.parquet file ready for ClickHouse!

Step 4: Ingesting into ClickHouse

Now for the easy part. Ingesting the Parquet file into ClickHouse is incredibly simple. First, you need a table with a matching schema.

CREATE TABLE my_data (
    id Float64,
    first_name String,
    last_name String,
    email String,
    is_active Boolean,
    created_at String,
    balance Float64
) ENGINE = MergeTree()
ORDER BY id;

Then, you can use the clickhouse-client to ingest the file directly.

clickhouse-client --query="INSERT INTO my_data FORMAT Parquet" < output.parquet

ClickHouse reads the Parquet file's schema and streams the data directly into the table. It's fast, efficient, and requires no complex INSERT statements with tons_of_values.

Conclusion

By automating schema detection and converting CSVs to Parquet, we've created a powerful and reusable ETL pipeline. This approach saves a massive amount of time, reduces manual errors, and leverages the high-performance capabilities of both Parquet and ClickHouse.

This proof-of-concept can be expanded with more robust type detection, error handling, and integration into a larger data workflow, but it's a fantastic starting point for streamlining your data ingestion process.

Uprading Ping Identity Platform to 8.0.0

DarkEdges — Fri, 22 Aug 2025 04:25:24 +0000

Ping AM

Issue with new config when running amupgrade

The following appears to be added in the wrong place with incorrect details

Error on Startup after migration

When PingAM Starts and you attempt to connect it fails with this in the log.

dfq-am  | {"timestamp":"2025-08-22T04:24:20.258Z","level":"WARN","thread":"http-nio-8080-exec-10","mdc":{"transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-38"},"logger":"org.forgerock.openam.core.realms.impl.DefaultRealmLookup","message":"DefaultRealms:lookup Unable to find Org name for: serverinfo","context":"default","transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-38"}
dfq-am  | {"timestamp":"2025-08-22T04:24:20.397Z","level":"WARN","thread":"http-nio-8080-exec-2","mdc":{"transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-43"},"logger":"org.forgerock.openam.core.realms.impl.DefaultRealmLookup","message":"DefaultRealms:lookup Unable to find Org name for: sessions","context":"default","transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-43"}
dfq-am  | {"timestamp":"2025-08-22T04:24:20.414Z","level":"WARN","thread":"http-nio-8080-exec-2","mdc":{"transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-43"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default","transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-43"}
dfq-am  | {"timestamp":"2025-08-22T04:24:20.523Z","level":"WARN","thread":"http-nio-8080-exec-2","mdc":{"transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-43"},"logger":"com.sun.identity.idm.plugins.internal.AgentsRepo","message":"AgentsRepo.getAgentGroupConfig: Unable to get Agent Group Config due to The instance agentgroup does not exist","context":"default","transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-43"}
dfq-am  | {"timestamp":"2025-08-22T04:24:20.637Z","level":"WARN","thread":"http-nio-8080-exec-1","mdc":{"transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-56"},"logger":"org.forgerock.openam.core.rest.authn.http.AuthenticationServiceV2","message":"Authentication encountered an error: [Status: 401 Unauthorized]","context":"default","transactionId":"4a84d7a7-8b56-4819-b5c6-6adc4d215e2e-56"}

it is missing the following entry in Ping DS

dn: ou=agentgroup,ou=Instances,ou=1.0,ou=AgentService,ou=services,ou=am-config
objectClass: top
objectClass: sunServiceComponent
objectClass: organizationalUnit
ou: agentgroup
sunserviceID: agentgroup

Creating a Workflow using your Connector

DarkEdges — Sat, 16 Aug 2025 06:34:34 +0000

In order to use connector you need to

Create a Connection
Create a Flow

Create a Connection

Connect to your Okta Workflow instance.
Select Connections.
Under Connections click the New Connection icon.
Select Your connector
Provide the details and click the Create button.

Create a Flow

Connect to your Okta Workflow instance.
Select Flows.
Click the New Flow button.
In the When this happens block click Add event and select API Endpoint.
Click save
In the Then do this block click Add function to add functions.
When finished it should look like.
Save it as the name you want to display when selecting the action. Ensure you select ``

Test Flow

Click the Run button and click Run
When it has completed the execution it should be successful.
Open the flow directly and it should return similair to

` { "output": { "length": 1 }, "statusCode": 200 } `

Creating a Connector using the Okta Workflow Connector Builder

DarkEdges — Sat, 16 Aug 2025 06:33:53 +0000

Okta Workflow is a no code development environment to create workflows to perform a large number of operations based on Connections. If there is not a Connector available you can create your own using their Connector Builder, and the best part is that you can try it out using their free Integration account.

Okta workflow no code approach is drag and drop between functions / actions. Most functions have Inputs and Outputs, so you can drag an Output onto an Input. Once done you can highlight Input or Output to see the connection.

For example:

Here are the basic steps.

Create a Connector.
Configure Authentication.
Create a httpHelper flow - used by the connector to make requests using the connection Acces token.
Create a _pingAuth flow - Validates the Access Token to see if it needs to be renewed.
Create a action flow- Performs an operation that can be used in Flow.
Deploy it

Create a Connector.

Connect to your Okta Workflow instance.
Select Connector Builder.
Under Connectors click the + icon.
Provide the name for the connector and click the Save button.

Configure Authentication.

Click the Add Authentication button and fill out the details

Create a Test Connection

Click Test Connections
Click +New Connection
Provide the details and click the Create button

Create a `httpHelper` flo.

Select Flows
Click +New Flow
In the When this happens block click Add event and select Helper Flow.
In the Then do this block click Add function to add functions.
When finished it should look like.
Save it as httpHelper

Create a `_pingAuth` flo.

Select Flows
Click +New Flow
In the When this happens block click Add event and select Authping.
In the Then do this block click Add function to add functions.
When finished it should look like.
Save it as _authPing

Create a `action` flo.

Select Flows
Click +New Flow
In the When this happens block click Add event and select Action.
In the Then do this block click Add function to add functions.
When finished it should look like.
Save it as the name you want to display when selecting the action.

Deploy

Click Deployment
Click Validate connector and then Done
Click Deploy test version
Click Deploy local connector

That gets a connector deployed and ready for using in an actual flow.

Finally testing the solution

DarkEdges — Sat, 16 Aug 2025 02:09:24 +0000

To recap, we have an Private Application Load Balancer, Fargate ECS Cluster and an AWS API Gateway v2. So know we can test the solution.

DEV Community: DarkEdges

VEX demo update: adding Docker Scout attestations (and three new gotchas)

What changed

Gotcha 4: Docker Scout won't match pkg:oci/

Gotcha 5: attestations and filesystem VEX are mutually exclusive (Scout)

Gotcha 6: Scout attestations need the containerd image store

The attestation flow end-to-end

Updated toolchain image

Summary: which mechanism to use when

From 70 CVEs to 0: a hands-on VEX suppression workflow with Trivy (and a path to Wiz)

The setup

Step 1, Baseline

Step 2, Generate OpenVEX statements

Step 3, The instant win: --vex flag

Step 4, Publish via a VEX repository

The three gotchas

1. The repository index needs repository_url for OCI purls

2. Embedding VEX in the image does nothing (for scanning)

3. VEX binds to the digest, derived images don't inherit it

What about Wiz?

Try it

Deploy Ping Identity Products on Kubernetes with a Single Operator

The design in one paragraph

Prerequisites

Installing the operator

Option A: from the OCI Helm chart

Option B: from source (Docker Desktop)

Creating your first environment

Adding PingFederate

Hostnames are derived automatically

Adding more products

PingDirectory

PingAuthorize (with a startup dependency on PingDirectory)

PingAuthorizePAP

PingAccess (waits for PingFederate)

Startup ordering with waitFor

Layered server profiles

Resource sizing with tier

Checking status

Removing a product

What's next

From Laptop to Cluster: Running darkedges-entraid-tokenexchange with Docker and Kubernetes

Why this setup works

Local path: Docker Compose

Step 1: Create your environment file

OIDC_PROVIDERS: quick configuration pattern

Entra app types you should provision

1-minute env mapping check

Step 2: Build and start

Step 3: Verify runtime

Step 4: Stop or reset

Cluster path: Kubernetes + Helm

Step 1: Build and push image

Step 2: Create namespace

Step 3: Add pull secret for private registry

Step 4: Update values

Step 5: Install or upgrade release

Step 6: Verify rollout

Step 7: Access the app

Lessons learned

Production checklist

Handy commands

PingFederate Token Exchange Processor Policy

Table of Contents

Overview

Complete Configuration Dependency Map

PROCESSORPOLICIES Policy

Core Configuration

Attribute Contract

Processor Mapping Selection Logic

Issuance Criteria

Processor Mapping 1 - PingFederate ↔ PingFederate

Overview

PFSubjectProcessor

PFActorSubject

Attribute Fulfillment - Mapping 1

Processor Mapping 2 - Microsoft Entra ID ↔ PingFederate

Overview

MSFTTOKENPROCESSOR

PFTOKENPROCESSOR

Gotcha 4: Docker Scout won't match `pkg:oci/`

Step 3, The instant win: `--vex` flag

1. The repository index needs `repository_url` for OCI purls

Startup ordering with `waitFor`

Resource sizing with `tier`

Demo Story 1: `app.idamaas.xyz` Enabled/Disabled

Our Example CSV: `sample-data.csv`