In the last blog, we were building and deploying inmutable oci packages from the pipeline to AWS EKS cluster using a model hub-spoke with argocd, kro y ack managed using KCL librarys in AWS EKS to create resources.
Check last blog
The next step is making it verifiable: signing every artifact, and having the cluster refuse to run anything it can't verify.
AWS Signer, the managed code-signing service, makes it possible to add this extra layer of security. It's used at two different points: inside the pipeline, before rendering the manifests (we verify that the KCL library being used is signed by an official AWS Signer profile), and inside the EKS cluster, through an ArgoCD PreSync hook that verifies the final artifact's signature just before it's applied. In both cases, the question is: the signature is the same, no one modified it after signing it?
Note: That an package is signed not means that is secure, only added an step for validate that never changed since was signed.
What is a signing profile, and what does it do across the supply chain?
A signing profile is the cryptographic identity that signs not the signer itself (that's the AWS Signer service), but the named entity with an AWS-managed key pair (you never see the private key) tied to a signing platform. In this case, Notation-OCI-SHA384-ECDSA, which fixes both the signature format (Notation, a CNCF project) and the algorithm (ECDSA P-384 + SHA-384).
It carries:
- A unique ARN (
arn:aws:signer:...:/signing-profiles/platform_xxxxx) the value that travels through the whole system: Notation's trust policy uses it astrustedIdentities, the pipeline's IAM role needs it to callsigner:SignPayload, and the cluster hook uses it to verify. - An AWS-managed root certificate, which is what Notation imports into its trust store (
signingAuthority:aws-signer-ts), so verification works without you managing any certificates yourself.
Across this pipeline's supply chain, the profile acts at three points:
-
Signing (automatic, asynchronous, invisible to the pipeline) this setup uses registry-wide managed signing (
AWS::ECR::SigningConfigurationwith a wildcardRepositoryFilters). Whenever something is pushed to any repository underkcl-modules/*, ECR automatically triggers signing with that profile. The pipeline never callsaws signer signit simply waits and confirms completion (describe-image-signing-status→COMPLETE). -
Frontier 1 (CodeBuild, before rendering) the pipeline imports a trust policy stating "I only trust signatures from this profile ARN, for the
blueprintsscope." Notation verification checks that the signature on the blueprints digest was created by that exact profile. If someone pushed a library without going through the signing configuration, or if the digest was altered after signing, verification fails and the build halts before it ever reacheskcl run. -
Frontier 2 (PreSync hook on the cluster, before applying) the same mechanism, but validating the
manifestsartifact right before ArgoCD applies it. The hook carries its own trust policy with the sametrustedIdentitiesARN. This closes the loop: what was signed in ECR is exactly what gets applied to the cluster, verified at the last possible moment.
The guarantee this gives and its limit: the profile proves origin (it came from this pipeline, not a third party) and integrity (the packahe hasn't changed since it was signed). It does not prove the content is safe, and it doesn't prevent replaying an old-but-still-validly-signed digest signing is not the same as authorizing this specific version, right now.
Prerequisites
Architecture
The signature flow and promotion: push to branch → build → signature of KCL library (
blueprints) verified before of render → render and push of package of manifests → PR of "Pinning the Exact Digest" promotion → humman approval → ArgoCD use the reference → PreSync hook verify the signature of package against that digest before apply.
Check all code in this repository for replicate this blog.
IAC Infraestructure
segoja7
/
aws_eks_capabilities_carm
Cross-Account Resource Management
Terraform Terragrunt Scaffold
Enterprise-grade AWS infrastructure scaffold using Terraform and Terragrunt with a layered architecture based on Domain-Driven Design (DDD) principles. Backs the GitOps Beyond Git blog series: a hub-spoke EKS Capabilities platform (ArgoCD, KRO, ACK, managed) delivered via OCI and GitOps, with AWS Signer–verified supply-chain security on top.
- Part 1 — Immutable Platform Delivery with OCI and Argo CD
- Part 2 — AWS Signer in the supply chain (signing, digest pinning, and why signing alone isn't enough) — this repo's
docs/blog-part2-final.md
Architecture
The project implements four infrastructure layers with strict dependency ordering:
Foundation → Platform → Application → Observability
It spans two AWS accounts in the same Organization — a hub that owns the EKS cluster, the CI/CD pipeline, and the ArgoCD capability, and a spoke that only trusts the hub's ACK controller to create resources on its behalf (CARM — Cross-Account Resource Management):
accounts/
├── hub/ # 700922282251KCL-YAML Kro-ack CRD.
segoja7
/
aws-eks-capabilities-kro-ack-argocd-oci-pipeline
OCI and Argo CD using EKS Capabilities (KRO, ACK )
With everything deployed, let's see how the pipeline works with additional signing layer.
The pipeline in Action
1. triggering the pipeline with the kro-ack-argocd-oci code.
$ git push codecommit main
remote: Validating objects: 100%
To https://git-codecommit.us-east-1.amazonaws.com/v1/repos/platform-engineering-from-scratch
* [new branch] main -> main
$ aws codepipeline list-pipeline-executions --pipeline-name base-eks-dev-cicd \
--max-items 1 --query 'pipelineExecutionSummaries[0].[pipelineExecutionId,status]' --output text
5f3eee4a-5030-48b7-85a7-360aa70fcf83 InProgress
2. The pipeline signs, renders, and opens a PR
3. Who has permission to approve and merge?
$ aws iam get-role-policy --role-name base-eks-dev-platform-approver \
--policy-name base-eks-dev-platform-approver \
--query 'PolicyDocument.Statement[0].Action'
[
"codecommit:UpdatePullRequestApprovalState",
"codecommit:MergePullRequestByFastForward",
"codecommit:ListPullRequests",
"codecommit:GetPullRequestApprovalStates",
"codecommit:GetPullRequest",
"codecommit:GetDifferences",
"codecommit:GetCommit",
"codecommit:EvaluatePullRequestApprovalRules"
]
The policy above defines what the role can do. The trust policy below defines who can become that role:
$ aws iam get-role --role-name base-eks-dev-platform-approver \
--query 'Role.AssumeRolePolicyDocument'
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "operator",
"Effect": "Allow",
"Principal": { "AWS": "arn:aws:iam::<>:root" },
"Action": "sts:AssumeRole",
"Condition": {
"ArnLike": {
"aws:PrincipalArn": "arn:aws:iam::<>:role/aws-reserved/sso.amazonaws.com/*AdministratorAccess*"
}
}
}
]
}
4. ArgoCD updates the reference — and the hook triggers.
the merge is exactly the last commit en main the file that argocd reads as its source of truth changes its targetRevision to the new digest.
Argo detect the new change in Git, synchronizes the application and the PreSync hook triggers before anything is applied.
segoja7@renderer:~$ kubectl get jobs -n argocd -w
NAME STATUS COMPLETIONS DURATION AGE
verify-manifests-signature Running 0/1 0s
verify-manifests-signature Running 0/1 0s 0s
verify-manifests-signature Running 0/1 1s 1s
segoja7@renderer:~kubectl logs -n argocd job/verify-manifests-signature -f
Trust policy configuration imported successfully.
Warning: existing trust policy configuration file will be overwritten
layer-1-rgds is pinned to: sha256:40992367b217695f8c516e6a47dea6759772697d52a7b19407203fcc238c27b5
Verifying <account_id>.dkr.ecr.us-east-1.amazonaws.com/kcl-modules/manifests@sha256:40992367b217695f8c516e6a47dea6759772697d52a7b19407203fcc238c27b5 ...
Successfully verified signature for <account_id>.dkr.ecr.us-east-1.amazonaws.com/kcl-modules/manifests@sha256:40992367b217695f8c516e6a47dea6759772697d52a7b19407203fcc238c27b5
The hook completed, and PreSync kept its promise: nothing was applied before that. As soon as it finished, ArgoCD proceeded with the actual apply the ResourceGraphDefinitions that layer-1-rgds exists to register:
$ kubectl get crd | grep kro.run
eksclusters.kro.run 2026-10-12T22:01:59Z
vpcnetworks.kro.run 2026-10-12T22:02:01Z
resourcegraphdefinitions.kro.run 2026-10-12T22:03:01Z
$ kubectl get resourcegraphdefinitions.kro.run
NAME APIVERSION KIND STATE READY AGE
eks-cluster v1alpha1 EKSCluster Active True 1m
vpc-network v1alpha1 VPCNetwork Active True 1m
One to three seconds after the hook finished, the CRDs were already registered and marked as Active/Ready. With EKSCluster and VPCNetwork available as types in the cluster, the component required for the next step is ready: an actual claim requesting a cluster, built upon the very CRDs we just saw come into existence under a verified signature.
every package with a signature. Now is time for validate that cluster(argocd PreSync hook) reject a package that canot verify.
5. Reproducing the rejection
The most accurate way to test this is to run exactly what the actual hook runs in an isolated Job, using the same identity and trust policy while pointing to a digest that never existed.
Note: change the placeholder account_id for your account id.
# verify-rejection-test.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: verify-rejection-test
namespace: argocd
spec:
backoffLimit: 0
template:
spec:
serviceAccountName: manifests-verifier # the same sa
restartPolicy: Never
containers:
- name: notation-verify
image: public.ecr.aws/amazonlinux/amazonlinux:2023
env:
- name: AWS_REGION
value: "us-east-1"
command: ["/bin/bash", "-c"]
args:
- |
set -e
dnf install -y tar gzip aws-cli >/dev/null
BASE="https://d2hvyiie56hcat.cloudfront.net/linux/amd64/installer"
curl -fsSL "$BASE/rpm/latest/aws-signer-notation-cli_amd64.rpm" -o /tmp/signer-notation.rpm
rpm -U /tmp/signer-notation.rpm
mkdir -p "$HOME/.docker"
TOKEN=$(aws ecr get-login-password --region "$AWS_REGION")
AUTH=$(printf 'AWS:%s' "$TOKEN" | base64 -w0)
printf '{"auths":{"%s":{"auth":"%s"}}}' "<account_id>.dkr.ecr.us-east-1.amazonaws.com" "$AUTH" > "$HOME/.docker/config.json"
# the SAME trust policy the real hook uses — same registryScope, same signing profile
cat > /tmp/trust-policy.json <<POLICY
{
"version": "1.0",
"trustPolicies": [{
"name": "manifests",
"registryScopes": ["<account_id>.dkr.ecr.us-east-1.amazonaws.com/kcl-modules/manifests"],
"signatureVerification": { "level": "strict" },
"trustStores": ["signingAuthority:aws-signer-ts"],
"trustedIdentities": ["arn:aws:signer:us-east-1:<account_id>:/signing-profiles/platform_87ab46385905556b0b0ce03852"]
}]
}
POLICY
notation policy import --force /tmp/trust-policy.json
# a digest that was never pushed to ECR
FAKE_DIGEST="sha256:3d13d104e47ed7ce4db079caa1f6f3a4027d25d5af34a2a480b207b3b8d55607"
echo "Intentando verificar un digest que nunca existio: $FAKE_DIGEST"
notation verify "<account_id>.dkr.ecr.us-east-1.amazonaws.com/kcl-modules/manifests@$FAKE_DIGEST"
apply and validate:
segoja7@renderer:~$ kubectl apply -f verify-rejection-test.yaml
job.batch/verify-rejection-test created
segoja7@renderer:~$ kubectl logs -n argocd verify-rejection-test-g99h4
Warning: existing trust policy configuration file will be overwritten
Trust policy configuration imported successfully.
Intentando verificar un digest que nunca existio: sha256:3d13d104e47ed7ce4db079caa1f6f3a4027d25d5af34a2a480b207b3b8d55607
Error: failed to get manifest descriptor: <account_id>.dkr.ecr.us-east-1.amazonaws.com/kcl-modules/manifests@sha256:3d13d104e47ed7ce4db079caa1f6f3a4027d25d5af34a2a480b207b3b8d55607: not found
The Job ended in error (1 failed) exactly what the actual PreSync hook would do against any digest it cannot verify. According to ArgoCD's documented behavior, a failed PreSync hook aborts the entire sync: nothing is applied.
CONCLUSION:
While AWS Signer signs our package OCI in ECR, and ArgoCD in the EKS cluster only runs packages whose signature it can verify, we must remember a fundamental rule: just because a package is signed doesn't mean it's secure. A signature only guarantees origin and integrity. To genuinely secure the entire deployment lifecycle, adopting supply chain frameworks like SLSA is the necessary next step.
Top comments (0)