Signing software artifacts has many obvious benefits such as code integrity or developer (author) authentication. Yet it's oftentimes neglected, creating a software ripe for supply chain attacks. One of the reasons why people can't be bothered to sign their code is that existing tools - such as PGP - aren't very user friendly and require extensive security and/or cryptography knowledge.
Signing software can be easy though thanks to sigstore and its
cosign CLI! In this article we will learn how
cosign works and integrates with other sigstore components (
rekor). More importantly, we will learn how to use it to sign container image the easy way, both with and without keys, as well how we can use it to verify produced signatures and integrity of the signed software.
Note: This is a "hands-on" followup to my previous article Sigstore: A Solution to Software Supply Chain Security, which explains what's sigstore and how its components work.
Before we sign anything, we first need all the CLI tools for each of sigstore's components - that is
rekor. The first of them -
cosign - which we need to actually sign anything, can be installed as binary or as Docker image. For the for first option, download the appropriate binary from release page and put it somewhere in your
$PATH. Additionally, considering that we're dealing with security tooling, it's recommended to verify authenticity and integrity of the binary. You can do that using the commands shown on release page.
If you prefer to use Docker image, then you can use the following:
skopeo inspect docker://gcr.io/projectsigstore/cosign:v1.0.0
docker pull gcr.io/projectsigstore/cosign:v1.0.0
docker run --rm gcr.io/projectsigstore/cosign:v1.0.0
cosign [flags] <subcommand>
For the second component -
fulcio - we won't need to install anything because we will be using the public instance of
fulcio. The public-good service is available at https://fulcio.sigstore.dev/api/v1 and API documentation can be found here.
We will however want to install the CLI so that we interact with the
rekor server. The binaries are available in GitHub release page. If you're on linux you can use the following:
wget -O rekor-cli https://github.com/sigstore/rekor/releases/download/v0.3.0/rekor-cli-linux-amd64
chmod +x rekor-cli
# Move it into $PATH directory...
Rekor command line interface tool
And again, as mentioned with
cosign, you should be careful with what binaries you're using. Therefore you might want to verify
rekor-cli binary using the process outlined here.
With all the tools ready, we can start signing artifacts. To get better understanding about what goes on under covers, we will first try doing it the "hard way", that is - without all the fancy tools.
First we will need an artifact. For this demo we will use "hello world" Docker image created using following
ENTRYPOINT ["echo", "Hello sigstore"]
We however cannot sign the image itself, instead we will sign its digest:
# Generate artifact
docker build -t dockerhub-username/sigstore-hello .
# Generate artifact digest for signing
cosign generate martinheinz/sigstore-hello > artifact
Next we need an ephemeral keypair to sign the digest with. We can use
cosign commands for this, but considering that this is the "hard way", let's use
openssl ecparam -genkey -name prime256v1 > ec_private.pem # Create keypair, same as `cosign generate-key-pair`
openssl ec -in ec_private.pem -pubout > ec_public.pem # Extract public key, same as `cosign public-key`
Now we're ready to sign it and while we're at it we can also verify the signature:
# Sign artifact digest, same as `cosign sign`
openssl dgst -sha256 -sign ec_private.pem artifact > artifact.sig
# Verify using public key
openssl dgst -sha256 -verify ec_public.pem -signature artifact.sig artifact
Now that we signed the artifact with our private key, we want to have a proof that we were the ones who really did it. For this we need code signing certificate from
fulcio. To get it, we have to authenticate with OIDC provider to get an ID token, which serves as proof of our identity for
After that, we sign our email address which we used to authenticate using the previously used private key. We do this to prove that we have possession of the private key at the time of signing.
Finally, we ask
fulcio for code signing certificate, by giving it ID token as form of authorization, the signed email address and our public key:
# ... Get token from OIDC provider
# ... Store ID token in `id_token` file
# Sign email address (to prove possession of private key)
echo "firstname.lastname@example.org" > email
openssl dgst -sha256 -sign ec_private.pem email > email.sig
# Submit token, public key and signed email to fulcio
curl -X POST "https://fulcio.sigstore.dev/api/v1/signingCert" \
-H "Authorization: Bearer $(cat id_token)" \
-H "accept: application/pem-certificate-chain" \
-H "Content-Type: application/json" -d \
"content": "$(base64 ec_public.pem)",
"signedEmailAddress": "$(base64 email.sig)"
One problem with this "hard way" approach is that it's not really feasible to simulate the authentication and retrieval of ID token. Therefore, in the above snippet this step is omitted and we skip directly to submitting everything to
Alternatively, you could also skip the interaction with
fulcio entirely and use your public key instead. This approach is shown in https://github.com/sigstore/rekor/blob/main/types.md#pkixx509.
Next we can proceed with uploading the record to the transparency log (
rekor). Here we show both the option with our public key and signing certificate from
fulcio. When using the certificate from
fulcio, we can also delete the keypair as we no longer need it:
# Delete keypair (if using signing certificate from fulcio)
rm -rf ec_private.pem ec_public.pem
rekor-cli upload --artifact artifact --signature artifact.sig --public-key=ec_public.pem --pki-format=x509 # With our public key
rekor-cli upload --artifact artifact --signatire artifact.sig --public-key sigingCertChain.pem --pki-format x509 # With cert from fulcio
Created entry at index 33612, available at: https://rekor.sigstore.dev/api/v1/log/entries/2f77b399fd8a162f44c75c596fb0e5917ed2f314348e135874fae9e14eff69e3
# Inspect entry
curl https://rekor.sigstore.dev/api/v1/log/entries/2f77b399fd8a162f44c75c596fb0e5917ed2f314348e135874fae9e14eff69e3 | jq .
rekor-cli get --uuid=2f77b399fd8a162f44c75c596fb0e5917ed2f314348e135874fae9e14eff69e3
In addition to the upload we can also check presence of the record in transparency log. Above snippet uses both
curl to directly access the public API.
All that's left to do is upload the signature to the registry to be stored alongside container image:
# Upload to Docker Hub
cosign upload blob -f artifact.sig index.docker.io/martinheinz/sigstore-hello:new-signature.sig
Uploading file from [artifact.sig] to [index.docker.io/martinheinz/sigstore-hello:new-signature.sig] with media type [application/octet-stream]
File [artifact.sig] is available directly at [index.docker.io/v2/martinheinz/sigstore-hello/blobs/sha256:5491a7ff9960236b4f0bc7311fc8dba8e1b9fadfef7f704ec54eddaac1977ecb]
Uploaded image to:
That's it. We have signed our image and added record of it to transparency log. This approach would work, but no one probably wants to do this on a daily basis, so let's see how the proper tools can make this easy.
The "hard way" wasn't really hard, but it gets much easier if we use the tools provided:
Enter password for private key:
Private key written to cosign.key
Public key written to cosign.pub
# We already uploaded signature in previous step so upload is set to false here
cosign sign -key cosign.key --upload=false martinheinz/sigstore-hello > file.sig
Enter password for private key:
# You can later upload the signature
cosign attach signature -signature file.sig martinheinz/sigstore-hello
All we need to do is generate a keypair and then sign the artifact. Upon signing,
cosign automatically uploads the signature to the registry where the image is located. In the above example we chose not to upload the signature and just save it to a file, because we did sign it in the previous section already. If we later decided to upload it anyway, then we can do it with
cosign attach as shown above.
It's also worth pointing out, that as of right now (
1.0), the above snippet will not upload the data to
rekor transparency log, for that to work, we would need to set
COSIGN_EXPERIMENTAL=1, so for example
COSIGN_EXPERIMENTAL=1 cosign sign -key cosign.key ....
There are also other ways to use
cosign to sign artifacts depending on your use case and workflow. These are described in detail in usage page in GitHub.
Even easier than the easy way is using the "keyless" method where only ephemeral keys are used, meaning you don't need to generate and maintain your own keys:
COSIGN_EXPERIMENTAL=1 cosign sign \
-oidc-issuer "https://oauth2.sigstore.dev/auth" \
-fulcio-url "https://fulcio.sigstore.dev" \
-rekor-url "https://rekor.sigstore.dev" \
Generating ephemeral keys...
Retrieving signed certificate...
Your browser will now be opened to:
tlog entry created with index: 33692
Pushing signature to: index.docker.io/martinheinz/sigstore-hello:sha256-af5909c54fe66d03dda41e93ca5db3f277bfc827b9758d8cfa9a5d8d60d85491.sig
All we need to do is run
cosign sign with
COSIGN_EXPERIMENTAL set to
1 while at the same time omitting the
-key argument. In the above example we also specified endpoints of OIDC provider,
fulcio server and
rekor server - these are the default values of the public-good services provided by sigstore, so they can be omitted, but are shown here for clarity and to highlight which services are being accessed/used. You could also replace those with your own instances - that would make sense if you wanted to run everything behind a firewall.
Now that we signed the artifact in all the ways possible we should also try verifying it, otherwise what would be the point of signing it in the first place, right?
First let's take the outputs of signing the image digest the "hard way". For that we can use
rekor-cli verify --artifact artifact --signature artifact.sig --public-key ec_public.pem --pki-format x509
rekor-cli verify --artifact artifact --signature artifact.sig --public-key sigingCertChain.pem --pki-format x509
Here we have 2 cases - if we signed the artifact with our public key, then we use that when verifying. On the other hand if we used the signing cert provided by
fulcio we would use that in place of the public key.
Next up is the verification using
cosign which is suitable for the basic signing with keys. All we need to do is run
cosign verify providing the key and image URL:
cosign verify -key cosign.pub docker.io/martinheinz/sigstore-hello:latest | jq .
Finally, for the keyless method - we can do essentially the same as above, but we need to add the experimental flag and we can skip the key argument:
COSIGN_EXPERIMENTAL=1 cosign verify docker.io/martinheinz/sigstore-hello:latest
Verification for docker.io/martinheinz/sigstore-hello:latest --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- Existence of the claims in the transparency log was verified offline
- Any certificates were verified against the Fulcio roots.
In this article I tried to outline and explain the basic use cases and approaches for signing container images using sigstore and more specifically
cosign. There are however, many more options and features of
cosign which might be useful to you, such as working with other types of artifacts, using hardware tokens or signing
git commits, so I encourage you to mess with the tool and see what else you can use it for. A lot of these options are described in very well written usage documentation here, so make sure to check that out too.
Also, if you want to dig even deeper, you can checkout "sigstore the hard way", which is a guide to setting everything up, for scratch - including
fulcio CA and
rekor transparency log server.