Ephemeral Jenkins Users + API Tokens using Hashicorp Vault

New year, new credentials! (well almost)

The Jenkins rabbit hole

Not long ago, I (like most people), went down the rabbit hole of working around some Jenkins limitations. I was experiencing some issues of hitting identity provider API rate limits due to heavy API usage of Jenkins. After some experimenting, I found that using Jenkins API tokens for machine to machine communication instead of user credentials doesn't require identity provider authentication which significantly cuts down on hitting said rate limits. "This is great!", I thought.

SDK's... those darn SDK's...

Next up, I was looking into the the gojenkins client library to see if there were some built-in methods to allow me to do my own API token management and rotation to solve the issue above. And of course, it didn't exist. It was at this point I sat back and thought, "Really? No one has yet to implement a decent solution for managing Jenkins tokens/users yet? This is the most widely used CI server out there. Surely it has to exist..."

Sure, I was already aware of the vault-jenkins-plugin which allowed Jenkins to fetch secrets from Vault, but what about accessing Jenkins itself?! Nope, it didn't exist. I then began to think about all the work needed to solve this problem in a non-crufty way. First of which included some PR's to the gojenkins client library to add some API/User management methods to make the overall solution cleaner, as well as give back to the community so that they can automate all of their problems away as well.

Time to do the needful

I've built other plugins for products within the Hashicorp ecosystem before, such as Terraform, but never a Vault plugin. I thought this would be a good opportunity to learn more about the inner workings of Vault and fill this gap in the community.

Low and behold, I created the vault-plugin-secrets-jenkins project!

Yes. That is a glorious chicken sandwich and I'm leaving it.

So what does this thingy do?

I'm glad you asked!

Once installed, the vault plugin takes in a configuration of a "root" Jenkins user. Then on behalf of this user, Vault can create short lived users as well as short lived API tokens for the configured user!

But why?

Dude, it's almost 2022. Hardcoding credentials as environment variables is not the best way to handle secrets these days. Now you can just request/rotate your own dynamic credentials by integrating with Vault.

So what does it look like to use this thing?

First detailed guidance/examples, you'll want to checkout the project README here, but while I have you, I'll show you some examples.

Enable the plugin

vault secrets enable -path=jenkins vault-plugin-secrets-jenkins
Success! Enabled the vault-plugin-secrets-jenkins secrets engine at: jenkins/

Configure the plugin

vault write jenkins/config url=http://localhost:8080 username=admin password=admin
Success! Data written to: jenkins/config

Bonus: The plugin validates your config before moving on 😁

Create a short lived API token

vault read jenkins/tokens/mytoken
Key                Value
---                -----
lease_id           jenkins/tokens/mytoken/fJ57afQZMyXDcJnm74BgLLt8
lease_duration     5m
lease_renewable    true
token              1184cb7b22c404efa1c293e9841b66f345
token_id           1c2864f3-4108-4417-807a-358357bc8432
token_name         mytoken

Create a short lived user

vault write jenkins/users/myuser password=password fullname="Jenkins the Butler" email=[e-mail]
Key                Value
---                -----
lease_id           jenkins/users/myuser/hTGbJhDFbAQpALv1FjJyJ4vz
lease_duration     5m
lease_renewable    true
email              [e-mail]
fullname           Jenkins the Butler
username           myuser

So what now?

Woohoo! We now have automatically expiring users and tokens we can manage from our app and can give our pals over in information security some piece of mind that even if our credentials do leak, they'll be useless within just a couple of minutes. Pretty slick right?!

Fin

Thanks for taking the time to checkout my article!

(gifs of more chicken sandwiches are encouraged in the comments section)

Comments

[Alex Pho]

Hey Caleb this was a great read for me; found your post from the Hashicorp Newsletter actually! I also took a quick peek at the repo too and the diagrams are helpful.

I don't use Jenkins, but use GitLab CI/CD instead for my current role and I also am familiar with managing Vault—I was hoping if you could confirm my understanding.

Would the flow look something like:

1. Jenkins pipeline job is executed and makes a request to Vault for ephemeral creation of either a user and/or token as defined by the particular pipeline.
2. Vault does the thing per the CLI and plugin integration.
3. Returns ephemeral token back and allows Jenkins to auth machine to machine to the IdaP avoiding rate limiting.

[Caleb Lemoine]

Hey @alexcfpho !

My coworkers actually created a vault plugin for GitLab CI: github.com/splunk/vault-plugin-sec...

The flow for usage with this plugin is actually outside of Jenkins pipelines. These are the 2 flows/use cases I built this for:

Jenkins Users:

1. If no external auth system is being used such as SAML/LDAP/Active Directory etc, Jenkins admins/platform team create vault policies to allow consuming teams to create Jenkins users to run Jenkins jobs/communicate with the API.
2. Dev team has automation to renew/create short lived users for their consumption of Jenkins.

Jenkins API Tokens:

1. External provider such as SAML/LDAP/Active Directory is used frequently and rate limits occur due to high Jenkins usage.
2. Jenkins consumers with their service account can then use vault to create short lived API tokens for said service account user which will not call external provider frequently and avoid rate limits.

[Caleb Lemoine]

@alexcfp would you mind linking me the newsletter you saw? I wasn't aware of it