Nowadays, it's hard to profoundly talk about security in the IT industry, since it has to be considered on so many different levels: from securing code chunks, securing containers, up to securing complex infrastructures and defining strong authorization and authentication policies across the enterprise.
Vault is a tool that let you store secrets, any kind of secret that your company requires to be secured, and also lets you generate them dynamically.
General Context
I will explain this with one simple example: let’s suppose that you have to secure a data mart, which contains some databases that are accessed by different microservices, each microservice has access to its own database and also could be scaled up and down dynamically according to the demand. New microservices are deployed every time the number of instances are increased and each instance needs to be provided with the proper configuration.
One important part of the configuration is the database credentials that usually are static values for all instances of the microservice. So there’s not much science in the typical ways that are commonly used to provide the credentials and secrets to each microservice at deployment time:
- Could be provided as environment variables either plain text or encrypted.
- Could be provided on a config file on each instance
- Could be provided as configmaps or a secret assuming you handle your orchestration over a Kubernetes environment.
I could mention some other examples, but handling your database secrets are always kind of a manual process; Think for a while what if for some reason you have to change the user and password of your databases because they were compromised? How about meeting compliance requirements for rotating passwords every 90 days for example? What are the implications of changing the credentials?
Just to mention a scenario, it may be the case that all instances of the microservice have to be re-deployed with the new credentials, or at least restarted in order to take the new values. Or you may have to change the configmaps or secret on your k8s cluster, etc. I don’t want to talk in deep about it but think over a wider scenario where not just one database but all your databases were compromised; the effort of fixing the issue could be immense.
This simple scenario of securing databases could be extended to some other areas like ssh keys, certificates, tokens, identities, credentials, encryption as service, etc.
If you manage all those in a centralized vault ecosystem, you can take prompt actions without disrupting your operation. You can have all your systems authenticating by themselves without human interaction, that’s the goal of a good implementation of DevSecOps.
So, going back to our database example, you can have all your microservices requesting credentials to vault who dynamically create them for each instance of microservice every time it is deployed, and rotate/revoke them over a well-defined security policy. The following diagram shows how vault implements this:
The authentication/authorization of the microservice
Before they can start requesting dynamic credentials to the database, each microservice should be authenticated against an internal or external system. Authentication is the process by which each microservice is verified and an identity is assigned to it. Multiple authentication methods are supported.
Once the microservice has been verified, it has to account for access control and permissions associated with such an identity, that’s called authorization. Vault handles authorization by the definition of policies in Vault which control what a user or microservice, in this case, can access. For example:
path "database/*" {
capabilities = ["create"]
}
path "database/creds/my-role" {
capabilities = ["read"]
}
Setup process
In order to set this up in a running environment, you have to follow the next simple steps:
- From the vault side: The admin user has to get authenticated against Vault, then he should enable the database secret and configure the postgres plugin, that requires to create a Role that creates dynamic credentials.
- From the application side: The application should be provided with a valid Token in order to get authenticated, then, through the API it can dynamically request postgres credentials as shown in the following diagram:
For enabling database secrets run the following command:
$ vault secrets enable database
Success! Enabled the database secrets engine at: database/
For configuring postgres plugin
$ vault write database/config/myapp plugin_name="postgresql-database-plugin" connection_url="postgresql://postgres:postgres@postgres:5432/myapp?sslmode=disable" allowed_roles="my-role, readonly"
To create a role that maps a name in vault to an SQL statement to execute to create the DB credential
$ vault write database/roles/my-role \
db_name=myapp \
creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' \
VALID UNTIL '{{expiration}}';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
default_ttl="5m" \
max_ttl="1h"
Success! Data written to: database/roles/my-role
To generate postgres secret
$ vault read /database/creds/my-role
Key Value
-------- -----
lease_id database/creds/my-role/6E252RJKZOid7rpUxHEeaabT
lease_duration 1m
lease_renewable true
password A1a-2mfSG9ClFPjt42Hi
username v-root-my-role-2PvoHLCRL3mK1kmE3w1J-1544215033
You also can generate the secret via API
$ curl --header "X-Vault-Token: ${ROOT_TOKEN}" http://localhost:8200/v1/database/creds/my-role | jq -r
{
"request_id": "cc08db9d-a665-0cb6-188d-20cf570d78b1",
"lease_id": "database/creds/my-role/5OidV08wNZLXkWgYTxR84n3u",
"renewable": true,
"lease_duration": 300,
"data": {
"password": "A1a-5xXk6hSwUfK3fzYW",
"username": "v-root-my-role-5njEgQlMloc4vqOZtUA6-1544217564"
},
"wrap_info": null,
"warnings": null,
"auth": null
}
This is a basic usecase for how Vault can be used for dynamically securing databases, among other much things, in order to provide a more secure deployment of your microservices and databases.
Top comments (6)
Great writeup, although I'm confused how this fixes the problem of having a hard-coded string somewhere (file, ENV var, whatever) that contains the database credentials. Unless I'm mistaken, doesn't this solution just "move" the problem? It seems like you still need to keep a VAULT_TOKEN somewhere, just as you had the database credentials without vault?
You would't have to handle DB credentials, instead the application will be requesting them via API. The trend is that the systems can be dynamically authenticated without human intervention.
In regards to the VAULT_TOKEN, Vault support some some other authentication backends, but having the VAULT_TOKEN doesn't give you access to the DB.
// , Other advantages which Francisco did not mention are that "moving" the problem in this way
Tokens are only obtained by an application which has non-token credentials. These credentials can be configured to your taste, and have varying degrees of non-repudiation protection, e.g. from IP address locking up to a combination of signed instance metadata and client certificates.
Because the tokens are not present in the source code, and only obtained when the workload starts, they never have to be stored on disk or in source code, removing the "hard-coded" problem.
Tokens give a massive increase in auditing, since they are unique to each authorized workload.
The tokens can be locked to a specific IP address.
Let me know if you'd like an example of this, Mike, since I can probably write a network state diagram from memory at this point!
Regardless, I'll have a separate post on this, later, since I've been asked this question by almost every smart security guy to whom I've presented Vault.
Update: Later, the separate post is here dev.to/v6/the-first-smart-question...
// , The metaphor I like to use for Vault Tokens is a session token for web applications.
While the token itself may not be all that special, security-wise, a user can only get the Vault Token by doing whichever song and dance you've set up for login.
And that token is time-limited, auditable, tied to a specific range of IP addresses, and specific to each client.
// , In light of @mdesjardin's question, Francisco, would you be willing to add a sentence or two describing an example of an initial authentication method by which one of the new micro-service instances would obtain its "secret 0"?
"Authentication is the process by which each micro-service is verified and an identity is assigned to it. Multiple authentication methods are supported."
Greate article. Which tool do you use to create that images (Diagrams, flow, etc) ?