just saying. .env can be transformed into .env.local.php with composer dump-env prod and used with preload (or just with opcache) to eliminate the need of re-reading and re-evaluating env variables on every request
Good tip for performance, but this transformation needs to be done each time vault-agent updates the credentials. If the PHP file is loaded with preloading, php-fpm have to be reloaded to get the updated values.
The .env file reader is quite fast, but it could be optimised by using a native format like .ini
Yeah. Sorry, I lost my touch with forums in slack/discord/etc era. I mostly was referring to the possibility that folks use not only .env
ini is not native format in any way. Parsing is quite simple and fast, but it still needs to be parsed. The only native format for PHP is array 😅
Let's say this .env file updating via vault-agent is atomic. How you handle change in credentials "mid-flight"? Just ignoring errors? Re-reading configuration mid-request?
Good question. I run the described solution on production for months, with high HTTP request throughout, without issue. Nothing in logs sound like the app read a partial file. I'll make some deep tests to understand how that works.
PHP brings the function parse_ini_file that uses the C implementation. That's what I was meaning with "native". .env and YAML parsers are written in PHP and uses regex (inherently slower).
First of all speed of regex (pcre2 library) may surprise your greatly) Second - to parse env on the most basic level you just need to split the line by = and process quotes on the right... but I digress
Nothing in logs sound like the app read a partial file
That's why I mentioned atomicity - this will guarantee that file always updates as a whole and instantaneously. Hypothetical scenario I'm talking about looks like this
credentials rotation is triggered by timer
credentials are changed in vault
now there is some time between actual update in central vault and vault-agent reaction
after that there is some time between vault-agent updating .env file and app reading it
even after that there is a possibility when app read old .env file (in the beginning of the request) and 1ms later .env was updated
In all last three scenarios credentials that app has in memory are already obsolete. That will not matter for already established connections but should matter greatly for new ones.
The only way I see to mitigate it is to rotate "whole credentials" as user/password pair altogether and keep old ones active for a little while after update. In which case I am quite curious about how it's handled say with database of any kind where user/password is not enough, you also need provide the same level of permissions (say to database / tables / operations)
After your remark, I found that vault-agent have an option to run a command to run composer dump-env prod when .env is actually modified.
command(string: "") - This is the optional command to run when the template is rendered. The command will only run if the resulting template changes. The command must return within 30s (configurable), and it must have a successful exit code. Vault Agent is not a replacement for a process monitor or init system.
just saying.
.env
can be transformed into.env.local.php
withcomposer dump-env prod
and used with preload (or just with opcache) to eliminate the need of re-reading and re-evaluating env variables on every requestGood tip for performance, but this transformation needs to be done each time
vault-agent
updates the credentials. If the PHP file is loaded with preloading, php-fpm have to be reloaded to get the updated values.The .env file reader is quite fast, but it could be optimised by using a native format like .ini
Yeah. Sorry, I lost my touch with forums in slack/discord/etc era. I mostly was referring to the possibility that folks use not only
.env
ini
is not native format in any way. Parsing is quite simple and fast, but it still needs to be parsed. The only native format for PHP is array 😅Let's say this
.env
file updating via vault-agent is atomic. How you handle change in credentials "mid-flight"? Just ignoring errors? Re-reading configuration mid-request?Good question. I run the described solution on production for months, with high HTTP request throughout, without issue. Nothing in logs sound like the app read a partial file. I'll make some deep tests to understand how that works.
PHP brings the function
parse_ini_file
that uses the C implementation. That's what I was meaning with "native"..env
and YAML parsers are written in PHP and uses regex (inherently slower).First of all speed of regex (pcre2 library) may surprise your greatly) Second - to parse env on the most basic level you just need to split the line by
=
and process quotes on the right... but I digressThat's why I mentioned atomicity - this will guarantee that file always updates as a whole and instantaneously. Hypothetical scenario I'm talking about looks like this
In all last three scenarios credentials that app has in memory are already obsolete. That will not matter for already established connections but should matter greatly for new ones.
The only way I see to mitigate it is to rotate "whole credentials" as user/password pair altogether and keep old ones active for a little while after update. In which case I am quite curious about how it's handled say with database of any kind where user/password is not enough, you also need provide the same level of permissions (say to database / tables / operations)
Ok. Nevermind, found my answer here learn.hashicorp.com/tutorials/vaul...
Sorry to bother😅
After your remark, I found that vault-agent have an option to run a command to run
composer dump-env prod
when.env
is actually modified.