DEV Community

Philip Meholm
Philip Meholm

Posted on

Dotenv files annoyed me into writing Polyenv

#programming #security #go #webdev

Tired of leaking secrets into .env files, and frustrated by the friction of existing tools, I built a simple CLI to separate config from secrets without the overhead of vaults.

If you don't live under a rock, you have at one time used dotenv files in some project to create/edit your configuration, or used it as a config file for something you made or cloned for self hosting.

Most frameworks and languages support them, and if not, its pretty easy to create something to import and use them (at least at a basic level). This is notable because it provides simplicity in a ocean of configuration frameworks while serving A LOT of the need for the developer and users.

But, trouble starts when you decide you want to store secrets inside dotenv files. Don't get me wrong, dotenv are great for simplicity. But when you start slipping secrets in there, it’s a bit like playing with matches in a fireworks factory. Commit + push the wrong .env file to git, and you’re basically performing version-control seppuku.

This isn't just paranoia, its happening a lot:

So what can you do about it?

Current Alternatives

  • Mozilla SOPS - Encrypts values while leaving keys readable. really flexible (multiple formats, key stores, rotation etc), Works well in teams comfortable with encrypt/decrypt as part of their pipeline.

  • Dotenvx - Encrypts the entire file with a private key stored alongside. Straight forward setup and tooling, though sharing keys with teammates/CI requires extra coordination (or a ~$19/yr offering from Radar).

  • Platform specific Secret managers - Hashicorp Vault, AWS/GCP Secret Manager, Azure Key Vault, CyberArk, etc. Centralized, battle tested and secure, Typically works best if your stack is tied to a given platform.

  • .env.local + .gitignore + ci secrets - Keeps sensitive values of version control while remaining simple and lightweight. Works fine for small projects, but contributors need some extra documentation or shared reference to know what variables exists.

  • Pulling secrets at runtime - Fetch secrets directly from your enterprise vault during CI or app startup. Secure, 'simple' and increasingly common in cloud native setups. may add complexity to local development workflows, especially where hot-reload or offline work may be important.

All of these approaches solve a real problem and each has its ideal context. For my everyday workflow though these solution either partially hit my use-case or i felt something was missing.

Problem statement: treating all envs as secrets is overkill

Many tools treat all values in dotenv files as if they are secrets by default, or have some platform specific approaches.

This often adds setup complexity and friction that may make developers or maintainers to take shortcuts "just for now" that end up being the default. This is really dangerous.

Not every env value is a secret.

Sometimes its just boring config like PORT=3000. why treat them all like secrets if only a handful need the proper protection?

That's the thinking behind polyenv: a tool I built to make handling secrets second nature and a natural fit for any dev workflow

Polyenv

The core idea: keep secret references in a dedicated config instead of the values. these references are safe to commit since the actual access is controlled by your identity or CI. any secrets then would get downloaded to a git-ignored file, keeping them safe from git's purview

  • For anyone dealing with .env and possibly secrets
    • if you have one or many .env files or deal with secrets in .env files this can help you
  • Not a replacement
    • this is meant to be used with your dotenv config, not instead of.
  • "Any" enterprise vault
    • its a thin cli wrapper for many different enterprise vault readers, so you could replace the different "vault readers" with this one tool
  • Separate secrets from config
    • Secrets are saved in a separate .env.secret file that can be git-ignored
  • References, not values
    • Instead of leaking values, a separate polyenv config file is created to host metadata about the secrets (where to pull from)
  • Pull secrets on demand
    • running pull will pull down secrets using your identity (eg. Azure key vault via your logged in az cli) or secrets stored on your local credential manager
  • Workflow-friendly and local-first
    • secret rollover? pull to get the latest version
    • need to add secret from defined vault? add secret and you can select what secret you want to add
    • Supports multiple vaults in same environment
    • (planned) CI support so you don't have to change tooling in a different environment

Config and references are stored in a .polyenv.toml file. I considered dotenv, but its not the right fit for structured metadata. TOML offers a nice balance: simpler than JSON, less fragile than YAML. While it lacks native schema support, the tradeoff feels worth it for readability and ease of use.

QuickStart

You have to download polyenv cli from github releases. Package manager and native code packages coming soon.

Every command is backed up with a interactive wizard in case you miss any arguments. I don't like argument-heavy cli's that just errors out if not specified correctly, so I'm trying a "yes, both" approach for this cli.

for now i just have to assume you know how to download a cli and set it as part of your session.

Init

When you have downloaded the cli and its part of your session, you can go to your git repo and type in polyenv init.

Init

it will ask you for environment name and give you some options for setting up and lead you through a wizard where you can add new vaults and secrets

Once you are done, you should have a config named {yourenv}.polyenv.toml in git root. If you want, you can move this file pretty much anywhere in your repo, except under .git,vendor or node_modules. the cli will search through your repo from git root to find it each time it spins up.
if you defined "no env".. ie .env the name will be .polyenv.toml

[options]
  hyphens_to_underscores = true
  uppercase_locally = true
  use_dot_secret_file_for_secrets = true

[vault]
  [vault."Manual Input"]
    service = "myservice"
    type = "local"
  [vault.mydev]
    store = "mystore"
    type = "devvault"

[secret]
  [secret.GITHUB_TOKEN]
    vault = "Manual Input"
    content_type = "text/plain"
    remote_key = "github-token"
Enter fullscreen mode Exit fullscreen mode

Add vault

adding a vault

to add a vault you can either use polyenv !{yourenv} add and select vault or polyenv !{yourenv} add vault to get to the add vault wizard
you will then be given a choice between several vault providers and then lead through a wizard for the vault of your choice.

Add secret

adding a secret

polyenv !{yourenv} add secret will add a new secret. you get to select from your defined vaults, when within that vault select your secret. the actual "wizard" for this is defined per vault for best use

Emitting data

polyenv !{yourenv} export

by default export will show the stats of all values so you don't accidentally show data in your current session.

in order to grant the best user-experience this single command can both format the data in many different ways and write it to many different "outs".

formatting

to specify a formatter, you can append --as with your given format:

  • json
    • {"MY_KEY":"MYVAL"}
  • dotenv
    • MYKEY="MYVAL"
  • azdevops
    • polyenv !{yourenv} export --as azdevops emits:
    • ##vso[task.setvariable..isSecret=true;MY_KEY]MYVAL
    • Loads all env vars into your current Azure Devops pipeline variable
  • pwsh
    • polyenv !{yourenv} export --as pwsh|Iex
    • Loads all env vars into your current PowerShell session.
  • bash
    • eval "$(polyenv !{yourenv} export --as bash)
    • Exports all vars into your current Bash session
  • stats (default)
    • shows where all your env variables comes from (what files or vaults) and if they should be deemed a secret.

all --as formatters have the ability of checking if each env value is a secret via regex rules and act accordingly to help you not omit any values polyenv deemes a secret. both stats and azdevops uses this feature.

writing

to specify a writer you can append --to with the set writer:

  • stdout
    • standard output
  • github-env
    • standard github env
  • github-out
    • standard github output

each writer have a set of preferred and allowed formatters to give you the best possible chances of success, so you don't need to give for example --as github-env a --as dotenv, as the only possible option for exporting to gh is dotenv, so this is implied when using the writer: polyenv !yourenv export --to github-out

Future work

the export topic is one of the core elements of this as it allows you to be really dynamic when setting up your pipeline or automation. I really want this to be like "rclone" of env management, and hopefully by focusing on writers and formatters and distribution, this will be easier to achieve than other methods

  • i want to create better and more formatters and writers so this could be used more places
    • export to onetimesecret: not affiliated, i just really like their service. planning a wizard where you can select keys and then format and get a link back you can share with collaborators
  • Addition of imports. How its going to work i dont know yet..
  • more implementations of vaults
    • currently only keyvault and local keyring/cred store
    • issue tracker
  • better control over secrets
    • add env to a vault
    • change vaults for a secret
    • update secret value
  • better documentation
  • better TUI elements for managing the environment
  • packages so it can be used
  • better handling of arguments->wizard actions

If you have any other suggestion, are interested to help out, no need to deliver actual code, please feel free to join the project.

Top comments (1)

Collapse
 
thekarel profile image
Charles Szilagyi •

Interesting project ⭐