DEV Community: Theo Ephraim

Level up your env var tooling in Next.js with DMNO

Theo Ephraim — Mon, 22 Jul 2024 19:20:11 +0000

Type-safety, validation, leak detection, and more

Dealing with config is Next.js has usually been an afterthought - use process.env, maybe a .env file, and call it a day. But the whole experience is clunky and leaves you open to some major security risks.

Over at DMNO, we've been building a framework agnostic config & secrets management tool that provides a unified way to manage your config across your entire stack. We're super excited to finally announce that our Next.js integration is ready - and we believe it is the BEST way to manage configuration in Next.js apps.

Here's a quick overview of the main problems and how DMNO solves them:

👷 Beyond type-safety

Problem: Environment variables are always plain strings, so you must coerce and validate them yourself (if at all). You also won't get any types on process.env unless you manually add them, and you probably shouldn't put a coerced non-string values back into process.env anyway.

Solution: DMNO lets you define a simple schema for your config. Aside from being more clear, this lets us do some magical things:

Easily add coercion and validation to your config, so you are guaranteed the data is what you think it is
Validates your config during your build and before boot, so you'll know exactly what's wrong before bringing down prod
Auto-generated TypeScript types that match your schema, including detailed JSDoc comments explaining what each config item does

🔐 Effortless secure collaboration

Problem: Every time we onboard a new team member or add a new external service, we often need to share some set of secrets. You've never sent anything like that over slack... right?? Not to mention needing to keep all the various platforms we build, test, and run our applications on all in sync.

Solution: With your schema as part of your repo, you'll never pull down the latest only to be met with unexpected crashes due to missing config. Inline documentation and validations let you know exactly what each config item does, and whether it is required. Plugins let you sync sensitive config securely, either encrypted in your repo, or with secure backends like 1Password. Plus unifying how config is managed makes it much easier to reason about and debug.

🤐 Leak detection

Problem: With the edges of client and server getting blurrier each day, it has become easier to accidentally leak sensitive config. Your secrets can end up in server-rendered pages and data, built javascript code, or be sent to the wrong 3rd party - especially logging and error tracking tools. With all the magic Next.js is doing under the hood, it can be very hard to predict what will end up being sent to the client.

Solution: DMNO does everything possible to protect you from accidental leaks. These features are opt-in but we think they are invaluable.

DMNO patches global console methods to scrub sensitive data from logs
Built client JS files and outgoing server responses are scanned for leaks before they are sent over the wire
outgoing HTTP requests are scanned to make sure secrets can only be sent to whitelisted domains - e.g., only send your Stripe key to api.stripe.com

📐 Dynamic configuration control

Problem: The NEXT_PUBLIC_ prefix controls whether config is public AND whether it will be static, meaning replaced at build time. If you want build once and deploy for multiple contexts, you'll have to awkwardly wire up fetching the config on your own.

Solution: DMNO decouples the concepts of "sensitive" and "dynamic", and supports both sensitive+static and public+dynamic config, and lets you set them explicitly in your config schema. This finer control is easier to reason about and we handle the hard part for you. See our dynamic config guide for more details.

🌲 Unified config system

Problem: In a larger projects, especially monorepos, each part of your system ends up with its own hacked together config tooling, and there's no way to share config across services. Keeping things in sync is error-prone and awkward, and dealing with many different tools is a pain.

Solution: DMNO lets you easily define config once and share it across your entire monorepo. Even outside of monoepos, using the same config system across your entire project will make your life much easier.

⛓️ Flexible value dependencies

Problem: Ideally we could set config values based on other values. While Next.js does have some built-in handling of multiple environment specific .env files (e.g., .env.production), this behavior is tied to the value of NODE_ENV. Since npm module installation behavior is also affected by this, often you'll end up running your build with NODE_ENV as development even when doing a non-production build. You can also use $ expansion to do some basic referencing, but it's extremely limited.

Solution: DMNO's config resolution logic allows you to reference other values however you like. You can define switching logic based on any value (not just NODE_ENV) and reuse values within arbitrary functions. Your configuration schema all forms a reactive DAG, which can also be visualized for simple debugging.

So what's Next

While much of what was described above is not specific to Next.js, a TON of specific work went into making sure our Next.js integration just works without additional setup on your part. We want this to become the default way that everyone deals with configuration in their Next apps, and for JS/TS in general. This stuff may seem like it's not a problem, and day to day it may not be - until it bites you. Setting up your schema isn't much harder than writing a .env.example file and you get SO much more, so please give it a try, and let us know what you think!

Ready to give it a try?

Hopefully the benefits are obvious, and we know once you try out DMNO, you'll love it 🥰.

Getting started is incredibly easy and it's 100% free and open-source.

All it takes is npx dmno init 🎉

For more detailed information, visit the DMNO Next.js Integration Guide.

Using eslint to its fullest -- and introducing 🛠️lint-fix-nodemon!

Theo Ephraim — Sun, 15 Mar 2020 20:37:31 +0000

There are lots of articles about why you should be using eslint (and prettier if you're into that) to lint your JavaScript code, so I'm not going to waste your time trying to convince you of that. Instead I'm here to tell you, don't just use them...

DEEPLY INTEGRATE THESE TOOLS INTO YOUR CODEBASE
make it easier to use the linter than not

Whether you're building a single page app, an API, or a library to publish on NPM, doing this will save you and your collaborators tons of time and effort. So what do I mean specifically?

Auto "fix" every time you save - trust me, this is magic and gives you superpowers
Don't rely on IDE plugins to do your linting - add the tools into your repo and dev workflow directly
Use git hooks to enforce rules before code gets committed - don't let the infractions build up

If you're already on board and just want some tooling recommendations, just skip down to the bottom :) Otherwise, let me briefly explain why.

Run eslint "fix" every time you save

I recently started working in a codebase that doesn't use semicolons. While I prefer (and am used to) using them and we can argue about it for an hour, the truth is it doesn't really matter these days. But I'm going to keep typing them because my fingers are in charge. But while I'm coding, if I realize I typed one, then have to go back and remove it, it's just a complete waste of time if I could let a robot do it for me. With auto-fix enabled, when I save, they just disappear.

I once had a coworker who refused to use the tooling I had set up (and insisted on using vim but was bad at it). I'd be helping him debug something, and watching him code. Everything would take an extra 20% longer, because I'd watch him write some code, then spot trivial formatting errors, and go back to fix them one by one.

BUT IT'S CHANGING THINGS?! That is the point. Learn to work with it and use it to your advantage. When I want to move a block of code into an adjacent if statement, I just move one bracket, hit save, and all the indentation is now magically fixed. Multiply each of those tasks times a few seconds and I've just saved a huge amount of time every day, not to mention the mental energy of thinking about these decisions. This is especially important when working on a project with many people who may have different habits.

Occasionally you may run into problems with one auto-fixing rule conflicting with another, like preferring short arrow functions vs keeping lines shorter than 100 characters. I'm not going to lie, it can be annoying, and sometimes I've had to add an eslint ignore, or write a statement a different way. But you fine-tune your rules, and you get used to it. The balance of time/effort saved is well worth the occasional hazard.

IDE plugins vs integrated tools in your repo

An IDE plugin can be great if you want to highlight linting errors in your code directly. But relying on the plugin to fix on save or keep the code clean for your team is a recipe for not everyone playing by the rules. Not everyone uses the same editor. Not everyone has the same plugins or plugin settings enabled. Build it into the repo and make it easier to use the linter than not.

Use git hooks to enforce your rules

I've seen countless codebases that have an eslintrc file, but the rules are not actually followed. If you're going to have linting rules, you should enforce them. The easiest way to make sure that is the case is to not let people commit/push code until they follow the rules. Yes it can be annoying, but by always following the rules, you avoid letting the linting errors/warnings build up to the point that they are overwhelming, which makes them less useful. You can always skip the checks during a crisis or set up your rules to be more relaxed for certain scenarios.

On a related note - also add a linting check in your CI tools!

Great - just show me how already

What tools to use really depend on what kind of project you are working on, but here are a few quick suggestions. I'll add more resources here as I find new great tools.

Git hooks

Husky is an awesome tool that adds configurable git hooks to your repo. It has a ton of options, and is super easy to use. After installing as a dev dependency, you define hooks in your package.json file.




  "scripts": {
    "lint": "eslint ./",
    ...
  },
  "husky": {
    "hooks": {
      "pre-commit": "NODE_ENV=production npm run lint",
      "pre-push": "npm run test"
    }
  },
 ...

Enabling lint+fix in a webpack project

Use eslint-loader to add eslint into your (dev) webpack build process. After installing as a devDependency, you need to add the following to the module.rules section your development config:



{ // enable eslint + fix
  enforce: 'pre', // runs on the original file
  test: /\.(js|vue)$/, // adjust for your project
  loader: 'eslint-loader',
  exclude: /node_modules/,
  options: {
    emitWarning: true, // false to show errors in browser overlay
    fix: true, // enable fixing!
  },
}

Enabling lint+fix in a vue-cli project

Vue cli just uses webpack under the hood, but abstracts away the webpack config. Use @vue/cli-plugin-eslint to add eslint-loader into your build process. Install using the vue-cli with vue add eslint. Lint on save is enabled by default.

Enabling lint+fix in a backend / api project

This is why I wrote this article - to share a new tool I've released :)
Say hello to lint-fix-nodemon!

Confused? Already using eslint with nodemon? It's not too hard, but here are the problems I encountered with all the different setups I tried:

Double restarts - nodemon detects a first change when you save the file, and then again after eslint fixes it. Restarting twice may not be a huge deal, but anything that slows you down that happens 100 times a day adds up quickly.
ESLint failures can get stuck - in certain circumstances (like starting up your npm run dev script at the beginning of a coding session) if eslint fails, it can fail the process so it doesn't start watching your files. Why should things work differently the first time I run it? If you have a lot of errors to fix, this can be a big pain...

This tool (which I've been using as an unpublished script for a long time) fixes both of these issues. It watches your files, runs eslint on change, and only restarts nodemon AFTER fixes have been applied. Also if anything fails on the first run, no problem - it will keep watching and try again when you save.

So after installing (npm i lint-fix-nodemon --save-dev), configure things in your package.json file. I usually add script called "dev" to start the tool. By default nodemon will run the "main" script (or you can pass one to the script), and it will respect the "nodemonConfig" settings it finds. By default it will ignore "node_modules" and any files/folders that start with ".". The same files that are "watched" will be linted.

Here is an example:



{
  "scripts": {
    "dev": "lint-fix-nodemon api/start.js",
    ...
  },
  "nodemonConfig": {
    watch: ["api/"],
    ignore: ["api/scripts"]
  },
  ...
}

So what's next?

I could probably add a lot more options to this library. Please open issues if you run into problems!
I use a similar script to run the linter (with fixes) and then run my tests (using jest) on each save. Hello TDD! I'll probably release something more generalized soon, maybe a tool called lint-fix-and :)
I still have been wanting to spend some time to set up automated linting to auto fix the css/less/sass and pug in my vue single file components.

I hope this article and lint-fix-nodemon are helpful for a few folks. Please let me know if you have any other suggestions to add, or are stuck on something.

Cheers!