How Should You Use an Auto-Formatter?

Auto-Formatter Repository Integration: On Save, on Pre-commit, or in CI

Introduction

Many code projects use an auto-formatter, such as Prettier. Auto-formatters are fantastic because they save developers an enormous amount of time and keep the codebase consistent. The rest of this article assumes that you want to use one.

But at what point should auto-formatters be applied to the code? There are three stages to choose from, and you can choose one or more of them:

1. On Save - The code editor (IDE) auto-formats the code as soon as a file is saved.
2. On Pre-Commit - A pre-commit hook ensures that all code is formatted before a commit.
3. In CI (Continuous Integration) - The CI pipeline checks formatting whenever a commit is pushed to the remote repository (using GitHub Actions, GitLab Runner, etc.).

Different projects use different stages. But what combination is best?

TLDR: format on save for immediate feedback, and check formatting in CI for enforcement. A pre-commit formatter adds little value between those two layers. The rest of this article will discuss the tradeoffs involved.

Goals

In this analysis, we want to optimize for developer experience (DX). In other words, we want our developers to be as productive as possible. At the same time, we still want to ensure that formatting violations do not slip through the cracks.

Time to Discrepancy Discovery

One way to think about DX is the amount of time that passes before code is auto-formatted. Or, more generally, before discrepancies between the written code and the correctly formatted code are discovered.

In general, we want to minimize that time because finding discrepancies as soon as possible allows us to fix them and iterate faster. Let's consider the three stages of auto-formatting checks with respect to time to discovery. We can plot them like so:

In this framing, we see that "on save" is the best.

Formatting on Save vs Formatting on Type

Auto-formatting on save has a short time to discovery because developers typically save a file once they are done modifying it. This means that the developer can immediately see what the auto-formatter did to their code, if anything. But there is a hypothetical option that is even faster: auto-formatting on type. In other words, you could imagine configuring the developer's IDE to summon the auto-formatter after every keystroke. (This would be a dot on the extreme left side of the chart above.)

It may go without saying, but this option would not result in better DX. Since auto-formatters move code around, having your cursor jump all over the place as you type would be confusing and would probably reduce your productivity. Since an auto-formatter should preserve the code's AST, the developer gains nothing by seeing the formatting changes that early in the process.

Manual Formatting vs Auto-Formatting on Save

Some developers do not like formatting on save and instead prefer to explicitly format the code with a hotkey. In this workflow, they would have to press both the formatting hotkey and the save hotkey once they are finished modifying a file. But this workflow has a failure mode: what if they forget to format the file? Now, formatting errors can leak into the codebase. The beauty of auto-formatting on save is that it avoids this failure mode.

We could also ask: what is the benefit of this extra, intermediate step? Since an auto-formatter should preserve the code's AST, there is no real reason to want unformatted code on disk. In unusual situations where you really don't want to summon the auto-formatter, such as when troubleshooting whitespace, it is easy to use the editor's "Save Without Formatting" command (mapped to Ctrl+K, Ctrl+Shift+S by default in VS Code).

For this reason, I think all developers should be encouraged to use auto-formatting on save. But if a developer really does not like it, that's okay too, because repositories should also check formatting in CI to ensure that nothing slips through the cracks. (More on that later.)

Formatting on Save vs Formatting on Pre-Commit

Both auto-formatting on save and auto-formatting on pre-commit accomplish the goal of preventing unformatted code from reaching the remote repository. So which is better?

Tool Visibility - Formatting on Save Wins

Auto-formatting on save is nice because the developer gets to immediately see what the auto-formatter did, if anything. With pre-commit formatting, however, the changes are performed less visibly.

Realistically, since an auto-formatter should preserve the code's AST, this distinction does not matter much: the developer probably won't care about formatting-related changes in either context. But there is an argument for making tooling as transparent as possible: it helps us understand what the tooling is doing. This is especially helpful when it fails and you need to debug it!

Installation Automation - It Depends

Turning on auto-formatting on save in an editor can be somewhat automated. For example, if the team uses VS Code, then .vscode/settings.json and .vscode/extensions.json can be committed to the repository to provide good defaults and extension recommendations, respectively. Other editors, such as WebStorm, offer similar functionality through shareable project-level files under .idea/.

The installation of pre-commit hooks can only be automated in certain cases:

• When using the Python pre-commit framework, each developer must manually run pre-commit install after cloning the repository.
• When using the Husky pre-commit framework, installation can be somewhat automated through an npm prepare script. In other words, the hooks are installed when the developer installs dependencies with a command such as npm ci.

So, with respect to automation, Husky is probably the clearest winner (contingent upon a JavaScript/TypeScript codebase in which developers reflexively install dependencies after cloning).

Editor Agnosticism - It Depends

When considering DX, it is desirable for a codebase to be editor-agnostic. In other words, if a codebase is not strongly coupled to a specific editor, developers are free to use whichever IDE makes them most productive. So, if we care about editor agnosticism, pre-commit hooks are the clear winner.

On the other hand, if everyone on the team uses the same editor, you aren't gaining much by being editor-agnostic. At the time of this writing (July 2024), VS Code is widely used across many language ecosystems. Some even consider it better than paid alternatives such as WebStorm. Languages such as Java won't necessarily have one editor to rule them all, but if your language does, the benefit of editor agnosticism is obviously reduced.

Configuration Maintenance - Tied

Both editor-specific files and pre-commit files must live inside of the repository. Thus, the maintenance burden between them is a tie.

The tie would be broken if the repository had to maintain files for two or more editors. This is uncommon, and I wouldn't recommend it.

Git Commit Velocity - Tied

It is common for skilled software developers to make lots of tiny commits. The basic idea is that smaller commits are easier for others on the team to understand and allow for more granular reverts when things go wrong. This practice is discussed in books such as Robert Martin's Clean Code.

Obviously, auto-formatting on save does not interfere with developers making commits. But pre-commit hooks do, and they are notorious for inhibiting productivity. Some developers have even said that they will quit outright when pre-commit hooks are mandatory within an organization.

However, while it is true that long-running pre-commit hooks inhibit productivity, this is not the case for hooks that run near-instantaneously. And code formatters fall into this category. For this reason, the negative stigma against pre-commit hooks is not relevant when considering auto-formatters. Thus, Git commit velocity is not really a concern for either option.

Complexity - Formatting on Save Wins

Ideally, we should choose the solution that introduces the least complexity into the developer workflow. This should probably be the most important consideration!

Overall, formatting on save is less complex than formatting in a pre-commit hook because most of the heavy lifting is done by the editor and its extension. Take Prettier, the most common formatter for JavaScript and TypeScript. Since it requires Node.js, we want to run it in a context that abstracts away the installation of the runtime.

• In the case of an IDE, the extension can bundle the formatter and its runtime dependencies. Thus, the Prettier extension can work without a separate Node.js installation.
• With pre-commit hooks, the runtime must either be installed by every user, as with Husky, or managed in an isolated environment by a framework such as pre-commit. That additional runtime-management layer can complicate troubleshooting, including OpenSSL compatibility issues.

Overall, formatting on save is much less likely to go wrong. And when things do go wrong, it is much easier to debug.

Using Formatting on Save and Pre-Commit Together

Even if you decide that formatting on save is slightly better than formatting on pre-commit, you might decide to use both solutions at the same time. But is that a good idea?

For - Defense in Depth

The argument to use both is a defense in depth strategy:

• If we use only formatting on save, a developer's editor can become misconfigured and allow unformatted code to slip into the codebase.
• If we use only formatting on pre-commit, a developer's pre-commit hook can become misconfigured and allow unformatted code to slip into the codebase.
• If we use both, then each step guards against failures at the other step.
• It is still technically possible that the developer's editor and pre-commit hook can both be misconfigured, but this is much less likely than just one component failing.

Against - We Have to Use CI Anyway

Consider the case where the developer's editor and pre-commit hook are both misconfigured. Because this case exists, if we want 100% coverage, we have no choice but to also check for formatting in CI.

Thus, lowering the probability of pushing unformatted code to the remote provides little additional protection because CI will catch the problem in either case.

(To be clear, this is not an argument for only having CI checks, because that is terrible DX. Rather, the argument is that formatting on pre-commit doesn't give us any additional DX wins on top of formatting on save.)

Against - More Complexity Is Bad

As engineers, we should strive to minimize complexity in our codebases and toolchains. It is pretty hard to justify the added complexity of a pre-commit hook when the payoff largely duplicates existing functionality.

The Downsides of CI Checks

In the earlier visualization, we called CI the "worst" option because it is the furthest from the developer. Let's color this in a little bit to illustrate exactly how the bad DX comes about.

CI runs can take a long time. Consider this common scenario:

• Alice finishes work on her feature branch. She pushes it to the remote repository and knows from experience that CI will take about 30 minutes to complete.
• To stay productive, Alice immediately switches to a new branch to work on another feature.
• Thirty minutes pass, and Alice is now thinking deeply about a problem related to the new feature.
• Alice gets an email notification that CI has failed.
• Alice now has to perform a very costly context switch, leaving her current problem and returning to work she set aside 30 minutes earlier.
• Alice finds and fixes the error. She pushes to remote.
• Alice now performs another costly context switch back to her new branch.
• Another 30 minutes pass, and Alice gets another email notification that CI has failed.
• The cycle repeats.

To avoid these kinds of time drains, we want to push checks closer to the developer whenever possible. Thankfully, auto-formatting allows us to do that with few other downsides.

CI Checks Are Still Mandatory

Using CI is not free: there is a modest maintenance cost, additional complexity, and the potential cost of hosting the CI platform and runners. But if we want to ensure that no formatting violations slip through the cracks, we don't have a choice: we must check formatting in CI, regardless of which other steps we use.

But the point is that we can use CI as a last-resort check instead of the primary check. By doing this, we get the best of both worlds: good DX and validation that is difficult to bypass.

Conclusion

Given the previous discussion, I think the following is true:

• We have to check formatting in CI to ensure proper coverage because the earlier checks can be bypassed.
• We have to check formatting before pushing to the remote repository to ensure good DX.
• Auto-formatting on save is arguably superior to auto-formatting on pre-commit, mostly for reasons of visibility and complexity.
• Using both auto-formatting on save and auto-formatting on pre-commit offers little additional benefit and greatly increases the complexity of the tooling.

Comments

[David Blass]

Agree! Format on save by default, always check in CI.

Do whatever else you want locally, but I don't see the upside to mandating precommit hooks. They're less immediate than on-save and less well-enforced than CI.