DEV Community: Doug Donohoe

Always Be Refactoring

Doug Donohoe — Tue, 06 May 2025 07:36:23 +0000

Great engineers refactor constantly, even if they have to slip it into their normal workflow.

“Always be closing.”

The hard-nosed sales mantra from Glengarry Glen Ross became shorthand for relentless improvement. In the same spirit (but with much less swearing), a good engineer should always be refactoring. Here’s why.

To Clean or Not to Clean

It’s generally agreed that clean code is easier to understand, cheaper to maintain, and much more extensible. At its core, refactoring is cleaning up existing code without altering its perceived functionality.

A common objection to refactoring is that there isn’t time to actually do it. Teams see it as a luxury. The relentless drive for new features simply doesn’t allow for refactoring, especially because refactoring changes nothing from an external point of view. That can be a hard sell to your product owner.

But clean code makes your life easier. Clean code pays for itself and slows the accumulation of technical debt.

So sneak it in.

That’s right. I’m suggesting a slight bit of subterfuge. A little misdirection even. Why not clean up a few things in every pull request?

Be Vigilant

A good engineer has a vision of where the codebase should be heading. It might take a while to get there, but you know the messy parts, the backlog, the roadmap, the technical debt, the security holes, and the documentation gaps.

As you go about your regular feature development, be on the lookout for refactorings that can advance your larger agenda. Like one of those hyper-observant TV show detectives surveying a crime scene, pay close attention to all the code you stumble across.

When you notice a code smell or something slightly off near your current focus, alarm bells should go off: “Don’t let this opportunity pass!” Take a few minutes and fix it now, before inspiration fades.

Don’t say it’s not your problem. Don’t pretend you can unsee it. Just roll up your sleeves and get it done.

A Simple Example

Refactoring doesn’t necessarily mean changing thousands of lines of code. It can be just a small chunk of code here and there. These little micro-refactorings add up over time. In fact, the more you get into the habit of constant cleanup, the less you will need to do major refactorings in the future.

To illustrate micro-refactoring, let’s look at an “extract method” Golang example.

Let’s say you are tackling a feature that requires knowing how many days it has been since a user last logged in. You notice an existing method that determines if a user is dormant by using that same information:

func IsDormant(user User, asOf time.Time) bool {
  days := int(asOf.Sub(user.LastLogin).Hours() / 24)
  return days >= 8
}

You want to re-use the last login calculation instead of copying and pasting it, so you take the internal variable, days, and extracting it into a separate method, DaysSinceLastLogin:

func IsDormant(user User, asOf time.Time) bool {
  return DaysSinceLastLogin(user, asOf) >= 8
}

func DaysSinceLastLogin(user User, asOf time.Time) int {
  return int(asOf.Sub(user.LastLogin).Hours() / 24)
}

This allows the last login logic to be tested and reused. If you write a unit test, you’ll spot an edge case that should be handled (a potential panic if a user has never logged in).

It also makes future enhancements easier. For example, it might make sense to make IsDormant and DaysSinceLastLogin methods on the User struct instead of being standalone. Likewise, consider replacing the hard-coded value 8 with something more descriptive like DormantDaysThreshold.

This is a simple example, just a few lines of code. But that’s the beauty. It shows a small refactoring can add value by revealing a potential bug and pointing towards future improvements.

To learn more about the craft of refactoring and see all the small ways to improve your code, check out online resources such as the refactoring catalog from Martin Fowler’s book or the Refactoring Guru.

A Refactoring Mindset

Having a vision for your codebase is easy. Knowing how to get it there is harder. Spotting refactoring opportunities takes practice. One way to get started is to consider which categories of refactoring are necessary to transform your code.

Here are some common refactoring themes to think about:

🧹 Remove Duplication — Collapse copy and pasted logic into a single function.

🧰 Extract Shared Utility or Library — Move common logic out of per-service implementations. Normalize integrations (e.g., all services use the same auth client).

🧱 Add Common Infrastructure — Introduce observability (logging, tracing, metrics). Centralize error handling or retries.

⚙️ Make Code More Testable — Break apart tightly coupled logic. Extract interfaces so dependencies can be mocked or stubbed.

🔄 Prepare for a Feature Change — Flatten complex conditionals or nested logic. Reorganize files or classes to fit new responsibilities.

🏗️ Support a New Architecture — Break apart a monolith. Introduce event-driven patterns, dependency injection, or async processing.

🚦 Reduce Cognitive Load — Split giant files or classes into logical, focused units.

🔐 Improve Security or Compliance — Centralize access control logic. Refactor insecure patterns (e.g., manual SQL concatenation → parameterized queries).

🚀 Improve Performance — Replace naive algorithms with optimized ones. Reduce memory churn or unnecessary computation in hot paths.

👥 Ease Onboarding or Handoff — Refactor code that only “Pat” understands into something team-readable. Introduce docs/comments/test coverage as part of structural cleanup.

The 20%

Some might object that surely not all refactorings can be snuck in. I’ll grant you that. Let’s apply the 80–20 rule and stipulate that 20% of refactorings need to be done on the up-and-up.

These should be an easier sell, however, because when you get to the point where you need a full, honest-to-goodness refactor, it is most definitely going to be in service of some larger goal.

For example, before working on a performance improvement ticket, you first need to add tracing and metrics so you can develop a finer picture of current performance and identify hotspots.

Refactoring Starts with Testing

It is probably self-evident, but I must point out that refactoring is much easier (and less nerve-racking) if your codebase has an accompanying suite of tests that pass before and after the refactoring. Having tests is also valuable for a bunch of other reasons. If your project doesn’t have strong testing, then add some tests first to unlock future refactorings.

Code Review Considerations

My advice in this article runs counter to the standard guideline to not mix refactoring work and regular feature development. The objection is that it makes it harder for the team to do code reviews if unrelated changes are included. That’s a fair point. When refactoring during feature work, keep it small and (ideally) related to the main thrust of the change.

A good rule of thumb for any pull request is to limit it to 500 lines of code changes. A second rule of thumb is that ancillary refactoring changes should be no more than 20% of the PR.

When you do have dedicated refactoring PRs, keep it focused and around this size. Sometimes a PR has to be greater than 500 lines of code, especially when working on a repo-wide refactor. In those cases, coordinate well with your colleagues to prepare them for the change and to minimize merge conflicts.

Make Every Commit Count

Every time you touch the codebase, you have a choice: leave it a little better, or leave it a little worse.

Refactoring doesn’t have to be a grand event. Small improvements (extracting methods, renaming confusing variables, breaking apart long methods, etc.) add up over time. They prevent technical debt from piling up and make future changes easier and safer.

If you see something, fix something.

Always be refactoring!

You Don't Have Time Not to Test

Doug Donohoe — Thu, 24 Apr 2025 06:58:04 +0000

Testing isn't a sunk cost. It's a compounding return that shapes better code and ultimately accelerates your team.

"We don't have time to test," stated the principal engineer matter-of-factly, with an expression that left no room for debate. It was a proclamation of absolute truth like "the sun always rises in the east" or "you always find the lost thing in the last place you look." A startup fact-of-life, or so it seems.

This was in response to my candid observation that test coverage was low, existing tests didn't always run locally for mysterious reasons, integration tests were non-existent, and pull requests could merge without passing the few tests that did exist.

I was shocked into silence. This anti-testing sentiment wasn't new to me, but it still astonished me to hear it said so forthrightly. Like getting insulted on the schoolyard playground by a bully, I was unable to swiftly come up with a clever retort. I kept silent but my brain was screaming "Oh yeah, you, you … you don't have time not to test!" Take that, principal engineer!

Some time has passed and this statement still bothers me. It seems that software engineers everywhere continue to have an aversion to writing tests and I don't understand why that is. Maybe the "move fast and break things" philosophy has gone too far. Testing is a really good tool. It's as indispensable as having turn-by-turn GPS guidance when navigating in an unfamiliar city.

In this post, I'll dig into this conundrum a little more. What is really going on here? What is testing, actually? Why should it be a tool in your professional toolkit?

Under Pressure

I get it. Leadership is pressuring you to ship some features so the company can find product-market fit in order to acquire customers and not die. Yes, that can be stressful.

It reminds me of the humorous rock monster scene from one of my favorite movies, Galaxy Quest. In it, a very angry rock monster is chasing the commander of a space ship. To fend off the monster, a fellow crew member helpfully suggests, "form some sort of a rudimentary lathe." This is an absurd suggestion and (at least to me) one of the funniest lines in the movie. What would make this even more absurd would be if another crew member were to yell out that he should make sure the lathe does its lathing properly by testing it out on a mock angry rock monster.

When the engineer said "we don't have time to test," in response to my no-doubt-aggravating observations, it was essentially equivalent to him shrugging his shoulders and pointing backwards at the looming monster. When under pressure you are certainly going to take some shortcuts in order to live another day.

I'm not here to fault any engineer. For all I know, they are also bummed about the state of affairs, but don't have the political capital to argue the point with management. You have to pick your battles.

I agree, yes you have to make tradeoffs, but I believe that abandoning tests and test automation - even at the very earliest stage - is a mistake.

What is a Test?

As a software engineer, you'll have heard of unit tests, integration tests, system tests, end-to-end tests, smoke tests, performance tests, chaos tests, regression tests, load tests, penetration tests and so on.

I'm going to intentionally ignore these distinctions. To me, the argument of "what is a unit test" vs "what is an integration test" is unhelpful. It's just bike-shedding [1]. The key question is whether a test exists at all, regardless of what dependencies a test might require to run.

So, what is a test? To me, a test is a chunk of code that directly verifies the most important concerns of another chunk of code. It can be run on demand, rapidly repeated, replicated elsewhere, and automated.

A test may be able to operate independently, which is usually the case for unit tests on purely functional bits of code. A test may require some help - like a database, an external service, or mocked dependencies - as is often the case with integration or end-to-end tests.

Most Important Concerns

What are the most important concerns? At its core, it validates that the code does what it claims to do. For example:

An MD5 hash function calculates the correct MD5 hash of a file.
A photo EXIF extractor properly fetches details like exposure and aperture from a JPG file.
A database repository returns exactly the same data that it saves in a database (looking at you, time zones and decimal numbers).
A third party library sanity check, demonstrating the API and verifying it does the thing it is supposed to do.
The right error is raised when something fails.
A log message is emitted.

Conversely, a test should fail if the code under test ceases to do the thing it is supposed to do. In the real world, you want to catch when:

A third party library changes its implementation.
A database subtly changes the semantics of rollbacks.
A rogue engineer on your team (you know the one) makes a bone-headed change.

I always ask myself, "If this critical assumption changes, will a test break?"

Invoke On Demand

A test should be easy to run, whether from the command line or right inside an IDE. I personally use IntelliJ, which has support for languages such as Java, Go, Python and Scala. With a single keyboard shortcut, it magically knows how to run the test under the cursor. Figuring out the command line equivalent of running an individual test is a task well suited for ChatGPT, as it varies widely by language.

Using an IDE also makes it easy to step through code in a debugger. This is especially crucial when you're learning a new codebase or tracking down a tricky bug. The ability to directly execute the code you're interested in - rather than triggering it indirectly through a webpage or API endpoint - is immensely valuable.

Rapidly Repeat

Running a test at the touch of a button significantly boosts your velocity. This is where tests genuinely help you deliver code faster: you reach a working state sooner and gain a companion that both validates your thinking and captures your intent - like executable documentation.

Replicate Elsewhere

Once your test can be run on demand, it becomes easy to try it out in other environments - like a teammate's laptop or a CI/CD pipeline. This is where your tests start to pay dividends for the team and unlock real automation.

Automate

Once you have a collection of tests, the next logical step is to run them all - either on demand (say, after a major refactor) or automatically during key points in the development lifecycle (like when opening a pull request). This safety net helps ensure your code continues to behave as expected, even as the team keeps moving forward.

In Reality

Now, to be fair, most engineers are not skipping testing altogether. They're probably doing some manual checks with Postman, watching logs, or visually confirming debug output. The resistance isn't to testing itself, it's to codifying and automating those tests.

Why the resistance? The reasons are many, but here are some I've heard (and maybe said myself):

I don't know how to write proper tests in [this new language] and - I'm afraid to admit it.
I barely got the code working on my laptop - I can't worry about onboarding or CI/CD right now.
We don't have time to build a framework for database tests.
We can't write integration tests - our CI doesn't support spinning up dependencies in Docker.

And so on …

Two things are happening here:

The time required to build testing infrastructure is vastly overestimated.
The accumulating, compounding time savings are vastly underestimated.

Time isn't budgeted to build it even though it will quickly pay for itself. Conversely, time not spent on infrastructure is only borrowing against the future when it will be more costly and time consuming to deal with bugs, production incidents and onboarding delays. I think of this as Time Technical Debt.

When this misunderstanding takes root, teams often enjoy a short burst of speed, only to slow down over time. They become increasingly less productive over time as they fight bugs and fragile systems.

But it doesn't have to be that way. With a small upfront investment - even just a bit of time learning, or scaffolding a few basic tests - teams can start reaping the rewards immediately. The savings scale across individuals, across the team, across months and years.

Yes, you do have to spend some time to build up your testing infrastructure. But this isn't an all-consuming task. I know this from my time at Retrofit, where I was CTO and employee number two. Two other engineers and I spent the first six weeks building not only the core architecture, but simultaneously putting in place comprehensive testing and CI/CD infrastructure. It paid dividends for years.

An old proverb states "The best time to plant a tree was 20 years ago. The second best time is now." The same is true for automated testing. The sooner you do it, the sooner you start to accrue the hockey stick benefits.

When you need to write a test to validate your code, you inevitably end up changing how you write your code. It's not just about checking behavior - it's about shaping it. Here are a few ways this happens:

You start returning more detailed information from methods - like the number of records changed or a modified data object with fields filled in (e.g., a generated timestamp).
Errors have more details, or better yet, become a return value instead of something that is raised or thrown.
You discover that something you start also needs to be cleanly stopped, so you can verify shutdown logic.
You also discover that things that start usually need to be initialized in some way, and that init step needs to be validated, so your code starts to have construct → init → start → stop → cleanup phases rather than just start-and-crash behavior.
You begin using interfaces so you can simulate failure scenarios - like a network outage - and confirm your code handles them gracefully.

Asking the question "How am I going to test this?" forces you to think critically and honestly about your code. You ultimately end up with better code, which means less time wasted in the future on bugs and dealing with unexpected behavior.

Testable code also tends to be more self-contained. It has a clear lifecycle. It's introspectable. It has better logging. And above all, it's modular.

Modularity is good.

You're building Lego bricks - reusable, well-formed pieces you can snap together to create great software.

Always Be Refactoring

Having a suite of tests facilitates another software development superpower: refactoring.

Your understanding of the requirements is constantly evolving, especially in a startup. An assumption you made yesterday might be invalidated tomorrow, forcing you to rethink a feature or rework some underlying architecture code.

Channeling the famous Always Be Closing (ABC) scene from Glengarry Glen Ross, you should "Always Be refaCtoring!"

A chunk of code is rarely ever "done-done," and rarely so on the first pass. You'll learn something new about your use case and realize you need to tweak it. A new version of your programming language comes out which forces a change. A faster third-party library offers a 25% performance boost. That handful of helper functions? It's now big enough to deserve its own library.

Whatever the case, you are bound to revisit a piece of code dozens or hundreds of times over a product's lifetime.

Having a suite of tests available to vet those refactorings is a gift that keeps giving. It gives you carte blanche to change any existing code with a minimal amount of stress or worry.

Pay It Forward

Some poor soul has to maintain your code. That person might just be you, months or years later. More likely, it'll be someone diving into it for the first time. Maybe under pressure, during an on-call incident.

Ideally, we all want to be kind to this future maintainer - just as we hope the engineers before us were kind to us.

A test that runs and directly invokes the target code is far more valuable than forcing someone to navigate an unfamiliar, sprawling codebase. Even better, that test becomes an example - making it easier to write new tests, reproduce bugs, and validate fixes.

So, do your future self (and others) a favor: Write some tests. Pay it forward.

Let Me Sleep

I don't want 2 a.m. wake-up calls because some code I wrote went off the rails. Been there, done that - and I'd really rather not do it again.

Pager duty is one of the least fun parts of being a software engineer. I'm generally okay supporting my own code, because I've taken the time to make sure it behaves well - not just in happy-path scenarios, but when everything goes sideways. What I dread is getting paged because someone else didn't test their code properly.

Good tests are a kindness - to your future self, your teammates, and whoever ends up on call next.

Vibe Coding

The hot new topic in software engineering is, of course, AI - and with it, the rise of "vibe coding," where you let the AI take the wheel and hope for the best.

There are plenty of issues with that approach - especially around maintainability and security - but this post isn't about that. What I will say is this:

AI can absolutely help you write tests. It can generate scaffolding, speed up repetitive work, and even suggest edge cases you hadn't considered. That's great.

But be careful about handing over the keys entirely.

You still need to think critically about your code - and make sure your tests are validating the most important concerns. That's not something you can vibe your way through.

Use AI for smaller, focused chunks. Review everything. Don't let it spew out thousands of unchecked lines and call it a day.

Would you drive across a bridge that was vibe architected?

Not Testing Is Not an Option

Automated testing isn't some extra layer you add after the fact - it's a habit you build as you write code. It helps you move faster, catch problems earlier, and write cleaner, more modular software. It helps your teammates. It helps future you.

You don't need a perfect test suite. You just need tests that verify the most important concerns, can be run on demand, and give you the confidence to change things without fear.

Write tests not because you were told to, but because they make your job easier.

Note: Originally published at Medium and discussed on Hacker News. Comments here are welcome and encouraged.

Multi-Architecture Google Cloud Builds on arm64 VMs

Doug Donohoe — Wed, 25 Jan 2023 14:37:07 +0000

As an engineer, I lean heavily on answers other engineers give at sites like dev.to or via blog posts. I like to return the favor when I can.

I just published a Medium article on a series of lessons I learned about building so-called "multi-architecture" Docker images with docker buildx.

This stems from the prevalence of Apple's new M1 / M2 chips. I got a new Mac laptop when I started my job in June. Practically none of our Docker images were available for arm64.

My article shows lessons learned getting to multi-architecture images, including using an arm64 VM on Google Cloud Build to speed up the arm64 half of the builds (as shown in this image):

There is also a companion git repo with full source code.

Hopefully it can be of use to someone else!