DEV Community: Nicolas Carlo

Can AI help me write tests on legacy code?

Nicolas Carlo — Thu, 09 Feb 2023 14:26:49 +0000

AI has become a hot topic in the past few months. Tools like ChatGPT have been released for the world to play with, and this has raised a lot of interesting questions about how it will change the way we work.

My interest is piqued when it comes to legacy code.

A lot of us are working on existing, poorly documented, untested software that we have to change. Evolving this code without breaking things is a daily challenge. Would AI does that job better? I would actually be excited if tools could do some of the grunt work, so I could spend more time understanding the Problem that is being solved instead of spending so long on the implementation details of the Solution that I need to change.

However, I’m suspicious.

I've already wrote about this topic and my experience wasn't great then. The generated tests weren't really helpful. I was mostly wasting time.

AI makes mistakes. Yet, it sounds confident. If it doesn’t know, it may just make things up. If I don’t know what the code is doing, how can I hope to detect the lies?

ChatGPT vs. Rubberduck

That being said, I’ve also played with ChatGPT. I gave it slices of code and asked it to simplify it, refactor it with patterns, etc.

It works… but not always. I’ve found it creates friction during my development process, namely:

Sometimes (often?) ChatGPT is not available
Sometimes, it stops generating code in the middle. You can reply “Continue” to resume the generation, but that’s still annoying (and you better know the trick)
Sometimes, the returned code formatting breaks down. It happened to me a few times already and that’s annoying.
It requires me to switch context: get out of my editor to use my browser. It’s not a big deal (browsing answers is probably the 2nd most common development activity after reading code), but it would be nice to have that from my editor…

And then, I saw Lars Grammel’s tweet about his new project: Rubberduck, an AI-powered coding assistant for VS Code.

It says it can help you:

Chat with OpenAI
Diagnose Errors
Explain Code
Generate Tests (!)
Edit Code (!!)
Find Bugs (!!!)

Right from VS Code.

Interestingly, its source code is open! Thus, I've started to contribute.

But here, I'll be using Rubberduck from my end-user point of view. So let’s try and find out what it can really do…

The Lift Pass Pricing refactoring kata

This is a coding exercise that captures real-life software complexity (HTTP requests, database calls, external data…), without adding too much noise. I really like to use it to practice tools and techniques for refactoring legacy code.

For testing Rubberduck, I will tackle the TypeScript version.

The very first step is to run a tool like Prettier to standardize the shape of the code. No more noise in follow-up changes: rbd-1-prettier

The end goal of the exercise is to change the API so the endpoint that returns lift pass prices can return the price for several passes instead of just one.

But the code is hard to re-use, so we would like to refactor it a little bit first.

But how do we know the code is still working as expected after we refactored it? In general, we want to write tests first.

So let’s see if the duck can help us write the missing tests faster… or not.

First, make the tests run

First of all, we need to write some tests before we touch the code. For this kata, I usually go with:

supertest-as-promised to run HTTP queries against the top-level application
testcontainers to set up the database and get it to run in the tests (it’s based on Docker)
A script to fill the database with tests data (these are already provided in the kata)

But let’s ask Rubberduck!

Ask Rubberduck to suggest how to test this code

To do so I need to:

Select the code I want to test
Use VS Code palette to trigger the command “Rubberduck: Start Chat 💬”

This started a new chat and I could type in my question:

👨‍💻 How can I write tests for this piece of code?

🤖 Writing tests for this code can be a bit tricky, as there are a lot of conditions to consider. To make sure that all the conditions are tested, you'll need to create a test for each condition. For example, you could create a test to check that the cost is 0 when the age is less than 6, and another test to check that the cost is reduced when the age is between 6 and 15. You could also create tests to check that the cost is reduced on holidays and on Mondays.

OK, this is not very helpful. It’s just generic guidance…

I followed up with more details about what’s blocking me and got some advice.

👨‍💻 This is an Express controller. There is also a database involved. Which TypeScript libraries can I use to test it?

🤖 You can use the Mocha testing framework with the Chai assertion library to write tests for your Express controller. You can also use the Sinon library to mock the database calls. This will allow you to test the controller without actually making a call to the database. You can also use the SuperTest library to make HTTP requests to the controller and test the response.

That’s more interesting!

✅ It’s good that it suggested SuperTest for testing the controller. If I wasn’t aware of this, it would have put me on the right track!

🚫 I think going for Sinon to mock the database calls is a mistake at this point. I will refactor the implementation details and I have no control over the connection API. Mocking it will make refactorings harder. I think a better approach here is to take a step back, consider this a black box, and set up an actual database.

I tried to ask follow-up questions. Rubberduck gave me an interesting option where we could use Sequelize in tests to setup the database:

👨‍💻 I don't want to mock the database calls. Is there a library to set up the database and get it to run in tests?

🤖 If you don't want to mock the database calls, you could use a library like Sequelize to set up the database and run it in tests. Sequelize is an ORM (Object-Relational Mapping) library that allows you to define models and perform database operations. You can use it to create a test database and run tests against it. For example, you could create a test database with the following code:
const Sequelize = require("sequelize")
const sequelize = new Sequelize("database", "username", "password", {
  host: "localhost",
  dialect: "postgres",
})

const User = sequelize.define("user", {
  username: Sequelize.STRING,
  birthday: Sequelize.DATE,
})

sequelize
  .sync()
  .then(() => console.log("Database is synced"))
  .catch(err => console.error("Error creating database"))
Once the database is set up, you can use Sequelize to perform database operations in your tests. For example, you could create a user in the test database with the following code:
User.create({
  username: "testuser",
  birthday: new Date(2000, 0, 1),
})
  .then(user => console.log(user.username + " created"))
  .catch(err => console.error("Error creating user"))
Do you have any other questions about setting up the database and running tests?

I think this would work, indeed.

Note however that I couldn’t get the AI to guide me towards testcontainers, despite mentioning “Docker in tests”. Without AI, I would usually search npmjs with the relevant keywords, which will expose me to it.

To me, this means you should not just blindly listen to AI—it would have been naive to think otherwise. But combining its suggestions with your own experience and other sources can get you further, faster.

If I didn’t know how to get started here, Rubberduck suggestions to use Sequelize would certainly have helped.

But let’s go with testcontainers anyway.

Use Rubberduck to write the code for me

There is an existing test. To make it run, we need to:

Get a MariaDB up and running
Run yarn start to get the server running
Run yarn test to run the tests against the server

It may work to get started, but that won’t go far. If anything, this test can’t easily run on CI. The idea would be to start a Docker container where the whole thing can run reliably for each test run. That’s what testcontainers can do!

Let’s ask Rubberduck to update the tests for us, and see if it can save us some time writing the boilerplate code:

Select the whole existing test code that I want to edit
Use VS Code palette to trigger the command “Rubberduck: Edit Code 💬”

Then, I type in my request, trying to give some specificities:

👨‍💻 Use testcontainers to set up a new GenericContainer before all tests. It should use mariadb version 10.4 and expose port 3306. After all tests ran, it should stop.

After ~10s it generates a diff next to my code. I can inspect it and decide to “Apply” by clicking on a button at the bottom. I must admit that’s a way better flow than having to copy-paste code on ChatGPT. It feels integrated within my development process 😄

Looking at the suggested diff, it’s not bad. There are some good things:

The imports are correct
It did declare the container variable along with the others and initiated it
It exposed the proper ports and started the container correctly
It did stop the container

There are also some errors:

The syntax to use MariaDB 10.4 is GenericContainer("mariadb:10.4"), not passing two arguments. Hopefully, TypeScript will catch that easily.
I wanted the container to be started before ALL tests and stopped after ALL tests. Doing it between each test is unnecessary and costly. AI didn’t listen to my instructions here.

At this point, I have 2 options:

Tell Rubberduck about the changes I want, so it can refine the suggestion.
Apply the suggestion and do the manual tweaks myself.

On real code, I would go for option #2 since it would be faster—the AI still takes a few seconds to generate the code and I feel I can do the change faster myself because I won’t wait. It also comforts me with the idea of using AI to assist me, but still own the changes and finalizes what needs to be done.

For the exercise though, I will refine the suggestion to see what it can do:

👨‍💻 The container should be created before ALL tests. It should be stopped after ALL tests.

And that worked!

It’s very interesting how you can follow up on the chat to get a refined suggestion. It means you don’t have to get the first input right on the first try. The main limiting factor today is the ~10s it takes to generate the code again, which makes the feedback loop too long for just developing like that—I can just make the changes myself.

Happy with these changes, I clicked “Apply” to get the code in. I make a few manual tweaks:

I forgot to ask Rubberduck to fix the syntax to call GenericContainer("mariadb:10.4")
I’m explicitly typing container when it’s declared to get type-safety wherever it’s used
I configure the DB environment variable and initialization script when starting the container

I guess I could have refined these points with Rubberduck, but I mostly realized them after I got the code in. That’s fine. Rubberduck got me the headstart I was looking for… TypeScript is getting me through the finish line 👍

Run the tests

Now it’s time to verify if all of that is working!

I don’t have a MariaDB up and running. Nor is the app running. I only have Docker started on my machine. I hit yarn test and wait…

Failure!

Apparently, something is wrong with the exposed ports. And indeed, in this specific case, AI’s suggested syntax won’t be enough: withExposedPorts(3306). The problem is that the port may not be mapped to the 3306 port on the host.

A quick search in testcontainers docs tells me that there is another syntax to bind the ports: withExposedPorts({ container: 3306, host: 3306 }). Another way would be to get the mapped port and pass it to the source code, but I don’t want to change the source code now. Let’s go with the first option.

I change the code and run yarn test again:

Success!

I think that illustrates something important: the code suggested by AI has a lot of unknowns and hypotheses. I need to combine it with other tools (static types, reading the API docs, my own knowledge, some sort of tests…) to validate them.

I think it’s important to find a way to verify the suggested code soon after it was merged in.

That being said, the suggested syntax was fine and it helped my lookup for the relevant docs. If I wasn’t familiar with testcontainers API, that would have saved me quite some time!

Commit, push, and we are here: rbd-2-testcontainers

Then, write the high-level tests

This is often a difficult part.

What do you test? How do you get started? There is so much going on… I generally recommend using the test coverage to identify parts of the code that are not tested yet. I’ve already detailed this process, the main idea is to vary the inputs to capture the existing behavior

For this exercise, we have 3 parameters: type, age, and date. Spoiler, there are at least 11 scenarios to cover:

type="1jour" => then cost is day cost
type="night" => then cost is 0
type="night" & age < 15 => then cost is night cost
type="night" & age > 64 => then cost is 40% of night cost
type="1jour" & age < 6 => then cost is 0
type="1jour" & age < 15 => then cost is 70% of day cost
type="1jour" & age = 15 => then cost is day cost
type="1jour" & age > 64 => then cost is 75% of day cost
type="1jour" & age = 15 & any date => then cost is day cost
type="1jour" & age = 15 & date is Monday => then cost is 65% of day cost
type="1jour" & age = 15 & date is Monday and Holiday => then cost is day cost

Let’s see if Rubberduck can speed that up!

The first test is already given, let’s just rename it.

- it("does something", async () => {
+ it("returns day cost when type is '1jour'", async () => {

Now, let’s ask Rubberduck to generate some tests for us, and see if it can suggest useful scenarios quickly from the source code.

Select the code I want to test
Use VS Code palette to trigger the command “Rubberduck: Generate Tests 💬”

After waiting ~1min it generates a new unsaved file with the test code:

The style doesn’t match the rest of the tests. It tried to write unit tests from scratch. But I can copy-paste the generated scenarios and adapt them. I feel this will be faster than re-generating new tests.

After copying the test cases, I select them and trigger another “Rubberduck: Edit Code 💬” action.

👨‍💻 Use supertest to request the app and don't mock the response. Use chai library for the expect API.

After ~10s I get a diff that looks good. I click apply and I got a code that looks like this:

// Original test
it("returns day cost when type is '1jour'", async () => {
  const response = await request(app).get("/prices?type=1jour")

  expect(response.body).deep.equal({ cost: 35 })
})

// Tests generated from Rubberduck
it("should return cost 0 when age is less than 6", async () => {
  const req = { query: { type: "day", age: 5 } }

  const response = await request(app)
    .get("/prices")
    .query(req)

  expect(response.body).to.have.property("cost", 0)
})

it("should return cost 0 when age is undefined", async () => {
  const req = { query: { type: "day" } }

  const response = await request(app)
    .get("/prices")
    .query(req)

  expect(response.body).to.have.property("cost", 10)
})

// …

But running the tests will fail. With a closer look, I realize there is indeed an error: the req object should actually be a query and not have a nested query attribute! I could use Rubberduck to fix all tests… but I decide to use VS Code multi-cursor instead since I can do it in a few seconds:

I decided to rewrite the original test to look similar to the others. This is when I notice something else is off: the type value is “day” but it should really be “1jour”.

That’s a legit mistake. The source code refers to the type “night” as a special one. AI figured another variant would be “day”. Except that the valid values are in the database, and AI isn’t aware of this yet.

I replace all “day” occurrences with “1jour” and run the tests again:

Well, well, well… When looking closer, there are some obvious mistakes in the generated tests.

One test isn’t testing what it says it is testing:

In fact, this test is even wrong. I can see the body of the test is the same as the original one. The expected cost should be the base cost, and I can confirm that with the test failure. Let’s scrap this one!

As for the other tests, they fail because they consider the base cost to be 10:

But here again: the base cost is set in the database, and depends on the ticket type.

Therefore, I need to correct these expectations manually. The test's failure helps me figure out the proper behavior. Sometimes the label is wrong, sometimes it’s the expected output that’s incorrect. However, the variation of inputs is the interesting part! Let’s see the scenarios that were properly covered:

✅ type="1jour" => then cost is day cost
type="night" => then cost is 0
✅ type="night" & age < 15 => then cost is night cost
✅ type="night" & age > 64 => then cost is 40% of night cost
✅ type="1jour" & age < 6 => then cost is 0
✅ type="1jour" & age < 15 => then cost is 70% of day cost
type="1jour" & age = 15 => then cost is day cost
✅ type="1jour" & age > 64 => then cost is 75% of day cost
type="1jour" & age = 15 & any date => then cost is day cost
✅ type="1jour" & age = 15 & date is Monday => then cost is 65% of day cost
type="1jour" & age = 15 & date is Monday and Holiday => then cost is day cost

That’s 8 scenarios out of 11!

Sure, the generated tests didn’t work. I also had to manually fix the syntax, most labels, and expected values. But I saved quite some time by not having to manually figure out the different parameters that have to change. Rubberduck found them all for me!

With this base, I would then use test coverage to see parts of the code that aren’t tested yet and figure out the missing variations to get them all.

From my experience, Rubberduck generates better tests when the source code doesn’t depend on external sources, like a database or a 3rd-party service. And yet, this was helpful to give me ideas and get me started. It may come in handy when dealing with unfamiliar code that I want to write tests for 👍

The final code is here: rbd-3-tests

Extra: Rubberduck helped me with a random question

While writing this article, I pushed all the code to a forked repository, so you can follow along.

At some point, I couldn’t remember the git syntax to push the tags to that specific remote… So I just open a new Rubberduck chat and asked. It gave me the command I was looking for. All of that without leaving VS Code ❤️

Well, thank you Rubberduck 👍 🦆

How is it different than ChatGPT? Ponicode?

It's very similar to what ChatGPT could do. The main difference to me is the UX:

No need to get out of my editor is nice
No fear of ChatGPT website being unavailable at the moment
No hassle crafting up the perfect prompt to get the answers I need (Rubberduck crafts the prompt under the hood)
And no little annoyances like AI stopping the prompt in the middle or messing up the code formatting
Oh, and 💯 for the diff view + "Apply" button

As for Ponicode, the experience is very different. Ponicode wasn't capable of generating useful unit tests on random code (and the code was simpler). The generated tests didn't help me figure out the different parameters I could use… It was also less versatile.

To be fair, OpenAI did bring a lot of new opportunities when they released their API. Their model is powerful, and the results are (finally) kinda usable on actual legacy codebases!

AI can help you write missing tests faster…

But don’t turn your brain off!

That would be my conclusion to this experiment. I was curious about what an AI assistant such as Rubberduck could do:

Can it just do the work for me? Nope.
Can it help if I’m not an expert at refactoring unfamiliar code? Yes, it gave valuable insights to get out of the analysis paralysis.
Can it make me go faster if I’m already familiar with the code/techniques? Surprisingly, yes. There are parts of the process that always take some time, such as figuring out the different parameters to vary. While Rubberduck couldn’t write the exact tests for me, it generated a useful base to get started.

As a bonus, it gave me easy input to query OpenAI, right from my editor. I was able to prompt random questions while coding without having to switch to my browser 🏆

I would conclude this article with 2 thoughts:

Rubberduck is open-source. I encourage you to give it a try, report bugs and suggest improvements. I'm now contributing to the tool myself, it's really cool!
I have used Rubberduck to help me write missing tests on existing code… But now I’m curious about what it can do to help me refactor the code! So that will be the topic of my next post about it. Stay tuned!

7 advice to help you inherit a legacy codebase

Nicolas Carlo — Tue, 15 Mar 2022 13:24:04 +0000

I wanted to make a recap of the most common and useful advice one can give to tackle Legacy codebases.

I have a few of my own, but I also went out to ask experimented developers about their opinions.

I ended up with 7 advice. Let's dig into these!

1. Stay curious

Approach the problem with an open mind.

Legacy systems are often frustrating to work with. Yet, you probably aren't the only one in this.

Ask questions to your colleagues, discuss with non-technical people, look up for information around. You may be surprised how much you can discover that wasn't obvious at first sight.

2. First, get it running

So many problems to address! An overwhelming mess! I know the feeling.

When you are dealing with many problems, the most efficient way I know is to start with one and focus on that.

Can you get the system running on your machine already? Start with that. Once you get it solved, it will help you tackle the other issues.

You can run it? Great! Then, get the tests running. There is no test runner? Put one in place. Then, make sure you can deploy new versions to production. And so on…

Pick one problem that will help solve others and focus on that. Get it done, then move to the next one. Don't try to tackle everything at once, that doesn't work well.

3. Take notes as you go

When you onboard on a new, legacy codebase, you are in a great position to notice problems!

As you learn the quirks of the system and how to solve them, take notes. These don't need to be polished and all. Raw Markdown notes would do.

Put these notes in a place where they are easy to search for. That may be Markdown files versioned along with the source code as we do for ADRs.

I find it useful to take notes for things like:

How I solved problem X (eg. I got this error when installing dependencies, so I did that and it worked).
Jargon, terms I'm not familiar with
The structure of the code, where to put things

A complimentary piece of advice is to train yourself to detect when you make assumptions.

4. Refactor as you go

This is very similar to the previous point, but turning the dial up a bit.

As you work with the code, your understanding will grow. You will have these aha moments. "Ooooh, that's what it does!"

You can capture these learnings in notes and code comments. I also recommend you take an extra step and improve the names.

Functions and variable names are important beacons for understanding. When something wasn't obvious at first and it finally clicked, take the time to improve the names so the intention is clearer!

This is easier to do with a fresh pair of eyes.

A technique I really like is called Exploratory Refactoring (or "Scratch Refactoring"). Allow yourself to do rogue changes in the code. You won't keep these changes, so you don't care if the code still compiles. The goal is to interact with the code instead of passively reading it. Timebox yourself for 30min. In the end, take notes of what you learned and scratch that work.

5. Read books

There is not a ton of resources on dealing with legacy systems. But there are a few that are quite good if you ask me.

Working Effectively with Legacy Code, from Michael Feathers, is a reference. I bet many of you have read it already. If you never had the chance, then it's the book I would recommend you to read.

If you have read this one already, I listed a few more that I found the most relevant.

Books, blog posts, talks, screencasts… These resources will expose you to new techniques and approaches. These will constitute a useful tool belt for when you are facing a new problem to solve, like identifying Seams in existing code so you can write tests for it.

6. Write the missing tests

Tests are particularly useful for 3 reasons:

Gain an understanding of what the code actually does
Reduce the risk of introducing a bug when making changes
Identify the main problems of the code design (eg. too many hidden dependencies)

Writing good tests is a skill to practice. Writing tests on existing code is even harder, because you may need to refactor this code… without tests to have your back!

That's where reading books and watching talks will help. You may already be doing Characterization Testing by intuition. There are more techniques and tools that can help you do it faster and safer.

Obvious candidates for me are Safe Refactorings and Approval Tests.

You don't need to write all the missing tests. Just write enough tests for the code you need to change. The time you invest here will be saved in the understanding you gain and the production support you won't have to do.

7. Make it better, not perfect

The most important advice of all, if you ask me.

This stuff is hard. It's overwhelming to work with an existing system, built on old technologies, patched everywhere, yet serving actual people in production! 🙀

It's fine to have a partial understanding of the system.

Your understanding will grow over time. You will have these aha moments. You can speed up that process by:

Talking to people
Taking notes
Using techniques such as Behavioral Analysis

Be mindful of yourself. This is a marathon. If you can make it just 1% better every single day, time will be on your side.

When you get overwhelmed by a problem, take a break and step away. To talk to people, take a step back. It may not solve your problem, but it may also give you new ideas.

I solved a lot of problems by simply "talking to others" about them.

Another good example of the "make it better, not perfect" mindset is this: consider you want to automate the deployment process (lovely). This can be an overwhelming task, with a lot of intermediate problems to solve…

Now, what about this:

You take notes of the steps that need to be performed to deploy
You put these steps in a script
You expose this script with a command, like if it would deploy it for you

This is a trick I learned from Arlo Belschee (I don't recall where, though).

The script won't do the deployment itself. In fact, it may just start with telling you the manual steps you need to do.

It's not fully automated yet but:

You now have a command to run to "deploy"
It reduces the risk of forgetting about a step
You can iterate to progressively automate each individual step.

Over time, this process will be more and more automated. But it doesn't have to be 100% automated or nothing. Only a Sith deals in absolutes 😉

What other advice would you give?

What's the difference between Regression Tests, Characterization Tests, and Approval Tests?

Nicolas Carlo — Mon, 23 Aug 2021 19:26:53 +0000

Before you start refactoring some Legacy Code, fellow developers would drop this advice:

Make sure it's covered with a test first!

Of course, this code isn't tested. Since you have little time and deadlines to meet, people like me would advise you to use a nifty technique: Approval Testing.

Approval Testing is the fastest way to put existing code under tests. You take some code, you throw a variety of inputs at it, and you capture outputs. Quickly, you build up a regression net that will tell you if you broke something when refactoring.

Sometimes, people are familiar with Working Effectively with Legacy Code from Michael Feathers. When that's the case, they usually pop the question:

Yeah, I've heard about Approval Tests. Is this the same as Characterization Tests? Asking for a friend…

For people who never read Feathers' book, "Regression Tests" seems to be a common name for approaching code that isn't tested. Is there a subtle difference? 🤔

Since I've been answering this question a few times already, I'm saving it here so we can all share it with our confused fellows 😉

Yes, Approval and Characterization Tests are the same thing

The reason for the different names is because there is no recognized standard—so far. Different people came up with different names for the same technique. In fact, most people seem to be using "Regression Tests" to refer to it!

What matters is the spirit of it.

First, you should understand the goal: have tests that fail if the system behaves differently.

The process has roughly 3 phases:

Get the code run into automated tests 👈 This typically is the hard part of the game. It involves cutting annoying dependencies. Feathers techniques will help you do that!
Capture interesting output. That may be the returned response. Or you may introduce a custom logger to capture interesting facts that happen during execution.
Vary inputs as much as possible to cover all possible scenarios. Test coverage can help you identify what you're not testing. I like to introduce deliberate bugs to verify I got at least one test fails.

"Characterization Tests" is the name that has been popularized among Feathers' audience. I think it's a good name, especially for Legacy Code.

"Approval Tests" is also a great name. I think it's even better.

Why? Because of what it suggests. Name matters. Let me tell you about the other names to illustrate that—yep, there are more!

The Golden Master and Snapshot Tests

If you've been around for a while, you may recall the Golden Master technique. Well, it's the same story. The goal is to capture existing behavior, so we can compare future changes against it.

"The purpose of characterization testing is to document your system's actual behavior; not to check for the behavior you wish your system had"

— Michael Feathers

The "Golden Master" name comes from the audio record-making industry, specifically the process of mastering. It's the one truth from which copies can be made.

While it embeds the spirit that what the software does is more important than what it should do, I don't like "Golden Master" as a name. To me, it implies you won't touch it, which I don't think is a good mindset to have when dealing with Legacy systems.

I prefer "Approval" because it suggests the behavior has been approved by a human, and we can change that. I think the human aspect is important to this process. We are in control of evolving the software. That means we have to make decisions on how the system behaves. "Approval Tests" expresses that!

The (unknown) importance of a good Printer

In September 2020, Emily Bache gave a talk on Approval Testing.

At 15:40, she presents the four parts of an Approval Test:

Arrange
Act
Print
Assert (compare with previous output)

The Print step matters to make these tests really useful. A Printer is what turns the output you're capturing into a string that will be convenient to compare new values with old values.

A good Printer will scrub irrelevant and flaky data, and format the output in a way that makes sense for a human to read.

For example, here's the output captured by an approval test in the Supermarket Refactoring kata:

----------------------------------------
Receipt date:           [date]
----------------------------------------
3 for 2(apples)                    -0.99

Total:                             -0.99

Notice how the data was formatted to look like an actual receipt?

Notice also that the actual date has been scrubbed (removed), because it may change between test runs.

And that's why "Snapshots Tests" also miss the point!

Snapshots have been popularized by Facebook and their testing library: Jest. They are useful when dealing with Legacy Code. They help you do Approval Testing.

The name "Snapshots Tests" focuses on the action of capturing the output, and doesn't suggest the importance of a good Printer.

I think Jest's snapshots are generally misused. It's common to reach a point where people just update snapshots without really understanding what's going on. Therefore, despite having tests, bugs appear because there's no easy way to tell whether the change is legit.

Although the misuse of the technique isn't Jest's fault, I think Facebook should have known better and re-use the "Approval Tests" name. Instead, they came up with their own 🤷‍♂️

"Approval Tests" will lead you to more resources

I would use this one over the other because the name leads you to convenient tools that help you do that. While you can write Characterization Tests by yourself, I recommend you have a look at approvaltests.com.

The "Approvals Tests" project has a lot of resources and documentation. It is implemented in a dozen of languages.

Going further than the approvaltests library, you may find products like Touca which aim to simplify the process for you. It supports applying Approval Testing at scale, fixing the typical issues you'd have collaborating with the captured outputs.

Call them "Approval Tests"

Now you know that the technique of capturing existing code behavior into tests has different names:

Characterization Tests
Approval Tests
Golden Master
Snapshot Tests
Locking Tests (thanks Maaret)
Regression Tests (that's how Touca describe itself)

Among all, I recommend referring to it as "Approval Tests". Acknowledge other names refer to the same technique.

Have a look at approvaltests.com and think about implementing a Printer to make your Approval Tests more useful.

Have you heard about another name for this technique?

Have you used Approval Tests without knowing it had a name?

Let me know! 🤠

Is it OK to change code for the sake of testing?

Nicolas Carlo — Tue, 18 May 2021 15:28:46 +0000

When you work with existing code that has no test, adding some is often a challenge. This code wasn't written to be executed in a different context than "production". It works™ under the right circumstances and environment, which are difficult to reproduce and automate.

I have techniques to get such code under tests. Some of them require to do minimal changes to the existing code, so we can write the tests.

When I present these, someone eventually raises this interesting question:

Is it fine that we create these extra parameters just for the tests?

That's a wonderful question to have! It shows that you care about the maintainability of the code in the future. You don't want to make the mess even worse, do you?

The code is in a poor state. We want to write tests so we can change it safely, faster. But doing so often makes the code looks… worse. How does it look like to turn this code:

export function getPurchaseExport(id) {
  // 200 lines of logic, http requests, logging, etc.
}

Into the same code, but with external dependencies passed as parameters?

export function getPurchaseExport(
  id,
  httpClient,
  sequelize,
  moment,
  logger,
  sentry
) {
  // The same 200 lines, but easier to test 🤷
}

Usually, the question pops at some point in your head. I think it's an important one to address.

Yes, it's fine to change code for the sake of testing

For once, I don't have an "it depends" answer to give here. I've experienced both approaches enough times to have a good opinion on this. Yes, it's OK to do the change so you can test the code. Even if it looks worse for now.

The rationale here is the following:

I don't know any other approach that would bring benefits in the future. Some tools may numb the pain for a while, letting you go in the wrong direction longer. Eventually, you have to come back from a worse state.
The remaining alternative is to… do nothing. That's dangerous. Doing nothing is the easiest approach, but it harms the codebase more because it makes it harder and harder to come back.

When it comes to Legacy Code, it's very easy to be paralyzed by the amount of work that is required to put things back on track. Ideally, you'd refactor the whole thing for a design that makes more sense. Maybe you'd rewrite all of it! In real life, you don't have time for that.

I've learned that you can't afford doing nothing because it wouldn't be perfect. That's not productive.

Don't make it perfect. Make it a little bit better, every day.

Why does it feel like we're making the code worse though? Because of the symptoms. Injecting dependencies explicitly to some code will make it clear that it does too much. We don't like seeing a class/function take 8 parameters. It will raise a red flag to most developers reading this code.

Yet, having 8 implicit dependencies is worse. It looks better because it's hidden. But that hidden complexity bites you whenever you need to change the code. It creates friction, preventing changes. The act of writing tests highlights this pain because you have to deal with the friction now. By making it visible, we create opportunities to address the root cause. Resist the temptation of hiding the problem.

You can preserve the API for clients

Note that you can always add some proxy, so you can start cleaning the code without impacting all the clients of this code:

export function getPurchaseExport(id) {
  return getPurchaseExportAndDoOtherStuff(
    id,
    httpClient,
    sequelize,
    moment,
    logger,
    sentry
  )
}

export function getPurchaseExportAndDoOtherStuff(
  id,
  httpClient,
  sequelize,
  moment,
  logger,
  sentry
) {
  // The same 200 lines, but easier to test 🤷
}

I did 3 things here:

I extracted the whole body of the original function into another one. This new function isn't used by clients and takes its dependencies as explicit parameters.
I gave an honest name to this function. It does more than getting the "purchase export". I don't know what yet, but I want to make that clear. I'll rename it as I understand this function better.
I exported it, so I can access it in tests. This is not ideal, but it is fine because it's my way out. If I'm concerned clients would start using this code, I can extract it in a different file that wouldn't be exposed at the module level. Or I could give it a more explicit name.

What matters is that we are addressing the root cause. Until we are done, we do compromises. It is fine as long as we make it clear, and we keep moving in the right direction.

It becomes worse before it becomes better

In her talk All the Little Things, Sandi Metz tackles the Gilded Rose kata. When she starts refactoring the code, she makes it look worse.

Actually, it doesn't just look worse. It is worse. The intermediate result is much more complex than the original code. And yet, this intermediate complexity leads to ultimate simplicity.

And here's the gist of it: you need to know (or believe) that the intermediates refactorings will lead you to simplicity to keep moving towards that direction. Otherwise, you won't dare to do the necessary changes, because they seem to make the code worse. You can't see it's only temporary.

At this point, here is the advice that helped me the most:

Practice in a safe playground. Use a coding exercise to practice refactoring techniques and experience how it feels—I've listed 5 great exercises here.
Clarify the end goal. Especially when working with a team. Explain what's the intention behind the intermediate refactoring, and how you plan to get there (roughly). Explain how the things that look worse, for now, will ultimately be simplified (e.g. responsibilities will be split, so this code will only compose other, well-tested functions).
Keep track of the work. Have an open issue for the remaining work. That helps to realize you already have work in progress before you decide to start another refactoring.
Make little refactorings part of your daily habits. Once you trust yourself that you will move the needle towards the end goal every time you touch the code, you don't need to keep track of the work so much. It takes time and practice, but ultimately makes working with legacy codebases less frustrating.

Give them a try, they may help you too 👍

7 tips for a successful code handover

Nicolas Carlo — Wed, 10 Feb 2021 11:32:25 +0000

Usually you would skim around this blog to learn tips and techniques to deal with some Legacy Code you inherited. In general, the original developers are long gone…

But, sometimes, you can be in a very particular position: a code handover!

Maybe you're inheriting a project from a different team and there's a dedicated knowledge transfer budgeted. Or maybe the original developer is still working with you, but he will leave soon.

Well you might even be the one who's handing over a Legacy codebase and you want to give the best chances to your successor!

Deadlines, deadlines, deadlines

In any situation, your time is limited. The codebase is huge. The technical debt backlog is bottomless. What should you document? Should you plan refactoring sessions with all developers? Is it smarter to focus on tests?

Imagine the last day of the handover: that's quite an event! What if you could live this moment relaxed, being confident that you set the team on the right track? You'll enjoy the very last moment of this transition as you'd have proven how efficient and professional you are!

But this moment is coming soon and there's so much to do… What should you prioritize now?

Hopefully, I've gathered 7 best practices that will help you go through this very particular period of your project.

7 best practices to hand over Legacy Code

1) Get everything under version control

Careful, this is very easy to overlook!

Get everything under version control, including the scripts that are on the leaving developers' hard drive.

Think about:

backup procedures
manual operations done periodically
manual operations done exceptionally
documentation of third-party softwares or APIs

Make sure you have all the passwords, licenses, contacts you need to keep maintaining the project.

New developers should be able to:

build the code
test the code
deploy the code

Make sure this is DONE before moving on!

2) Get management support

You don't have infinite time for this transition.

A great transition will make a tremendous difference in the new team productivity. That's crucial for the business!

Make sure that you get management on your side. Ask them to prioritize the knowledge transfer over "just fixing those last few bugs before they leave".

3) New developers should write the docs

Writing documentation can feel boring. Yet, it's really helpful and does wonder for passing knowledge!

The developers who will maintain the code after the transition are in the best position to document: they know nothing.

When you don't know, you have plenty of questions. This is where you can detect what's missing in the existing docs.

Original developers can't unlearn what they know, they won't realize things are not obvious to you!

Ask questions to the leaving developers. Document that knowledge. Make the leaving developers review the documentation.

Documentation can (should) take many forms:

formal documents
comments in the code to clarify the intent of the tricky bits
class and interaction diagrams at different levels of abstraction

You can make this massively easier by pairing: the leaving developer explains and the new developer writes the documentation. Reformulating is the best indicator of successful knowledge transfer 👌

4) Make it fast and fun for the leaving developers

Did you notice how I suggested making the previous practice easier? It's deliberate.

You'll realize that leaving developers' motivation will go down as their last day comes over.

Leverage that by making them do the exciting stuff. E.g. let them code while pairing with them if that's what they like.

5) Write more tests to understand the system

Most of the time, the system will be lacking tests.

Find out what are the critical parts of the application:

what is critical for the business?
what parts are often impacted by bugs?
what parts will be impacted by incoming features?
where are the Hotspots of the system?

Writing these tests is a great investment for your future work. Not only, but it will pay off later, it's also a great way to understand how the system works and discover the most important quirks you'll have to deal with.

Write more tests to understand the system and reveal potential bugs.

6) Try to predict where fixes should be made

Take the list of old bugs and practice this exercise with new developers: can they accurately predict where the problem was?

If they can't, you probably have identified important knowledge that needs to be transferred.

That's a great way to prioritize knowledge to transfer outside of doing a guided tour of the codebase.

You'll likely have to deal with subparts of the system that fail to stabilize. That is, bugs keep appearing in the same subparts of the code. Identifying these will make the new team much more efficient.

7) Don't refactor code without a specific goal

This final advice is more of a warning.

Sure, there is plenty of Tech Debt in this Legacy Code that's handed over. And maybe you've negotiated 2 Sprints of knowledge transfer. But spending this time refactoring the code isn't a good idea.

Refactoring code without a clear goal quickly becomes a bottomless pit that ends up using way more of your available time.

You can't afford it!

Efficient refactoring is oriented towards a goal: implementing a new feature, fixing a bug, etc. Don't refactor for the sake of it. Refactoring isn't cost-effective to transfer knowledge.

Focus on tests and documentation instead. This will set you in the best possible tracks 🚂

Remember the 7 practices

Next time you participate in a code handover, these are the things that will get you the most bang for the buck:

Get everything under version control
Get management support
New developers should write the docs
Make it fast and fun for the leaving developers
Write more tests to understand the system
Try to predict where fixes should be made
Don't refactor code without a specific goal

Transitions are exciting because they mean "change". I know you'll make the best out of yours 👍

Does refactoring Legacy Code pay off in your career?

Nicolas Carlo — Fri, 22 Jan 2021 02:14:54 +0000

Legacy software is everywhere. In fact, if you are working on code that was deployed to production and bring money to your company, you're certainly dealing with it every day. And yet, it's not seen as fun 😬

Most developers I've talked to were more excited to work on new features, new projects, new concepts, new integrations… Something new. Something to make. Mending existing code might be a necessary chore, but building new things is what's exciting.

Companies are looking for makers. Developers who will architect, design, build, and deliver the next generation of features with exciting new technologies! Shipping, with high-quality for sure, brand new products to customers and outperform the competition!

And there you are, stuck maintaining that 10-year-old software weeks after weeks. You wish you could try out some GraphQL like the cool kids. But you've to compose with these PostgreSQL databases that were ~~patched~~ fine-tuned over the years.

If only you could sell the idea of rewriting part of the app to use a more modern stack… Unfortunately, there's no budget for that. Some managers can't see outside of the "what works" area. Now you're afraid to be left behind with the tools and frameworks, relegated to minor bug fixes instead of honing your skills!

Now, you have been committed to writing high-quality, well-tested software. Relentlessly improving the status quo of this codebase. But this doesn't seem to lead to career advancement and that's frustrating.

Managers with non-technical background don't see the benefits of maintainable code!

I must admit that seeing developers who rack up mountains of technical debt get praised and promoted feels unfair, to say the least.

But the truth is: you've been honing extremely valuable skills across these months working on this legacy codebase. What you need now is:

To realize what these skills are
To learn how to communicate them to non-technical people

Let's go through this together.

The skills you've been practicing without realizing

First of all, persistence in front of challenges. Having to constantly go in and change unfamiliar, undocumented, and poorly tested code is hard. And yet, you have been doing that for weeks, months, maybe years! It's normal to feel like quitting, and most developers do. We're lucky enough to have the option to jump from a company to another every year or so. If you have been maintaining a codebase for more than 12 months, you surely have a good lump of tenacity in you 👏

After all this time working with a convoluted codebase, you surely have developed a spider-sense to detect code smells. I won't pretend that being exposed to bad code makes you a great developer—it doesn't, exposure to great code will teach you that. But now, you have experienced the maintenance issues some patterns can cause. I think that after spending weeks playing whack-a-mole with a bug that should have taken 2 days to be solved, you have a sense of how painful it is for the logic to be spread across the codebase. You may not know how to solve that issue, but you can anticipate this will happen, from experience.

Then, my own history of working with Legacy software showed me that you learn to think out of the box. Because side-effects can happen everywhere, and you've been burned a few times already, you have learned to ask relevant questions when change requests come by:

Is this bug occurring when there is only 1 element in the list?
Can this ID field change in the partner system for some reason?
What if I try to access this field with the keyboard?
etc.

Coupling is subtle and spread like fire. It makes everything more complex than it looks. Hopefully, this has made you a more discerning developer.

Finally, working with Legacy Code teaches you pragmatism. Because everything is broken and you have limited time, you have to make tradeoffs. Sure this codebase isn't perfect, but can it run decently and serve customers in production? Sure a lot of things are broken, but what should be fixed first? Working with such a system is learning to embrace better than yesterday over perfection. At some point, you have to focus on what matters: delivering value to people.

How to get credit for your work

OK, I suspect at this point you may be cheered up, but doubtful this would make a difference in the real world.

I've lost many jobs due to my wanting to push a culture of quality over sloppy hacks…

Have you heard such stories? Or maybe you have lived in such a situation.

So what? What can you do when companies seem to prefer developers who have a semblance of high productivity over those who really know about "craftsmanship" and "best practice"?

Actually, you can do something. It's not about the skills—you already have that. It's about focusing on what matters.

And this starts with you grasping something critical: it's not about doing craftsmanship, it's about being a craftsperson.

Let's be honest here:

Most employers don't care about craftsmanship. You'll hardly find job descriptions mentioning refactoring skills, despite these being critical to evolve existing systems.
Most developers ignore there are better ways than the way they've done things for years. There are always new tools, new frameworks to learn. How come making it harder to access the global state would help them ship faster?

And yet, I've seen a lot of folks willing to do better and fall for this last one:

Many software developers use "craftsmanship" and "best practices" as an excuse to over-engineer.

"Pushing a culture of quality" isn't the strategy to adopt, unless you're in a leading position to have a privilege. I don't mean you should stop writing automated tests neither.

My point is that the best way to spark a change towards a culture of quality is the same as getting credit for your hard work on Legacy Code: focus on the business and lead by example.

Focus on the business

It is usually the missing part in developers' argumentation.

Why all of this matter for the end-user? For the company? Dig into the root cause. Use the 5 whys technique to explore the potential impact of your actions. Maybe you can find data on how much some problem is costing the company (e.g. time spent on bugs per iteration).

Taking care of this Legacy Code has a much more direct impact on existing customers (and company wallet!) than a greenfield project—that is generally a risky bet. So use the data you have to demonstrate the value of your work!

Lead by example

It works, even if you aren't in a leadership position.

I don't mean being stubborn in forcing your coding techniques over the rest of your team. I mean introducing better ways progressively in the code you write, proving the added benefits of doing so are greater than the cost of changing The Way It Has Always Been Done™.

If your team doesn't like to pair-program as a rule, that's fine! Take the opportunity of a non-trivial question to dome some without naming it. If your colleague doesn't know how to test some part of the code, propose to spend a 30min timebox on this together. While you collaborate, share your tips of "doing it better" with them.

Actually, this is how I introduced my teammates to using automated refactorings: "I see you've extracted this variable. If you want, I can show you how to do that almost instantly with your editor. Let's try?". That felt like sharing a handy trick. After a couple of mentions across a few weeks, they started doing it by themselves. But most of all, they realized there were better ways to do that kind of work. This sparked the idea that there might be more… ☀️

Working on Legacy Code can pay off in your career

Having to deal with a Legacy codebase feels like a chore for many. Yet, it teaches you skills that are extremely valuable for your career.

There will probably always be opportunities to jump from greenfield projects to greenfield projects. "Makers" are everywhere.

There will certainly always be open positions to maintain applications that are running in production and serving actual customers. "Menders" are needed. Good ones are not so common, because it's so tempting to always chase for the latest technology, leaving very little time to focus on the basics of building maintainable software.

Being persistent, creative, and pragmatic: Legacy Code teaches you all of that.

You can't make it perfect, but you know how to constantly make it better. That's why companies will desperately need you!

All big, successful companies have a whole load of legacy. If they are still alive after all these years, they are not running on the latest tech. They require experienced developers to keep this working while growing. This is where you can make a key difference compared to any other candidate. "I'll rewrite everything with X" is less sexy than "I can migrate your system smoothly to new high standards". That is a valuable skill, generally overlooked.

Can you put an existing software into automated tests? Can you minimize risks while making changes to such software? Then you'll have plenty of jobs. Legacy code isn't about COBOL. There are already plenty of Rails and JS legacy codebases. Not everyone can afford to rewrite everything from scratch…

Working on Legacy software does pay off. You need:

To realize what you learned, what you are capable of
To explain these skills to non-technical people, focusing on your impact on the business
To inspire your fellow developers by solving the annoying problems with your techniques

Don't stay alone

My final advice would be to get in touch with other developers like you. You'll find plenty of great persons you can talk to in these 2 communities:

If you join there, come and say hi to me 👋

In any case, take care of yourself!

When is it the best time to refactor?

Nicolas Carlo — Tue, 01 Dec 2020 13:13:27 +0000

One of the hardest things to teach is probably when to refactor.

With expertise, that's an answer you may feel in your guts: sometimes a piece of code should be refactored, and sometimes it doesn't worth it. But it's hard to stand by the "gut check" when you have to justify your score in a poker planning session!

Different approaches, different problems

Some people follow the TDD cycle: red-green-refactor. There's a dedicated moment for refactoring the code that has just been written.

But what about Legacy Code? When you inherited a large pile of tangled code, how do you know the cost of refactoring would be less than the cost of not refactoring?

You can't calculate these costs. You can at best estimate them. People can do a decent job of comparing things, but we are not very good at estimating them. How do you know if the code you see will have to change? How do you estimate the number of bugs it will generate? Just like all estimation techniques, there are theories, sophisticated algorithms to try to tell you how much hours each code smell would take to clean up.

But it's not because some code could be refactored that it should be refactored.

What's good in refactoring if it takes you more time to deliver the value every single time? If you constantly argue that the Story the team sized is larger because you need to refactor everything that might come across, is it really helpful?

On the other hand, if you're barely able to see the whole function in your big widescreen monitor in portrait mode, then any change to this code is considerably longer and riskier to make.

Could there be some common understanding of when to refactor, so you don't waste time doing it when it doesn't worth it, but don't skip it when it does?

Refactor continuously, but only what needs to be

I think the best moment to refactor is before you need to change the code.

You shouldn't refactor every tangled spaghetti you're coming across. You don't have to refactor the first time you write the code. You can't just plan days of refactorings for the sake of it.

Instead, make refactoring part of implementing the feature or fixing the bug. It doesn't mean you should size every story larger because you want to refactor everything first—although you should probably add some points if you don't have a clear understanding of the change that needs to be made, that's another topic.

Refactor continuously. But only refactor what needs to be.

What if you don't know the code enough to see what needs to be refactored? I suggest you first do some Exploratory Refactoring to learn more about it. You won't keep these changes. You do that so you have a clearer idea about what actually needs to be changed.

Wouldn't that make the code inconsistent if you only refactor a part of it? It probably will and that's fine. Your team can keep track of ongoing refactorings in a Markdown file, versioned with the source code.

The rationale is that if you have a hard time selling refactoring time to your stakeholders, you really shouldn't. Refactoring isn't a separate activity, it's part of your work. It doesn't mean that every single task should take twice the time because you will clean up everything. Take a pragmatic approach, refactor as you go, focus your refactoring on what needs to be changed.

Don't ask for time to refactor. Don't make refactoring sounds like a constant waste of time because you wouldn't be able to work on the existing code. You refactor because it will make changes faster and safer. Don't get side-tracked by all the refactorings that could be made though. Practice cleaning up what's in the way, and only that.

The best way to start testing untested code

Nicolas Carlo — Tue, 17 Nov 2020 12:53:05 +0000

Are you working on a large codebase, with no test whatsoever, and no time is given to add some?

"Automated tests will help you work faster", people say, "they reduce your time-to-fix when an issue occurs". But when there's no test, someone needs to take hours to write them first. If you ask management, of course, there's no time for that: the project is already behind schedule! Next iteration, maybe 🤷‍♂️

The truth is: you are not really sure where to even start.

Should you write E2E tests first, so you can cover more critical cases faster? Or should you focus on unit tests, because you've heard they are faster to run and more useful feedback while you're coding? Or maybe something in between?

Changing Legacy Code is a specific situation that's very different from writing new code from scratch. You need to adapt your approach.

Hopefully, there's a way to quickly add tests on the code you need to change, so you don't even need to ask management permission first.

E2E tests or unit tests?

End-to-end (E2E) tests exercise your application as a black box, from the end-user point of view. They typically require your app to be built and run in a production-like environment.

At first sight, you may think that they would be ideal since they would cover a lot of code with every single test.

In practice though, I find they take a lot of time to set up if you don't have some already. They also tend to run more slowly than you'd like and fail for many different reasons. Therefore, they are most suited to run on a CI than on your own machine while you're coding.

Their feedback is too long to be useful during development.

"Unit tests" definition is more debatable. I've found that people think differently about what a unit is. I prefer to talk about isolated tests. Nevertheless, I'm referring to the tests that are closer to the code, run fast, and are reliable.

Such tests are great while developing. Done properly, they don't focus on implementation details and help you guide the design of the code. They provide you with extremely quick feedback on the code. At first sight, they seem the companion you need to deal with this legacy!

In practice though, it's hard to retrofit existing code into unit tests. It's easy to get the "unit" level wrong and test implementation details you wish to refactor in a moment. Trying to force unit tests on code that wasn't designed with test feedback can be time-consuming and solidify the design too much: tests get in the way, making your refactoring attempts even more difficult!

Legacy Code is a special beast. In general, the most practical approach should be more nuanced. E2E or unit tests aren't the best candidates.

So, what's the best way to add tests on untested code?

Close to the code, but not too much

In this old-still-gold video on "Testing and Refactoring Legacy Code", Sandro Mancuso is showing you how to address existing, untested code that's doing calls to a database:

In particular, he presents a strategy for adding tests to Legacy Code. I think it is excellent:

I generalized this tip and recommend the following mindset:

Start testing from the outside of the code, then refactor from the inside.

OK, let's detail that a bit…

Start testing from the outside

You don't know much about untested code.

Start writing tests from the highest level you want to work with is easier. It also helps you not tie tests to the implementation details too much. It's a similar mindset to the E2E tests, except that the level can be anywhere in the code that you need to work with.

The key idea is to quickly build a safety net to ensure that you capture the existing behavior of the code. The best way to do that is to use Approval Testing.

Approval tests are also known as Golden Master, Characterization tests, or, more recently, Snapshot tests.

Whatever name and tool you use to write them, Approval tests are composed of 4 steps:

An Arrange step, where you set up what's necessary to run the test
An Act step, where you execute the code
A Printer step, where you generate an output string from the effects of the code
An Assert step, where you compare the new output with the existing one

It's like writing unit tests, but much faster. You don't need to understand everything the code is doing at the moment. But you're building a test suite you can rely on, so you can safely refactor code and know instantly when something goes wrong.

It is a perfect compromise.

Refactor from the inside

Once you have covered enough cases to be safe, you can start refactoring code.

The simplest (and usually fastest) way to do it is to tackle small chunks. Extract meaningful parts of the code that's covered with tests, and redesign them.

Because the tests are more coarse-grained, you have more flexibility to craft new classes/functions. Tests don't get in the way, but they verify the overall behavior still works as expected. You can even replace some code with a whole new function that produces the same result!

As you're refactoring the code, it's a good time to write better tests for the "units" you extract. It's easier to do because the chunks are smaller. These tests will capture your understanding of what these smaller pieces do, making it easier to work with this code, and potentially re-use it in a different context.

When you don't have much time, capturing the behavior of existing code with Approval Tests is the best way to get started. Pick a coarse chunk of code that covers what you need to change. It will help you refactor this code safely so you don't waste time debugging when something breaks.

With practice, doing this may even be faster than not testing at all and getting things done with no bug!

What you can do when code is really hard to review

Nicolas Carlo — Wed, 04 Nov 2020 11:06:50 +0000

Code reviews are one way to improve the quality of the code by having more than one pair of eyes on the code. When you're dealing with a Legacy codebase, it's a helpful practice to spread knowledge, get more people familiar with the codebase, and reduce risks of breaking things.

But sometimes, you may stumble upon a very hard code review. Maybe the pull-request is big, or the change involves many parts of the code.

Maybe the change isn't really big, but it's a risky one!

You feel that this code is fragile and complex. You don't feel safe poking too hard on it. The team doesn't know much about the possible side-effects of such change. The test suite isn't really exhaustive neither. The change seems to work, but it would take an excessive amount of time to verify the absence of any regression. Perhaps even more time than what it took to do the change itself!

So here's the dilemma:

Do you try to spot obvious flaws and get it merged, hoping nothing slips through the cracks of your imperfect knowledge?
Or do you block the merge until you've tested extra-thoroughly the change against all scenarios you can imagine, risking the deadline?

What do you do when code is too hard to review, but you don't have time to do everything again?

If it's hard to review, it can't skip the review!

First, if you feel the change is so difficult that you won't have time to review it properly, then it's the very last code you should spend less time reviewing.

"Merge and cross fingers nothing bad happens" is a notoriously bad idea. When doing so, you rely on luck. When you're out of luck, you fail. Some day, you fail badly.

Think about it: if the code is hard to review now, it will harder to maintain in 6 months! You'll lose the context around the decisions that were taken here. Changes without context stay untouched because we're afraid of unintended consequences! Hopefully, there's a simple solution for that.

Yet, it's hard to come up with a different approach when you're under the pressure to ship.

What can you do if you don't understand the impact of the change you're reviewing? I think you can reverse the steam and move the project in the right direction if you follow these 5 tips.

5 tips to review difficult code changes

1. Focus on readability

I always introduce the Software Crafters Montreal meetup with a slide saying:

Code is not how you tell the computer what to do; it is how you tell another programmer what you want the computer to do.

I think this makes the difference between something that works and something that can be changed.

Therefore, the first thing you should focus on in your review is readability. Do you understand the big picture of the change? Forget about trying to find all possible bugs. Do you get what's going on here? What is the problem being solved, and how your team has decided to solve it with this change?

2. Ask for proofs

Can you see what will happen to the software after this change? Do you understand the intentions behind the changes?

Manual tests could do. Often, a legacy codebase requires quite a lot of changes before you can add automated tests. In the short term, it might be faster to simply test manually. Before/after screenshots or a video reproducing the scenario in the PR description will do a lot more than trusting the code. Doing a QA of changed code before you merge it, while it's fresh, is a great moment to spot issues.

Focus on that, require to see the QA of the final result.

3. Ask for automated proofs

Manual tests could do, but if you really need to save time you should automate these tests. You won't have time to manually reproduce all scenarios every time you do a change. This grunt work is for the computer. This is why you should push your team to get the code under tests.

If there is no test, ask for them.

If there are automated tests along with the change, make sure you understand what they verify. It's easy to write tests for the sake of it and overlook their readability. Good tests will help you understand what the code is supposed to do. They act as living documentation of the software.

What if code needs a lot of changes to be testable and you don't have time? If you don't know how to do better, go on survival mode: do the best review you can do, and fall back on manual tests. But you should make a top priority for the next iterations to address this issue.

Among all the technical debt you want to pay off, being able to write automated tests on the codebase is one of the most important ones! It's the one that will save you time and make deadlines easier to hit:

Automated tests will execute faster than you can manually test
Changes covered with automated tests will less likely introduce a regression, so you spend less time debugging and fixing code
Tests will pressure you to improve the design of the code—pay attention to what's painful to test, and make that easier

4. Ask for smaller changes

A common clue for difficult code reviews is when the diff is more than 500 LOC, yet the description is as tiny as "Implement cancellations".

In such a case, ask the change to be broken down into smaller chunks. A series of small, self-contained changes is less risky. One formula that usually works is to first ship all the updates that don't change how the code works, then ship a smaller change that implements the feature.

Here's what you can say:

I'm having trouble following everything going on here. Could you break this patch into smaller steps?

It will also make previous points easier. That's less room for mistakes!

5. Speed up the process with a synchronous code review

Here's the secret trick if you can't afford spending days on this review: take notes and go through the review with the author synchronously. Do it IRL or schedule a call, share your screen, and go through it together.

It will help you understand the logic faster. It will help you discuss the trade-offs. It will help you explain what is critical to be changed vs. what is merely a nice to have.

Having a synchronous code review is a great way to save all the waiting time between each back-and-forth of asynchronous comments. The discussion is smoother and the end result is much more productive! You will come up with a concrete plan to get this merged safely.

Bonus tip: spark the change!

If you have the power and the budget, you should really reject such a change. If the level of quality can't be proven, it's a very risky move to get such an uncontrolled change in the codebase.

Maybe you're in a Senior or technical leading position. If so, you can help your team do a better job on the log run. You can make the situation progressively improve instead of letting things getting worse and worse. What will happen if you don't hit the deadline? Does it worth delivering software that may or may not work? Can you buy some time to clean things up for the next iteration?

Investing in people understanding the code pays off over time.

If you don't have such power, you can use my 5 previous tips to deal with such changes. Don't block, but ask the right questions. Focus on what matters: not just producing code, but delivering value to the end-user.

The good part of Legacy software is that you already have people depending on it. So anything you do to improve it will have an actual impact. It's your opportunity to use your skills to help people. I find that to be a motivating challenge, don't you?

Approval Tests, TCR, and Menders

Nicolas Carlo — Mon, 28 Sep 2020 12:24:41 +0000

On September 9, 2020, I co-organized the 3rd edition of The Legacy of SoCraTes with my partner in crime Adrian Bolboacă.

It's a half-a-day remote conference where 5 speakers share their tips to deal with Legacy Code.

This edition had an accent on bringing existing code under tests. We talked about Approval Testing a lot. I have a dedicated page for that. These talks are a great opportunity to learn more and see it in action.

Without further ado, let me tell you how each talk can help you deal with your existing codebase.

A deep dive into the Gilded Rose

Barney Dellar kick-started this conference by demonstrating the Approval Testing technique, on the Gilded Rose refactoring kata, in C++.

I like this talk because it shows the technique in action, on concrete code.

Also, Gilded Rose is an exercise I recommend doing if you want to get better at refactoring existing code.

Watch him get the code under test in no-time:

Approval Testing

Approval Testing, again?!

Well, this was intentional. Emily Bache follow-up on Barney's talk to go further with the techniques.

She has been using this technique in lots of situations for many years. It's especially useful when working with legacy code. But there are common pitfalls to avoid and best practice to learn.

Oh and if you don't know Emily, she's the expert in coding exercises. Most of the refactoring katas I would recommend can be found in her GitHub, in many different languages: github.com/emilybache?tab=repositories&q=refactoring

Watch her talk to see how she compares unit tests and approval tests methodologies:

Did you noticed she uses a similar definition of Legacy code of mine:

Valuable code that works, developers find it difficult and we need to change it safely.

7 Techniques to understand Legacy Code

This time, these techniques are not from me 😁

Jonathan Boccara is the author of The Legacy Code Programmer's Toolbox. It's one of the rare books that give you practical skills when you have to deal with Legacy code.

In this talk, Jonathan presents 7 techniques that can be handy:

Know your I/O frameworks
Find a stronghold
Analyze call stacks
Start reading from the outputs
Count words (I wrote about this one in details)
Filter on control flow
Scan for the main action

See them in action in his talk:

TCR is your best friend against legacy

Test, Commit, or Revert (TCR) is a technique that was popularized by Kent Beck. You can think about it as a variant of TDD, pushing the programmer to do very small changes.

My friend Khaled Souf had the opportunity to actually use it on a Legacy rescue project. In this talk, he shares his experience and explains why it's really efficient when you're dealing with existing, untested code:

Makers vs Menders

Finally, I had the pleasure to host M. Scott Ford to talk about developers' mindsets.

We're all familiar with "makers". But it takes a different state of mind to really enjoy mending code.

Scott presents 4 different traits:

The rapid prototyper
The feature builder
The emergency responder
The software remodeler

He goes into the details of what are the strengths and weaknesses of each, and in which case each is best suited.

I personally don't like putting people inside boxes. We made that clear with Scott as we discussed how each of us can be in many boxes at the same time, or switch depending on the context.

This is a really inspiring talk I recommend you to watch if you feel alone, fighting with your codebase:

If you want to connect with other menders (like me!), I recommend you join the Legacy Code Rocks community. There is a Slack group and a weekly virtual meetup.

If you join there, come and say hi to me 👋

"These talks are really great, can we have more?"

Sure thing!

There's no fixed schedule, but I like to organize this short conference every few months. I reach out to a few speakers to build an interesting agenda, then we set up the event.

Everything is FREE. Attending is the opportunity to directly ask questions to these speakers, from the comfort of your home.

If that's something you'd like to hear about, subscribe to my newsletter below 👇

I publish my weekly tips on Legacy Code here. I also tell my subscribers in advance about the conference and the agenda, so they don't miss out. Join them!

4 developers working their way through Legacy Code

Nicolas Carlo — Thu, 24 Sep 2020 12:57:38 +0000

I'm wondering how my way of working through legacy code may be different from someone more seasoned…

Ooooh I feel you. That is a very, very common thought to have my friend 😃

In fact, most developers are only exposed to Legacy Code at work. You have to maintain an existing codebase because that's what is bringing money to the company. After all, Legacy Code is valuable code you're afraid to change.

If you are lucky enough, you'll be working with seasoned developers who can teach you their tricks. But this is not very common.

So how do you know if you're doing it right? What can you do to make it easier to deal with Legacy Code?

Let me rephrase that: what can you do to make it easier to perform working on Legacy Code?

You practice.

Have you ever felt impressed by how acrobats make their moves look so easy, but you know it's damn hard?

The thing is: performers spend most of their time practicing.

We, developers, spend most of our time performing. Little time is dedicated to actual practice. You know, trying some new moves, repeating them, perfecting them. Exercising our skills. Making it so when we are on the battlefield, we instinctively know what could work.

Practicing is key in making Legacy Code easier to deal with.

Let's briefly cover the few questions that are popping in your head:

Where to practice? On coding exercises that are designed for this. Here's the list of 5 refactoring katas I recommend.
When to practice? We don't all have the luxury to take on our personal time to do so. But your employer certainly has to train you every year. They have a budget to spend on this! Use it to develop skills that will help you in the long run 😉
What to practice? This is a very good question. Practicing is good, but it's better if someone can show you the moves…

4 developers who show you what to practice

When you're trying to learn new skills, it's a good idea to be looking for exposure to more experienced people on the topic.

If you're a visual learner, then it can be really helpful to see someone practice. Personally, it helps me understand how to apply techniques in front of actual code. Theory is good, but I prefer when it gets in contact with reality.

It's not easy to find good resources online on that very topic.

To help you save some time (and get you into practicing), here's my selection of 4 resources that will expose you to seasoned developers working their way through Legacy Code!

"Testing and Refactoring Legacy Code" from Sandro Mancuso

This one is in the "old, still gold" category 🏅

It shows you useful refactoring moves on the Trip Service kata, dropping some tips doing so.

A 6-parts series on refactoring Legacy Code, from Jovche Mitrejchevski

I really like this one because Jovche is demonstrating how to behave in front of a code you never saw before.

A lot of good refactoring techniques are presented in these videos, including different approaches to the same problem.

A step-by-step refactoring in 2 parts, from Dave Farley

You may already know Dave Farley. He's the co-author of "Continuous Delivery". You can tell he's a very seasoned developer.

In these 2 videos, he shows how to address Legacy Code. Specifically:

Using Approval Tests to quickly cover existing code with tests
Removing clutter from the code with basic refactorings
Reducing complexity by extracting methods that got composed together

Refactoring techniques, from Gregor Riegler

Gregor has a few videos like this on his channel.

I like how they are straight to the point and digestible. It shows you a technique (e.g. Subclass & Override) that you can reproduce on a refactoring kata I mentioned previously.

Do you have videos you recommend for learning to work with Legacy Code? Let us know in comments 👇

Where should you put the documentation?

Nicolas Carlo — Wed, 09 Sep 2020 20:27:26 +0000

"Of course, there is never any documentation…"

This is the typical experience of a Legacy codebase.

Sure, documentation is not fun to write. It often feels like a waste of precious time, because we know. The problem is: we forget.

Documentation is never a priority for your project stakeholders. There are features to ship and bugs to fix! And then, one day, the lead developer who was working on the project for the last 5 years announces she's leaving in 2 months. You now have 60 days to learn how to live and work without her.

So you start thinking about documenting everything she knows, all of the implicit knowledge. And very quickly comes up the question:

"Where should we put the documentation?"

Should it be specific to the team? Should it live in the company's Confluence? Should you use the wiki that comes along GitHub/Gitlab? Are there tips that can help you? Traps you should avoid?

Let's pause a moment here to think about what makes great documentation.

The 3 traits of great documentation

1. Easy to find

If knowledge is spread across wikis, google docs, emails, and Jira tickets, people will have a hard time finding the information they need.

How do you know if documentation is easy to find? Well, there's a typical symptom when it's not: people frequently ask questions about things your team knows. Often, questions are the same. Because it's not clear where they can find the information by themselves. Hunting for the answer will take longer (and more efforts) than raising the question and wait. That's a waste of time. Good documentation would fix that!

When it's clear where the documentation is, and it's easy to search for specific information, you see less of that.

Aim for having your documentation in a single place. A powerful search feature is a must.

2. Easy to read

Basic formatting matters (and is usually given by most tools).

But think about images, diagrams, and code. Are they easy to read? Is there a way to make some information stand out? Is there a way to hide details that would not be relevant for all readers?

Adding information is simple. Keeping it relevant takes practice. Going to the point is a hard skill to earn, but it definitely worth it. Remove from your documentation what turns out to be unnecessary.

Keep information short, easy to skim. Make people practice writing docs.

Speaking of writing…

3. Easy to write

This is usually overlooked, especially when it gets spoiled by heavy validation processes.

Let me be clear on this one:

The more friction you add to the process of updating the doc, the less updated it will be.

Out-of-date documentation is untrustworthy, making people fall back to asking others instead of trusting it. Out-of-date information should be updated or deleted.

Get rid of useless information. Add missing ones. Make newcomers update the onboarding documentation. Take questions as chances to clarify the doc!

If updating the doc is frictionless, you'll save your team a loooot of time.

You don't need a validation stamp from the Tech Lead to update the doc. Anyone should feel entitled to do it.

2 things can help here:

Change history will make you stop being afraid of losing information by someone accidentally removing it.
Frequent backups, in case something goes wrong

Trust people in doing their job. Let them update the doc as if it was a living document.

OK, but WHERE to put the documentation?

Following these 3 traits, I'd recommend keeping your documentation in 2 places:

Everything that is codebase specific should be versioned along with the source code. Instructions to help people work with the codebase should be here, in some README. I recommend you to document your architectural decisions here, with ADRs.
Everything else should be in a single place. The exact solution doesn't matter. What matters is that it's consistent.

That is, if your company uses Confluence, then use Confluence. If there's no company standard on this, then make a decision for your team. Make it clear that all the documentation would be here.

What would be the best solution for documentation?

If you don't have a place to put your documentation yet, or if you've decided to migrate from your existing solution, you should be looking for something that makes documentation:

easy to find
easy to read
easy to write

As of 2020, I recommend going for Notion. It ranks best in most of the characteristics that matter for building handy documentation, especially if you're working in software.

Listing all reasons why Notion is my preferred choice would fill another blog post. But in short:

their search feature is powerful enough to make it easy to find information
they handle formatting, embedding images, code syntax highlighting, calling out important bits, and hiding less relevant ones
the content is editable in place, making it a breeze to write
anything can be turned into anything else, so you can just dump what's in your head and then turn it into a more sophisticated page later
comments and assignments make it great for collaborating
they backup your content with change history, so you can go back in time if some information was deleted

If you're curious, have a look at my showcase of useful Notion features for building company documentation.

Regardless of the solution you choose, don't get caught with perfectionism. Done is better than perfect. As long as you start putting useful knowledge in a single place that anyone can update, you're doing great!