When we find a new bug in our applications or a test randomly fails in CI, frequently, the bug came to exist as the unintended side effect of another change.
There are a lot of blog posts and documents out there about avoiding these unintended consequences, but since none of us are perfect it's important to know how to diagnose and locate the root cause of this category of bug.
The good news is, if you're using git, it's easy. If you're not using git, stop reading this and go run git init in your project's directory.
Because developers frequently need to find the change that introduced an undesired behavior, there is a git command called bisect. Bisect means, "to cut or divide into two equal or nearly equal parts." That's pretty much what this command does.
Bisect means, "to cut or divide into two equal or nearly equal parts."
Before we get into git bisect, let's set up a basic project to illustrate this problem. I'm going to do this with Ruby, but it's not super important what language the example is in.
If at any point, you are confused by the code in this article, just skip it. I'm just using it to explain why you might need to use git bisect.
I'm going to make a directory and run git init, then I'll write a function and test for that function. That's our starting point.
mkdir bisect-example && $_
git init
vim main.rb
# main.rb
def square(number)
number * number
end
# Testing Code
def assert_equal(actual, expected)
unless actual == expected
raise "Assertion failed: #{actual} does not match expected #{expected}"
end
end
def test_square
assert_equal(square(2), 4)
end
test_square
So, we've got a function called square that has one parameter and it does what you think it does (multiplies the argument passed to it by itself). We've got a couple of different functions testing the square function.
Everything is good, so we make a commit.
git add .
git commit -m "Add square function"
The world is in perfect balance until we realize that math has changed, so our stakeholders come to us and say, "With this new math when you square something, you aren't allowed to multiply it by itself, you need to use addition instead."
So we refactor to our function.
# main.rb
def square(number)
result = 0
number.times do
result += number
end
result
end
...
Now we have this gross implementation, but our stakeholders are happy and tests are passing. So we commit it.
git commit -am "Refactor square function to meet requirements"
This happens a dozen more times (and our coworker who doesn't see the value in good commit messages does some of the work), then some new features are added to help us get extra VC. Now the project is really hard to wrap our heads around, but because we are smart developers we wrote tests! We can rely on those to help us identify when there is a problem.
One day, we are happily working in our main.rb file and we run tests to find that somehow our square function is no longer functioning the way our tests expect it to function! Oh no!
main.rb:15:in `assert_equal': Assertion failed: 5 does not match expected 4 (RuntimeError)
Our current changes aren't even touching the square function, we are working on a whole new function called overthrow_humanity which doesn't even use square. Because we are smart developers, we decide to look at the git history for insight into how this could have happened:
git log --oneline --no-merges
f25411d9e9 Add README
9a8225f6b3 Remove unused code in RobotOverlord#simulate_mercy
708a4955e0 workaround for legacy T-800 model
9a2f2805f6 this probably works
360689a0ec Add missing tests for AI features
cfcb762e91 fix whitespace
32ca0fa720 implement features because PO said to
26dd1ab393 Fix bug in cube function
11e118d981 Fix bug in square function that causes hostility
01da7be780 Add test coverage for RobotOverlord#enslave
59415871b2 Add dependency on skynet
31bb9ac6c2 do some work
92f4123391 Add RobotOverlord class
a1e8a90763 Improve test coverage for math functions
069ab50172 Refactor square_root? function
18512a7fef Add square_root? function
a15700afb3 Add cube function
c25f8ef431 Refactor square function to meet requirements
12e4ebffce Add square function
The last four or five commits don't seem to be very helpful in diagnosing the issue, in fact, at a glance none of them even touched the main.rb file. That doesn't mean those changes couldn't have unintended side effects, so we need to identify which change actually broke the test.
We have finally arrived at git bisect.
The first step in fixing the code that broke this test (we are going to assume the test is right), is identifying how exactly it got broken. In one of these commits, a piece of code was added to the project that broke this test. How do we determine which commit housed that code? Easy, run the tests on each commit until we find the one! What if it was 200 commits? Not quite as easy.
The command git bisect takes advantage of binary search to quickly find a commit that introduced a given bug. Let's test it out on this project.
git bisect start
We've begun our bisect, but we need to tell git that the current state of the project isn't working as intended. To do this, we mark the current commit as bad.
git bisect bad
The next piece of information git needs is a commit where we know our code was working as it was intended. Well, we remember running the tests and being satisfied the first time we had to refactor the square function, so let's go with the commit where did that: c25f8ef431 Refactor square function to meet requirements. We need to tell git, that c25f8ef431 was good.
git bisect good c25f8ef431
You should see a response that resembles this:
Bisecting: X revisions left to test after this (roughly Y steps)
[some_commit_hash] Some commit message
Now it's as simple as walking through each revision, running our tests, and marking those commits as good or bad.
When you've run your tests and identified that the commit you're on is good, just type: git bisect good. When you find a commit that is bad, type: git bisect bad. The software is going to do the hard work of finding the commit that actually broke our test.
In our example project, we are going walk through several revisions and we will find that in one of our commits (11e118d981 Fix bug in square function that causes hostility) someone messed with the square function.
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[11e118d981] Fix bug in square function that causes hostility
When we look a little deeper to see what changed in this commit, we will see that some developer (probably us) tried to fix a bug elsewhere in the project by modifying our square function.
# main.rb
def square(number)
result = 1 # 🚨 This is the bug!
number.times do
result += number
end
result
end
...
Now we know exactly what caused our bug. We can ask git to take us back to where we started the bisect by typing:
git bisect reset
Now we just have to fix the bug and commit the change.
That's how you use git bisect to take advantage of binary search and find a commit that introduced a bug! This is a contrived example, but it really is that easy.
It's on you to write tests that allow you take advantage of git bisect in this way.
Top comments (13)
But what if I am not trying to find the transition between a good and a bad state. What if I am trying to find some other change, like when we added some particular function?
Well, the "test" to execute would be 'git grep theFunctionName'. But the 'good' followed by 'bad' is exactly the opposite logic to use.
You can change the names of what you use to indicate the transition. In this case:
git bisect start --term-old=dne --term-new=exists
git bisect exists
git bisect dne someOldhash
git grep -q theFunctionName && git bisect exists || git bisect dne
Repeat the last command until you get output.
Cool! Yeah this is a contrived example, but for people unfamiliar it's useful. The cool thing about git is that it can be used really creatively to solve problems like this really quickly.
Thanks Jacob, really cool blog post! 👍
git bisectis great, but it's even better when combined with micro commits. This really helps to pin down faulty code.Thanks for explaining it so well! I really struggle to understand when I'd use this in my workflow, and you did a great job showing examples of that.
So, if all the tests ran on every commit, this would be less useful- unless they were something like performance tests that were relatively subjective.
Usually I try and find the bug, then look at the history for that line or file to see why that change was made, in effect doing almost the opposite of bisect.
I'll try and use bisect, see if it improves my!
Yes, the real beauty of bisect is that it does a binary search, which makes it way faster and easier to find the broken commit
Git bisect is amazing. It has saved me several times!
I think it's great, but I've found a lot of developers are afraid of it because it seems complicated.
It definitely intimidated me when I was first getting familiar with Git. But once I was trying to find a big one commit at a time, and decided to give it a try. Now it’s my go to tool for finding a bad commit!
Nice summary! Gives insight beside listing the necessary commands. Would be nice to have a part 2 that shows the automation of the whole thing by passing a shell script to bisect.
Very easy to grasp! Thanks
Great tutorial - very easy to follow. Thanks!
Thanks so much for this!
No problem, glad someone finds it useful. 🤠