Automating It: git bisect run

Learning Objectives

By the end of this lesson you will be able to:

• Write a test script that returns the exit codes bisect expects
• Run a complete unattended bisect with git bisect run
• Use exit code 125 to skip commits that cannot be built
• Recognise which bugs can be automated and which cannot

The Contract

git bisect run takes a command and runs it at every step, reading its exit code instead of asking you.

Anything that follows that convention works, and most test runners already do: they exit 0 when tests pass and non-zero when they fail. Which means, very often, the automation is one line.

The One-Liner

If you have a failing test that captures the bug:

git bisect start
git bisect bad
git bisect good v4.1
git bisect run npm test -- tests/payments/refund.spec.ts

Then walk away. Git checks out a commit, runs the test, reads the exit code, narrows the range, and repeats until it prints:

4c1f8ab2e94b7d0c1f5a2836b4c9d0e1f2a3b4c5d is the first bad commit
bisect run success

Nine builds, no interaction, one commit identified.

Write the test first. A failing test that reproduces the bug is worth writing even if you were not going to automate — it makes the bug objective, and it becomes the regression test afterwards.

The Script Version

When the check is not a single test command, write a script. Anything is allowed as long as it exits correctly.

#!/usr/bin/env bash
# bisect-refund.sh — exit 0 if the refund total is correct, 1 if not

npm ci --silent || exit 125          # can't install deps here: skip
npm run build --silent || exit 125   # can't build this commit: skip

RESULT=$(node scripts/calc-refund.js --amount 100 --discount 20)

[ "$RESULT" = "80.00" ]              # exit 0 when equal, 1 when not

Make it executable and hand it over:

chmod +x bisect-refund.sh
git bisect run ./bisect-refund.sh

Two details worth internalising.

Keep the script outside the repository, or ignore it. Bisect checks out old commits, and a script tracked in the repo will vanish at any commit made before you wrote it. Put it in /tmp, or add it to .gitignore so it survives every checkout as an untracked file.

The last line is the verdict. A bash script exits with the status of its final command, so ending with a test expression gives you 0 or 1 for free. Being explicit — if …; then exit 0; else exit 1; fi — is fine too and often clearer.

Exit Code 125: Skipping Unbuildable Commits

This is the code that makes automation practical on a real repository.

Some commits will not build. A dependency was added in the next commit, a migration is half-written, a wip commit does not compile. Marking those bad would poison the search.

exit 125 says I cannot judge this one. Git picks a nearby commit instead and carries on, and if it ends up unable to isolate a single commit it reports a small range instead:

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
4c1f8ab refactor: extract discount calculation
9d3e072 wip

Two candidates is still an excellent answer, and it is honest about the uncertainty.

This is also where Module 2 pays off: a branch of clean, buildable commits produces far fewer skips than a branch full of wip.

What Can and Cannot Be Automated

Good candidates: anything a program can check. A wrong calculation, an API returning the wrong status code, a missing field in a JSON response, an error in a log file, a performance threshold, a file that should exist.

Poor candidates: anything needing a human. A visual regression, a layout that "feels wrong", a race condition that appears one time in five, a bug requiring manual interaction with a third-party system.

For the poor candidates, run the manual loop from the previous lesson. Everything else about the method is identical — bisect run only replaces who answers.

Pro Tip: Test your script by running it by hand on the known-bad commit and on the known-good commit before starting. ./bisect-refund.sh; echo $? should print 1 on one and 0 on the other. If it does not, the automation will be confidently wrong nine times in a row, very quickly.

Key Takeaways

• git bisect run <command> reads exit codes instead of asking you
• 0 is good, 1–124 is bad, 125 is skip
• A failing test that reproduces the bug is usually the whole automation
• Keep the script outside the repository or gitignore it, so old checkouts do not remove it
• exit 125 handles commits that will not build, and Git reports a range if it cannot narrow further
• Automate anything a program can judge; bisect manually for anything needing a human
• Verify the script against both boundaries before starting the run