Skips, Flaky Tests, and Reporting the Result

Learning Objectives

By the end of this lesson you will be able to:

Use git bisect skip for commits you cannot judge
Narrow a bisect to the commits that touched a particular path
Recognise when a flaky test has corrupted a search, and restart correctly
Write a bisect result into a bug report a developer can act on

`git bisect skip`

The manual equivalent of exit code 125:

git bisect skip

Use it when the commit cannot be judged — it does not build, a required service is unavailable, a migration is half-applied. Git chooses a different commit near the midpoint and continues.

You can skip a whole range in one go:

git bisect skip 9d3e072..4c1f8ab

Handy when you already know a stretch of the branch was mid-refactor and will not run.

If skips prevent full isolation, Git says so honestly:

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
4c1f8ab refactor: extract discount calculation
9d3e072 wip: half-migrated schema

Report both. "One of these two" is a genuinely useful answer, and it is far better than picking one and sounding certain.

Narrowing by Path

If you know the bug is in the payments code, you can tell bisect to consider only commits that touched it:

git bisect start -- src/payments/

Commits that changed nothing under that path are excluded from the search, which can cut a 380-commit range to 40 and save most of the builds.

Use this only when you are confident about the scope. A bug that was actually caused by a shared utility or a dependency bump will be missed entirely, and the search will end pointing at the wrong place — with no indication that anything went wrong.

When the Test Is Flaky

This is the failure mode that wastes an afternoon and produces a wrong answer, so it is worth handling explicitly.

Symptom: the result names a commit whose diff is obviously unrelated — a README edit, a change in a different module, a comment fix.

That usually means one verdict during the search was wrong, and everything after it searched the wrong half.

Prevention: run the reproduction several times on the known-bad commit before you start. If it fails 5 times out of 5, the check is stable enough. If it fails 3 times out of 5, stop and stabilise it — no amount of bisecting will fix a check that lies.

Handling it in a script: repeat the test and require consistency:

#!/usr/bin/env bash
FAILURES=0
for i in 1 2 3; do
  npm test -- tests/payments/refund.spec.ts || FAILURES=$((FAILURES + 1))
done

[ "$FAILURES" -eq 0 ] && exit 0      # never failed: good
[ "$FAILURES" -eq 3 ] && exit 1      # always failed: bad
exit 125                             # inconsistent: cannot judge

That third line is the important one. An inconsistent result is honestly a skip, not a guess. Three runs per step triples the build time and is almost always worth it.

Recovering from it: git bisect log, find the verdict you now doubt, correct it, and git bisect replay.

Writing the Report

The result is only worth what your report makes of it. A good one has five parts:

Regression: refund totals wrong for discounted items

Introduced by: 4c1f8ab — "refactor: extract discount calculation into a helper" (Maria Kovalenko, 12 March 2026)

How it was found: git bisect between v4.1 (good) and main (bad), 9 steps, automated with the failing test tests/payments/refund.spec.ts. Bisect log attached.

Reproduction: POST /api/refunds with amount: 100, discount: 20. Expected total 80.00, actual 100.00.

Observation: the commit moves the rounding step in applyDiscount() to after the subtraction. This looks like the relevant change, but the fix may belong elsewhere — I have not verified the cause, only the boundary.

The last line matters. You are reporting where the behaviour changed, which is a fact, and separating it from what should be fixed, which is the developer's call. Stating that distinction explicitly is what makes the report trustworthy rather than presumptuous.

Attach the git bisect log file. It turns your conclusion into something anyone can re-run.

Pro Tip: After the developer fixes it, keep the failing test you wrote for the bisect. It cost you nothing extra, it is already proven to catch this exact regression, and it is the reason this bug does not come back next quarter.

Key Takeaways

git bisect skip handles commits you cannot judge; a range can be skipped at once
If skips prevent isolation, Git reports several candidates — report all of them
git bisect start -- <path> narrows the search but silently misses out-of-scope causes
Verify the reproduction is stable before starting; a flaky check produces a confident wrong answer
Run the test three times in the script and return 125 on an inconsistent result
Report the commit, the method, the reproduction, and the log — and separate "boundary" from "cause"
Keep the failing test afterwards; it is the regression test for free