Learning Objectives
By the end of this lesson you will be able to:
- Define what it means for a test set to achieve pairwise coverage
- Manually construct a pairwise test set for a small parameter space
- Explain why pairwise coverage is considered the sweet spot between exhaustive and minimal testing
What "Pairwise" Means
A test set has pairwise coverage if for every pair of parameters, every combination of their values appears in at least one test case.
For two parameters P1 (values: A, B) and P2 (values: X, Y), pairwise coverage requires:
- (P1=A, P2=X) in at least one test
- (P1=A, P2=Y) in at least one test
- (P1=B, P2=X) in at least one test
- (P1=B, P2=Y) in at least one test
With 2 parameters and 2 values each, pairwise and full coverage are the same: 4 tests.
The savings appear with 3+ parameters.
The Savings: A 3-Parameter Example
Parameters: P1 (A, B), P2 (X, Y), P3 (M, N)
Full coverage: 2³ = 8 tests
Pairwise coverage (all pairs covered):
| Test | P1 | P2 | P3 |
|---|---|---|---|
| T1 | A | X | M |
| T2 | A | Y | N |
| T3 | B | X | N |
| T4 | B | Y | M |
4 tests. Every pair is covered:
- P1×P2: (A,X)T1, (A,Y)T2, (B,X)T3, (B,Y)T4 ✓
- P1×P3: (A,M)T1, (A,N)T2, (B,N)T3, (B,M)T4 ✓
- P2×P3: (X,M)T1, (Y,N)T2, (X,N)T3, (Y,M)T4 ✓
Half the tests. All pairs covered.
Why the Savings Grow With Complexity
For our 5-parameter, 3-value checkout example (243 full combinations):
A pairwise set typically requires 17–25 tests — roughly 10% of full coverage while catching ~95% of bugs.
The exact count depends on the algorithm used, but the compression ratio improves dramatically as parameter count increases. With 10 parameters, full coverage is millions of tests; pairwise is typically under 30.
How Pairwise Tables Are Generated
The math behind pairwise table generation is non-trivial (it involves Latin squares and orthogonal arrays), but you don't need to do it by hand for real projects. Tools that generate pairwise tables include:
- PICT (Microsoft, free, command-line) — industry standard
- allpairs (Python, open source)
- Any spreadsheet-based pair coverage checker
For BugEater practice challenges, the parameters are small enough to construct pairwise tables manually — which builds intuition.
What Pairwise Doesn't Cover
Pairwise testing by definition does not cover 3-way or higher interactions. If a bug only manifests when three specific parameter values coincide, pairwise will miss it. In practice, this accounts for ~2–5% of bugs — acceptable for most risk profiles.
When you need higher confidence, use 3-wise or n-wise testing — but recognize that each step up multiplies your test count significantly.
Pro Tip: Use pairwise as your default for multi-parameter features with more than 3 parameters. Use full coverage for security-critical combinations (authentication, payment) and for features with 2–3 parameters (the overhead is minimal). Use 3-wise when you know from defect history that three-way interactions are a real risk.
Summary
Pairwise testing is a principled, research-backed strategy for reducing a massive combinatorial space to a manageable test set. The key insight: most bugs come from 2-way interactions, so proving every pair is covered gives you most of the value at a fraction of the cost. In the next lesson, you'll apply this to a real feature.