Pairwise Testing

What is pairwise testing?

Pairwise testing, also called all-pairs testing, is a test design technique that covers every possible pair of input values using the smallest practical set of test cases. Instead of trying every combination of every parameter, you guarantee that each pair of values appears together at least once. The result is typically a 70-95% reduction in test count with very little loss of defect detection.

The technique rests on an empirical finding from NIST fault studies: the large majority of software failures are triggered by a single parameter, or by an interaction between exactly two parameters. Three-way and higher-order interactions do cause failures, but they are the long tail. Pairwise buys you the first two tiers of that curve cheaply, and leaves the tail to exploratory and risk-based testing.

A worked example: 54 combinations down to 9

Suppose you are testing a checkout page across four parameters:

Exhaustive coverage is 3 × 3 × 3 × 2 = 54 test cases. That is already more than most teams will run on every release, and real features have more than four parameters. Feed the same four parameters into the generator above and it returns 9 test cases: the exact output below:

Nine rows, an 83% reduction, and every pair is still covered: Chrome with Linux, Safari with ja-JP, Pro with de-DE, and every other two-value combination appears somewhere in the set. Nine is also the theoretical floor here: with two parameters of three values each, you cannot cover all 9 of their pairs in fewer than 9 rows, so no algorithm can do better on this input.

How to use this pairwise generator

1. Add one parameter per dimension you actually vary: browser, OS, user role, payment method, feature flag.
2. Add the values for each parameter. Keep them to real, distinct equivalence classes, not every possible string.
3. Generate. The tool runs an IPOG-style greedy algorithm: it seeds a full table for the first two parameters, then extends one column at a time, always choosing the value that covers the most still-uncovered pairs.
4. Read the result as your test matrix: each row is one test case.
5. Export and paste it into your test plan, or use it as the data table for a parameterised automated test.

Everything runs in your browser. Nothing you type is uploaded, so you can use real parameter names from an unreleased feature without clearing it with anyone.

When NOT to use pairwise testing

This is the part most pairwise articles skip. The technique has real limits, and applying it blindly produces a test suite that looks rigorous and misses the obvious.

The parameters are not independent

Pairwise assumes any value of parameter A can combine with any value of parameter B. If your system forbids some combinations, Safari on Windows, a Free plan with an enterprise SSO provider, the generator will still emit them and you will burn time triaging impossible test cases. Either split into separate runs per valid region, or drop the invalid rows manually after generating.

The bug you are hunting is a three-way interaction

Pairwise guarantees two-way coverage and nothing more. If a defect only appears with Safari AND ja-JP AND a Pro plan together, an all-pairs set may never produce that row. When a subsystem has known three-way risk, raise the strength for those parameters specifically rather than generating higher-strength combinations across the board.

Order and state matter

All-pairs is a technique for combinations of independent inputs, not for sequences. Workflow bugs, add to cart, apply coupon, change currency, then check out, are a sequencing problem. Use state transition testing or scenario-based testing there; a pairwise matrix will not model order at all.

There are only a handful of combinations

If exhaustive coverage is 12 test cases, run all 12. Reducing 12 to 7 saves very little and costs you full-combination confidence plus the effort of explaining the technique in your test report.

The values are not real equivalence classes

Garbage in, garbage out. Listing 40 country codes as one parameter inflates the matrix without adding information. Reduce to classes that behave differently, LTR versus RTL, a locale with non-Latin script, a locale with a comma decimal separator, then run pairwise on those.

Pairwise testing tools compared

The established options each solve the algorithm well. They differ mainly in how much friction stands between you and a test matrix.

Honest summary: if you need constraints or three-way coverage, use PICT or ACTS, they do things this tool does not. If you need an all-pairs matrix right now without installing a runtime, requesting a download, or creating an account, generate it above and move on.

How pairwise relates to the other test design techniques

Pairwise answers which combinations to run. It does not answer which values are worth putting in the matrix in the first place: that comes from equivalence partitioning and boundary value analysis. And it does not model business rules, which is what a decision table is for. In a mature test design pass you use all three: partition the inputs, pick boundary values from each partition, then combine what is left with all-pairs.

Frequently asked questions

How many test cases does pairwise testing save?

It depends on the shape of the input, not on a fixed ratio. The reduction grows with the number of parameters: four parameters in the example above went from 54 to 9 (83%), while ten parameters of four values each drop from over a million combinations to roughly 30 test cases. The floor is always the largest product of any two parameters’ value counts.

Is pairwise testing the same as orthogonal array testing?

They solve the same problem differently. Orthogonal arrays come from experimental design and require every pair to appear an equal number of times, which forces padding when your parameters have uneven value counts. Pairwise only requires each pair to appear at least once, so it produces smaller sets for real-world inputs. For QA purposes, all-pairs is almost always the better fit.

What is the difference between 2-way and 3-way coverage?

2-way (pairwise) covers every pair of values. 3-way covers every triple, which catches interactions that need three specific values to align, and costs far more test cases. NIST data shows 2-way coverage catches most failures and 3-way catches most of the remainder, with steeply diminishing returns above that. Start at 2-way and raise strength only for high-risk areas.

Can I use pairwise output as automated test data?

Yes, and it is one of the best uses. Export the matrix and feed it to a parameterised test: pytest parametrize, JUnit ParameterizedTest, Playwright forEach over a data array. One test body, one row per generated case.

Does pairwise testing replace exploratory testing?

No. Pairwise is systematic coverage of combinations you already know about. It cannot find the parameter you forgot to list, the state-dependent bug, or the usability problem. Treat it as the combinatorial floor of your test plan, not the ceiling.

Do the generated rows change between runs?

No. The algorithm is deterministic: the same parameters in the same order always produce the same matrix, so you can paste the result into a test plan and reproduce it later. Reordering your parameters can change the output while still covering every pair.