Skip to content

Determine Test Impact from a Git Diff with AI

Updated 2026-06-08·advanced·CI/CD for QA

Reads a git diff and an import graph and returns the minimal test set to run for that diff — direct hits, transitive impact (1-2 levels), and explicit full-suite triggers (config / migration / lock file changes).

When to use it

  • Optimizing CI runtime by avoiding running all tests when only one file changed.
  • Implementing PR-time test selection for monorepos.
  • Onboarding to a CI system where flaky 'run everything' is the current default.
  • Debugging why tests run / don't run for specific PRs.

The prompt

XML-tagged — best for Claude 4.x

<role>
You are a build-systems engineer. You know test impact analysis (TIA) is a tradeoff — false negatives (skipping a test that would catch a bug) are MUCH worse than false positives (running a test you didn't need). You err on the side of running more, not less.
</role>

<context>
TIA categories:
- **Direct hits** — tests that import the changed file directly
- **Transitive impact** — tests that import a file that imports the changed file (1-2 levels deep is the practical limit; deeper transitives lose meaning)
- **Full-suite triggers** — changes to: package.json / lockfile, build config (tsconfig, vite.config), CI config, db migrations, env files, anything in shared/ or lib/

False negatives in TIA cause production incidents. Default to RUN MORE when uncertain.
</context>

<task>
For the diff and import graph below:
1. Classify each changed file: source / test / config / migration / other.
2. Identify direct test hits (tests importing the changed source file).
3. Identify transitive impact (1-2 levels).
4. Identify full-suite triggers and flag them.
5. Emit a recommended test command (e.g., `vitest --findRelatedTests <files>` or specific test files).
6. Note any uncertainty — files where the import graph is incomplete or ambiguous.
</task>

<input>
Git diff (paths and brief change context): {diff}
Import graph (or sample of imports per file): {graph}
Test framework + TIA command (vitest, jest, Playwright): {tooling}
</input>

<constraints>
- Distinguish DIRECT vs TRANSITIVE in output.
- Full-suite triggers are P0 — surface immediately if any change matches.
- Transitive impact limited to 1-2 levels — beyond that, false-positive cost dominates.
- Emit CI command, not just a list of tests.
- Acknowledge uncertainty (incomplete import graphs are common).
</constraints>

<output_format>
Five sections:
1. **File classification** — table: File | Type | Notes
2. **Direct hits** — list of test files
3. **Transitive impact** — list (1-2 levels) with depth indicator
4. **Full-suite triggers** — list with reasoning (or "None — selective testing applies")
5. **CI command** — exact command to run
</output_format>

Before writing, scan the diff for ANY full-suite trigger — if present, all other analysis is moot.

Example

Common pitfalls

  • Model treats package.json bump as 'minor — no impact' — every dependency change can break code. Always full-suite.
  • Transitive analysis goes 4+ levels deep — meaningless beyond 2; ranks every test as 'affected'.
  • Selective testing skips integration tests that don't import the changed file but DO test its consumers — false negative.
  • Model produces a list of tests without a CI command; engineers can't run it as-is.

Tips

  • Default to running MORE tests when uncertain. False negatives cost production incidents.
  • Use Vitest `--changed` for git-based TIA; `--findRelatedTests` for explicit file-based.
  • For Playwright (no native TIA), tag specs with feature names (`@checkout`, `@billing`) and grep.
  • Pair with `parallel-test-execution-config` — TIA reduces what to run; parallelization reduces how long.

FAQ

Vitest computes import graph and runs only tests transitively related. Provide the changed source file as input. Best for non-monorepo Vite/Vitest projects. Less mature than Bazel's TIA but free.

Related prompts