Determine Test Impact from a Git Diff with AI
Reads a git diff and an import graph and returns the minimal test set to run for that diff — direct hits, transitive impact (1-2 levels), and explicit full-suite triggers (config / migration / lock file changes).
When to use it
- Optimizing CI runtime by avoiding running all tests when only one file changed.
- Implementing PR-time test selection for monorepos.
- Onboarding to a CI system where flaky 'run everything' is the current default.
- Debugging why tests run / don't run for specific PRs.
The prompt
XML-tagged — best for Claude 4.x
<role>
You are a build-systems engineer. You know test impact analysis (TIA) is a tradeoff — false negatives (skipping a test that would catch a bug) are MUCH worse than false positives (running a test you didn't need). You err on the side of running more, not less.
</role>
<context>
TIA categories:
- **Direct hits** — tests that import the changed file directly
- **Transitive impact** — tests that import a file that imports the changed file (1-2 levels deep is the practical limit; deeper transitives lose meaning)
- **Full-suite triggers** — changes to: package.json / lockfile, build config (tsconfig, vite.config), CI config, db migrations, env files, anything in shared/ or lib/
False negatives in TIA cause production incidents. Default to RUN MORE when uncertain.
</context>
<task>
For the diff and import graph below:
1. Classify each changed file: source / test / config / migration / other.
2. Identify direct test hits (tests importing the changed source file).
3. Identify transitive impact (1-2 levels).
4. Identify full-suite triggers and flag them.
5. Emit a recommended test command (e.g., `vitest --findRelatedTests <files>` or specific test files).
6. Note any uncertainty — files where the import graph is incomplete or ambiguous.
</task>
<input>
Git diff (paths and brief change context): {diff}
Import graph (or sample of imports per file): {graph}
Test framework + TIA command (vitest, jest, Playwright): {tooling}
</input>
<constraints>
- Distinguish DIRECT vs TRANSITIVE in output.
- Full-suite triggers are P0 — surface immediately if any change matches.
- Transitive impact limited to 1-2 levels — beyond that, false-positive cost dominates.
- Emit CI command, not just a list of tests.
- Acknowledge uncertainty (incomplete import graphs are common).
</constraints>
<output_format>
Five sections:
1. **File classification** — table: File | Type | Notes
2. **Direct hits** — list of test files
3. **Transitive impact** — list (1-2 levels) with depth indicator
4. **Full-suite triggers** — list with reasoning (or "None — selective testing applies")
5. **CI command** — exact command to run
</output_format>
Before writing, scan the diff for ANY full-suite trigger — if present, all other analysis is moot.Example
Common pitfalls
- Model treats package.json bump as 'minor — no impact' — every dependency change can break code. Always full-suite.
- Transitive analysis goes 4+ levels deep — meaningless beyond 2; ranks every test as 'affected'.
- Selective testing skips integration tests that don't import the changed file but DO test its consumers — false negative.
- Model produces a list of tests without a CI command; engineers can't run it as-is.
Tips
- Default to running MORE tests when uncertain. False negatives cost production incidents.
- Use Vitest `--changed` for git-based TIA; `--findRelatedTests` for explicit file-based.
- For Playwright (no native TIA), tag specs with feature names (`@checkout`, `@billing`) and grep.
- Pair with `parallel-test-execution-config` — TIA reduces what to run; parallelization reduces how long.
FAQ
Vitest computes import graph and runs only tests transitively related. Provide the changed source file as input. Best for non-monorepo Vite/Vitest projects. Less mature than Bazel's TIA but free.
Related prompts
GitHub Actions QA Pipeline Generator
Returns a complete `.github/workflows/qa.yml` with unit → integration → E2E stages, Playwright browser matrix, dependency + browser caching keyed on lockfile, artifact retention with explicit period, and failure notification via webhook / Slack.
Open →GitLab CI Test Pipeline Configuration
Returns a `.gitlab-ci.yml` with stages (build, test:unit, test:integration, test:e2e), parallel matrix for E2E browsers, cache keyed on lockfile, rules section for MR vs main differences, and artifact reports with explicit expiration.
Open →Configure Parallel Test Execution
Returns a parallel-execution config tailored to your framework (Playwright or Jest), CI runner count, average test duration, and flakiness rate — including shard count, worker count per shard, test ordering strategy, and a reasoning paragraph.
Open →Refactor Flaky Test to Stable
Takes a flaky test and its failure history, identifies which of the canonical root causes (race, hard sleep, shared state, network dependency, ordering, animation) is responsible, and produces a rewritten test that fixes the specific cause — no blanket retries.
Open →