Claude vs Codex vs DeepSeek vs Gemini: Code, Autotests, Business Plans & Reasoning (2026)

Most "Claude vs GPT vs Gemini" pages on the internet just paste a vendor's own launch-blog table and call it a comparison. This one is built differently: four separate research passes (one per lab) cross-checked vendor claims against independent trackers — Artificial Analysis, NIST's CAISI, LMArena, aider's Polyglot leaderboard, and hands-on practitioner reviews — specifically for four use cases QA and dev teams actually care about: writing code, writing and running automated tests, drafting business plans and analytical documents, and working across large, long-lived codebases. We compare Anthropic's Claude (Opus 4.8), OpenAI's Codex (the agentic coding product, running on GPT-5.5), DeepSeek (V4-Pro), and Google's Gemini (3.1 Pro — the current publicly available flagship; Gemini 3.5 Pro was still in enterprise-only preview as of this snapshot). Every number below is labeled vendor-reported or independent, because that distinction changed the ranking more than any single benchmark did.

Key takeaways

• No model wins every category: Claude Opus 4.8 is most balanced for daily coding, Codex/GPT-5.5 leads independent math/abstract-reasoning benchmarks, Gemini 3.1 Pro leads GPQA Diamond and Workspace integration, DeepSeek V4-Pro is 5-10x cheaper and the only self-hostable option.
• Vendor-reported and independently-measured scores consistently disagree — NIST CAISI measured a 7-point gap between DeepSeek's own SWE-bench claim and its independently-tested score; Claude Fable 5's flagship benchmark was publicly disputed within 24 hours of launch.
• DeepSeek V4-Pro has an independently-measured, quantified tendency to confidently fabricate answers rather than abstain (94-96% on AA-Omniscience) — a real risk for business-plan and analytical writing use cases specifically.
• All four labs now ship 1M-token context windows, but long-context retrieval accuracy at the true 1M edge drops sharply for at least Gemini (84.9%→26.3%) and DeepSeek (0.82→0.59) per their own published long-context benchmarks.
• Every score here is a July 2026 snapshot from a single research pass per model — re-check primary sources (vendor system cards, Artificial Analysis, NIST CAISI) before citing an exact percentage, since several figures in this space were found to conflict across secondary sources.

Why vendor benchmarks and independent benchmarks disagree

Every lab publishes SWE-bench Verified, GPQA Diamond, or Terminal-Bench numbers under its own best-case configuration: highest reasoning effort, full tool access, and often multiple attempts averaged together. Independent evaluators — Artificial Analysis, NIST's CAISI, Epoch AI's FrontierMath, and the community-run aider Polyglot and LMArena leaderboards — reproduce the same benchmark under one fixed harness, and the numbers are consistently lower. The gap isn't noise: NIST CAISI measured DeepSeek V4 at 74% on comparable software-engineering tasks against DeepSeek's own ~80.6% SWE-bench claim, and separately found DeepSeek's real-world capability trails the frontier by roughly eight months. Anthropic's newest Claude Fable 5 had its SWE-bench Pro claim publicly disputed by evaluators within 24 hours of launch. Treat any single vendor-reported number as a ceiling, not an expectation.

Code writing: how they actually compare

On vendor-reported SWE-bench Verified, Claude Opus 4.8 (88.6%) leads this group of four, with Codex/GPT-5.5, DeepSeek V4-Pro, and Gemini 3.1 Pro clustered near 80-83%. But the one independent, government-run evaluation found here (NIST CAISI) only tested GPT-5.5 and DeepSeek V4 head-to-head, and found a real 7-point gap (81% vs 74%) in GPT-5.5's favor on comparable tasks — a useful reminder that self-reported rankings and independently-measured rankings can diverge. Real-world developer sentiment (Hacker News, dev blogs) consistently describes Claude's agentic coding style as disciplined and concise, Codex as strong on terminal/DevOps workflows but weaker on frontend work, Gemini's raw output as powerful but frequently over-engineered, and DeepSeek as good value rather than best-in-class on the hardest bugs.

Autotests and test automation: how they actually compare

No vendor publishes a dedicated "writes good tests" benchmark — this is qualitative territory for all four models. The most useful data point found is an independent, hands-on QA-practitioner review (not vendor-affiliated): Claude Code generated a 23-test Java suite with 95% line coverage and 91% mutation coverage in minutes, but missed real edge cases (an HTTP 500 path, a boundary condition on interest calculations) and produced some redundant tests — the reviewer's conclusion was cautious optimism, not blind trust. Codex's advantage is architectural: it runs tests inside a sandboxed cloud container and can iterate autonomously before opening a PR, though community reports flag a tendency toward over-mocking and happy-path bias. Gemini ships first-party test generation inside Android Studio and Code Assist, but Hacker News threads on Gemini's agentic CLI environments specifically flag harness-escape and reliability issues relevant to unattended CI runs. DeepSeek's test-automation evidence is limited to community tutorials rather than evaluation — plausible but unverified that its cheaper Flash tier lags Pro on the multi-step debug-and-retest loop that test automation actually requires.

Business plans and analytical writing: how they actually compare

For long-form business documents, the most actionable finding isn't a benchmark score — it's a documented failure mode. DeepSeek V4-Pro scores well on GPQA Diamond (90.1%, vendor) and MMLU-Pro, but Artificial Analysis's AA-Omniscience benchmark found that when it doesn't know an answer, it guesses confidently wrong 94-96% of the time rather than admitting uncertainty — corroborated independently by a hands-on reviewer who noted it "will happily invent statistics if you let it." That's a real risk for stakes-bearing business writing. Gemini's differentiator here isn't a reasoning score at all — it's native integration into Gmail, Docs, Sheets, and Slides, letting it draft directly from existing Workspace files, which neither Claude, Codex, nor DeepSeek can do natively. Claude and Codex both have strong anecdotal reputations for long-form business writing and instruction-following, backed mainly by their large context windows rather than a dedicated business-writing benchmark, since no such standardized benchmark exists industry-wide.

Analytical reasoning approach: how they actually compare

The four labs solve reasoning differently, and it shows in where each one wins. Codex/GPT-5.5 leads the independently-run FrontierMath benchmark (Epoch AI, one of the hardest public math evals) and posts the highest ARC-AGI-2 abstract-reasoning score found in this research (85.0%). Gemini 3.1 Pro's standard mode scores 77.1% on ARC-AGI-2, but its "Deep Think" mode — which spends extra inference-time compute rather than using a bigger model — jumps to 84.6%, and separately posts the highest GPQA Diamond score found here. DeepSeek's RL-driven reasoning lineage (the same technique that produced R1) shows real, independently-verified strength on competition math (97% on OTIS-AIME-2025, per NIST CAISI) but a genuine, independently-measured weakness on novel abstract reasoning (46% ARC-AGI-2 — barely half of GPT-5.5's score). Claude's current reasoning architecture (adaptive thinking with a soft effort dial) trails GPT-5.5 on ARC-AGI-2 specifically, even though it leads Artificial Analysis's broader, agentic-weighted Intelligence Index in other configurations. The honest takeaway: "best at reasoning" depends entirely on which kind of reasoning you mean.

Context retention and project understanding: how they actually compare

All four now ship a 1M-token context window, but "1M tokens" and "reliable recall at 1M tokens" are different claims. Gemini 3.1 Pro's own model card shows retrieval accuracy on its MRCR long-context test collapsing from 84.9% at 128K tokens to 26.3% at the full 1M mark — a headline context size that significantly overstates practical reliability at the far end. DeepSeek shows the same pattern at a smaller scale (0.82 at 256K down to 0.59 at 1M) but backs it with real architectural efficiency gains (a new sparse-attention design cutting KV-cache memory to roughly a tenth of its prior generation at 1M-token scale). Codex has a genuinely confusing story here: GPT-5.5 supports 1M tokens via the API but only 400K tokens inside the Codex product itself, a gap worth checking directly before assuming your Codex session gets the full window. Claude's Sonnet 5, Opus 4.8, and Fable 5 all ship 1M tokens at flat per-token pricing with no long-context surcharge — a genuine, verifiable pricing differentiator — though the specific multi-needle benchmark numbers claiming Claude's lead over competitors traced back to vendor blog claims that couldn't be independently confirmed in this research pass.

Which model should you actually pick

For day-to-day coding and test automation, Claude Opus 4.8 is the most balanced choice across this research — strong independently-adjacent coding scores, a disciplined agentic style developers describe favorably, and a flat 1M-token context window. If your team is standardized on GitHub and wants autonomous PR-open-and-review workflows, Codex (GPT-5.5) is a credible, arguably deeper agentic-coding product, with the caveat that its Codex-product context window (400K) is smaller than its API window (1M). If cost is the binding constraint, or you need to self-host for compliance reasons, DeepSeek V4-Pro is 5-10x cheaper than the other three and the only genuinely open-weight option — accept that it independently trails the frontier by a real margin on hard software-engineering tasks and has a documented tendency to fabricate rather than abstain. If your organization already runs on Google Workspace or needs the largest practical context window for very large document sets, Gemini 3.1 Pro is the natural fit, with the caveat that its next flagship (3.5 Pro) was not yet generally available as of this snapshot. Within Claude's own lineup specifically, Sonnet 5 keeps most of Opus 4.8's coding score at roughly a third of the price for teams that don't need the absolute ceiling.

Honest caveats and risks worth knowing before you commit

Each vendor carries a real, current caveat that a purely benchmark-driven comparison would miss. Anthropic's newest top-tier model (Fable 5) had its flagship SWE-bench Pro claim publicly disputed by independent evaluators within a day of launch, and its own safety classifier automatically reroutes certain sensitive-domain requests to Opus 4.8 instead — meaning Fable 5's headline scores don't apply to every real request. OpenAI's Codex has conflicting reports across sources about current MCP (Model Context Protocol) support, which matters if your QA tooling depends on it — verify directly against current docs before building around it. DeepSeek's hosted app and API store consumer data on servers in mainland China, subject to Chinese data-access law, and it is currently restricted on government systems in several countries (Italy, Taiwan, Australia, and others) — a real consideration for regulated industries, even though the open-weight model can in principle be self-hosted outside that jurisdiction. Google discontinued Gemini CLI in mid-2026 in favor of a rewritten Antigravity CLI that reportedly launched without full feature parity — worth checking before building CI automation on top of it. None of these caveats make a model unusable, but an honest comparison should surface them rather than only comparing headline scores.

FAQ

Sources

• Anthropic — Claude models overview
• Anthropic — Claude pricing
• OpenAI — Codex models
• OpenAI — API pricing
• OpenAI — GPT-5.3-Codex system card
• DeepSeek — API pricing
• DeepSeek — V4 announcement
• Google DeepMind — Gemini 3.1 Pro model card
• Artificial Analysis — Intelligence Index
• Artificial Analysis — DeepSeek V4 analysis
• NIST CAISI — DeepSeek V4 evaluation
• Epoch AI — FrontierMath Tier 4
• vals.ai — SWE-bench Verified leaderboard
• On Test Automation — independent hands-on review of Claude Code test generation