Research / Claims
Claims table
Every claim (C01. C13) from the original research brief, independently verified against primary sources.
Full detail for each claim, including every source URL opened, is in claims.md.
MQT QCEC capabilities / overlap with proposed idea
FALSE (overlap claim); CONFIRMED (existence/capabilities)
Mature equivalence checking library (decision diagram / ZX calculus engines), Qiskit + OpenQASM only. Its own CI is an OS/compiler matrix only, no SDK version matrix, no CI/CD product for user projects.
Source: github.com/munich-quantum-toolkit/qcec (README + CI YAML); mqt.readthedocs.io verification handbook
MQT Debugger capabilities / overlap with proposed idea
FALSE (overlap claim); CONFIRMED (existence/capabilities)
Assertion based, single run simulation debugger with DAP IDE integration, OpenQASM input. No cross version or CI regression capability.
Source: github.com/munich-quantum-toolkit/debugger; arXiv:2412.12269
“Q-Trace” / other quantum debugging systems overlap with proposed idea
FALSE (“Q-Trace” itself); real systems found operate at single run fault localization level only
“Q-Trace” is not a real named tool. QMon, TraceQ, the Microsoft QDK Trace Simulator, CUDA-Q statistical assertions, Proq, Bloq/AutoBloq, and MorphQ were all verified, none do cross version/cross SDK regression testing or CI integration.
Source: arxiv.org/html/2512.13422; arxiv.org/pdf/2508.14533; arxiv.org/abs/2507.16255; arxiv.org/abs/2506.18458; arxiv.org/pdf/2206.01111
≈31% of quantum developers use quantum specific testing tools (practitioner survey)
CONFIRMED
Exact quote “Only eight of 26 respondents (31%)…” verified in the full PDF text, Finding 2, Section 5.2, p.18. N=26 survey, IRB approved, fielded in 2024. Authors flag the small N limitation themselves.
Source: arXiv:2506.17306 (Zappin, Stalnaker, Chaparro, Poshyvanyk; to appear ACM TOSEM)
First alleged arXiv paper (evidence for the gap)
PARTIALLY CONFIRMED, no pre given ID existed to check; best reconstructed candidate is highly material
No specific ID was supplied. Best candidate found: QUTest (arXiv:2605.19736), a published cross Qiskit version regression testing tool with CI compatible (JUnit/xUnit) output. This partially contradicts the originally claimed gap rather than supporting it.
Source: arxiv.org/abs/2605.19736 (full HTML verified)
Second alleged arXiv paper (evidence for the gap)
PARTIALLY CONFIRMED (same caveat as C05)
Best candidates: the 31%-survey paper itself (arXiv:2506.17306, supports the need but is not a tool paper) and arXiv:2410.00650 (broad testing/analysis survey, no CI/CD content, general background only).
Source: arxiv.org/abs/2506.17306; arxiv.org/abs/2410.00650
No dedicated standard bug corpus exists for quantum SDK versions / regression testing
FALSE
Bugs4Q is a real, “widely used” corpus (36 bugs in the 2021 preprint; 42 in the DOI resolved JSS 2023 version); arXiv:2606.27124 already ran 37 of its artifacts across 21 Qiskit core library versions / 77,700 executions, found reproducibility collapsed 62.2%→16.2%, and restored it to 78.4% via a patched “Bugs4Q-Robust” fork. QBugs, a 32,296-report mined dataset, and QBugLM also exist. No corpus is a universal standard, but the “none exists” framing is false.
Source: arXiv:2108.09744; arXiv:2606.27124; arXiv:2103.16968; arXiv:2512.24656; github.com/Z-928/Bugs4Q
An open source pytest plugin already does full cross SDK version regression discovery/execution/comparison in CI
PARTIALLY CONFIRMED
pytest-quantum, qtest-quantum, and qc-assert are real, installable, but do framework to framework (not version to version) equivalence or single run assertions only; all are under 9 months old, 0-2 stars. No plugin executes across multiple SDK versions with automated diffing.
Source: github.com/qbench/pytest-quantum; github.com/metin-5115/qtest; github.com/JMORAF87/qc-assert; PyPI release metadata
Existing GitHub Actions products combine quantum SDKs with matrix builds / regression / equivalence testing
CONFIRMED (narrow)
Qiskit’s official ecosystem CI template (test_latest / development / minimum_versions.yml) is real, reusable, and active across ~8 repos, but only checks pass/fail of existing tests per version tier, it doesn’t diff behavior. Qiskit core’s qpy.yml does genuine but narrowly scoped (serialization only) cross version regression testing, internal only.
Source: github.com/Qiskit/qiskit-addon-cutting; github.com/Qiskit/qiskit/blob/main/.github/workflows/qpy.yml
Cross SDK or cross version quantum regression testing already exists as reusable open tooling
PARTIALLY CONFIRMED
quantum-transpiler-regression-testing (“cart,” Zenodo DOI 10.5281/zenodo.21020113, June 2026) found 38% of real Qiskit transpiler bug fixes are regressions invisible to equivalence oracles, single SDK, CLI based, brand new, 0 stars. QUTest (C05) does cross Qiskit version testing with CI output. No cross SDK regression/equivalence tool was found. Benchpress (C14) is pytest native and cross SDK but benchmarks capability/performance only.
Source: github.com/furqan-nr/quantum-transpiler-regression-testing; Zenodo 10.5281/zenodo.21020113; arXiv:2605.19736
Other equivalence/regression tooling exists beyond MQT QCEC
NOT FOUND (beyond items already catalogued under C01/C05/C10)
GitHub search for “quantum equivalence checking” returned only unrelated/unverified SAT solver student projects.
Source: gh search repos “quantum equivalence checking”
A dedicated product combines quantum testing + state awareness + autonomy + CI/CD + cross version regression
NOT FOUND AFTER SEARCH
The individual pieces exist separately (QBugLM is agentic and quantum specific but has no CI/CD or state awareness; generic autonomous testing agents exist only for classical software). No system combines all elements.
Source: arXiv:2606.07314; general web search
Overall research gap: does a genuine, defensible gap remain?
PARTIALLY CONFIRMED, a gap exists but is much narrower than originally framed
The narrow, defensible gap is: pytest native + cross SDK (not just cross version) + automated regression/equivalence detection (not hand written assertions) + reusable drop in GitHub Actions packaging. No single system combines all four; several systems solve one or two of the four pieces (Benchpress. C14, covers pytest native + cross SDK, but only as a performance benchmark).
Source: Synthesis of C01. C12 and C14, see the Gap Analysis page
Benchpress (IBM) already provides a pytest native, cross SDK harness overlapping the proposed infrastructure
PARTIALLY CONFIRMED, the harness pattern exists; the detection capability does not
Benchpress is a pytest based suite of 1,000+ benchmarks run across 8 SDKs (Qiskit, Braket, Cirq, Tket, BQSKit, Staq, pyqpanda3, Qiskit IBM transpiler) on circuits up to 930 qubits, proving pytest native cross SDK orchestration works at scale. But every test measures capability/performance (pass/skip/fail/xfail + timings); it performs no equivalence checking, no cross version regression detection, and ships no reusable GitHub Actions packaging. Apache-2.0, ~155 stars.
Source: github.com/Qiskit/benchpress; arXiv:2409.08844; Nation et al., Nat. Comput. Sci. 5, 427-435 (2025)
Raw table with full per claim source links: research/claims.md in the GitHub repository