Research / Gap Analysis
Gap analysis & conclusions
Written last, after every claim in the claims table and evidence record was checked. It does not protect the original hypothesis — where the evidence contradicts it, that is stated plainly.
Our synthesis — not an independently verified source
Everything on this page is our own interpretation of the verified evidence documented on the claims, prior-art, and evidence pages — not itself an independently verified primary-source fact. Treat the conclusions below as reasoning built on top of that evidence, open to revision if new prior art surfaces.1. What already exists?
A genuine, active prior-art landscape exists across five layers:
- Equivalence checking of individual circuit pairs (MQT QCEC) — mature, citable, widely usable.
- Single-run assertion-based debugging (MQT Debugger, CUDA-Q assertions, Proq, Bloq/AutoBloq) — several independent implementations.
- SDK-internal version-tier CI (Qiskit's official CI template) — real, reusable, actively used across ~8 Qiskit-ecosystem repos.
- Cross-Qiskit-version regression testing with CI-compatible output (QUTest, May 2026) — published and working, though not pytest-native and Qiskit-only.
- Bug corpora and cross-version reproducibility studies (Bugs4Q + its 21-Qiskit-version replication study) — already used for exactly this kind of cross-version regression analysis.
- Immature pytest plugins with cross-framework (not cross-version) equivalence assertions — real but young, unadopted.
- A single-SDK transpiler-regression pilot (“cart,” June 2026) finding ~38% of real transpiler bug-fixes are invisible to equivalence oracles.
2–5. What do MQT QCEC, MQT Debugger, quantum testing frameworks, and existing quantum CI/CD each solve?
MQT QCEC solves the algorithmic core of pairwise circuit equivalence checking — not orchestration, CI/CD packaging, cross-version execution, or cross-SDK testing. MQT Debugger solves assertion-driven fault localization within one execution of one program — not cross-run or CI-level regression detection. pytest-quantum / qtest-quantum solve framework-to-framework (Qiskit↔Cirq↔pytket) equivalence assertions inside pytest — the pytest-native piece has (immature) precedent, but neither executes the same test across multiple versions of one SDK. Qiskit's official CI template solves “run my existing tests against latest/dev/minimum Qiskit” as reusable GitHub Actions — but only reports pass/fail of tests already written; it does not diff behavior between versions.
6–8. Does the proposed infrastructure already exist?
Not as a unified, adopted product. No single system is simultaneously pytest-native, cross-version, cross-SDK, and automated in its regression/equivalence detection. QUTest covers cross-version + CI output but explicitly rejects pytest and is Qiskit-only. Cross-version regression testing exists in at least two independent 2026 efforts (QUTest; the Bugs4Q replication study). Cross-SDK regression/equivalence testing does not — no working tool of any kind was found. No single bug corpus is a universal standard, but real, citable, actively-used corpora exist — the “no corpus exists” framing is false.
9–10. Does the 31% statistic exist, and do the alleged papers support the gap?
Yes, the 31% statistic is confirmed with exact provenance (Zappin et al., arXiv:2506.17306, Finding 2, p. 18) — but it measures practitioner adoption/awareness of testing tools, not the existence or absence of relevant tooling; it is evidence of unmet need, not proof of a gap. No specific arXiv IDs were ever supplied for the “two papers” originally cited as evidence for the gap — that claim is unverified as stated. Independent search surfaced QUTest, which partially contradicts rather than supports the original framing, since it already implements a meaningful chunk of the proposed system.
11. What part of the proposed idea is genuinely novel, if any?
The narrowest defensible novel combination — not found anywhere in this research:
- pytest-native test discovery and fixtures (some precedent exists, e.g. pytest-quantum)
- cross-SDK (Qiskit vs Cirq vs PennyLane, not just cross-version-within-Qiskit) — no precedent found anywhere
- automated regression/equivalence detection (not hand-written assertions) as the comparison mechanism
- packaged as reusable, drop-in GitHub Actions tooling for arbitrary third-party quantum projects (not internal-only CI, not a bespoke research pilot)
No system in this research combines all four. The strongest partial overlaps are QUTest (3 of 4, missing pytest + cross-SDK) and pytest-quantum (2 of 4, missing cross-version + automated detection).
12. What parts are NOT novel and should be removed from the thesis/product claim?
- “No one does equivalence checking for quantum circuits” — false, MQT QCEC solves this well.
- “No one does quantum program debugging” — false, MQT Debugger and several assertion-based tools exist.
- “No bug corpus exists for quantum software” — false, Bugs4Q and others exist and are actively used.
- “No one does cross-Qiskit-version regression testing” — false as of May/June 2026 (QUTest, cart, the Bugs4Q replication study).
- “No one has built quantum-aware CI/CD in GitHub Actions” — false, Qiskit's own ecosystem template is real and reusable.
- Any claim built on “two arXiv papers proving the gap” with unverified/unsupplied IDs — must be replaced with the papers actually verified here.
13. The narrowest defensible research gap
This is materially narrower than the original framing. The original framing is largely already addressed piecewise: Qiskit's CI template handles version-matrix execution; QUTest handles cross-version behavioral drift with CI output; pytest-quantum handles pytest-native cross-framework equivalence. What is not addressed anywhere is the cross-SDK, automated-detection, pytest-native, reusable-packaging combination specifically. The state-aware/autonomous-agent angle remains genuinely open, but it is the least load-bearing part of the idea (no adjacent prior art at all, positive or negative) and should be treated as a possible extension, not the primary thesis.
14. What evidence would still be required before a publication-quality novelty claim?
- Direct confirmation or retraction of the originally-cited “two arXiv papers” — the actual intended IDs, if they exist, were never supplied to this research pass.
- A systematic, non-English-limited literature search — active Chinese- and German-language quantum software engineering research communities may hold unindexed prior art.
- Citation-graph tracing from QUTest and the “cart” pilot — both are very recent and may have follow-up versions or related-work sections that further narrow or widen the gap.
- DOI-resolver-verified checks of the QCEC and Bugs4Q journal citations (currently verified only via repo citation blocks / citing papers).
- An adoption/usage audit (download counts, dependent repos, citation counts) for QUTest, cart, pytest-quantum, and qtest-quantum.
- A working prototype comparison: run the same test suite against two SDK versions of two different quantum SDKs and confirm no existing tool does this end-to-end.
Recommendation
Our synthesis — not an independently verified source
The evidence does not support the original framing of the research gap as stated. Multiple substantial pieces of the proposed system already exist, some published within the last three months of this research date. The 31% statistic is real but was being used, in the original framing, to imply an evidentiary gap in tooling when it actually measures a gap in practitioner adoption/awareness — a different claim. The “two arXiv papers” as originally referenced could not be verified at all.
The idea is not dead, but it must be reframed narrowly around the cross-SDK + automated-detection + pytest-native + reusable-packaging combination identified in Section 13, explicitly positioned as building on — not competing with — MQT QCEC, Bugs4Q, and QUTest.
Full document with inline evidence citations: research/gap_analysis.md in the GitHub repository