AIWG 2026.9.6 — Verifiable test conformance

Release date: 2026-09-08

Channel: stable

AIWG 2026.9.6 expands the Testing Quality addon with a repeatable workflow for assessing an existing test suite, reviewing its evidence, and applying reversible normalization changes.

Review what your tests actually prove

Ask your assistant:

Assess this codebase's test conformance. Inventory its runners, collect execution

evidence, review test oracles, and propose a normalization plan.

The addon includes conformance, oracle-review, and normalization agents; three new skills; YAML flows; and standard schemas and templates for protocols and outputs. The supporting `aiwg test-conformance` CLI provides twelve commands covering inventory, evidence collection, assessment, planning, changes, and rollback.

Assessment binds evidence to source and configuration. Runner discovery and execution receipts help reconcile declared tests with executed cases. Negative controls check whether specified tests fail when reviewed source behavior is deliberately changed, then restore the source and rerun the baseline. Missing evidence blocks a conformance claim; a green test run alone does not establish that the tests are adequate.

Normalize a suite with a reviewable plan

Platform templates support development and deployment planning. Apply and rollback record exact file contents and modes, detect conflicting edits, and guard path and symlink boundaries. Research guidance can search a configured local corpus and suggest platform testing tools through primary-source references.

Profiles cover Vitest, Jest, native Node, pytest, Go, Cargo, JUnit/Maven, VSTest, and generic runners. Real Vitest and pytest lifecycles were qualified for this release. The other profiles have adapter fixtures and research guidance; they still need validation against the target project's runner and reporting conventions.

See the conformance workflow and acceptance matrix for the evidence requirements and qualification limits.

AIWG uses the workflow on its own tests

The release audit restores omitted Vitest tests, separates mixed Node/Vitest contract lanes, makes validation setup fail closed, strengthens schema and fixture oracles, and adds multi-target negative controls. A real nested-suite Vitest report also exposed and fixed a suite-versus-file counting error in the conformance adapter.

AIWG now includes a runnable, source-bound conformance example. It collects real runner evidence, applies a reviewed source mutation, requires the named test to reject it, restores the source, and produces an assessment. Its review pins become stale when reviewed code changes. A separate measured library coverage gate proves that adding uncovered source fails the threshold even while behavior tests pass.

See AIWG's test conformance example for commands, scope, and the audit's before-and-after findings. Whole-repository semantic certification and live-platform qualification are outside this example.

Update and deploy

npm install -g [email protected]
aiwg use testing-quality --provider codex

Select your project's provider when deploying. The full `aiwg` package includes the addon corpus; `@aiwg/cli` provides the lightweight runtime. All three npm packages, including `@aiwg/cockpit`, share this release version.