SDLC evidence readiness screening

Status: experimental, default-off. Issue: #2622 / D29.

The SDLC screening pack is an advisory layer over existing deterministic artifact, citation and phase-gate checks. It never retrieves evidence, creates locators, approves publications, waives failed tests, or changes a gate result. Hosts must opt in by calling the `aiwg/decision` SDLC screening APIs and then decide separately whether to show the advisory receipt to a reviewer.

What the module trusts, and what it does not

The evidence facts in a subject are caller-asserted: `present`, `passed`, `expiresAtEpochMs`, `locatorExists`, `retrieved`, `provenanceVerified`, `publicationAuthorized` and the source `content`. The module cannot check them against the real artifacts. What it guarantees is fail-closed handling of what it is told, combined with the trusted gate policy: any reported defect, missing item or malformed value is non-ready. A host integration must source these facts from the deterministic validators (file/ID existence, schema, signature, test, approval and retrieval checks) and must not let a model or a request author supply them.

The gate policy is the other trusted input. `evaluateSdlcEvidenceScreening(request, trust)` takes a separate `SdlcScreeningTrustContext` from host configuration:

• `gatePolicyPin` and `gatePolicies`: the criterion-to-required-evidence map and

evidence ownership (including each citation source's trust and sensitivity) are an `SdlcGateEvidencePolicy` artifact. It is resolved by id/version from the host registry and verified against the pin with `assertArtifactPin`. The request may only reference it through `gatePolicyPin`; a request pin that does not match, a registry entry whose digest does not match, or an inline request policy produces a non-ready receipt.

• `calibration`: a `CalibrationRegistry`, compatibility request and policy.

Compatibility is resolved through the D09 registry. A missing, non-`allow`, model-mismatched or non-registry calibration input routes to review.

Runtime boundary

`evaluateSdlcEvidenceScreening()` accepts exactly one subject:

• one citation claim/source/locator pair; or
• one phase-gate criterion with its enumerated evidence bundle.

The subject is checked against a caller-owned inventory of known claim, requirement, source, locator, criterion and evidence IDs. `mode` must be `disabled`, `shadow` or `advisory`; any other value throws `SdlcScreeningValidationError`, and `disabled` returns `null` without reading the rest of the request. In `shadow` and `advisory` mode every other malformed input (unknown or unsupported fields, null or missing subject/evidence/inventory, missing requirement IDs, non-integer or non-finite numbers, broken trust context) produces a `FAIL` or `REVIEW` receipt instead of an exception. Each finding carries an explicit reason code. A model observation must point back to the exact same subject and evidence IDs; it cannot introduce a citation, locator, artifact, criterion, approval or requirement.

Deterministic preflight runs before model evidence is considered:

• Phase-gate criteria take their required evidence from the trusted policy, not

from bundle `required` flags. Every bundle item is inspected, not only the required ones. A missing required item, an item owned by another criterion or not owned at all (`foreign-evidence`), an item that is not present or not passed, an expired item (expiry at exactly the screening clock counts as expired) or an invalid expiry is non-ready. Duplicate evidence IDs are rejected outright (`duplicate-evidence-id`, `FAIL`); there is no "latest version wins" rule.

• Citations require a policy ownership entry for the exact claim/source/locator.

Unretrieved sources, missing locators, unverified provenance and unauthorized publication are non-ready. Trust and sensitivity come from the policy entry; a caller value that differs is `evidence-trust-mismatch`. Omitted `content` cannot be digest-verified and is `content-unverified`; content whose digest does not match is `content-digest-mismatch`. The D10 projection policy is validated with the policy's trust and sensitivity.

Semantic observations go through the published closed rulesets with D08 primitive acceptance. Each choice question needs a native provider distribution over its closed options; the module never builds one from a single confidence number. A missing or invalid distribution, low probability, low native confidence or low margin routes to review. For injection, only an explicit `no` with at least 8,000 bps confidence is clear: `unclear`, `yes` and a less-confident `no` route to review, both in the published ruleset and in a separate code guard. A supported citation also needs a support strength of at least 8,000 bps.

Gate outcome integration

AIWG has no programmatic SDLC phase-gate evaluator. `flow-gate-check` is an agent-executed skill, and there is no programmatic citation-verification path for SDLC artifacts. This pack therefore does not wire into, or make a byte-identity claim about, an existing gate path. `applySdlcScreeningToGateOutcome()` is a pass-through over whatever outcome value a host supplies: in `disabled` and `shadow` modes it returns a copy of that outcome with `publication: 'unchanged'`, keeps the host's audit receipt pins and, in `shadow`, attaches the screening receipt as `alternateScreening`. The tests check that the returned outcome serializes identically and that mutating it does not touch the host's object.

Durable review

High-risk, unclear, contradictory, injection-flagged or deterministic non-pass receipts are converted with `sdlcScreeningReviewInputFromReceipt()` whenever screening is enabled; the legacy `enabled` flag is retained in metadata only and does not suppress required review creation. The trusted gate policy pin is carried as a policy pin. The presentation sent to D13 is metadata-only and carries a digest of any requester presentation instead of raw content. This is only a D13 `CreateReviewInput`; the caller still creates the review with `DecisionReviewService`, authorization, retention and resume handling. The review effect remains idempotent through D13 continuation identity and cannot be dispatched twice by repeated resume.

Qualification scaffold

The repository includes closed schemas for:

• `SdlcEvidenceScreening.v1` (the request)
• `SdlcScreeningPreregistration.v1` (paired non-inferiority required)
• `SdlcScreeningPreregistration.v2` (paired non-inferiority optional: a null

margin skips the NI gate; passage baselines stay reported diagnostics)

• `SdlcScreeningRelease.v1` (carries a v1 or v2 preregistration)

The preregistration pins the held-out test split digest, the preregistered slices and gate-blocking slices, maximum false-support and false-ready rates, minimum total, per-slice and gate-blocking-slice support, the confidence-interval method/level, the paired non-inferiority margin and any efficiency claim. `evaluateSdlcScreeningPreregistration(plan, trustedDigest, records, nowEpochMs)` requires the plan's canonical digest (`sdlcScreeningPreregistrationDigest`) to match a separately anchored trusted digest, so a plan edited after freezing is a failure.

Held-out metrics are computed by `computeSdlcScreeningHeldoutReport()` from per-sample records (adjudicated gold label, candidate route/label/probability, usage, the paired baseline outcome and cost, and reviewer agreement). They are never caller-asserted. Calibration/risk-coverage, latency, tokens, cost and per-slice counts come from `evaluateBinaryHeldout` over the frozen split. Per-class support/contradiction/unclear precision and recall and the false-support and false-ready counts are computed from the records. False rates are bounded by the upper limit of the Wilson score interval (`wilsonScoreInterval`) at the preregistered level.

Quality is a paired non-inferiority test against the baseline screening path on the same items (v1, or v2 with a numeric margin; a null v2 margin skips this test and contributes no gate reason). An item is correct when the readiness route matches gold and, for citations, the support label also matches gold; each record's `baseline.correct` must use the same definition. The paired table (both / candidate-only / baseline-only / neither correct) goes to `pairedBinaryDifferenceInterval` (Newcombe method 10, candidate minus baseline) at the preregistered level, and `pairedNonInferiority` reads it with `marginBps = -qualityNonInferiorityBps`: a preregistered 300 bps margin lets the candidate be at most 3 points worse. A lower bound below the margin is `quality-not-non-inferior` (fail).

The result is `insufficient-evidence` for missing records or missing total/slice/class/gate-blocking support, including preregistered slices that are absent. It is `fail` for a plan digest mismatch, a plan frozen after the evaluation clock, a missing or unparseable `evaluatedAt`, an evaluation before the freeze or after the clock, a split mismatch, invalid or unregistered-slice records, an exceeded false-support or false-ready bound, a candidate that is not non-inferior to the baseline, or non-positive net economics against the baseline when an efficiency claim is made. Each reason is an explicit code with a fixed disposition; unknown codes fail.

Only the `wilson` method is implemented, at levels strictly between 5,000 and 9,999 bps (the range the shared helpers accept). A preregistration that asks for `exact-binomial` or another level gets `confidence-interval-unsupported` and `quality-non-inferiority-insufficient` (insufficient evidence).

`buildSdlcScreeningReleaseReport()` carries #2037/#2048 eval-integrity metadata unchanged and preserves `PROMOTE`/`HOLD`/`ROLLBACK`. Integrity is an allowlist (`qualificationIntegrityAllowlistProblems` in `qualification/release.ts`): only `verified` state with a strict mode (`fresh`, `locked`, `full-locked`) and that mode's own trusted score source, plus uncertainty, a paired baseline, positive `sample_n`, no weak signal, no compromise labels and an upstream `PROMOTE`, can promote. Unknown or malformed values hold. D29 cannot upgrade an upstream `HOLD` or `ROLLBACK`. The release is `PROMOTE` only when the preregistered evaluation passes, integrity has no problems and the upstream gate is `PROMOTE`; a not-non-inferior candidate is `HOLD`.

Offline tests

`test/unit/decision/sdlc-screening.test.ts` and `test/unit/decision/sdlc-screening-review-regressions.test.ts` cover:

• a supported citation with verified locator/provenance/content;
• unverified, unowned, trust-spoofed, restricted, content-omitted and

digest-mismatched citations;

• prompt-injection `yes`/`unclear`/low-confidence `no`;
• spoofed `required:false`, omitted, duplicated, foreign, extra-failed,

expired and invalid-expiry criterion evidence;

• a request that tries to narrow or replace the pinned gate policy;
• malformed requests that must produce receipts, not exceptions;
• calibration through the registry and native-distribution-only acceptance;
• a property check across criterion and citation evidence defects, every

bounded model response and a range of confidence values, proving no defect yields `ADVISORY_READY`;

• the host-outcome pass-through in disabled and shadow modes;
• record-computed held-out metrics, post-hoc and future-dated evaluations,

missing slices, integrity allowlisting and release HOLD/ROLLBACK;

• a report meeting every preregistered threshold reaching `PROMOTE`, and a

candidate that regresses against the paired baseline (or fails a tighter margin or a wider preregistered level) held as not non-inferior;

• schema conformance for the request, preregistration and release shapes;
• D13 restart/resume idempotency through the real file review store.

Pending live inputs

The implementation does not claim live Jev quality, human reviewer agreement, held-out false-support/false-ready bounds, production latency/cost, or reviewer time savings. Those require a frozen held-out corpus, adjudication guide, reviewer identities/rationales, live provider credentials, deployment egress approval and positive total-economics evidence.

Synthetic study module

The D29 study runbook describes source-only preparation, frozen tuning/calibration/test splits, the deterministic oracle and visible-text baseline, native receipt mapping and the additional coverage/conditional-rate report. Its 132-item operator template is unfilled. Live model observations, a compatible D09 artifact, protected integrity/access records and actual operator review remain required. No measured synthetic or representative SDLC-quality result is claimed by the module's offline fixtures.