Decision compilation and provider-prefix caches
The decision runtime has separate namespaces for definition/schema compilation, adapter or grammar compilation, provider-prefix evidence, invocation…
Decision compilation and provider-prefix caches
The decision runtime has separate namespaces for definition/schema compilation, adapter or grammar compilation, provider-prefix evidence, invocation replay, and semantic result caching. Only the first two store compiled artifacts. They never store or reuse a prior answer.
Compile keys cover pinned source digests, compiler/runtime/schema/canonicalizer versions, adapter and prompt versions, backend capability mode, model compatibility, feature flags, project scope, and data class. Reads verify identity, value integrity, authorization, project scope, expiry, and tombstone state. Cold concurrent requests use a single fill; failed fills are not published. Bypass compiles through the same callback and therefore preserves normalized request semantics.
Provider-prefix identity additionally pins the ordered-prefix digest, provider/backend/model/API revision, cache policy and TTL, workspace, region, data class, and egress policy. `hit` and `miss` require explicit provider reports. An absent report is `unknown`; timing, token patterns, and local compile hits are never used as evidence.
Reusing a provider prefix across an identity change needs an explicit, pinned `decision-provider-prefix-compatibility/v1` record. `evaluateProviderPrefixReuse` allows reuse only for an identical identity inside the shorter of the two TTLs, or for a change covered by one unexpired record that names the exact from/to identity digests and permits every changed dimension. Only `requestedModel`, `actualModel`, `apiRevision`, `backend` and `policy` can be permitted; tenant, workspace, data class, region, egress policy, provider and the prefix bytes never transfer. A model or alias move also needs the record's D09-style alias snapshot (alias, snapshot ID, approved actual models, validity) to name the new requested alias and actual model and to be confirmed by the host registry callback. TTL expiry always invalidates. Jev does not expose prompt-prefix handles or cache usage, so nothing in the Jev transport consumes this yet (tracked in #2683).
Cache telemetry is metadata-only. It uses bounded layer/outcome labels and must not contain raw cache keys, prompts, schemas, credentials, provider handles, private locators, or tenant-controlled labels.
Both stores bind to the D10 `decision-lifecycle/v1` contract. The constructor requires a complete `lifecyclePolicy`, and the `cache` surface rule governs the store: a fill TTL longer than the rule's `retentionMs` is rejected, `backup` is refused when the rule says `not-persisted`, and restore is gated by `mayRestoreDecisionReference` (retention window and tombstones). Legal hold is a `DecisionLifecycleHold` whose scope includes `cache`; while it is unexpired, tombstone and deletion return `false`. Tombstone and deletion both replace the entry with a body-free `decision-compile-cache-tombstone/v1` record that holds only the key digest and a D10 tombstone reference, with no value, value digest or identity. The tombstone is persistent: the key never fills again, and no earlier backup of it can be restored. `FileCompileCache` can also take an independent `lifecycleJournal` (for example `FileDecisionLifecycleStore`), which is written before the local tombstone and consulted on every restore, so a cache directory restored from media still cannot resurrect a deleted entry.
Scope and authorization are decided from the requested identity before any storage lookup. For an out-of-scope or unauthorized caller, `read` and `backup` return `null`, `delete` and `tombstone` return `false`, and `setLegalHold` does nothing, exactly as for an absent key, and `getOrCompile` rejects in both cases. An authorized caller still gets `CompileCacheRejectedError` for an expired, tampered, wrong-version or tombstoned entry.
Compilation or prefix reuse does not imply a cached answer, correctness, calibration, deterministic inference, semantic equivalence, or semantic freshness. Deploy compile caching disabled first, compare the normalized requests from bypass and cache paths, and enable provider-prefix handling only for a pinned backend revision with documented semantics and approved privacy policy.
Runtime compilation is opt-in through `DecisionEvaluationRequest.compileCache`. Adapters that expose `compile` receive the same cloned definition and target on both paths, and the resulting immutable artifact is attached to the adapter request. When the policy is absent or disabled the compiler runs directly; enabled policies may use the memory or atomic filesystem store. A rejected cache read recompiles by default, while `failureMode: "fail"` makes rejection a pre-dispatch failure. Neither path resolves credentials or starts transport during compilation.
Both shipped adapters implement `compile`, so the cache is reachable in production when a host enables it. `JevDecisionAdapter.compile` returns a `jev-question/v1` artifact and `LlmSubagentDecisionAdapter.compile` returns a `llm-subagent-decision-prompt/v1` artifact (the input-independent prompt frame and output schema). Each keeps its payload as JSON text, so key order and therefore the request bytes survive canonical cache storage, and the adapter decodes the same artifact on the cached and uncached paths. A malformed artifact fails before dispatch (Jev: `invalid-request`, `not-sent`; LLM subagent: `invalid-definition`). Both compilations are local: Jev has no server-side compile or prompt-prefix API, so this saves only local request preparation. `CCP-013` drives `evaluateDecisionRuleset` with a fake Jev transport and shows byte-identical request bodies and identical `RulesetResult` semantics for no policy, a disabled policy, a cold fill and a verified hit.
Compile-layer telemetry is emitted through `DecisionCompileCachePolicy.onTelemetry` as a `decision-cache-telemetry/v1` record (layer, outcome, bounded reason, compiler version, preparation latency, expiry, invalidation reason) with no alias, key or identity. `mapCacheTelemetry` turns any layer's record into fixed `aiwg.cache.*` attributes, and the bounded metrics recorder accepts only the enumerated layer, result and reason values as dimensions.
`FileCompileCache` persists entries using digest-only filenames and atomic rename, revalidates content and identity on every read, and exposes scope-authorized backup, restore, tombstone, legal-hold, and deletion operations under the lifecycle rules above. Digest-scoped exclusive lock files single-flight cold fills across processes; waiters re-read and revalidate the published entry, dead owners are reclaimed, bounded waits fail closed, and failed compilation removes its owned lock without publishing an entry. Backup media encryption remains the caller's responsibility.
Paired benchmarks must pin their configuration digest, warmup and measured calls, minimum benefit target, and confidence interval. They report preparation latency, authoritative provider token/cache usage when available, total token/cost observations, hit and invalidation rates, and memory/storage. Unknown provider economics remain `null`, not zero. Total task tokens are reported only if authoritative input and output token counts are both supplied. The benchmark rejects unequal mode counts and invalid/negative measurements rather than manufacturing savings. A provider-reported prefix hit with an expiry at or before observation time is treated as `unknown`, not a current hit. `CCP-011` now executes paired bypass/cache-hit runs against the filesystem store and reports measured preparation time, serialized artifact bytes, actual file size and a deterministic 95% paired-bootstrap interval. This is an offline compiler fixture, not a provider-performance or minimum-benefit qualification: no Jev request is made, so provider tokens, cache tokens, cost and actual process heap remain unknown (the `memoryBytes` sample is serialized artifact size only). No production latency or savings claim follows from the fixture. `CCP-011` also exercises a pinned synthetic provider-report sequence (miss, hit, expired hit) and paired benchmark aggregation: only reported, unexpired cache tokens count; expired reports and unreported costs remain unknown. This fixture does not authenticate any live provider's cache implementation, pricing or savings. Provider-qualified production economics require an approved backend/model/API revision, documented prefix behavior, permitted egress, bounded synthetic calls and authoritative usage/cost reports; leave the issue open if those are unavailable.
`evaluateCacheBenchmarkTarget` enforces the preregistered minimum benefit as `pass`, `fail` or `unknown` from measured preparation latency only. The retained D30 CCP evidence lives in `docs/decision/evidence/compile-cache-ccp-v1/`: a preregistered benchmark plan, the executed offline benchmark report with its target verdict, and a qualification run and evidence manifest at an exact, clean source commit that map the CCP suites to G0 (contract), G1 (runtime), G2 (security and privacy) and G5 (performance). The retained run at the recorded commit fails its preregistered G5 target: a verified filesystem hit took about 0.16 ms against about 0.02 ms to recompile the Jev question locally (95% paired interval roughly -0.15 to -0.13 ms, benefit about -65000 bps against a +1000 bps target). Because Jev compiles by serializing a small question, integrity-checked cache reads cost more than they save; keep `compileCache` disabled for Jev unless a later, heavier compiler or a provider-side cache changes that result. G0, G1 and G2 pass. Regenerate it from a clean tree after `npm run build:cli` with `node tools/decision/ccp-qualification-evidence.mjs`.