Detect and route stale correctness evidence
What staleness means
Staleness means a downstream claim may no longer be supported by the current upstream state. It is a review signal, not automatic deletion and not a claim that historical evidence was invalid when produced.
Teammately objects change independently. A revised Policy may affect linked Rubrics and current benchmark interpretation without changing the exact evidence captured in an earlier Snapshot. A changed Harness may require a new Run while leaving the Benchmark Version unchanged.
Common triggers
- A Case input, context, source attachment, or supported reference output changes.
- A Policy rule, scope, approval, or applicable boundary changes.
- A Rubric criterion or link changes.
- Coverage facets or selected Case membership change.
- A source becomes superseded or the target product changes.
- A saved Harness version changes before candidate comparison.
Prerequisites
- The changed object and its earlier and current versions can be identified.
- The team can trace current downstream artifacts that rely on the changed fact.
- An owner can decide whether current evidence needs qualification, replacement, or no action.
Assess and route staleness
- Name the changed object, its earlier and current versions, and the reason for change.
- Identify downstream artifacts that rely on the changed fact: linked standards, selected Cases, Snapshots, Runs, comparisons, or customer-owned human review context.
- Classify each artifact as historically valid, current and unaffected, current but requiring qualification, or requiring replacement.
- Route the correction to the owning workflow: edit a Case, govern a new Policy or Rubric version, refresh coverage, create a Snapshot, or run a saved Harness again.
- Preserve the old version and its evidence. Add a note that states which boundary the evidence still supports.
- Confirm that current navigation and handoff material point to the new canonical version.
Object and state changes
A changed object does not make every connected artifact unusable. If a Policy wording change does not affect a particular Case or Rubric, document that determination. If a candidate configuration changes, create a new Run rather than a new Benchmark Version. If selected membership changes, create a new Snapshot rather than editing an old one.
Comparison Directions have a narrower advisory state. Potentially stale applies to untouched AI-suggested directions considered inconsistent with newer project context. It does not automatically apply to user-created or user-edited directions, and dismissing the label does not delete the direction or govern any downstream artifact.
Success criteria
- The trigger and affected version boundaries are named.
- Historical evidence remains interpretable under its original boundary.
- Current artifacts are explicitly unaffected, qualified, or routed to the owning workflow.
- New Snapshots or Runs are created only when their respective evidence boundary changed.
Common failure modes
- Treating every connected artifact as invalid after one upstream change.
- Editing a historical Snapshot or Run to resemble current state.
- Creating a new Benchmark Version when only the Harness changed.
- Confusing the advisory Comparison Direction label with governed-object staleness.
Worked example
Source document superseded
A new service policy supersedes the source used by twelve refund Cases. The operator preserves the old Snapshot and its Runs, updates affected current Cases, checks the linked Policy and Rubrics, and creates a new Snapshot. Two Cases describe historical behavior and remain unchanged with an explicit time boundary; ten move to the current version.
Source confidence
Code-backed: Policy and Rubric types, Benchmark Version and Snapshot surfaces, and Run detail preserve the version boundaries needed for this assessment. Comparison Directions explicitly expose their narrower advisory stale label. Cross-object dependency assessment and the decision to rerun or revise remain owner-reviewed work.