# Weak Applicability Logic Generated: 2026-09-13T04:39:32.043Z Source build: local Canonical docs: https://teammately.ai/docs --- id: troubleshooting.weak-applicability-logic title: Weak Applicability Logic summary: Fix standards that are applied to the wrong cases or skipped where they matter. kind: error product_area: troubleshooting status: stable updated: 2026-08-23 canonical: /docs/troubleshooting/weak-applicability-logic --- # Weak Applicability Logic Use this when policies or rubrics fire on irrelevant cases or miss cases where they should apply. ## Symptom Benchmark results show failures that reviewers consider irrelevant, or important cases skip the standards that should govern them. The issue appears as false positives, false negatives, or confusing policy-level result counts. ## Likely causes - The applicability condition uses a broad keyword or metadata field that does not prove the behavior is in scope. - Cases lack the metadata or source context the applicability rule depends on. - A policy boundary changed but applicability was not revised. - Conversation context or retrieved-source state is not represented in the case. ## Diagnostic checks - Inspect included and excluded cases side by side. - Identify the exact source signal the applicability rule depends on. - Check whether case metadata, dimensions, or context fields are missing or stale. - Review recent policy or rubric revisions for boundary changes. ## Fix - Rewrite applicability around source-backed signals, not broad topic labels. - Add missing metadata or context before relying on the rule. - Create examples that should be included and excluded, then test the boundary. - Approve and version the corrected governed objects, then create the Benchmark boundary and Runs needed to evaluate the revised applicability. ## Prevention - Define applicability before benchmark runs, not after reading failures. - Keep case metadata and dimensions current. - Review applicability whenever a policy or rubric version changes. - Keep included and excluded example Cases linked to the governed standard where the current surface supports them. ## Related task pages {% related-card-grid title="Related task pages" %} - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) - [Work with Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Applicability logic](/docs/object-model/applicability-logic) - [Policies](/docs/object-model/policies) - [Coverage dimensions](/docs/object-model/coverage-dimensions) {% /related-card-grid %} ## Source confidence Code-backed: Policy and Rubric detail routes expose governed scope and linked Cases, while Benchmark Evaluation results expose which evaluator outcomes appeared for Cases. The boundary-testing method remains a human interpretation of those inspectable artifacts. --- id: object-model.applicability-logic title: Applicability logic summary: Explain when a policy or rubric should be used for a case, output, or coverage segment. kind: reference product_area: object_model status: stable updated: 2026-08-23 canonical: /docs/object-model/applicability-logic --- # Applicability logic ## Definition Applicability logic is the boundary that decides whether a policy or rubric should be considered for a specific case, output, or coverage segment. It prevents a good standard from being applied to the wrong behavior. Use this reference when a benchmark result is hard to explain because a standard seems relevant in some cases but not others. The question is not only whether the policy is correct; it is whether the policy was eligible to judge that output. ## Fields, states, or lifecycle rules - Applicability sits between the case/output and the policy/rubric that may judge it. - Weak applicability makes benchmark failures noisy: a candidate can fail a good rubric on a case where the rule should not have applied. - Strong applicability names the behavior condition, source context, or case segment that brings the standard into scope. - Applicability changes can make old benchmark evidence stale because the same output may be judged by a different standard boundary. - This page explains the public object relationship, not a public rule language, API schema, or export contract. ## Related objects Applicability logic should be read with [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Cases](/docs/object-model/cases), and [Coverage Dimensions](/docs/object-model/coverage-dimensions). Use the Correctness Governance workflow to write the boundary and troubleshooting when the observed Case set is wrong. {% example-demo title="Applicability logic boundary" %} Raw case: A user asks whether a product works with equipment they already own. Policy: Compatibility claims require explicit source support. Applicability logic: The policy applies only when the answer recommends, validates, or compares a product for a concrete use context. Benchmark interpretation: If the case only asks for a product description, the compatibility rubric should not judge it. If the answer claims the product will work with the user's equipment, the rule is in scope. {% /example-demo %} ## Source confidence Code-backed: Policy and Rubric types and their detail routes expose applicability fields and the links among governed standards, Cases, and evaluation checks. This page describes those product relationships; it does not define a portable rule language. ## Related task pages {% related-card-grid title="Related task pages" %} - [Work with Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics) - [Fix weak applicability logic](/docs/troubleshooting/weak-applicability-logic) - [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} --- id: correctness.policies-rubrics title: Policies and Rubrics summary: Understand the governed relationship between behavior policies, applicability, binary rubrics, linked cases, and expert provenance. kind: reference product_area: correctness_governance status: stable updated: 2026-08-23 canonical: /docs/correctness-governance/policies-and-rubrics --- # Policies and Rubrics ## Definition A **Policy** is a reusable statement of expected specialist AI behavior. Its applicability explains the situations in which the rule controls. A **Rubric** is an evaluation criterion that turns the policy into observable evidence for a case and candidate response. Correctness Governance owns both artifact types. Expert Contributions can supply proposed or accepted policy and rubric material, while the governance surfaces preserve the artifact's current state, links, activity, and provenance. ## Fields, states, or lifecycle rules - Policies have identity, descriptive rule content, scope or applicability, linked cases, linked rubrics, activity, and approval context. - Rubrics have identity, criterion wording, policy or case relationships, evaluation relevance, and lifecycle context. - A policy can connect to several rubrics when its behavior requirements need separate checks. - A rubric should express one inspectable criterion wherever independent diagnosis matters. - Linked cases demonstrate applicability or behavior; benchmark dataset membership remains a separate benchmark-scoped decision. - Proposed applications and agent suggestions remain proposals until the owning workflow records acceptance. - Expert contribution provenance should remain visible when contributed material becomes a governed artifact. - Editing a project-level standard does not retroactively change the standard boundary used by an already recorded Run. ![Correctness Governance rows showing Policy titles, lifecycle state, required behavior force, linked-count columns, and accountable owners.](/docs-assets/assets/screenshots/policies-rubrics-neutral-rows.png) Read the rule, state, links, and owner together; a plausible title alone does not establish governed authority. ## Reading the pair Begin with the policy when deciding what should happen and why. Inspect applicability before assuming the policy governs a case. Then read the linked rubric as the testable question applied to candidate behavior. If the rubric cannot be answered from the response and visible case evidence, revise the criterion or the case rather than relying on reviewer intuition. When standards overlap, distinguish complementary criteria from contradictory authority. Preserve unresolved conflict until an accountable expert contribution or governance action settles the intended rule. {% example-demo title="Example: escalation policy and rubrics" %} A policy states that unresolved eligibility exceptions must be escalated. One rubric checks that the response does not promise the exception; another checks that it gives the correct escalation path. Separating the checks lets an evaluation show whether a candidate avoided the unsupported promise but still failed to guide the user correctly. {% /example-demo %} ## Source confidence Code-backed: active policy and rubric detail routes expose linked cases, linked rubrics, approval and activity context, and evaluation relationships. Exact editable fields can vary by artifact state. ## Related task pages {% related-card-grid title="Related task pages" %} - [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics) - [Write binary rubrics](/docs/correctness-governance/binary-rubrics) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) {% /related-card-grid %} --- id: coverage.boundary-cases title: Boundary Cases summary: Use edge and ambiguous cases to sharpen policies, applicability logic, and rubrics. kind: concept product_area: coverage_engineering status: stable updated: 2026-09-07 canonical: /docs/coverage-engineering/boundary-cases --- # Boundary Cases ## Definition A Boundary Case sits near a meaningful change in expected behavior: a Policy exception, an applicability edge, conflicting source authority, missing information, an escalation threshold, or another condition where a small contextual change should alter the correct output or action. Boundary Cases are not simply unusual examples. They are deliberate tests of a named distinction in the benchmark's coverage and correctness model. ## Why it matters Ordinary Cases can make a weak rule appear reliable because they never force the team to decide where it stops applying. Boundary Cases expose vague Policy scope, overlapping Rubrics, missing exceptions, and unsupported assumptions before those problems are hidden inside aggregate results. ## Where it appears in the product Coverage Stories can name a boundary as a missing or thin behavior slice. Case Foundry or another sourcing workflow can prepare candidate Cases against that need. Case Review checks whether the input, materials, and contrast are realistic. If the expected judgment is unclear, route the selected Cases into an Expert Contribution instead of inventing the answer during case construction. ## Artifacts it affects A useful Boundary Case records the exact contextual feature that changes the expectation, the controlling source material, the relevant Coverage Facets, and the Policy or Rubric question it exercises. Admission to the current Benchmark Dataset is a separate selection decision; admission to a Snapshot is a later immutable evidence decision. ## Operational check Compare the Boundary Case with a nearby ordinary Case. The pair should differ in the intended condition, not in several unrelated dimensions. Confirm that specialists can explain why the expected behavior changes, that the Rubric tests one observable criterion, and that the Case materials make the boundary visible to the evaluated system. {% example-demo title="Current versus superseded source" %} Two Cases ask the same eligibility question. One contains only the current policy document; the other also contains a plausible superseded document with conflicting terms. Specialists confirm that the response must follow the controlling current source or state uncertainty. The pair becomes a Boundary Case set for source authority, and its Rubric fails unsupported blending rather than generic factual error. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Manage Coverage Stories](/docs/coverage-management/coverage-stories) - [Review prepared Cases](/docs/coverage-management/case-review) - [Applicability Logic](/docs/object-model/applicability-logic) - [Comparison Directions](/docs/assets/comparison-directions) - [Product quickstart](/docs/quickstart) - [Task index](/docs/operating-manual/task-index) {% /related-card-grid %} ## Source confidence Code-backed: Coverage Stories can represent named coverage needs, Case Review exposes prepared Cases and materials, and Case sourcing types preserve coverage targets and source context. Specialist authority and evaluator approval remain separate workflows.