# Rubric Points and Benchmark Scores
Generated: 2026-11-02T05:44:29.321Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: correctness.scoring
title: Rubric Points and Benchmark Scores
summary: Interpret earned points, penalties, mandatory gates, negative percentages, and incomplete benchmark results.
kind: reference
product_area: correctness_governance
status: stable
updated: 2026-10-30
canonical: /docs/correctness-governance/scoring
---
# Rubric Points and Benchmark Scores
## Definition
A binary rubric judgment describes observed behavior. The Regime determines how that judgment contributes to a Case and Benchmark score. Positive points, penalties and mandatory gates have distinct meanings.
## Fields, states, or lifecycle rules
- Positive points are earned when the relevant Rubric passes.
- Penalty points are deducted when its Rubric fails.
- A mandatory gate determines eligibility in a gated framework.
- Missing or incompatible judgments remain incomplete.
- Exact recorded fractions are displayed as percentages.
## Case score
For current point-based configuration, the Case score is:
```text
(earned positive points - triggered penalty points)
/ total applicable positive points
```
If there are no applicable positive points, the denominator is the total applicable penalty points. If that denominator is also zero, the Case is unscored.
A penalty-only Case ranges from 0% when no penalty triggers to -100% when all trigger. A mixed Case can score below -100%. There is no per-Case floor, cap or output scaling in the current configuration.
{% example-demo title="A valid negative score" %}
A Case has 2 applicable positive points and 6 penalty points. It earns both positive points and triggers every penalty. Its score is (2 - 6) / 2, or -200%. The negative result preserves the configured consequence rather than being clamped to zero.
{% /example-demo %}
## Gates and incomplete judgments
A completed mandatory-gate failure contributes zero to the benchmark score. Its rubric calculation remains available for diagnosis. Missing judgments do not become zero merely because another gate failed.
Benchmark and repeated-Trial means retain the entire frozen population. An unresolved Case or Trial makes the evaluation incomplete and unrankable. Technical execution failure is not a valid rubric failure.
## History and configuration
Project Settings → Regime configures point ranges, increments and guidance. New settings affect future captured work. Earlier Benchmark Versions retain their accepted scoring configuration; historical scaling rules are interpreted under that configuration.
Use **Correct judgment** for an attributable correction to an exact result. Use the governing Policy, Rubric or Regime settings when the intended standard is what needs to change.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Regime Settings](/docs/project-settings/regime)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Write Binary Rubrics](/docs/correctness-governance/binary-rubrics)
{% /related-card-grid %}
---
id: project-settings.regime
title: Regime Settings
summary: Configure scoring frameworks, point ranges, increments, and guidance with one current-settings save.
kind: reference
product_area: project_settings
status: stable
updated: 2026-10-30
canonical: /docs/project-settings/regime
---
# Regime Settings
## Definition
Open **Project Settings → Regime** to configure how applicable rubric judgments contribute to Case and benchmark results. The framework, point ranges, increments and guidance form one current configuration.
## Fields, states, or lifecycle rules
- Positive and penalty limits can be edited independently.
- Point increments are positive decimals; the editor offers common increments and custom entry.
- Zero is permitted where the configured treatment allows it.
- Gate and point guidance explain assignment expectations.
- One Save applies the configuration and guidance together.
- A concurrent change is reported instead of silently overwritten.
New numerical configurations start with a signed point range from -5 to +10. There is no configurable minimum magnitude. Existing rubric assignments must remain compatible with the chosen ranges and step.
## Save current settings
Review the framework and ranges together with the current assignments. Save after checking whether the values still express the intended consequence of each rubric.
A failed save preserves the editor draft. Resolve the reported conflict or invalid configuration and save the intended current values. Saving settings does not itself approve a Rubric, replace an assignment rationale or confirm expert review.
## Point interpretation
Current point scores preserve negative values and have no per-Case floor or cap. A gated framework also distinguishes mandatory-gate eligibility from the point calculation. Missing judgments remain incomplete.
Read the score guide before interpreting a large penalty or a zero score. A zero caused by a completed gate failure has a different meaning from an unscored Case with no applicable points.
## Historical work
Saving current settings preserves existing assignments and their attribution. Accepted Benchmark Versions, evaluation tasks and results retain the configuration they captured.
Changes are recorded automatically with their actor, time, scope and before/after content. Users do not create, select, publish or promote a separate configuration version to make the change take effect.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
- [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
{% /related-card-grid %}
---
id: benchmark-evaluations.inspect
title: Inspect Evaluation Results
summary: Trace scores to exact responses and rubric judgments, distinguish incomplete results, and record authorized corrections.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/inspect-results
---
# Inspect Evaluation Results
Start with identity and completeness, then explain the score from the actual response and applicable Rubrics.
## Prerequisites
- Access to the selected Benchmark Version and Run.
- Available responses or evaluation progress.
- Project-management permission if a judgment correction is needed.
## Steps
1. Open Runs and confirm the candidate, saved Harness Version where present, machine and planned Trials.
2. Identify whether this is the Run used for score or another independent execution.
3. Open Results and inspect the Case population, applicable criteria and incomplete count.
4. Read failed gates, earned positive points and triggered penalties separately.
5. Open the response and rubric detail. Compare the criterion with the actual behavior and visible Case materials.
6. Inspect individual Trial outcomes when reading a repeated result.
7. Decide whether the responsible change belongs to candidate code, Case content, applicability, rubric wording or the observed judgment.
## Scores and missing results
Point scores can be negative, including below -100% for mixed positive/penalty Cases. A completed mandatory-gate failure contributes zero to the benchmark score. Missing or incompatible judgments stay incomplete even when another gate has failed.
The full planned population remains the denominator. An incomplete evaluation cannot be made comparable by ignoring its difficult or failed executions.
## Correct a judgment
An authorized project manager can choose **Correct judgment** on the exact Trial's rubric result and supply the supported corrected judgment and rationale. The original AI judgment and manual change remain recorded. Reopen the refreshed result to interpret its resulting scores consistently.
Changing the judgment is different from editing the Rubric or the Case. Correct the owning artifact when its meaning is wrong.
## Object and state changes
Inspection and filtering do not mutate results. A judgment correction records an attributable authoritative change. Starting a Contribution or an improvement Session creates separate work and preserves the inspected Run.
## Success criteria
- The conclusion identifies its Run, Trial and Benchmark Version.
- Scores are explained by applicable criteria and the configured Regime.
- Missing telemetry and judgments remain unknown.
- Corrections retain original judgment provenance.
## Common failure modes
- Reporting a rank without completeness.
- Assuming all results are read-only.
- Treating a gate failure as permission to replace unavailable results with zero.
- Inferring hidden model reasoning from a captured trajectory.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Run Details and Metadata](/docs/benchmark-evaluations/run-details)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
- [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics)
{% /related-card-grid %}
---
id: correctness.binary-rubrics
title: Write Binary Rubrics
summary: Write atomic pass-or-fail criteria grounded in governed policies, applicable cases, and observable candidate behavior.
kind: task
product_area: correctness_governance
status: stable
updated: 2026-10-30
canonical: /docs/correctness-governance/binary-rubrics
---
# Write Binary Rubrics
Write a rubric when a governed policy needs an observable pass-or-fail check for benchmark evaluation. A strong rubric identifies one behavior, the cases where it applies, and the supporting information that distinguishes pass from fail.
## Prerequisites
- A policy or expert-grounded correctness statement.
- Representative passing, failing, and boundary cases.
- Clear applicability for the behavior being checked.
- Access to Correctness Governance → Rubrics.
## Steps
1. State one behavior that can be inspected in the candidate response and visible case supporting information.
2. Name the policy or specialist judgment that authorizes the criterion.
3. Define applicability before writing exceptions into the pass condition.
4. Write explicit pass supporting information and fail supporting information. Avoid “good,” “appropriate,” or “high quality” without observable conditions.
5. Link representative cases and test whether two informed reviewers would reach the same binary result.
6. Split independent requirements into separate rubrics when each failure should be diagnosed separately.
7. Inspect contribution provenance and approval state before relying on the rubric in benchmark interpretation.
## Object and state changes
This task creates or updates a project-level rubric and can change its wording, policy relationship, linked cases, evaluation use, activity, and approval context. Linking a case does not add it to a benchmark dataset. Editing a rubric does not alter historical Run supporting information that used an earlier benchmark boundary.
## Success criteria
- The rubric tests one behavior and can be answered from visible supporting information.
- Applicability excludes irrelevant cases without hidden reviewer judgment.
- Pass and fail conditions are explicit.
- Linked cases include at least one meaningful boundary.
- Policy authority and expert provenance are inspectable.
## Common failure modes
- Combining several behaviors into one criterion.
- Restating the policy without defining observable supporting information.
- Encoding applicability only as exceptions inside the rubric.
- Using a suggested or contributed draft as if it were already governed.
- Changing rubric wording and comparing Runs without checking the benchmark version boundary.
{% example-demo title="Example: grounding rubric" %}
Policy: material claims must use the controlling source or state uncertainty. Rubric: pass only when every material claim is supported by the current controlling source, or the response explicitly says the available sources do not resolve the claim. Unsupported blending of current and superseded sources fails.
{% /example-demo %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Applicability logic](/docs/object-model/applicability-logic)
- [Rubrics](/docs/object-model/rubrics)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics)
- [Weak Applicability Logic](/docs/troubleshooting/weak-applicability-logic)
- [Low expert agreement](/docs/troubleshooting/low-expert-agreement)
{% /related-card-grid %}