# Evaluation Results
Generated: 2026-11-02T06:37:53.908Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: benchmark-evaluations.compare
title: Evaluation Results
summary: Inspect Case and rubric outcomes and compare exact managed or imported Runs within a Benchmark Version.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/results
---
# Evaluation Results
Results presents a matrix of selected Runs and the Cases or criteria they evaluated. Use one Run for inspection or several comparable Runs to locate gains, regressions and missing judgments.
## Prerequisites
- At least one Run for the intended Benchmark Version.
- Permission to inspect its Cases, responses and judgments.
- Comparable recorded conditions when interpreting differences.
## Steps
1. Open **Benchmark Evaluations → Results** and confirm the Benchmark Version.
2. Select the visible Runs. Managed and imported Runs can both appear; an imported Run does not need a Harness Version to be inspected.
3. Choose Cases, Policies, Rubrics, Dimension ontology values, Topics, Topic Groups or Scenarios as the row grouping.
4. Read completeness before comparing scores. An empty or incomplete result is not a failed judgment.
5. Open a Case or rubric detail to inspect the response, criterion, observed judgment and rationale.
6. For repeated Trials, inspect the selected aggregation rule and the individual Trial outcomes.
7. Route an actual candidate problem to Harness improvement, a standard problem to Correctness Governance, or an input-coverage problem to Coverage Design.
## Representative and historical Runs
Default candidate results use the Run selected before its outcomes were known. **Used for score** identifies that reference. Other executions remain in Run history and can be inspected without silently replacing the representative result.
Selecting matrix columns changes the analysis view. It does not activate a Harness, execute a candidate or declare a winner.
## Object and state changes
Filtering, row grouping and inspection are read-only. An authorized project manager can separately use **Correct judgment** on an exact rubric result. That operation records a correction and preserves the original AI judgment; refreshed aggregates come from the authoritative result snapshot.
## Success criteria
- Every compared column has an identified Run and Benchmark Version.
- Differences resolve to actual Cases and applicable criteria.
- Missing outcomes stay distinct from negative scores or failed gates.
- Manual corrections remain attributable.
## Common failure modes
- Treating an imported Run as an executable Harness.
- Combining later successful Trials into an earlier reference score.
- Reading a filtered population as the entire benchmark.
- Changing a rubric judgment when the real problem is an incorrect Case or standard.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Evaluation Dashboard](/docs/benchmark-evaluations/dashboard)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
- [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics)
{% /related-card-grid %}
---
id: benchmark-evaluations.inspect
title: Inspect Evaluation Results
summary: Trace scores to exact responses and rubric judgments, distinguish incomplete results, and record authorized corrections.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/inspect-results
---
# Inspect Evaluation Results
Start with identity and completeness, then explain the score from the actual response and applicable Rubrics.
## Prerequisites
- Access to the selected Benchmark Version and Run.
- Available responses or evaluation progress.
- Project-management permission if a judgment correction is needed.
## Steps
1. Open Runs and confirm the candidate, saved Harness Version where present, machine and planned Trials.
2. Identify whether this is the Run used for score or another independent execution.
3. Open Results and inspect the Case population, applicable criteria and incomplete count.
4. Read failed gates, earned positive points and triggered penalties separately.
5. Open the response and rubric detail. Compare the criterion with the actual behavior and visible Case materials.
6. Inspect individual Trial outcomes when reading a repeated result.
7. Decide whether the responsible change belongs to candidate code, Case content, applicability, rubric wording or the observed judgment.
## Scores and missing results
Point scores can be negative, including below -100% for mixed positive/penalty Cases. A completed mandatory-gate failure contributes zero to the benchmark score. Missing or incompatible judgments stay incomplete even when another gate has failed.
The full planned population remains the denominator. An incomplete evaluation cannot be made comparable by ignoring its difficult or failed executions.
## Correct a judgment
An authorized project manager can choose **Correct judgment** on the exact Trial's rubric result and supply the supported corrected judgment and rationale. The original AI judgment and manual change remain recorded. Reopen the refreshed result to interpret its resulting scores consistently.
Changing the judgment is different from editing the Rubric or the Case. Correct the owning artifact when its meaning is wrong.
## Object and state changes
Inspection and filtering do not mutate results. A judgment correction records an attributable authoritative change. Starting a Contribution or an improvement Session creates separate work and preserves the inspected Run.
## Success criteria
- The conclusion identifies its Run, Trial and Benchmark Version.
- Scores are explained by applicable criteria and the configured Regime.
- Missing telemetry and judgments remain unknown.
- Corrections retain original judgment provenance.
## Common failure modes
- Reporting a rank without completeness.
- Assuming all results are read-only.
- Treating a gate failure as permission to replace unavailable results with zero.
- Inferring hidden model reasoning from a captured trajectory.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Run Details and Metadata](/docs/benchmark-evaluations/run-details)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
- [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics)
{% /related-card-grid %}
---
id: benchmark-evaluations.arena-rankings
title: Evaluation Dashboard
summary: Read benchmark Progress, Leaderboard, and Frontier using the recorded candidate population, score meaning, and completeness.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/dashboard
---
# Evaluation Dashboard
Dashboard summarizes benchmark performance through **Progress**, **Leaderboard** and **Frontier**. Use Results when a chart needs Case-level explanation.
## Prerequisites
- Runs for one identified Benchmark Version.
- A clear understanding of the selected metric and Regime.
- Complete results for claims that require a rankable comparison.
## Read the dashboard
1. Confirm the Benchmark Version and candidate identities.
2. Choose Progress to inspect movement over recorded executions, Leaderboard for the selected performance ordering, or Frontier to inspect visible tradeoffs.
3. Read the metric's population, aggregation and completeness.
4. Follow a surprising value into Results and inspect its Cases and Rubrics.
5. Preserve regressions and resource tradeoffs alongside gains.
A chart is a summary, not a new approval state. The newest candidate or the highest displayed aggregate does not automatically become the selected implementation.
## Repeated behavior
Repeated Trials support different questions. At-least-one success and all-Trials success are different reliability statements. Read pass@n or pass^n only with the recorded Trial count and cohort. Ordinary scores and means answer different questions again.
Uncertainty belongs to the measured sample. A small numerical lead does not remove missing judgments, a failed mandatory gate or concentrated regressions.
## Object and state changes
Dashboard controls read existing results. They do not launch a Run, alter representative score selection, correct a judgment or retain a Lemon Goal candidate.
The reference Run used for a candidate score is selected before outcomes are known. Additional independent Runs remain available in history rather than silently changing that reference population.
## Success criteria
- The metric, population and repeated-Trial rule are explicit.
- Incomplete results remain visibly incomplete.
- A meaningful change can be traced to exact responses and criteria.
- Cost, tokens and latency are used only when actually captured.
## Common failure modes
- Treating a ranking as an approval.
- Substituting later successful Trials for missing reference results.
- Reading a negative point score as an impossible result.
- Treating absent telemetry as zero consumption.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
- [Missing Outputs](/docs/troubleshooting/missing-outputs)
{% /related-card-grid %}
---
id: correctness.scoring
title: Rubric Points and Benchmark Scores
summary: Interpret earned points, penalties, mandatory gates, negative percentages, and incomplete benchmark results.
kind: reference
product_area: correctness_governance
status: stable
updated: 2026-10-30
canonical: /docs/correctness-governance/scoring
---
# Rubric Points and Benchmark Scores
## Definition
A binary rubric judgment describes observed behavior. The Regime determines how that judgment contributes to a Case and Benchmark score. Positive points, penalties and mandatory gates have distinct meanings.
## Fields, states, or lifecycle rules
- Positive points are earned when the relevant Rubric passes.
- Penalty points are deducted when its Rubric fails.
- A mandatory gate determines eligibility in a gated framework.
- Missing or incompatible judgments remain incomplete.
- Exact recorded fractions are displayed as percentages.
## Case score
For current point-based configuration, the Case score is:
```text
(earned positive points - triggered penalty points)
/ total applicable positive points
```
If there are no applicable positive points, the denominator is the total applicable penalty points. If that denominator is also zero, the Case is unscored.
A penalty-only Case ranges from 0% when no penalty triggers to -100% when all trigger. A mixed Case can score below -100%. There is no per-Case floor, cap or output scaling in the current configuration.
{% example-demo title="A valid negative score" %}
A Case has 2 applicable positive points and 6 penalty points. It earns both positive points and triggers every penalty. Its score is (2 - 6) / 2, or -200%. The negative result preserves the configured consequence rather than being clamped to zero.
{% /example-demo %}
## Gates and incomplete judgments
A completed mandatory-gate failure contributes zero to the benchmark score. Its rubric calculation remains available for diagnosis. Missing judgments do not become zero merely because another gate failed.
Benchmark and repeated-Trial means retain the entire frozen population. An unresolved Case or Trial makes the evaluation incomplete and unrankable. Technical execution failure is not a valid rubric failure.
## History and configuration
Project Settings → Regime configures point ranges, increments and guidance. New settings affect future captured work. Earlier Benchmark Versions retain their accepted scoring configuration; historical scaling rules are interpreted under that configuration.
Use **Correct judgment** for an attributable correction to an exact result. Use the governing Policy, Rubric or Regime settings when the intended standard is what needs to change.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Regime Settings](/docs/project-settings/regime)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Write Binary Rubrics](/docs/correctness-governance/binary-rubrics)
{% /related-card-grid %}
---
id: troubleshooting.benchmark-results-changed
title: Benchmark Results Changed Unexpectedly
summary: Diagnose result changes across target behavior, benchmark cases, standards, and versions.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-10-30
canonical: /docs/troubleshooting/benchmark-results-changed-unexpectedly
---
# Benchmark Results Changed Unexpectedly
Use this when benchmark results change and the team is not sure whether the cause is model behavior or an artifact change.
## Symptom
A pass rate, policy-level result, rubric result, or case-level outcome changes between runs even though the expected candidate behavior did not obviously change.
## Likely causes
- Benchmark membership changed through case import, promotion, removal, or refresh.
- Policy, rubric, or applicability versions changed between runs.
- A different saved Harness Version, execution setting, or imported output-only Run was inspected.
- Run metadata or benchmark version selection differs from the previous run.
## Diagnostic checks
- Compare benchmark version, case count, policy versions, rubric versions, and applicability boundary.
- Inspect changed Cases and confirm whether each result came from a managed Run or imported output-only Run.
- Check run metadata for candidate, prompt, retrieval, or model differences.
- Open policy/rubric result changes and trace them to exact cases.
## Fix
- If artifacts changed, label the comparison as an artifact-boundary change rather than a pure behavior regression.
- If the candidate identity or settings changed, run the intended saved Harness Version with the intended settings.
- If the Benchmark boundary changed, create and name the appropriate Snapshot and Benchmark Version; do not rewrite the older Run.
- If the cause remains unclear, hold downstream action until the changed supporting information can be explained.
## Prevention
- Record benchmark, case, policy, rubric, and candidate versions for every run.
- Use comparison views before summarizing score movement.
- Treat mapping, coverage, and standard changes as review-context boundaries.
- Keep previous runs reproducible for audit.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Evaluation Execution Settings](/docs/benchmark-evaluations/execution-settings)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Versions, staleness, and resolution](/docs/object-model/versions-staleness-and-resolution)
- [Benchmark Evaluations](/docs/benchmark-evaluations)
- [Benchmark Versioning](/docs/governance/benchmark-versioning)
{% /related-card-grid %}
---
id: troubleshooting.overlapping-rubrics
title: Overlapping Rubrics
summary: Resolve rubrics that ask the same question or create contradictory supporting information.
kind: error
product_area: troubleshooting
status: stable
updated: 2026-10-30
canonical: /docs/troubleshooting/overlapping-rubrics
---
# Overlapping Rubrics
Use this when multiple rubrics appear to fail the same behavior or produce contradictory supporting information.
## Symptom
A single output triggers several rubric failures that all describe the same defect, or two rubrics disagree about whether the same behavior should pass. Review context becomes noisy because failures are counted more than once.
## Likely causes
- A new rubric was created without checking existing approved rubrics.
- One rubric tests several behaviors and overlaps narrower checks.
- Policy revisions changed the behavior boundary but old rubrics remained active.
- Applicability rules bind multiple rubrics to the same irrelevant cases.
## Diagnostic checks
- Group failures by output and identify rubrics that cite the same supporting information.
- Compare rubric pass/fail wording against linked policies.
- Check rubric versions and whether old versions are still bound to benchmark cases.
- Inspect applicability rules for rubrics that should be mutually exclusive.
## Fix
- Choose the Rubric definition that should govern future evaluations and revise or version the affected governed objects through their available controls. Do not claim a merge or retire action unless it is visible in the current surface.
- Split multi-behavior rubrics into one binary check each.
- Revise applicability so rubrics apply only to the intended cases.
- Rerun affected benchmark slices after approved rubric changes.
## Prevention
- Search existing rubrics before approving a new one.
- Require a passing and failing example for every rubric boundary.
- Review overlap during policy/rubric approval.
- Treat rubric edits as versioned changes that can affect benchmark comparisons.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Rubrics](/docs/object-model/rubrics)
- [Rubric Versioning](/docs/governance/rubric-versioning)
- [Applicability logic](/docs/object-model/applicability-logic)
{% /related-card-grid %}