# Run Details and Metadata
Generated: 2026-11-02T07:47:46.323Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: project-settings.run-metadata
title: Run Details and Metadata
summary: Inspect the identity, recorded settings, source, telemetry, and status that explain an evaluation Run.
kind: reference
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/run-details
---
# Run Details and Metadata
## Definition
A Run's details explain which candidate was measured, against which Benchmark Version, under which execution conditions. Read the recorded Run rather than today's project defaults.
## Fields, states, or lifecycle rules
- Managed Runs identify a saved Harness Version and captured runtime.
- Imported Runs identify the external submission and its exact Cases.
- Planned Trials, machine allocation and evaluation settings describe the recorded execution.
- Status distinguishes output progress from rubric evaluation progress.
- Labels and descriptive metadata help interpretation but do not replace issued identities.
- Tokens, latency and cost are nullable measurements. Missing capture remains unknown.
## Run history and score use
Use Run history to inspect independent executions of the same candidate. **Used for score** identifies the reference selected before outcomes were known. **Run evaluation** reopens matching work; **Run again** deliberately adds an independent Run.
A later label or metadata change does not make two different execution populations identical. Retain the exact Run and Trial links when recording a comparison or asking an expert about a response.
## Settings versus observations
Machine allocation and planned Trial count are accepted execution commitments. Measured latency, token use and actual responses are observations produced afterward. Keep both when explaining an operational difference.
Historical descriptive fields remain attached to their Runs. They do not imply that there is a current project-level Run Metadata configuration tab or that editing today's settings changes accepted history.
## Using details in follow-up work
Link an expert to the relevant response and criterion rather than only an aggregate. When copying an External Agents task, confirm whether its optional reference Run represents the code the agent will change. When starting Lemon Goal, inspect the selected starting candidate and available measurements.
Never put credentials in labels, metadata or Case content. Grant secrets through the owning settings and Harness controls.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results)
- [Work with External Agents](/docs/improve/external-agents)
{% /related-card-grid %}
---
id: benchmark-evaluations.run
title: Run a Benchmark Evaluation
summary: Choose saved Harness Versions, an execution machine, and Trials, then launch or reuse a comparable Run.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/run-evaluation
---
# Run a Benchmark Evaluation
Use a managed evaluation to execute a saved Harness Version against every Case in the selected Benchmark Version.
## Prerequisites
- A Benchmark Version with the intended Cases and eligible evaluation rules.
- A saved Harness Version with its runtime and required secret grants.
- Permission to run evaluations and an available execution machine.
## Steps
1. Open the Benchmark's **Benchmark Evaluations → Runs**.
2. Choose **Run evaluation** and select the saved Harness Versions. Confirm each Version rather than relying on the Harness name.
3. Select the execution machine. Its CPU and memory allocation must meet the Harness requirements.
4. Keep one Trial for a single execution, or select **Repeat Trials** and enter the total Trials per Run, including the first.
5. Check the displayed Case executions and confirm the launch.
6. Follow output preparation and rubric evaluation separately. Responses can finish before their judgments.
7. Open Results after checking the planned population and completeness.
The service accepts 1–50 Trials per selected candidate. The launch form checks a positive whole number; the service enforces its upper bound. This is a per-launch choice rather than a request to change a benchmark-wide sampling profile.
## Reuse and deliberate repeats
Run evaluation reopens a Run with matching immutable conditions when one already exists, including a queued, running, completed or failed Run. It does not start extra Trials merely because the action was selected again.
Use **Run again** when another independent execution is intended. A repeat keeps the referenced conditions. Changed conditions must be handled explicitly rather than silently joining a different population.
## Object and state changes
A new admission records one Run per selected candidate, its Trials, exact Harness and Benchmark Versions, execution allocation and evaluation settings. A matching admission returns the existing Run. Recorded responses, judgments and telemetry accumulate independently.
The Run automatically selected as **Used for score** is chosen before results are known. Later repeats remain in Run history and do not silently replace it.
## Success criteria
- The selected Versions, machine and Trial count match the intended experiment.
- Every planned Case and Trial remains accounted for.
- Missing judgments remain incomplete.
- A repeat is distinguishable from reopening existing work.
## Common failure modes
- Choosing a different saved Version than intended.
- Mistaking output completion for completed evaluation.
- Treating Run evaluation as an unconditional repeat.
- Reading only successful Trials from an incomplete Run.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Evaluation Execution Settings](/docs/benchmark-evaluations/execution-settings)
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Run Details and Metadata](/docs/benchmark-evaluations/run-details)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark run troubleshooting](/docs/troubleshooting/benchmark-runs)
- [Missing Outputs](/docs/troubleshooting/missing-outputs)
{% /related-card-grid %}
---
id: benchmark-evaluations.inspect
title: Inspect Evaluation Results
summary: Trace scores to exact responses and rubric judgments, distinguish incomplete results, and record authorized corrections.
kind: task
product_area: benchmark_evaluations
status: stable
updated: 2026-10-30
canonical: /docs/benchmark-evaluations/inspect-results
---
# Inspect Evaluation Results
Start with identity and completeness, then explain the score from the actual response and applicable Rubrics.
## Prerequisites
- Access to the selected Benchmark Version and Run.
- Available responses or evaluation progress.
- Project-management permission if a judgment correction is needed.
## Steps
1. Open Runs and confirm the candidate, saved Harness Version where present, machine and planned Trials.
2. Identify whether this is the Run used for score or another independent execution.
3. Open Results and inspect the Case population, applicable criteria and incomplete count.
4. Read failed gates, earned positive points and triggered penalties separately.
5. Open the response and rubric detail. Compare the criterion with the actual behavior and visible Case materials.
6. Inspect individual Trial outcomes when reading a repeated result.
7. Decide whether the responsible change belongs to candidate code, Case content, applicability, rubric wording or the observed judgment.
## Scores and missing results
Point scores can be negative, including below -100% for mixed positive/penalty Cases. A completed mandatory-gate failure contributes zero to the benchmark score. Missing or incompatible judgments stay incomplete even when another gate has failed.
The full planned population remains the denominator. An incomplete evaluation cannot be made comparable by ignoring its difficult or failed executions.
## Correct a judgment
An authorized project manager can choose **Correct judgment** on the exact Trial's rubric result and supply the supported corrected judgment and rationale. The original AI judgment and manual change remain recorded. Reopen the refreshed result to interpret its resulting scores consistently.
Changing the judgment is different from editing the Rubric or the Case. Correct the owning artifact when its meaning is wrong.
## Object and state changes
Inspection and filtering do not mutate results. A judgment correction records an attributable authoritative change. Starting a Contribution or an improvement Session creates separate work and preserves the inspected Run.
## Success criteria
- The conclusion identifies its Run, Trial and Benchmark Version.
- Scores are explained by applicable criteria and the configured Regime.
- Missing telemetry and judgments remain unknown.
- Corrections retain original judgment provenance.
## Common failure modes
- Reporting a rank without completeness.
- Assuming all results are read-only.
- Treating a gate failure as permission to replace unavailable results with zero.
- Inferring hidden model reasoning from a captured trajectory.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Run Details and Metadata](/docs/benchmark-evaluations/run-details)
- [Rubric Points and Benchmark Scores](/docs/correctness-governance/scoring)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Benchmark Results Changed Unexpectedly](/docs/troubleshooting/benchmark-results-changed-unexpectedly)
- [Overlapping Rubrics](/docs/troubleshooting/overlapping-rubrics)
{% /related-card-grid %}
---
id: improve.work-evolve
title: Work with External Agents
summary: Copy a benchmark-scoped task for your coding agent to implement and evaluate in your real application repository.
kind: task
product_area: improve
status: stable
updated: 2026-10-30
canonical: /docs/improve/external-agents
---
# Work with External Agents
External Agents gives your coding agent a concrete improvement task grounded in the selected Benchmark Version. The agent works in your real repository; Teammately supplies the applicable Cases, Policies, Rubrics and recorded results.
## Prerequisites
- A selectable Benchmark Version.
- Access to the application repository and authorization for the intended changes.
- A coding agent connected through the setup supplied with the task.
- An optional reference Run that genuinely represents the application starting point.
## Steps
1. Open **Harness improvement → External Agents**.
2. Select the Benchmark Version. Changing it clears local criterion and reference-Run selections.
3. Focus on the whole Benchmark or choose up to 20 approved and applicable Policy and Rubric Versions.
4. Optionally select one individual reference Run. No Run is silently selected.
5. Choose the allowed improvement direction: supported practices, prompt only, model only, the same model family, or model configuration only.
6. Copy the task, including setup instructions when needed, into your coding-agent conversation.
7. Have the agent inspect the actual repository, retrieve the pinned criteria and Run context, implement the change and evaluate it.
8. Inspect resulting Runs through ordinary Benchmark Evaluations.
The copied task includes the Project Agent Brief when present. Credentials do not belong in that text.
## Execution choices
A Teammately Harness and source upload are optional. Customer-executed responses are the default path for the agent's real application. Managed Harness execution requires a deliberate choice.
The product data MCP supplies read-only context, instructions, results and comparisons. The CLI owns local files and submission operations. The documentation MCP only retrieves these manuals.
Start with [CLI authentication](/docs/cli/authentication) and [product MCP setup](/docs/cli/mcp), then follow [external output submission](/docs/cli/external-outputs) or the optional [managed Harness workflow](/docs/cli/managed-harness).
## Object and state changes
The composer keeps selections locally and copies text. It creates no hosted agent Session, transcript, durable handoff object, automatic code adoption or Harness activation. Later submitted responses or managed execution create their own ordinary Run records.
## Success criteria
- The task preserves the intended Benchmark Version and criteria.
- A selected reference Run is checked against actual code.
- Implementation and comparison use actual pre-change and post-change behavior.
- Accepted responses and candidate identity remain attributable.
## Common failure modes
- Stopping after tool setup or context retrieval.
- Assuming a Harness upload is mandatory.
- Treating copied text as an executed Session.
- Using a reference Run from unrelated code.
- Claiming private agent work before returned artifacts or results exist.
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Harness Improvement](/docs/improve)
- [Import External Evaluation Outputs](/docs/benchmark-evaluations/output-mapping)
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Project Context](/docs/agent-setup/project-context)
{% /related-card-grid %}
## Related troubleshooting pages
{% related-card-grid title="Related troubleshooting pages" %}
- [Output mapping troubleshooting](/docs/troubleshooting/output-mapping)
- [Permissions troubleshooting](/docs/troubleshooting/permissions)
{% /related-card-grid %}