# Configure Run Metadata Generated: 2026-11-13T08:01:26.403Z Source build: local Canonical docs: https://teammately.ai/docs --- id: workflow.project-settings.run-metadata title: Configure Run Metadata summary: Preserve the candidate and execution context needed to interpret a Run. kind: task product_area: project_settings status: stable updated: 2026-11-11 canonical: /docs/project-settings/run-metadata --- # Configure Run Metadata Preserve the candidate and execution context needed to interpret a Run. ## Prerequisites The intended Benchmark Version and the managed or external execution workflow you will use. ## Steps 1. Identify the candidate Version or external candidate context before submission. 2. Supply metadata supported by the chosen execution or import contract. Keep application labels distinct from Case input, scoring configuration, and immutable version identities. 3. Inspect the accepted Run details and its recorded execution conditions. 4. Use those records when comparing results so a descriptive label is not mistaken for proof that candidate or experiment conditions matched. ## Object and state changes Run metadata describes the recorded operation. It does not change the selected Case population, activate a Harness, or rewrite accepted results. ## Success criteria A teammate can identify what ran, against which Benchmark Version, and under which recorded conditions. ## Common failure modes Using a friendly candidate name as its only identity; assuming metadata overrides the actual submitted candidate or input. ## Related reference pages {% related-card-grid title="Continue this workflow" %} - [Run Details and Metadata](/docs/benchmark-evaluations/run-details) {% /related-card-grid %} ## Related troubleshooting pages {% related-card-grid title="Resolve a problem" %} - [Permissions troubleshooting](/docs/troubleshooting/permissions) - [Benchmark run troubleshooting](/docs/troubleshooting/benchmark-runs) {% /related-card-grid %} --- id: project-settings.run-metadata title: Run Details and Metadata summary: Inspect the identity, recorded settings, source, telemetry, and status that explain an evaluation Run. kind: reference product_area: benchmark_evaluations status: stable updated: 2026-10-30 canonical: /docs/benchmark-evaluations/run-details --- # Run Details and Metadata ## Definition A Run's details explain which candidate was measured, against which Benchmark Version, under which execution conditions. Read the recorded Run rather than today's project defaults. ## Fields, states, or lifecycle rules - Managed Runs identify a saved Harness Version and captured runtime. - Imported Runs identify the external submission and its exact Cases. - Planned Trials, machine allocation and evaluation settings describe the recorded execution. - Status distinguishes output progress from rubric evaluation progress. - Labels and descriptive metadata help interpretation but do not replace issued identities. - Tokens, latency and cost are nullable measurements. Missing capture remains unknown. ## Run history and score use Use Run history to inspect independent executions of the same candidate. **Used for score** identifies the reference selected before outcomes were known. **Run evaluation** reopens matching work; **Run again** deliberately adds an independent Run. A later label or metadata change does not make two different execution populations identical. Retain the exact Run and Trial links when recording a comparison or asking an expert about a response. ## Settings versus observations Machine allocation and planned Trial count are accepted execution commitments. Measured latency, token use and actual responses are observations produced afterward. Keep both when explaining an operational difference. Historical descriptive fields remain attached to their Runs. They do not imply that there is a current project-level Run Metadata configuration tab or that editing today's settings changes accepted history. ## Using details in follow-up work Link an expert to the relevant response and criterion rather than only an aggregate. When copying an External Agents task, confirm whether its optional reference Run represents the code the agent will change. When starting Lemon Goal, inspect the selected starting candidate and available measurements. Never put credentials in labels, metadata or Case content. Grant secrets through the owning settings and Harness controls. ## Related task pages {% related-card-grid title="Related task pages" %} - [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation) - [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results) - [Work with External Agents](/docs/improve/external-agents) {% /related-card-grid %} --- id: troubleshooting.permissions title: Permissions troubleshooting summary: Separate Project membership from Expert Contribution assignment and readiness when an expert has no tasks available. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/permissions --- # Permissions troubleshooting Use this when project role, reviewer assignment, or approval ownership prevents someone from completing the intended work. ## Symptom A user can access Teammately but cannot open the expected Project, or an expert reports **no tasks available** after opening their Contribution link. These symptoms belong to different owning surfaces. ## Likely causes - The user lacks the required Project permission through a direct grant, workspace group, or workspace role. - The Contribution was requested for a different recipient. - The Contribution exists but its next Task or Checkpoint is not ready. - The current IP is blocked before Project membership is evaluated. ## Diagnostic checks - Check direct grants in **Project Settings → Project Members**, the user's workspace-group membership, and the group's permission for the intended Project. Absence from the direct member list alone does not establish lack of access. Inspect workspace status and role in **Admin → Directory → Members** as well. - Open the Contribution from the sender-side Expert Contributions workspace and verify its recipient and status. - Check whether the expert has an executable Task or Checkpoint, not merely whether the Contribution exists. - If **Access Restricted** is visible, resolve the IP allowlist first. ## Fix - Add the correct individual workspace user under **Project Settings → Project Members** when direct Project membership is missing. - If access is intended through a workspace group, confirm both the user's membership in that group and its permission for the intended Project. - Correct the Contribution recipient through the owning Contribution workflow when the request went to the wrong person. - Resolve lifecycle or readiness problems in Expert Contributions when access succeeds but no Task can be entered. - Do not infer a permission from a role label or edit governed artifacts merely to make a control appear. ## Prevention - Separate reviewer access from approval authority in review setup. - Check assignments before launching a review session. - Use a small pilot Contribution before assigning a larger specialist cohort. - Keep project membership changes visible to review owners. ## Related task pages {% related-card-grid title="Related task pages" %} - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) - [Expert Contributions](/docs/expert-contributions) - [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Permissions](/docs/reference/permissions) - [Roles and Permissions](/docs/governance/roles-and-permissions) - [Expert Contributions](/docs/expert-contributions) {% /related-card-grid %} --- id: troubleshooting.benchmark-runs title: Benchmark run troubleshooting summary: Diagnose a Run that cannot start, has no usable outputs, or produces results that cannot be compared safely. kind: error product_area: troubleshooting status: stable updated: 2026-11-06 canonical: /docs/troubleshooting/benchmark-runs --- # Benchmark run troubleshooting ## Symptoms - The evaluation surface has no Benchmark Version to run. - Output import or mapping cannot identify a Case or output column. - A Run is created but remains empty, incomplete, or failed. - Results appear under the wrong candidate label or Harness Version. - Two Runs show a score change but do not share a comparable snapshot boundary. ## Likely causes - No immutable Snapshot exists for the intended evaluation set. - Imported outputs are mapped to the wrong Case or column. - The saved Harness Version or candidate metadata does not match the evaluated system. - The Run is partial, failed, or being compared across different Benchmark Versions. ## Check the boundary before the failure 1. Confirm the URL and page identify the intended Project, Benchmark, and Benchmark Version. 2. Open the Version or Snapshot and verify it contains the expected Cases and evaluators. If it does not, repair coverage and create a new Snapshot; do not edit the historical Run. 3. For managed execution, confirm the saved Harness Version and recorded settings. For imported results, confirm that the external candidate context describes the system that produced the submitted responses. 4. If importing outputs, inspect the mapping preview. Match the Case identifier and candidate-output column deliberately; do not use a reference-output column as candidate behavior. 5. Open Run detail and inspect status, Case count, errors, metadata, and per-Case results before trusting aggregates. ## Fix - **No runnable version:** finish Case selection and create a Snapshot first. - **Missing imported outputs:** append only missing Cases to an uncompleted Trial using the exact manifest IDs. Correcting an accepted response or adding Cases after completion requires a new Trial in the external Run or a new Run. Preserve the original results. - **Wrong Harness or metadata:** create a correctly configured Run. Do not relabel completed results to represent a different system. - **Partial managed failure:** inspect and preserve the existing Run's recorded outcomes. For another independent execution under unchanged conditions, use **Run again**. **Run evaluation** reopens matching work, including failed work. Publish a new Harness Version when its code changes, or create a new Snapshot and Benchmark Version when Case content or criteria change. - **Confusing comparison:** compare the Benchmark Version, Harness Version, candidate metadata, and evaluator boundary. Qualify or avoid the comparison when more than the intended variable changed. ## Prevention Create a Snapshot before evaluation. For managed execution, save the exact Harness Version. For imported results, preserve external candidate context and preview the mapping on representative Cases. Before comparing, confirm that every difference between the two Runs is intentional and visible. ## Recovery check Open the recovered Run and sample several Case results. Confirm the displayed input, candidate output, reference output where present, applicable Rubrics, and metadata all belong together. A completed status alone does not prove correct mapping. {% example-demo title="Example: scores drop after output import" %} A team imports a new candidate file and sees a sudden score collapse. Run detail shows that the column containing reference outputs was mapped as candidate output. The operator creates a new Run, maps the actual candidate column, preserves the mistaken Run as an identifiable failed attempt, and compares only the corrected Run with the prior candidate under the same Benchmark Version. {% /example-demo %} ## Related task pages {% related-card-grid title="Related workflows" %} - [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation) - [Import External Evaluation Outputs](/docs/benchmark-evaluations/output-mapping) - [Inspect Evaluation Results](/docs/benchmark-evaluations/inspect-results) - [Evaluation Results](/docs/benchmark-evaluations/results) - [Run Details and Metadata](/docs/benchmark-evaluations/run-details) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference" %} - [Benchmarks and versions](/docs/concepts/benchmarks-and-versions) - [Benchmark snapshots](/docs/coverage-engineering/benchmark-snapshots) - [Outputs](/docs/object-model/outputs) {% /related-card-grid %}