# Teammately Reference Context
Generated: 2026-09-13T04:31:28.741Z
Source build: local
Canonical docs: https://teammately.ai/docs
---
id: admin-console.overview
title: Admin Console
summary: Understand the organization-level administration surfaces available at admin.teammately.ai.
kind: reference
product_area: admin_console
status: stable
updated: 2026-09-07
canonical: /docs/admin-console
---
# Admin Console
The current Teammately product exposes the Admin Console as a workspace-administration mode under `/admin`. Use this page to distinguish organization-level administration from project-level correctness work in the project workspace. The shared product shell switches between project navigation and workspace administration while preserving the organization-level boundary.
> Administration boundary
>
> Use the Admin Console for organization controls. Use project and benchmark workspace docs for Cases, Expert Contributions, Policies, Rubrics, coverage, evaluations, and improvement evidence.
## Definition
The Admin Console groups organization administration into Directory, Security, Insights, and Organization Settings. The code-backed navigation includes Members, Groups, Roles & Permissions, Domain Control, IP Address Control, Audit Log, Usage Statistics, Profile, Support Settings, Integrations, and API Keys.
This page documents that those surfaces exist. It does not claim detailed compliance, billing, deployment, rate-limit, or security behavior unless a linked source file exposes that behavior directly.
The Admin Console is the organization administration surface. Group management is separate from project-level correctness work.
The Integrations screen shows notification and partner connection surfaces that belong to organization administration. Visible contact-support states should not be documented as self-serve integrations.
API Keys are managed in the Admin Console and should be documented as an organization administration surface, separate from project correctness artifacts.
> API key boundary
>
> Creating an API key establishes a credential; it does not make every internal endpoint a supported customer API. Endpoint availability, bearer authentication, required scopes, request and response schemas, and error behavior belong to the versioned Public API contract.
## Fields, states, or lifecycle rules
- Members and Groups are organization directory surfaces. Project-level membership and project permissions remain separate surfaces in the main product.
- Roles & Permissions in the Admin Console manage workspace roles and permission keys exposed by the admin application. Do not treat role names in orientation pages as exact permission contracts.
- Domain Control stores whether domain enforcement is enabled and which email domains are allowed for invitation.
- IP Address Control stores whether IP limiting is enabled and the configured allowlist.
- API Keys can be listed, created, copied at creation time, scoped, and deleted from the admin app surface. Treat the full value as a secret and create separate keys for separate integration boundaries.
- Slack is the currently clickable self-serve notification integration in the Admin Console. Microsoft Teams and the other listed partners are visible as contact-support or coming-soon surfaces in the current integrations index, so these docs should not describe them as self-serve integrations.
- Audit Log and Usage Statistics appear as admin console surfaces, but this page does not promise exact event schemas, retention periods, analytics definitions, or export behavior.
Domain Control and IP Address Control are organization-level security settings. They should not be described as project review or benchmark approval mechanisms.
IP Address Control exposes enforcement settings and should be documented only as an admin access-control surface.
## Product boundary
Use the Admin Console when the question is about organization access, security controls, API keys, notification integrations, or organization-level activity. Use project settings and product-area docs when the question is about cases, reviews, policies, rubrics, benchmarks, and project-specific permissions.
A reviewer persona, a Project member, a Workspace role, and an organization administrator are related but not identical. Read each access or authority claim from its owning surface.
Likewise, an API key, its Project access, and an endpoint's action scope are related but not identical. Key administration belongs here; the external service contract must define what a caller can actually do.
## Source confidence
Code-backed: this page is grounded in the current `/admin` workspace route, shared navigation mode, admin directory, integrations index, and representative admin route implementations listed in the frontmatter source_refs.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Manage Project Members](/docs/project-settings/project-members)
- [Resolve an IP access restriction](/docs/troubleshooting/authentication)
- [Troubleshoot permissions](/docs/troubleshooting/permissions)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Workspace administration](/docs/governance/workspace-administration)
- [Roles and permissions](/docs/governance/roles-and-permissions)
- [Permissions](/docs/reference/permissions)
{% /related-card-grid %}
---
id: agent-context.index
title: Agent context index
summary: Use Teammately docs safely from AI agents, retrieval tools, and coding assistants.
kind: reference
product_area: agent_context
status: stable
updated: 2026-08-23
canonical: /docs/agent-context
---
# Agent context index
Use this page to choose the smallest Teammately docs context package that can answer a product, workflow, object, troubleshooting, or code-assistance question. This page is about retrieval surfaces and source boundaries, not autonomous agent behavior policy.
## Task intent
Select a context source before reading or summarizing Teammately docs. Agents should start with the smallest pack or retrieval query that matches the task, then load page-local context only when the answer needs exact wording.
The decision rule is conservative: load less context first, cite stable public pages, and stop before inferring product contracts from draft docs or internal file paths.
## Decision checkpoint
| Agent task | Start with | Escalate only when... |
| --- | --- | --- |
| Route a human to the right docs | `/docs/llms.txt` or machine maps | The user needs page-local wording. |
| Explain category or first path | `/docs/llms-core.txt` | A specific workflow page is needed. |
| Answer an operating question | `/docs/llms-operating.txt` or `/api/docs/context` | The answer depends on exact source confidence. |
| Explain object semantics | `/docs/llms-reference.txt` | Exact page retrieval is needed. |
| Diagnose a symptom | `/docs/llms-recovery.txt` | The symptom spans multiple artifacts. |
| Work through an MCP client | MCP `search` | Exact page or block Markdown is needed through `fetch`. |
| Build an index or audit | `/docs/llms-full.txt` | Full-corpus context is intentionally required. |
## Required context
- Root map: [/docs/llms.txt](/docs/llms.txt) for the concise docs map and recommended loading strategy.
- Core pack: [/docs/llms-core.txt](/docs/llms-core.txt) for category, lifecycle, product boundaries, quickstart, and core model concepts.
- Operating pack: [/docs/llms-operating.txt](/docs/llms-operating.txt) for task and workflow guidance.
- Reference pack: [/docs/llms-reference.txt](/docs/llms-reference.txt) for object model and reference pages.
- Recovery pack: [/docs/llms-recovery.txt](/docs/llms-recovery.txt) for troubleshooting pages.
- Retrieval APIs: [/api/docs/search](/api/docs/search), [/api/docs/context](/api/docs/context), and [/api/docs/page](/api/docs/page).
- Machine maps: [/docs/tasks.json](/docs/tasks.json), [/docs/objects.json](/docs/objects.json), [/docs/surfaces.json](/docs/surfaces.json), and [/docs/troubleshooting.json](/docs/troubleshooting.json).
- Private Docs MCP: `https://docs-mcp.teammately.ai/mcp`, with `search`, `fetch`, and `list` tools after connection with `docs:read` access.
## Context pack strategy
1. Start with [/docs/llms.txt](/docs/llms.txt) to understand the docs map and source-confidence policy.
2. Use one focused pack: core for positioning, operating for workflows, reference for objects, or recovery for symptoms.
3. Use [/api/docs/context](/api/docs/context) for query-scoped context when the task is narrow.
4. Load /docs//docs/llms-full.txt only when a specific page needs full context.
5. Treat [/docs/llms-full.txt](/docs/llms-full.txt) as an intentional full-corpus dump for indexing or exhaustive review, not as the default context.
## MCP retrieval strategy
1. Call `search` with the user's question and an optional `kind` or product-area filter.
2. Read the returned search mode and build metadata. Semantic retrieval can fall back to lexical retrieval without making results invalid.
3. Use cited `blockId`, `pageId`, or `slug` with `fetch` before quoting exact guidance or applying an instruction.
4. Use `list` only for navigation or filtered inventory; it is not a substitute for page content.
5. Treat AI Overview text as a cited routing aid. The referenced blocks and fetched page remain the evidence boundary.
The MCP resources `teammately-docs://manifest`, `teammately-docs://docs/{slug}`, and `teammately-docs://blocks/{blockId}` provide the same active corpus at manifest, page, and block granularity.
## Agent answerability check
| Before answering, verify... | If not... |
| --- | --- |
| The cited page is stable and public. | Say the source boundary is not public rather than quoting draft files. |
| Source confidence matches the claim strength. | Downgrade the answer or ask for human confirmation. |
| The route answers the user's task without loading unrelated packs. | Use search or `/api/docs/context` before full-corpus context. |
| MCP overview claims resolve to returned block references. | Fetch the cited block or page before relying on exact wording. |
| The claim is not about API, auth, billing, compliance, retention, deployment, or provider handling unless a source-backed page says so. | State that the docs do not currently confirm the contract. |
## Available routes
- [/docs/.md](/docs/quickstart.md): page-level markdown export.
- [/docs//docs/llms.txt](/docs/quickstart/llms.txt): compact page-local context.
- [/docs//docs/llms-full.txt](/docs/quickstart/llms-full.txt): page-local full context.
- [/api/docs/manifest.json](/api/docs/manifest.json): stable public page metadata, source confidence, source refs, and related IDs.
- [/api/docs/page?slug=quickstart](/api/docs/page?slug=quickstart): exact public page retrieval.
## Allowed assumptions
- Stable public docs are the public source of truth for published guidance.
- Draft and removed pages are intentionally excluded from public routes and root context packs.
- Source confidence metadata tells the agent how strongly to treat a claim.
- Page-local routes are useful for exact wording; root packs are useful for routing and broad orientation.
## Forbidden assumptions
- Do not load draft or removed docs from repository files as public facts.
- Do not treat /docs/llms-full.txt as mandatory for every answer.
- Do not infer public API, auth, rate-limit, compliance, deployment, billing, or integration guarantees from internal evidence paths.
- Do not collapse doctrine-backed positioning into code-backed behavior, or code-backed product behavior into public external contracts.
## Related human docs
{% related-card-grid title="Related human docs" %}
- [What is Teammately?](/docs/introduction/what-is-teammately)
- [Task index](/docs/operating-manual/task-index)
- [Reference index](/docs/reference)
- [Troubleshooting](/docs/troubleshooting)
{% /related-card-grid %}
## Related reference docs
{% related-card-grid title="Related reference docs" %}
- [Agent instructions](/docs/agent-instructions)
- [Object model](/docs/object-model)
- [Permissions](/docs/reference/permissions)
{% /related-card-grid %}
## Safe operating boundaries
Agents should answer from stable public pages and cite the page URL. If a needed claim is only draft, inferred, or absent from the context packs, say the source boundary is unconfirmed and ask for human confirmation.
When a task needs exact behavior, prefer `/api/docs/page?slug=` or page-local markdown over a broad pack. When a task needs a route into the docs, prefer `/docs/llms.txt`, focused packs, search, and machine maps before loading full-corpus context.
## Source confidence
Docs-backed: this page describes routes and files implemented in the docs repository. The source_refs frontmatter lists the docs loaders, llms generators, and machine-readable route files.
---
id: agent-instructions
title: Agent instructions
summary: Rules coding agents should follow when using Teammately public docs as source context.
kind: reference
product_area: reference
status: stable
updated: 2026-08-23
canonical: /docs/agent-instructions
---
# Agent instructions
Use this page as the behavioral policy for agents operating with Teammately docs. For route selection and context packaging, read [Agent context index](/docs/agent-context).
## Task intent
Answer Teammately product and docs questions safely without inventing unsupported product claims. The agent should preserve source confidence, cite stable pages, and ask for confirmation when a claim depends on draft, inferred, or missing evidence.
## Required context
- Read [/docs/llms.txt](/docs/llms.txt) first for the recommended context loading strategy.
- Use [Agent context index](/docs/agent-context) to choose the smallest relevant context pack.
- Use page-local markdown or llms routes for exact wording.
- Use the manifest or page API when source confidence, source refs, related IDs, or metadata matter.
- In an MCP client, search first and fetch the cited page or block before relying on exact wording.
## Allowed assumptions
- Code-backed pages can support object, state, and workflow claims when source_refs are present.
- Doctrine-backed pages can support positioning, lifecycle, category language, and public narrative.
- Docs-backed pages can support claims about routes, llms files, manifests, search, context APIs, and validation behavior.
- Stable public pages supersede older draft or removed pages.
## Forbidden assumptions
- Do not present draft schema pages as public schemas.
- Do not infer public APIs, SDKs, API keys, auth behavior, rate limits, billing, compliance, tenant isolation, deployment modes, support guarantees, customer names, model-provider integrations, or production monitoring behavior.
- Do not treat AI-assisted suggestions as approved policies, rubrics, or review context unless a stable page says a human approved them.
- Do not collapse Teammately into eval dashboards, observability, annotation, prompt testing, generic LLMOps, or a replacement for adjacent tools.
- Do not use internal source file names as public UI promises unless the public docs already state the behavior.
## How to answer safely
1. Identify whether the user is asking for positioning, operating steps, object behavior, recovery, or implementation context.
2. Load the smallest matching context pack or query-scoped context.
3. Check source confidence before making claims.
4. Prefer stable page citations over broad corpus summaries.
5. If search returns an AI Overview, verify its block references with `fetch`; treat the overview as routing assistance rather than independent authority.
6. State uncertainty when a claim is inferred, draft-only, or outside the docs.
7. Ask for human confirmation before advising changes that depend on permissions, compliance, deployment, billing, customer-facing APIs, or unsupported integrations.
## When to ask for human confirmation
- The user asks whether an inferred or draft schema is a public contract.
- The answer would require exact UI labels not present in stable docs or source refs.
- The user asks about admin, auth, security, compliance, deployment, billing, or support commitments.
- Related evidence conflicts across docs, product code, and doctrine.
- The requested action could change review context, approval state, or benchmark interpretation without accountable human review.
## Related human docs
{% related-card-grid title="Related human docs" %}
- [Product boundaries](/docs/introduction/product-boundaries)
- [Human approval boundaries](/docs/governance/human-approval-boundaries)
- [What AI Features Can and Cannot Do](/docs/governance/what-ai-features-can-and-cannot-do)
{% /related-card-grid %}
## Related reference docs
{% related-card-grid title="Related reference docs" %}
- [Agent context index](/docs/agent-context)
- [Reference index](/docs/reference)
- [Object model](/docs/object-model)
{% /related-card-grid %}
## Safe operating boundaries
Agents may help readers find pages, summarize stable docs, compare source-confidence labels, and draft operational next steps. Agents should not approve policies, approve rubrics, assign reviewers, declare a candidate ready for rollout, or convert draft/inferred material into public product commitments.
Search mode is diagnostic metadata, not a confidence label. Lexical fallback remains usable when semantic embeddings are unavailable; source confidence still comes from the fetched page metadata and prose.
## Source confidence
Docs-backed: this page is a behavior policy for using the docs system safely. The docs loaders, context routes, and MCP tools establish the available retrieval surfaces, cited block references, and read-only tool boundaries.
---
id: agent-setup.project-context
title: Project Context
summary: Maintain the Project Agent Brief that gives Teammately agents stable, project-wide understanding.
kind: reference
product_area: agent_setup
status: stable
updated: 2026-08-22
canonical: /docs/agent-setup/project-context
---
# Project Context
## Definition
Project Context is the Agent Setup surface that edits the **Project Agent Brief**. The brief gives Teammately agents stable project-wide understanding: what the specialist AI is for, which behavior matters, important constraints, terminology, and other context that should carry across coverage, contribution preparation, case construction, and improvement work.
The Project Agent Brief is different from the Project Memo in General settings. The memo is administrative project text and explicitly is not used as prompt or agent context. Put agent-relevant project understanding in Project Context.
## Fields, states, or lifecycle rules
- The brief is scoped to the project and reused across benchmark workspaces.
- Editing the brief changes future agent context; it does not rewrite completed Contributions, Runs, or Improvement Session history.
- The brief provides orientation and constraints, not governed correctness authority. Policies and rubrics remain in Correctness Governance.
- Controlling source material belongs in Reference Materials. Summarize stable project intent in the brief and keep source-backed detail in the indexed material.
- The brief should state product-specific meaning directly. Avoid copying transient benchmark goals, one expert's unconfirmed opinion, or a temporary candidate hypothesis into permanent project context.
## Writing a useful brief
Describe the specialist AI's purpose, users, important domain vocabulary, expected interaction shape, and constraints that affect many workflows. Include explicit boundaries where agents might otherwise make unsafe assumptions. Name controlling authorities without duplicating entire manuals.
Review the brief when the product purpose, domain, input architecture, or correctness boundary changes materially. If only one benchmark needs a special objective, put it in that benchmark's setup or Contribution. If only one Improvement Session needs a constraint, put it in the Goal Contract.
{% example-demo title="Example: project context boundary" %}
The brief states that a procurement assistant supports internal buyers, must distinguish current agreements from expired ones, and should expose uncertainty rather than invent an exception. The current agreements themselves remain indexed Reference Materials. The exact evaluation target for expired-agreement cases belongs to the benchmark and Improvement Session, not the brief.
{% /example-demo %}
## Source confidence
Code-backed: the active Agent Setup Project Context route renders the Project Agent Brief editor. The distinction from General settings is supported by the current project settings UI.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Product quickstart](/docs/quickstart)
- [Use Reference Materials](/docs/agent-setup/reference-materials)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
{% /related-card-grid %}
---
id: agent-setup.reference-materials
title: Reference Materials
summary: Connect project knowledge, inspect indexing state, and verify the blocks available to Teammately agents.
kind: reference
product_area: agent_setup
status: stable
updated: 2026-09-07
canonical: /docs/agent-setup/reference-materials
---
# Reference Materials
## Definition
Reference Materials is the Agent Setup workspace for project knowledge that Teammately agents may use. **Materials** is the unified inventory for websites, Git repositories, and uploaded files. **Indexed Reference** shows the published reference blocks available after extraction and indexing.
Reference Materials is project understanding, not governed correctness. A manual can support an agent's reasoning or an expert Contribution without automatically becoming a policy, rubric, case, or approved statement.
## Fields, states, or lifecycle rules
- A Material identifies a source and its ingestion or freshness state.
- Websites, Git repositories, and files follow source-specific discovery and processing paths in the unified Materials inventory.
- Indexed Reference presents published blocks rather than a second editable copy of the source.
- Publication is atomic: agents should see a coherent published generation rather than a partially updated index.
- Reconnecting, refreshing, or processing a source can create a newer indexed generation. Completed Runs and Contributions keep their own recorded evidence boundaries.
- Removing a Material or source does not imply that previously materialized policies, rubrics, cases, or contribution records should be silently deleted.
- File upload acceptance depends on file type, size, content verification, and the active source-processing path.
## Materials and Indexed Reference
Use Materials to answer: which sources are present, when were they processed, and does a source need attention? Use Indexed Reference to answer: what text or blocks can agents actually retrieve now? A material can be present without the expected controlling content appearing in Indexed Reference.
When sources conflict, retain the conflict in the indexed material and resolve correctness through an expert contribution or governed policy. Do not rewrite Project Context to hide source disagreement.
{% example-demo title="Example: reference publication check" %}
A team adds a policy website and a Git repository containing operational rules. After processing, the operator opens Indexed Reference and searches for the current exception clause. The website block is current, while the repository still contains an older rule. The operator keeps both materials visible and requests an expert Contribution to establish the controlling policy.
{% /example-demo %}
## Source confidence
Code-backed: the active Agent Setup routes expose Materials and Indexed Reference, and the backend publication service supports coherent indexed-reference publication. Exact connector availability may depend on the current product configuration.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Product quickstart](/docs/quickstart)
- [Maintain Project Context](/docs/agent-setup/project-context)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
{% /related-card-grid %}
---
id: assets.cases
title: Cases
summary: Understand canonical project cases, their input and materials, and how they become members of benchmark datasets.
kind: reference
product_area: assets
status: stable
updated: 2026-08-22
canonical: /docs/assets/cases
---
# Cases
## Definition
A Case is a project-level situation used for expert contribution, benchmark coverage, or candidate evaluation. It has canonical input content and may include declared supporting materials. Cases live in the Assets pool and can be selected into one or more benchmark datasets.
The Project Input Schema determines how the primary input and materials are represented. A benchmark snapshot determines which selected cases belong to one versioned evidence boundary.
## Fields, states, or lifecycle rules
- Canonical primary input is stored under `content.input`.
- Optional supporting values or artifacts are stored under `content.case_materials` according to the project's declared keys.
- `record_content.case_view` is a rendered projection used for inspection and delivery; it is not a second editable payload.
- Inputs can use plain-text, chat, or structured architecture as configured by the project.
- Materials can include admitted artifact families and must satisfy any required-field and file-extension rules.
- A project Case is not automatically part of every benchmark. Benchmark Datasets owns selection and snapshots.
- Generated or imported Cases should be reviewed for clarity, source authority, and schema conformance before they are trusted as benchmark evidence.
- Static case materials and executable Worlds remain separate. A document supplied to a Harness does not become a world merely because it affects execution.
## Case identity and change
Treat the persisted case identity as opaque. Do not construct IDs in client code or documentation. When case content changes materially, benchmark interpretation must use a snapshot or version boundary that makes the selected content clear.
Responses produced by a Harness are evaluation outputs attached to a Run. They are not the primary case input. Expert-authored acceptable examples can inform standards or contribution work, but the current evaluation contract should remain explicit about which candidate produced each response.
{% example-demo title="Example: multimaterial case" %}
A chat case asks whether an exception applies. Its required `current_policy` PDF and optional `account_history` table are stored as case materials admitted by Project Input Schema. The rendered case view presents the conversation and both materials. A benchmark snapshot selects the case, and a Run records the evaluated Harness response separately.
{% /example-demo %}
## Source confidence
Code-backed: the active Assets Cases route and case-content services define canonical input, case materials, and the rendered case view. Public import or export APIs are outside this reference unless separately documented.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Configure Project Input Schema](/docs/project-settings/input-schema)
- [Work with Benchmark Datasets](/docs/benchmark-datasets)
- [Manage benchmark coverage](/docs/coverage-management)
{% /related-card-grid %}
---
id: assets.comparison-directions
title: Comparison Directions
summary: Create reusable guidance for meaningful candidate-output differences in comparative expert work.
kind: reference
product_area: assets
status: stable
updated: 2026-09-07
canonical: /docs/assets/comparison-directions
---
# Comparison Directions
## Definition
A Comparison Direction is a reusable project Asset that describes how candidate outputs should differ during comparative expert work. It can focus attention on a meaningful contrast such as evidence grounding, uncertainty handling, or response strategy without declaring which candidate is correct.
Comparison Directions are project-scoped. A Contribution can select or allow a pool of directions for its comparative component, while the benchmark Contribution still owns the objective, cases, candidates, and expert task.
## Fields, states, or lifecycle rules
- A direction has a name or label and a description of the intended contrast.
- Users can create, edit, pin, archive, and remove directions from **Assets → Comparison Directions**.
- Pinned directions are surfaced when a Contribution request selects comparative output guidance.
- The suggestion experience creates draft candidates in a suggestion run. Nothing enters the reusable library until a user accepts it.
- A direction can carry a staleness advisory when its source context has changed. Dismissing that advisory records a review decision; it does not approve a Policy, Rubric, Case, or Benchmark.
- A direction guides comparative presentation or generation. It does not create a Case, change coverage structure, or replace expert judgment.
## Correct scope
Use Dimensions, Project Topics, and Case Construction Patterns for the behavior space a benchmark should represent. Use Comparison Directions for how candidate outputs should be contrasted within a comparative Contribution. Use Correctness Governance for the approved standard that determines how an output is judged.
{% example-demo title="Example: source-grounding contrast" %}
A project creates one Comparison Direction asking for a response that cites the current source conservatively and another asking for a focused clarification when the source hierarchy is unresolved. A comparative Contribution can use those directions to elicit an expert preference. The direction does not approve either response or create the governing rubric.
{% /example-demo %}
## Source confidence
Code-backed: the active Assets routes expose the Comparison Directions library, detail controls, suggestion runs, accept or dismiss decisions, pinning, and staleness review. The API keeps legacy compatibility names internally, but this page uses the current product label.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Assets](/docs/assets)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Manage benchmark coverage](/docs/coverage-management)
{% /related-card-grid %}
---
id: assets.harnesses
title: Harnesses
summary: Build versioned Python Harness bundles, validate and debug Drafts, publish exact Versions, and activate them for benchmark evaluations.
kind: reference
product_area: assets
status: stable
updated: 2026-08-22
canonical: /docs/assets/harnesses
---
# Harnesses
## Definition
A Harness is a project-level, versioned Python bundle used to produce behavior for benchmark cases. It implements `generate_result(request)` at its configured entrypoint and returns the result expected by the case contract. The workspace provides a multi-file editor, one editable Draft, immutable saved Versions, Coding Agent proposals, secret grants, debug runs, runtime preparation, export, and benchmark activation.
A Harness can represent prompt logic, retrieval, tools, orchestration, or other candidate behavior supported by the managed runtime. It is the executable candidate boundary, not a general model registry entry.
## Fields, states, or lifecycle rules
- A newly created Harness begins with one blank editable Draft.
- Draft source and runtime changes autosave after a short delay. Concurrent edits use a Draft revision and can surface a conflict that must be reloaded.
- The Draft contains source files, an entrypoint, Python version, dependency specifications, sandbox profile, and target-platform/runtime facts.
- Draft validation errors block publication. Warnings remain visible for operator review.
- **Publish** creates an immutable saved Version from the exact Draft bundle and runtime configuration. Publication also uses that Version for the currently selected benchmark.
- A previously saved Version can be selected with **Use this Version** for the current benchmark.
- Existing Runs keep their referenced Harness version even after later Draft or Saved changes.
- Archived Harnesses remain visible but cannot be edited, debugged, activated, or granted new secrets.
- Credentials belong in workspace secrets and are explicitly granted to the Harness. Secret values do not belong in source, case content, or Run Metadata.
- Coding Agent proposals are staged as file diffs. Accept or reject the proposed hunks before they modify the canonical Draft.
- Runtime delivery can include the rendered case view, static `case_material_refs`, and an optional supported `world_instance_ref`.
- Debug runs execute only a saved Version against a `harness_contract.v1` request. They are authoring checks, not benchmark evaluation evidence.
## Draft, saved version, and activation
Use Draft for active editing. Publish when a candidate is coherent enough to identify, compare, or evaluate. If a benchmark needs a different saved Version, select it explicitly. Do not describe a Run as evaluating “the current Harness” because the Draft and benchmark activation may have changed after the Run began.
The Evaluations execution settings show project Harnesses with saved Versions and control which are active for future managed Runs. Activation is benchmark-scoped. Every admitted Run still records the exact saved Harness Version, so later activation changes do not rewrite existing evidence.
## Runtime, debug, and output mapping
Publishing preserves bundle and dependency hashes and can prepare the immutable runtime image. A failed or missing runtime can be prepared or retried from the saved Version. Use Debug to send a single contract-shaped request before spending a benchmark Run.
Output mapping is a separate boundary. A Harness return must match the benchmark's declared result contract. For imported reference outputs, Benchmark Evaluations maps uploaded rows to immutable benchmark cases. Neither debug success nor a syntactically valid upload proves rubric correctness.
Improvement Sessions can generate or receive candidate Harness versions. Those candidates become meaningful in a session only when their identity and canonical evaluation receipt are observable.
{% example-demo title="Example: exact candidate identity" %}
An engineer edits retrieval filtering in a Harness Draft and publishes Version 12 for the selected benchmark. A managed Run records that exact Version. The engineer then continues editing the Draft toward Version 13. The existing Run still refers to Version 12, so its case responses and rubric results remain attributable even though the editor shows newer work.
{% /example-demo %}
## Source confidence
Code-backed: the active Harness list and editor establish creation, archive state, Draft autosave and conflicts, validation, publication, activation, secrets, debug, Coding Agent proposals, and runtime preparation. The runnable adapter supports the execution boundary without making its internal API public.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
- [Configure Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
---
id: assets.project-tools
title: Project Tools
summary: Understand the current Project Tools asset surface and its deliberately limited public behavior.
kind: reference
product_area: assets
status: stable
updated: 2026-09-07
canonical: /docs/assets/project-tools
---
# Project Tools
## Definition
Project Tools is a visible project-level Asset category intended for reusable callable capabilities. In the current product, the page presents an empty state and does not expose creation, editing, versioning, permissions, activation, or execution controls.
> Current capability boundary
>
> Do not document a Project Tool lifecycle or claim that a Harness can select a Project Tool merely because the tab exists. No durable public workflow is exposed on the current page.
Harness source and dependencies remain in Harnesses. Connected knowledge remains in Reference Materials. Static Case inputs and materials remain governed by Project Input Schema and the Case.
## Fields, states, or lifecycle rules
There are currently no public fields, durable states, or lifecycle actions on this page. The absence of controls is itself the verified boundary.
## What to use today
Keep candidate-specific code and declared dependencies inside the Harness bundle. Use Workspace Secrets for credentials granted to the exact Harness, Reference Materials for connected knowledge, and Project Input Schema for Case inputs and materials. Do not relabel those objects as Project Tools because their ownership and evidence boundaries differ.
The empty state does not provide a Project Tool ID, activation action, persistence contract, or Harness-selection control.
{% related-card-grid title="Available alternatives" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Configure Reference Materials](/docs/agent-setup/reference-materials)
- [Configure Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Configure Reference Materials](/docs/agent-setup/reference-materials)
{% /related-card-grid %}
## Source confidence
Code-backed: the Assets layout exposes the Project Tools tab and its current route renders “No Project Tools yet.” No additional public behavior is claimed.
---
id: assets.review-screens
title: Review Screens
summary: Configure reusable project templates for the context and presentation experts see during Contribution work.
kind: reference
product_area: assets
status: stable
updated: 2026-09-07
canonical: /docs/assets/review-screens
---
# Review Screens
## Definition
Review Screens are reusable project Assets for designing how experts see Case context and Contribution questions. A screen can present selected inputs, Case materials, Dimensions, candidate outputs, and review controls in a consistent layout.
Review Screens control presentation. They do not change Case content, the Contribution objective, Policy meaning, Rubric semantics, or expert authority.
## Fields, states, or lifecycle rules
- The library is available at **Assets → Review Screens**.
- A screen has a project-owned identity and can be opened in the visual designer for editing and preview.
- The designer can use current Dimensions, context keys, and declared Project Input Schema material fields as presentation inputs.
- A screen can be selected for Contribution work; the Contribution still supplies the benchmark-specific objective, expert, Cases, attachments, and task components.
- Creating or editing a screen affects future presentation. It does not rewrite completed responses, Checkpoints, or historical Contribution evidence.
- A visible field is not automatically required by Project Input Schema, and a required Case material is not automatically appropriate for every screen.
## Designing for judgment
Show the smallest context set that lets an expert make and explain the requested decision. Include source conflicts, Case materials, candidate responses, and relevant coverage dimensions when they affect correctness. Keep administrative metadata and unrelated fields out of the primary judgment surface.
Test a screen against representative and boundary Cases before using it for broad Contribution work. If an expert must rely on private knowledge or locate a missing source, correct Project Context, Reference Materials, the Case, or the Contribution before changing the layout.
Preview the expert-facing Case and question layout with representative data before using it in a Contribution.
{% example-demo title="Example: source-grounding screen" %}
For a grounding Contribution, the screen displays the user request, candidate response, current source document, superseded source document, and source-freshness Dimension. It omits internal ingestion metadata so the expert can compare the response with both documents and explain which source controls.
{% /example-demo %}
## Source confidence
Code-backed: the active Review Screens library and detail routes provide paginated browsing, usage filtering, creation, editing, preview, and screen-authoring controls. The stable responsibility boundary is documented here without claiming every visual control is permanent.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Assets](/docs/assets)
- [Configure Project Input Schema](/docs/project-settings/input-schema)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
{% /related-card-grid %}
---
id: assets.weights
title: Weights
summary: Understand the current Weights asset surface and the absence of a public model-weight lifecycle.
kind: reference
product_area: assets
status: stable
updated: 2026-09-07
canonical: /docs/assets/weights
---
# Weights
## Definition
Weights is a visible project-level Asset category for model-weight artifacts. The current page presents an empty state and does not expose upload, registration, versioning, activation, deployment, or evaluation controls.
Model provider, model name, configuration, and execution receipts can appear in Harness Version or Run evidence. Those fields are not a Weights asset lifecycle.
> Current capability boundary
>
> Do not claim that model weights can be imported, fine-tuned, deployed, or compared from Assets until those actions and their durable identities are exposed by the product.
## Fields, states, or lifecycle rules
The current page defines no public weight fields, artifact states, or lifecycle actions. Model and provider facts in candidate or Run evidence remain owned by those records.
## What to record instead
When candidate behavior depends on a model, preserve the exact Harness Version and the model or provider configuration recorded by its source, runtime facts, Run Metadata, or evaluation receipt. Do not create a fictional Weights asset to make an experiment look versioned. Candidate code and dependency changes belong in Harness publication; evaluation evidence belongs to the Run and immutable Benchmark Version.
Run telemetry can report tokens, cost, or latency when captured. Those values describe execution and do not establish that a model checkpoint has been registered in Weights. Likewise, an Improvement Session candidate remains a saved Harness Version even when its behavior depends on model selection.
The empty state does not provide a weight artifact ID, supported format, upload action, version, Harness attachment, fine-tuning job, or deployment control. Use active Harness, Run, and provider-evidence surfaces for currently inspectable candidate identity.
{% related-card-grid title="Current related boundaries" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Configure Run Metadata](/docs/benchmark-evaluations/run-metadata)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
{% /related-card-grid %}
## Source confidence
Code-backed: the Assets layout exposes the Weights tab and the current route renders “No Weights yet.” No additional public behavior is claimed.
---
id: assets.worlds
title: Worlds
summary: Understand the current Worlds asset surface without inferring an environment lifecycle that the product does not expose.
kind: reference
product_area: assets
status: stable
updated: 2026-09-07
canonical: /docs/assets/worlds
---
# Worlds
## Definition
Worlds is a visible project-level Asset category for executable or queryable environments. The current Worlds page presents an empty state and does not expose creation, configuration, versioning, activation, or lifecycle controls.
The Harness execution contract can carry an optional supported World instance reference, but that runtime boundary does not establish a user-facing Worlds management workflow.
> World versus case material
>
> Static files, documents, images, and values supplied with a case are case materials. Do not call them Worlds. A World requires an observable executable-environment identity and lifecycle, which the current Assets page does not yet expose.
## Fields, states, or lifecycle rules
The current page defines no public World fields or states. The optional runtime reference is the only verified adjacent contract and must not be expanded into a management lifecycle.
## What the runtime reference does establish
A saved Harness can receive an optional supported `world_instance_ref` with its request. That establishes a boundary between executable environment context and static `case_material_refs`. It does not reveal who creates the instance, how long it lives, which benchmark owns it, whether state resets between attempts, or how a user chooses one.
Treat the reference as optional runtime input. It does not promise deterministic reset, persistence, query APIs, tool availability, or replay. If a Run depends on a World reference, preserve the exact Harness and Run evidence that Teammately exposes and state any known environment limitation separately.
## What to use today
Use Project Input Schema and Case materials for static evidence. Use Harness source for supported candidate orchestration. Do not upload a directory or service endpoint to the Worlds empty state, invent a World ID, or describe a Case fixture as a managed environment.
The empty state does not provide a World ID, configuration, version, instance-creation action, reset control, Harness attachment, telemetry view, or archive action.
{% related-card-grid title="Current related boundaries" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Configure Project Input Schema](/docs/project-settings/input-schema)
- [Understand Dataset Snapshots](/docs/benchmark-datasets/snapshots)
{% /related-card-grid %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Configure Project Input Schema](/docs/project-settings/input-schema)
{% /related-card-grid %}
## Source confidence
Code-backed: the Assets layout exposes the Worlds tab and its current route renders “No Worlds yet.” Runtime types support optional World references but do not justify additional public management claims.
---
id: benchmark-evaluations.execution-settings
title: Evaluation Execution Settings
summary: Configure machines and choose saved Harness Versions and Run counts at launch.
kind: reference
product_area: benchmark_evaluations
status: stable
updated: 2026-09-13
canonical: /docs/benchmark-evaluations/execution-settings
---
# Evaluation Execution Settings
## Definition
Evaluation Settings contains machine settings for managed execution. Harness selection and repetition are choices made when launching an evaluation.
## Fields, states, or lifecycle rules
- Choose an exact saved project Harness Version when launching. Benchmark activation is not required.
- Choose a whole number of Runs from 1 through 50 for each selected Harness. The default is one, and different Harnesses may have different counts.
- Each managed Harness launch creates one Run Group containing the requested Runs, including a one-Run group.
- Existing Runs keep their immutable configuration and evidence. Launching more Runs creates another group rather than rewriting previous membership.
## Reliability choices
Dashboard, Compare, and Arena aggregate compatible observed Runs for each saved Harness Version across launch groups. Average score weights individual Runs equally. Passed at least once and passed every time summarize observed binary case outcomes where the evaluation framework supports them.
Run counts can differ. The product displays the counts and coverage and warns about unequal evidence without requiring another launch. Missing or infrastructure-failed observations are not numerical successes or failures.
## Before launching
Check the exact saved Version, machine selection, and requested execution volume. Runtime preparation and required access still apply. Imported external outputs remain a separate flow because they have no executable Harness Version.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Manage Harnesses](/docs/assets/harnesses)
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
{% /related-card-grid %}
## Source confidence
Code-backed: Evaluation Settings defines machine configuration; the shared Run modal defines saved-Version selection and per-launch Run counts.
---
id: benchmark-evaluations.run-metadata
title: Benchmark Run Metadata
summary: Interpret benchmark-level descriptive fields without confusing them with project settings or version identity.
kind: reference
product_area: benchmark_evaluations
status: stable
updated: 2026-09-07
canonical: /docs/benchmark-evaluations/run-metadata
---
# Benchmark Run Metadata
## Definition
Benchmark Run Metadata is descriptive context attached to benchmark-level evaluation work. It helps operators interpret a Run without replacing the exact Benchmark Version, Harness Version, Run Group, or published Regime Version that defines the evidence boundary.
Project-level Run Metadata templates are retired. When the current evaluation surface offers metadata fields, manage them at the benchmark or Run setup boundary and keep the values specific to the evidence being created.
## Fields, states, or lifecycle rules
- Metadata describes a benchmark evaluation context; it is not a Policy, Rubric, Case, Harness Version, Benchmark Version, or Regime Version.
- Existing Runs retain the metadata and exact version identities recorded with their evidence.
- Benchmark-level fields can be managed from the benchmark context when that surface exposes the control.
- External model configuration is declared when the Run is created rather than through a project-level template.
- Metadata can help compare or interpret Runs, but it does not make an external metric a Teammately-verified result.
## Choose the right boundary
Put executable candidate behavior in a Harness and its saved Version. Put Case content and materials in Assets and Benchmark Datasets. Put evaluation scoring behavior in the published Regime and governed Policies and Rubrics. Use Run Metadata only for descriptive context that should travel with a particular benchmark evaluation.
{% example-demo title="Example: benchmark-level experiment context" %}
Two Runs use the same Benchmark Version but different saved Harness Versions. Their benchmark-level metadata records the experiment labels and external model configuration needed to interpret the comparison. The metadata does not change either candidate identity or the published Regime used to score the evidence.
{% /example-demo %}
## Source confidence
Code-backed: the current Run presentation and setup surface expose benchmark-level metadata context and explicitly fence off retired project-level templates. Exact fields depend on the benchmark evaluation surface in use.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
- [Compare evaluation results](/docs/benchmark-evaluations/compare)
{% /related-card-grid %}
---
id: concepts.benchmarks-versions
title: Benchmarks and versions
summary: Understand benchmarks, benchmark versions, snapshots, and why versioned measurement matters.
kind: concept
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/concepts/benchmarks-and-versions
---
# Benchmarks and versions
## Definition
A Benchmark is the durable workspace for one evaluation program. Its current Dataset and coverage work can change over time. A Dataset Snapshot freezes selected Case membership; a Benchmark Version fixes the evidence boundary used by an Evaluation Run.
## Why it matters
This separation keeps evidence interpretable. A score can change because the saved Harness Version changed, the Dataset changed, or the governed evaluator boundary changed. Naming the Benchmark Version prevents those causes from being collapsed into “the model changed.”
## Version sanity check
| Before comparing runs, confirm... | Otherwise... |
| --- | --- |
| The same benchmark version was used, or the version change is intentional and named. | Score movement may be caused by changed cases or standards. |
| The Dataset Snapshot and governed evaluator boundary are identifiable. | A comparison can hide the evidence that actually changed. |
| The saved Harness Version, settings, and Run Metadata are attached. | The result cannot explain which candidate configuration was evaluated. |
## Where it appears in the product
Use the Benchmark selector to enter a Benchmark workspace. **Benchmark Datasets** owns selected Cases and Snapshots. **Coverage Management** explains representation and gaps. **Expert Contributions** scopes specialist work. **Benchmark Evaluations** runs and compares saved Harness Versions. **Improve** explores evidence-backed candidate changes.
## Artifacts it affects
The Benchmark connects Dataset membership, Snapshots, coverage work, Contributions, Benchmark Versions, Runs, comparisons, and Improvement Sessions. Project Cases, Coverage Facets, Policies, Rubrics, and Harnesses remain reusable project objects rather than becoming owned by the Benchmark.
{% example-demo title="Coverage refresh version" %}
Reviewers discover an unsupported compatibility boundary after a candidate passes Benchmark Version 3. They add reviewed Cases to the current Dataset and create a new Snapshot and Benchmark Version 4. The original Run remains evidence for Version 3. A new Run against Version 4 answers the expanded question, and any customer-owned human review context names both versions.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Run a benchmark](/docs/benchmark-evaluations/run-evaluation)
- [Compare Harness Versions](/docs/benchmark-evaluations/compare)
- [Read run results](/docs/benchmark-evaluations/inspect-results)
- [Work with Benchmark Datasets](/docs/benchmark-datasets)
- [Refresh coverage after product change](/docs/coverage-engineering/coverage-refresh)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Benchmarks](/docs/object-model/benchmarks)
- [Dataset Snapshots](/docs/benchmark-datasets/snapshots)
- [Benchmark Versioning](/docs/governance/benchmark-versioning)
- [Reproducibility](/docs/governance/reproducibility)
{% /related-card-grid %}
## Source confidence
Code-backed: the Benchmark type establishes durable identity; Dataset Snapshots freeze selected Case membership; the version-scoped evaluation workspace consumes an exact Benchmark Version; current navigation establishes the benchmark workspaces named above.
---
id: concepts.dimensions-ontology
title: Dimensions and ontology
summary: Learn how dimensions and ontology values define the coverage space for a Teammately project.
kind: concept
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/concepts/dimensions-and-ontology
---
# Dimensions and ontology
## Definition
Dimensions and ontology describe how Teammately classifies cases into behavior segments that humans can reason about. Dimensions name the axes that matter, while ontology values provide the controlled labels used for coverage planning, case review, and result analysis.
## Why it matters
This matters because aggregate benchmark results can hide an unsafe gap. A candidate may pass common cases while missing a stale-source segment, a boundary condition, a product tier, or a policy exception that reviewers care about.
## Coverage vocabulary check
| Good coverage vocabulary does... | Weak vocabulary does... |
| --- | --- |
| Names behavior slices that change judgment or risk. | Uses labels that only describe where the case came from. |
| Keeps ontology values consistent enough for comparison. | Lets free-form tags drift until segment results are meaningless. |
| Makes missing or thin segments visible before review. | Treats a large case count as representative coverage. |
## Where it appears in the product
Create and maintain Dimensions and ontology values under **Coverage Facets → Dimensions and Ontology**. Benchmark Datasets uses them to inspect representation, while Benchmark Evaluations can group existing evidence by supported Coverage Facets. The Case Pool and Case Review surfaces use the same vocabulary when classifying and preparing Cases.
## Artifacts it affects
Dimensions affect Case classification, Benchmark Dataset representation, Coverage Stories, synthesis targets, Case Review, and result grouping. Changing the current vocabulary does not rewrite the labels or interpretation of historical Snapshots and Runs.
{% example-demo title="Stale-source segment" %}
An enterprise-search team adds a **Source freshness** Dimension with current, superseded, and unknown values. Benchmark Datasets then reveals that superseded-source Cases are thinly represented. The team prepares additional Cases, reviews their classifications, and creates a new Snapshot before using that segment in evaluation interpretation.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema)
- [Inspect Dataset representation](/docs/benchmark-datasets/representation)
- [Review prepared Cases](/docs/coverage-management/case-review)
- [Diagnose classification](/docs/troubleshooting/dimension-classification)
- [Refresh changed coverage](/docs/coverage-engineering/coverage-refresh)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Coverage Dimensions](/docs/object-model/coverage-dimensions)
- [Ontology](/docs/object-model/ontology)
- [Project Topics](/docs/coverage-engineering/project-topics)
- [Case Construction Patterns](/docs/coverage-engineering/case-construction-patterns)
{% /related-card-grid %}
## Source confidence
Code-backed: Dimension types and the Dimensions and Ontology surfaces define the project vocabulary and editable fields; Benchmark Dataset Representation shows how that vocabulary is used to inspect selected Cases.
---
id: concepts.policies-rubrics
title: Policies and rubrics
summary: Learn how Teammately turns product judgment into reusable policies and scoring rubrics.
kind: concept
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/concepts/policies-and-rubrics
---
# Policies and rubrics
## Definition
Policies state governed expectations for behavior and the situations to which those expectations apply. Rubrics are evaluator definitions used to judge Case responses. A Policy can link relevant Cases and Rubrics, but the objects remain separately versioned and reviewable.
## Why it matters
This separation lets a team correct the right layer. A mistaken rule belongs in the Policy; an overbroad scope belongs in applicability; an unreliable check belongs in the Rubric. Evaluation evidence should show which applicable Rubric produced each outcome rather than treating an aggregate score as the standard itself.
## Standard pair check
| The pair is healthy when... | Rework it when... |
| --- | --- |
| The policy states the product behavior rule. | The policy is only tone, preference, or broad quality advice. |
| Applicability names the cases where the rule belongs. | The same rubric could apply to nearly everything. |
| The Rubric tests one observable requirement. | The Rubric combines several decisions into one unclear result. |
## Where it appears in the product
Create and inspect Policies and Rubrics under **Correctness Governance**. Expert Contributions can supply attributable candidate artifacts, but contributed content is not automatically approved. Benchmark Evaluations reports applicable Rubric outcomes for the frozen Benchmark Version.
## Artifacts it affects
Policies and Rubrics affect Case links, applicability, Benchmark Versions, evaluator coverage, Run results, failure clusters, and staleness. Changing either governed object requires a new version boundary before the revised standard is treated as current evaluation evidence.
{% example-demo title="Eligibility policy to must-level rubric" %}
A support Policy requires entitlement answers to use the controlling contract or state uncertainty. Its applicability is limited to plan limits, contract exceptions, and admin-controlled access. A linked binary Rubric checks whether the response identifies that source or explicitly withholds an unsupported eligibility claim. Evaluation results can then show the failed Rubric on the affected Cases without broadening the rule to unrelated setup questions.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Work with Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Design binary Rubrics](/docs/correctness-governance/binary-rubrics)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Applicability logic](/docs/object-model/applicability-logic)
- [Cases](/docs/object-model/cases)
- [Policies](/docs/object-model/policies)
- [Rubrics](/docs/object-model/rubrics)
- [Human Approval Boundaries](/docs/governance/human-approval-boundaries)
{% /related-card-grid %}
## Source confidence
Code-backed: the current Policy and Rubric list and detail routes define their separate identities, editable fields, links, versions, and approval state. Expert Contribution and Benchmark Evaluation pages define how attributable input and evaluator outcomes enter those objects' wider lifecycle.
---
id: concepts.workspaces-projects
title: Workspaces and projects
summary: Learn how Teammately organizes teams, projects, product goals, and access boundaries.
kind: concept
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/concepts/workspaces-projects
---
# Workspaces and projects
## Definition
The Workspace is the organization-level container. A Project is the operating boundary for one body of correctness work: its Project Agent Brief, Coverage Facets, Cases, Policies, Rubrics, Harnesses, Agent Setup, Project Settings, and Benchmarks.
## Why it matters
The distinction matters because Workspace membership and Project participation are not interchangeable. A person can belong to the organization without having access to every Project, and a role label must not be treated as proof of a specific permission.
## Where it appears in the product
Project Settings exposes General configuration, Regime, Input Schema, and Project Members. Organization-level administration belongs to the Admin Console. Benchmark work remains nested inside the selected Project and reuses that Project's governed objects.
## Artifacts it affects
The boundary affects navigation, identifiers, membership, permissions, Project Agent Brief generations, Cases, governed standards, Harnesses, Benchmarks, Runs, and Contributions. Moving or copying artifacts between Projects is not implied by shared Workspace membership.
## Boundary check
When a user cannot reach an object, confirm the Workspace, Project ID, Project membership, and object-specific assignment separately. When an artifact appears reusable across Projects, verify its source context, owner, and correctness boundary before recreating it. Shared organizational membership is never evidence that two Projects use the same Cases, standards, or Benchmark Versions.
{% example-demo title="Same policy shape, different projects" %}
One Workspace contains a billing-assistant Project and a security-assistant Project. The teams may use similar Rubric-writing practices, but their Project Agent Briefs, Cases, Policies, Harnesses, reviewers, and Benchmark evidence remain project-scoped. A user who can administer the billing Project is not assumed to have the same access in the security Project.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [Configure Project settings](/docs/project-settings)
- [Manage Project Members](/docs/project-settings/project-members)
- [Understand organization administration](/docs/admin-console)
- [Troubleshoot permissions](/docs/troubleshooting/permissions)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [IDs and keys](/docs/reference/ids)
- [Permissions](/docs/reference/permissions)
- [Workspaces, Projects, and target systems](/docs/object-model/workspaces-projects-and-target-systems)
- [Project Context](/docs/agent-setup/project-context)
{% /related-card-grid %}
## Source confidence
Code-backed: Workspace and Project types establish the container boundary; Project Settings, Members, and Permissions expose the current project-scoped configuration and access surfaces. Organization-level behavior is intentionally left to the separately bounded Admin Console documentation.
---
id: correctness.policies-rubrics
title: Policies and Rubrics
summary: Understand the governed relationship between behavior policies, applicability, binary rubrics, linked cases, and expert provenance.
kind: reference
product_area: correctness_governance
status: stable
updated: 2026-08-23
canonical: /docs/correctness-governance/policies-and-rubrics
---
# Policies and Rubrics
## Definition
A **Policy** is a reusable statement of expected specialist AI behavior. Its applicability explains the situations in which the rule controls. A **Rubric** is an evaluation criterion that turns the policy into observable evidence for a case and candidate response.
Correctness Governance owns both artifact types. Expert Contributions can supply proposed or accepted policy and rubric material, while the governance surfaces preserve the artifact's current state, links, activity, and provenance.
## Fields, states, or lifecycle rules
- Policies have identity, descriptive rule content, scope or applicability, linked cases, linked rubrics, activity, and approval context.
- Rubrics have identity, criterion wording, policy or case relationships, evaluation relevance, and lifecycle context.
- A policy can connect to several rubrics when its behavior requirements need separate checks.
- A rubric should express one inspectable criterion wherever independent diagnosis matters.
- Linked cases demonstrate applicability or behavior; benchmark dataset membership remains a separate benchmark-scoped decision.
- Proposed applications and agent suggestions remain proposals until the owning workflow records acceptance.
- Expert contribution provenance should remain visible when contributed material becomes a governed artifact.
- Editing a project-level standard does not retroactively change the standard boundary used by an already recorded Run.
Read the rule, state, links, and owner together; a plausible title alone does not establish governed authority.
## Reading the pair
Begin with the policy when deciding what should happen and why. Inspect applicability before assuming the policy governs a case. Then read the linked rubric as the testable question applied to candidate behavior. If the rubric cannot be answered from the response and visible case evidence, revise the criterion or the case rather than relying on reviewer intuition.
When standards overlap, distinguish complementary criteria from contradictory authority. Preserve unresolved conflict until an accountable expert contribution or governance action settles the intended rule.
{% example-demo title="Example: escalation policy and rubrics" %}
A policy states that unresolved eligibility exceptions must be escalated. One rubric checks that the response does not promise the exception; another checks that it gives the correct escalation path. Separating the checks lets an evaluation show whether a candidate avoided the unsupported promise but still failed to guide the user correctly.
{% /example-demo %}
## Source confidence
Code-backed: active policy and rubric detail routes expose linked cases, linked rubrics, approval and activity context, and evaluation relationships. Exact editable fields can vary by artifact state.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics)
- [Write binary rubrics](/docs/correctness-governance/binary-rubrics)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
{% /related-card-grid %}
---
id: coverage.candidate-in-use-cases
title: Candidate and In-Use Cases
summary: Distinguish examples under consideration from cases that actively support benchmark evidence.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/candidate-and-in-use-cases
---
# Candidate and In-Use Cases
## Definition
Candidate Cases are Cases or prepared Case proposals still being considered for reuse or benchmark membership. In-use Cases are already selected by one or more Benchmarks. The distinction describes current use, not intrinsic quality: a Candidate can be excellent but not yet selected, while an in-use Case can later require correction or removal from the editable dataset.
## Fields, states, or lifecycle rules
- Case Pool rows can retain source type, source label, coverage targets, review notes, and the Benchmarks that currently include the Case.
- Adding a Case to a Benchmark changes editable membership. It does not retroactively add the Case to an existing Snapshot or Benchmark Version.
- Removing a Case from a Benchmark does not delete the reusable project Case.
- A Case can be in use by several Benchmarks at once; membership must be interpreted per Benchmark.
- Generated or contributed Cases still require review for schema conformance, materials, realism, and intended coverage before selection.
- Historical Runs continue to reference the Case versions frozen into their Benchmark Version.
## Related objects
Use [Case Pool](/docs/coverage-engineering/case-pool) to inspect reusable Cases and sourcing work. Use [Benchmark Dataset Cases](/docs/benchmark-datasets/cases) to inspect or change membership for one Benchmark. Use [Dataset Snapshots](/docs/benchmark-datasets/snapshots) when that membership must become immutable evaluation evidence.
{% example-demo title="Promotion without rewriting history" %}
Case Foundry prepares a Case for a missing high-impact exception tuple. It remains a Candidate while reviewers inspect its input and materials. After approval, the operator adds it to the current dataset for the support Benchmark, so it becomes in use there. Earlier Snapshots remain unchanged; the Case enters evaluation evidence only after the team creates a new Snapshot and Benchmark Version.
{% /example-demo %}
## Source confidence
Code-backed: Case Pool navigation, benchmark Dataset Case actions, and Case sourcing types expose candidate counts, in-use counts, source context, coverage targets, and benchmark inclusion. Snapshot pages define the separate immutable evidence boundary.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Case Pool](/docs/coverage-engineering/case-pool)
- [Case pool](/docs/object-model/case-pool)
- [Importing cases](/docs/operating-manual/import-and-prepare-cases)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: coverage.case-construction-patterns
title: Case Construction Patterns
summary: Define reusable mechanisms for constructing cases and steer how benchmarks use or avoid them.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/case-construction-patterns
---
# Case Construction Patterns
## Definition
A Case Construction Pattern describes a reusable mechanism for building cases. It answers “how should this situation be constructed?” while Dimensions describe differentiating values and Project Topics describe subject matter.
Examples include conflicting authorities, missing prerequisite evidence, ambiguous user intent, multi-step state change, or a plausible but superseded source. A good Pattern is portable across Topics rather than tied to one case's wording.
## Fields, states, or lifecycle rules
Each Pattern has a name, description, origin, usage counts, examples, and benchmark statistics. Origins currently distinguish manual, AI-generated, expert-input, and imported Patterns. Usage can show Case Pool cases, benchmark cases, requirements, and benchmark steering.
Pattern suggestions can be grounded in Project Topics, source material, existing cases and Patterns, the Project Agent Brief, Dimensions and ontology, or expert input. Generated candidates include the proposed definition, why they were suggested, and source references. Accept or reject each candidate explicitly.
## Benchmark steering
Get Started can use one of three modes:
- **System choose:** allow construction to select appropriate Patterns from the project library.
- **Prefer:** encourage named Patterns when building Coverage Stories and cases.
- **Avoid:** prevent named Patterns from shaping this benchmark's intended case supply.
Coverage Story tuples can also reference specific Patterns. The tuple reference is the concrete construction instruction for that part of the story; the project Pattern remains reusable and independently editable.
## Design rules
- Name the mechanism, not the domain example. “Conflicting authorities” travels better than “Conflicting HR policies.”
- Explain what makes a case instantiate the Pattern.
- Keep Topics out of the Pattern definition unless they are only examples.
- Do not encode an expected answer or rubric verdict as a construction pattern.
- Inspect examples and usage before deleting or materially changing a Pattern.
- Treat AI and expert suggestions as reviewable proposals. Origin is provenance, not approval.
> Pattern versus requirement
>
> A Pattern describes a reusable structure. Benchmark requirements describe constraints for one benchmark, and a Coverage Tuple combines the relevant Dimensions, Topic Groups, Patterns, proof statement, and target case count.
{% example-demo title="Example: portable construction pattern" %}
The project defines **Plausible superseded authority**: construct a case where an older source appears credible but a newer source controls. Coverage Stories reuse the Pattern for procurement, support, and compliance Topics while applying different source-condition and impact ontology values.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Work with Project Topics](/docs/coverage-engineering/project-topics)
- [Configure benchmark Get Started](/docs/coverage-management/get-started)
- [Edit Coverage Stories](/docs/coverage-management/coverage-stories)
- [Review prepared cases](/docs/coverage-management/case-review)
{% /related-card-grid %}
## Source confidence
Code-backed: the current Pattern list, detail, generation, recommendation, usage, examples, classification, and benchmark-steering contracts establish this behavior.
---
id: coverage.dimensions-ontology
title: Dimensions and Ontology
summary: Define reusable behavior axes and their allowed values, then inspect how cases and benchmarks cover them.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/dimensions-ontology
---
# Dimensions and Ontology
## Definition
Dimensions are reusable project-level axes for describing how cases differ. Each Dimension contains ontology values: the named members used to classify cases and measure representation. A Dimension might be **Source condition**, with values such as **Current**, **Superseded**, **Conflicting**, and **Missing**.
Use **Project Foundations → Coverage Facets → Dimension** to create, generate, inspect, and maintain them.
## Fields, states, or lifecycle rules
### What a Dimension contains
| Element | Purpose |
| --- | --- |
| Name and description | Explain the behavior axis and its boundary |
| Ontology values | Define the values used for classification |
| Examples | Show classified cases and the reason for a value assignment |
| Statistics | Show case-pool and benchmark distribution by ontology value |
| Benchmark focus | Show whether values are required, sampled, diagnostic, or ignored in benchmark setup |
Dimensions and ontology values are project foundations. A benchmark does not copy them. **Get Started** assigns benchmark-specific roles to the project values, and **Representation** reports how the selected cases cover them.
Use the table to compare each Dimension's definition with its ontology and current classification footprint before opening the detail view.
### Create or generate a schema
Create a Dimension manually when the axis and vocabulary are already understood. Use the dimension-schema generator when project context or source material should produce a reviewable proposal. Generated proposals can include a definition, why the Dimension matters, proposed ontology members, and warnings.
A proposal is not the active schema. Review each proposed Dimension and value before accepting it. Avoid accepting near-duplicates simply because they use different wording.
> Classification boundary
>
> Creating or editing a Dimension does not instantly classify every existing case. Missing or stale classifications can be queued and monitored separately. Treat unclassified cases as missing evidence, not as an implicit ontology value.
### Design rules
- Make the Dimension answer one stable question. Split axes that mix several independent concerns.
- Give every ontology value a definition that distinguishes it from neighboring values.
- Prefer values that can be applied consistently to real cases.
- Do not use a Dimension to encode case quality, policy approval, or a desired model score.
- Review distributions after changing values. A clean schema can still leave important cases unclassified.
- Delete only after checking Case Pool and benchmark usage; removal changes the project coverage vocabulary.
### Benchmark roles
In benchmark Get Started, each Dimension and ontology value can receive a focus role:
- **Required:** the benchmark is expected to cover this value.
- **Sampled:** include it as part of the desired mix.
- **Diagnostic:** track it for analysis without making it part of the main denominator.
- **Ignored:** exclude it from the benchmark coverage intention.
- **Unset:** no explicit benchmark instruction has been recorded.
Those roles shape Coverage Story generation and interpretation. They do not alter the project-level definition of the value.
{% example-demo title="Example: source condition" %}
The project defines a Source condition Dimension with Current, Superseded, Conflicting, and Missing values. One benchmark marks all four as required; another marks Current as required and the remaining values as diagnostic. The same project vocabulary supports different benchmark intentions without duplicating the Dimension.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema)
- [Work with Project Topics](/docs/coverage-engineering/project-topics)
- [Plan Benchmark Coverage](/docs/coverage-engineering/plan-benchmark-coverage)
- [Analyze dataset representation](/docs/benchmark-datasets/representation)
{% /related-card-grid %}
## Source confidence
Code-backed: the current Dimension list, detail, settings, proposal, classification, examples, and statistics surfaces establish these fields and lifecycle boundaries.
---
id: coverage.project-topics
title: Project Topics
summary: Maintain source-grounded subject areas, editable groups, and Atlas relationships used to organize project and benchmark coverage.
kind: reference
product_area: coverage_engineering
status: stable
updated: 2026-09-07
canonical: /docs/coverage-engineering/project-topics
---
# Project Topics
## Definition
Project Topics are reusable subject areas derived from project knowledge, entered by an administrator, or contributed by an expert. They organize cases by what they are about. Unlike a Dimension, a Project Topic is not one value on a fixed behavior axis: a case can belong to several Topics.
The Project Topics workspace contains **Topics**, **Groups**, and **Atlas**.
## Fields, states, or lifecycle rules
### Topics
Each Topic has a name, description, active state, author type, case usage, benchmark usage, and source mentions where available. Source mentions retain evidence such as the originating Reference Material, Project Agent Brief, manual entry, or expert suggestion.
The Topic detail shows its definition and source grounding, linked and example cases, coverage confidence, nearby-topic signals, coverage by Dimensions and ontology values, and editable membership in Topic Groups.
An AI-created or source-extracted Topic remains an editable project artifact. Review its definition and grounding before using it to steer a benchmark.
### Groups
A Project Topic Group is an editable bundle of Topics. It does not merge or replace its members. Groups let benchmark setup express intent at a useful scale while retaining Topic-level traceability.
In **Get Started**, a Group can be handled as:
- **Cover every topic:** the benchmark should represent each member Topic.
- **Cover the group:** the group should be represented without requiring every member.
- **Use as guidance:** it can guide story construction without becoming a coverage obligation.
- **Do not use:** exclude the Group from this benchmark's setup.
- **Unset:** no explicit instruction.
Groups can start from manual work, AI suggestions, source material, or Atlas exploration. Manual edits remain significant; regenerating a suggestion should not be treated as authority to overwrite the reviewed group.
### Atlas
Atlas visualizes Topics, relationships, and Groups. Use it to inspect neighborhoods, redundancy, missing nearby Topics, and possible groupings. Relationships are analytical evidence, not a taxonomy merge. A close position or strong relation score does not mean two Topics are interchangeable.
The Topic coverage view can distinguish direct case grounding from breadth, Dimension spread, and binding confidence. When a Topic looks thin, inspect the linked cases before generating more. The problem may be missing cases, weak classification, an overly broad definition, or a duplicate Topic.
### Refresh and review
Project Topics can be refreshed from current project context and indexed sources. Refreshing can create or update Topics, source mentions, evidence, and relationship analysis. Review the resulting changes and recommendation runs before incorporating them into Groups or benchmark setup.
{% example-demo title="Example: group without flattening" %}
The project has Topics for Contract renewal, Price adjustment, and Termination notice. An administrator groups them as Agreement lifecycle. One benchmark chooses Cover every topic because each action has distinct risk. Another chooses Use as guidance because it only needs broad agreement-related examples. The individual Topics remain traceable in both benchmarks.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Work with Dimensions and Ontology](/docs/coverage-engineering/dimensions-ontology)
- [Define Case Construction Patterns](/docs/coverage-engineering/case-construction-patterns)
- [Configure benchmark Get Started](/docs/coverage-management/get-started)
- [Analyze dataset representation](/docs/benchmark-datasets/representation)
{% /related-card-grid %}
## Source confidence
Code-backed: the current Topic, Group, source-mention, statistics, relationship, Atlas, suggestion, and benchmark-handling contracts establish this model.
---
id: expert-contributions.artifacts
title: Contributed Artifacts
summary: Inspect policies, rubrics, cases, and coverage observations produced through attributable expert contribution work.
kind: reference
product_area: expert_contributions
status: stable
updated: 2026-09-07
canonical: /docs/expert-contributions/contributed-artifacts
---
# Contributed Artifacts
## Definition
Contributed Artifacts is the benchmark workspace for inspecting durable material produced through Expert Contributions. It organizes contributed **Policies**, **Rubrics**, **Cases**, and **new coverage observations** while preserving their relationship to the Contribution and expert work that produced them.
The view is a provenance and reconciliation surface. The final owner of a materialized artifact remains Correctness Governance, Assets, or Coverage Management according to artifact type.
## Fields, states, or lifecycle rules
- Policy contributions represent expert-grounded behavior rules or revisions.
- Rubric contributions represent proposed or accepted evaluation criteria tied to specialist judgment.
- Case contributions represent situations supplied or corrected through expert work.
- Coverage observations identify missing, thin, conflicting, or newly important benchmark behavior.
- A coverage observation preserves its source, proposed facet applications, and application status so an operator can distinguish a recorded observation from one incorporated into coverage structure.
- Each artifact should remain traceable to the Contribution, expert, selected evidence, task responses, and checkpoints that support it.
- Contribution completion and artifact governance are separate transitions. Inspect the artifact's owning surface before treating it as active policy, active rubric, benchmark dataset membership, or resolved coverage.
- Reconciliation can accept, revise, route, or leave material unresolved according to the active workflow.
## Interpreting contributed material
Use the artifact type to choose the next surface. A contributed policy or rubric belongs in Correctness Governance. A contributed case belongs in the project Assets pool before benchmark selection. A coverage observation belongs in Coverage Management and may motivate a Coverage Story, case construction, or another focused Contribution.
Preserve disagreements. Two experts can contribute conflicting Policy interpretations, and the artifact view should help an operator trace each interpretation rather than merge them into an invented consensus. Materialization should keep the Contribution, activity, checkpoint, expert, and scoped evidence links needed to explain why the artifact exists.
{% example-demo title="Example: contribution provenance" %}
An expert contributes a policy limiting compatibility claims, a rubric for explicit uncertainty, and a new case involving an unsupported adapter. The policy and rubric move to Correctness Governance for their lifecycle. The case enters Assets and is later selected into a benchmark dataset snapshot. All three retain the Contribution as their provenance.
{% /example-demo %}
## Source confidence
Code-backed: the active Contributed Artifacts workspace exposes policy, rubric, case, and new-coverage groupings. This page preserves the separation between contribution provenance and the lifecycle of each owning artifact.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics)
{% /related-card-grid %}
---
id: expert-contributions.lifecycle
title: Contribution Lifecycle and Status
summary: Interpret Contribution, activity, task, expert-runtime, and checkpoint states without inferring completion.
kind: reference
product_area: expert_contributions
status: stable
updated: 2026-08-22
canonical: /docs/expert-contributions/lifecycle-and-status
---
# Contribution Lifecycle and Status
## Definition
Contribution status, task status, expert-runtime state, and checkpoint state describe different layers. Read them together before diagnosing delay or declaring completion.
## Fields, states, or lifecycle rules
### Contribution status
| Status | Meaning |
| --- | --- |
| `PREPARING_DIRECTION` | Teammately is planning the direction and requested work |
| `AWAITING_DIRECTION_ALIGNMENT` | An administrator decision or alignment is required |
| `MATERIALIZING_TASKS` | The confirmed plan is becoming executable expert activities |
| `READY` | Work is prepared and available to the expert |
| `IN_PROGRESS` | At least part of the expert work is active |
| `COMPLETED` | Required expert work and finalization reached the durable terminal state |
| `CANCELLED` | Further work was stopped while preserving the record |
Planned activities can be Case Review, Form, Chat, or Interview. An individual review task can be preparing, blocked, ready, in progress, completed, skipped, or superseded. A superseded task remains provenance for what changed; it should not be counted as current outstanding work.
### Runtime and checkpoints
The expert runtime moves through preparing, ready, active, final checkpoint, completed, or exhausted. `EXHAUSTED` means the current executable path has no further work; it is not a synonym for a successfully completed Contribution.
A checkpoint moves from preparing to ready to reconciled. Its requirements may be pending, retryable, materialized, terminal-empty, or failed. Checkpoints can consolidate requirements, capture Policy meaning, prepare or record an Interview, or materialize Rubric meaning. A ready checkpoint still needs the expert's reconciliation.
### Diagnosing state
If the Contribution is `IN_PROGRESS` but the expert sees no action, inspect the current task, runtime, and checkpoint. A blocked task may be waiting for evidence; a final checkpoint may need confirmation; an exhausted runtime may require administrator follow-up. Use Logs & Status for durable timing and engagement evidence rather than relying on a stale open browser view.
> Completion boundary
>
> `COMPLETED` makes contributed evidence available for downstream governance. It does not automatically approve a Policy or Rubric, select a Case into a benchmark, or apply a coverage observation.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
{% /related-card-grid %}
## Source confidence
Code-backed: administrator and expert runtime sources define the layered Contribution, task, runtime, checkpoint, and requirement states described here.
---
id: expert-contributions.logs-status
title: Logs & Status
summary: Inspect contribution reviews, sessions, interviews, engagement, and operational records within the benchmark scope.
kind: reference
product_area: expert_contributions
status: stable
updated: 2026-09-07
canonical: /docs/expert-contributions/logs-and-status
---
# Logs & Status
## Definition
Logs & Status is the benchmark-scoped operational record for Expert Contributions. It exposes reviews, review sessions and plans, interviews, expert or contribution engagement, and related activity tables. Use it to answer when work happened, which record is active, and where a stalled or disputed result came from.
## Fields, states, or lifecycle rules
- Every table remains scoped to the selected benchmark.
- Review, session, interview, engagement, and artifact records keep separate identities and statuses.
- Filters, sorting, and detail views change presentation without rewriting the underlying Contribution.
- Available transcript and response detail follows the user's project permissions.
## Choose the right record
- **Reviews** shows review work and its Case or artifact relationship.
- **Review Sessions** and plans show the bounded sequence that presented work to an expert.
- **Interviews** exposes interview identity, targets, state, and available transcript detail.
- **Expert Engagement** and **Contribution Engagement** show participation and operational progress.
- Specialized tables can expose new-Case, output, draft-Policy, Case-specific Rubric, or review-question activity.
Use progress counts to locate work operationally; they do not establish whether contributed judgment was accepted.
Open a session when you need its task and result provenance rather than inferring it from aggregate progress.
Filter and sort within the benchmark scope before comparing counts. A session, interview, review, and contributed artifact are separate objects and may have different completion times. Do not add their row counts together as a measure of unique Contributions or experts.
## Provenance and diagnosis
Start from the Contribution when explaining intent, then follow its planned activity to the session, review, interview, checkpoint, and materialized artifact. Transcript or response evidence may explain a decision, but the owning Policy, Rubric, Case, or coverage surface determines the artifact's current governance status.
When work appears stuck, compare durable status with the latest task or checkpoint rather than inferring from engagement alone. An expert can open a Contribution without completing a task, and a completed interview can still be waiting for checkpoint reconciliation or materialization.
> Operational evidence, not correctness proof
>
> Logs demonstrate activity and provenance. They do not prove that contributed content was accepted, that experts agreed, or that a candidate system passed an evaluation.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
{% /related-card-grid %}
## Source confidence
Code-backed: the active Logs & Status routes define benchmark-scoped review, session, interview, and engagement tables and their detail surfaces.
---
id: orientation.key-objects
title: Key objects and relationships
summary: Understand how project foundations, contributions, datasets, evaluations, and improvement artifacts connect.
kind: concept
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/getting-oriented/key-objects-and-relationships
---
# Key objects and relationships
Teammately's evidence is trustworthy when a reader can move from project understanding and specialist authority to the exact Case, Benchmark version, Harness version, Run, and Improvement Session involved. This page gives the shared object graph.
> Artifact state matters
>
> An object name is not enough. Preserve project or benchmark scope, version or snapshot identity, provenance, and lifecycle state whenever the object supports evaluation or improvement.
## Definition
A **Project Agent Brief** and published **Reference block** give agents project understanding. **Project Input Schema** governs canonical **Case** input and materials. **Dimensions**, **Project Topics**, and Case Construction Patterns define reusable coverage structure. A saved **Harness version** identifies an executable candidate.
A benchmark selects Cases into a **Dataset snapshot** and combines them with governed **Policies** and **Rubrics** through a **Benchmark version**. An **Expert Contribution** requests specialist judgment through one or more Tasks and Checkpoints. Its **Contributed artifact** can become a policy, rubric, case, or coverage observation while retaining provenance.
A **Run** evaluates a saved Harness Version against a Benchmark Version. Its response, Rubric outcomes, settings, mapping, and metadata form evaluation evidence. An **Improvement Session** pins target evidence through a Goal Contract, creates or receives candidates, records evaluation receipts and safe session narration, and maintains a **Current frontier**.
## Decision checkpoint
| Object | Scope | Relationship that must remain visible |
| --- | --- | --- |
| Project Agent Brief / Reference block | Project | What agents understood and which source generation was available |
| Case / Harness version | Project | Which reusable asset and exact candidate state was selected |
| Contribution / Checkpoint | Benchmark | Which expert supplied or confirmed the judgment |
| Policy / Rubric | Project governance | Which authority, applicability, cases, and provenance support it |
| Dataset snapshot / Benchmark version | Benchmark | Which cases and correctness boundary define evidence |
| Run | Benchmark version | Which Harness, settings, mapping, and metadata produced results |
| Improvement Session / Current frontier | Benchmark version | Which goal and evaluation receipts justify retained candidates |
{% artifact-map title="How correctness artifacts connect" %}
{% /artifact-map %}
## Static materials and executable worlds
Canonical Case content separates `content.input` from optional `content.case_materials`. Static execution support uses case-material references. A `world_instance_ref` represents an executable or queryable environment and follows a separate capability and lifecycle boundary. The rendered case view helps people and adapters inspect canonical content; it does not create another authoring source.
## Provenance across scopes
Project assets can be reused across benchmarks, while dataset snapshots, Contributions, Runs, and Improvement Sessions remain benchmark-scoped. Materializing a contributed policy moves its governed owner to project scope without erasing the benchmark Contribution that supplied it. Evaluating a candidate records the saved Harness version rather than whichever Draft is currently open.
{% example-demo title="Contribution to frontier" %}
An Expert Contribution confirms a source-authority Policy and Rubric from selected Cases. The Cases enter a Dataset snapshot and the standard enters a Benchmark version. A Run evaluates Harness version 8 and exposes three failures. An Improvement Session pins those failures, evaluates versions 9 and 10, and retains version 10 in the Current frontier with canonical evaluation receipts.
{% /example-demo %}
## Related workflows
{% related-card-grid title="Related workflows" %}
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Object model](/docs/object-model)
- [Project Input Schema](/docs/project-settings/input-schema)
- [Product map](/docs/getting-oriented/product-map)
{% /related-card-grid %}
## Source confidence
Code-backed: active navigation, canonical case contracts, Contribution surfaces, versioned evaluation routes, and Improvement Session contracts support this object graph.
---
id: orientation.user-roles
title: User roles
summary: Understand the responsibilities of project operators, domain experts, AI engineers, accountable owners, and organization administrators.
kind: reference
product_area: reference
status: stable
updated: 2026-08-22
canonical: /docs/getting-oriented/user-roles
---
# User roles
## Definition
Teammately work is coordinated across responsibilities rather than one universal operator. A **project operator** maintains reusable context, assets, standards, and benchmark work. A **domain expert** supplies specialist judgment through Contributions. An **AI engineer** owns candidate Harness behavior and evaluation interpretation. An **accountable owner** decides how customer-owned downstream processes use the evidence. An **organization administrator** manages membership and organizational controls.
Product roles and permissions determine access, but responsibility can still vary by team. The important rule is to preserve who supplied judgment, who changed an artifact, and who owns the next action.
## Fields, states, or lifecycle rules
- Project operators configure Agent Setup, Project Input Schema, Coverage Facets, Assets, and benchmark work according to their access.
- Domain experts use the focused Contribution experience and do not require the full project workbench to complete assigned work.
- Expert profiles are global rather than a project-specific roster; a Contribution selects an appropriate expert for its application domain.
- AI engineers save Harness versions, start or inspect Benchmark Evaluations, and participate in Improvement Sessions.
- Accountable owners inspect versioned evidence and retain authority over downstream product, governance, or operational choices.
- Organization administrators manage organization-level access and controls without becoming the automatic approver of every policy, rubric, or Contribution.
- AI agents prepare and coordinate work but do not inherit human authority.
## Responsibility handoffs
An operator can prepare a Contribution, but the expert owns the specialist judgment. The resulting material can be reconciled into Correctness Governance, Assets, or Coverage Management. An AI engineer can change the Harness and produce new Runs, but cannot rewrite the expert provenance behind a rubric. An accountable owner can act on the evidence without turning that action into a Teammately artifact unless a source-backed workflow exists.
{% example-demo title="Example: role handoff" %}
A project operator requests a grounding Contribution from the policy owner. The expert confirms source authority and contributes a rubric. The operator reconciles it in Correctness Governance. An AI engineer evaluates a saved Harness version and starts an Improvement Session from the failures. The accountable product owner later decides what downstream action to take from the evidence.
{% /example-demo %}
## Source confidence
Code-backed: current project navigation, Contribution expert routes, and administration surfaces support these responsibility boundaries. Exact permissions remain governed by the active role configuration.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
{% /related-card-grid %}
---
id: governance.approval-history-reviewer-activity
title: Approval History and Reviewer Activity
summary: Understand how approvals and reviewer actions support explainable correctness decisions.
kind: reference
product_area: governance
status: stable
updated: 2026-08-23
canonical: /docs/governance/approval-history-and-reviewer-activity
---
# Approval History and Reviewer Activity
## Definition
Approval history records attributable decisions about a governed artifact. Reviewer activity records observable actions such as contribution progress, edits, comments, Checkpoint decisions, or materialization. They answer different questions: history explains which decision established the current governed state, while activity explains what work occurred around it.
Neither should be inferred from a final label alone. A Policy marked approved does not reveal every preceding suggestion, and activity does not become approval merely because an expert performed it.
## Fields, states, or lifecycle rules
- Approval belongs to the exact artifact or version shown by its owning surface.
- A Checkpoint decision can authorize Contribution progress or materialization without approving every related project object.
- Comments, interviews, task answers, and agent suggestions remain inputs until the owning workflow records an accepted or approved result.
- Current state and chronological activity should be read together; an older approval does not automatically govern a newer version.
- Reviewer identity, timestamps, and rationale are useful only when the product exposes them. Do not reconstruct missing history from private memory or internal logs.
- This documentation does not promise audit-log completeness, retention, export, or compliance behavior.
## Related objects
Read this page with [Contribution Lifecycle and Status](/docs/expert-contributions/lifecycle-and-status) for Contribution state, [Contributed Artifacts](/docs/expert-contributions/contributed-artifacts) for materialized learning, and [Human Approval Boundaries](/docs/governance/human-approval-boundaries) for accountable decisions. Policy and Rubric detail pages remain the authority for their current governed state.
{% example-demo title="Contribution work versus policy approval" %}
An expert completes an interview, edits a proposed exception, and approves a Contribution Checkpoint. The Contribution activity shows that work and the Checkpoint decision. When the accepted statement materializes as a Policy, its Policy detail records the governed version and approval context. A later reader can distinguish the expert's working history from the Policy version that actually entered benchmark evidence.
{% /example-demo %}
## Source confidence
Code-backed: Policy detail exposes approval and activity sections, while Contribution detail exposes lifecycle and Checkpoint state. These surfaces support attributable current-state interpretation, not a general compliance or audit-retention guarantee.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Using Checkpoints](/docs/expert-contributions/complete-contribution)
- [Human Approval Boundaries](/docs/governance/human-approval-boundaries)
- [Expert Contributions](/docs/expert-contributions)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.benchmark-versioning
title: Benchmark Versioning
summary: Preserve benchmark snapshots so evidence can be compared across target and standard changes.
kind: reference
product_area: governance
status: stable
updated: 2026-08-23
canonical: /docs/governance/benchmark-versioning
---
# Benchmark Versioning
## Definition
A Benchmark Version is the immutable evidence boundary used by Runs. It identifies the frozen dataset state and admitted evaluator relationships that make a result interpretable. The Benchmark remains a durable program; its versions preserve successive evidence boundaries as Cases, materials, coverage, Policies, or Rubrics change.
## Fields, states, or lifecycle rules
- Create a new Snapshot and resulting Benchmark Version when changed evidence would alter what a Run claims to test.
- Existing Runs remain attached to their original Benchmark Version.
- Current Dataset edits do not mutate a historical version.
- A new Harness Version alone does not require a new Benchmark Version; candidate and evidence versions move independently.
- Comparisons within one Benchmark Version isolate candidate differences more cleanly.
- Cross-version comparisons must name the changed Cases, evaluators, or representation boundary.
- Version identity does not prove that coverage is sufficient or that every admitted Rubric is correct.
## Related objects
Use [Dataset Snapshots](/docs/benchmark-datasets/snapshots) to create the frozen dataset boundary. Use [Benchmark Evaluations](/docs/benchmark-evaluations) to inspect Runs inside one exact version, and [Compare Harness Versions](/docs/benchmark-evaluations/compare) to interpret candidate movement without hiding version changes.
{% example-demo title="Separating candidate change from standard change" %}
Harness Version 12 improves retrieval and is evaluated against Benchmark Version 4, the same boundary used for Version 11. That comparison isolates candidate behavior. Later, experts approve a stricter source-authority Rubric and the dataset gains conflict Cases. The team creates Benchmark Version 5 and reports subsequent Runs under that new boundary instead of presenting the lower score as a regression against unchanged evidence.
{% /example-demo %}
## Source confidence
Code-backed: Benchmark Datasets → Snapshots preserves immutable Dataset boundaries, and the version-scoped Evaluation route binds Runs to one selected Benchmark Version. Coverage quality and downstream decisions remain outside version identity itself.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Benchmark snapshots](/docs/coverage-engineering/benchmark-snapshots)
- [Benchmarks](/docs/object-model/benchmarks)
- [Compare Harness Versions](/docs/benchmark-evaluations/compare)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.case-versioning
title: Case Versioning
summary: Track meaningful changes to cases, metadata, context, and attached outputs.
kind: reference
product_area: governance
status: stable
updated: 2026-08-23
canonical: /docs/governance/case-versioning
---
# Case Versioning
## Definition
A Case has durable identity and versioned canonical content. The exact Case version records the input shape and Case Materials that an evaluator or Harness received. Customer provenance and classification context can help find and interpret the Case, but they must not obscure which canonical version a historical Benchmark Version froze.
## Fields, states, or lifecycle rules
- A change to canonical input or required Case Materials creates a new evidence meaning and must remain distinguishable from the earlier version.
- Project Input Schema determines the accepted input kind and named Case Material slots.
- Correcting a filename label or customer-owned filter may be operational metadata; do not claim that every metadata edit has the same consequence as changed evaluated content.
- Accepted or golden output changes affect reference-output interpretation and should be traced separately from the Case input version.
- Existing Snapshots and Runs continue to identify the Case version they used.
- A current Case can be selected by several Benchmarks; version review must consider each future dataset that uses it.
- Case IDs and version IDs are backend-issued opaque values.
## Related objects
Read [Cases](/docs/assets/cases) for canonical Case structure, [Project Input Schema](/docs/project-settings/input-schema) for input and material requirements, and [Dataset Snapshots](/docs/benchmark-datasets/snapshots) for the immutable benchmark boundary. Use Benchmark Versioning when corrected Cases need to enter future evaluation evidence.
{% example-demo title="Correcting a controlling document" %}
A Case asks whether an opened product is returnable, but its attached policy PDF is discovered to be superseded. Replacing that Case Material changes the evidence presented to the candidate and evaluator, so the corrected Case must remain distinguishable from the old version. Runs against the old Benchmark Version stay interpretable as historical evidence; the corrected Case enters a new Dataset Snapshot for future Runs.
{% /example-demo %}
## Source confidence
Code-backed: canonical Case content and Project Input Schema services define input and material structure, while Case view types preserve backend-issued Case identity for product surfaces. Exact persistence mechanics remain internal; Snapshot pages define public historical evidence behavior.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Cases](/docs/assets/cases)
- [Dataset Snapshots](/docs/benchmark-datasets/snapshots)
- [Benchmark Versioning](/docs/governance/benchmark-versioning)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.human-approval-boundaries
title: Human Approval Boundaries
summary: Define which AI-assisted suggestions require accountable human review before becoming standards.
kind: reference
product_area: governance
status: stable
updated: 2026-09-07
canonical: /docs/governance/human-approval-boundaries
---
# Human Approval Boundaries
## Definition
Human approval boundaries separate preparation from governed correctness evidence. A suggestion, reviewer comment, interview answer, or draft standard can inform the loop, but it should not govern benchmark interpretation until the relevant human approval state is clear.
Use this page when prepared or contributed material is about to become a governed Policy, Rubric, Case-scoped reference output, or selected Benchmark Dataset evidence.
Comparison Directions are a narrower configuration object, not a governed standard. An AI-suggested Comparison Direction can be active without a separate approval step, but generated Cases, Benchmark membership, Policies, Rubrics, and reference outputs still follow their owning review or approval boundaries. A customer-owned human review packet is assembled from evidence; it is not a Teammately approval state.
> Approval is a state transition
>
> AI-assisted suggestions do not become governed standards until an accountable human approves the relevant artifact.
## Fields, states, or lifecycle rules
- Draft suggestions and reviewer notes are preparation material.
- Approved Policies and Rubrics, reviewed Case changes, selected Dataset membership, and supported Case-scoped reference outputs can affect governed evidence.
- AI-suggested Comparison Directions can affect future variant generation as active directions, but they do not approve the generated cases or standards they help explore.
- Approval should name the artifact being approved, not only the discussion that produced it.
- Stale or superseded approvals should be visible before older benchmark evidence is reused.
- This page does not claim external compliance approval, legal signoff, or production deployment authorization.
## Related objects
Read this with [What AI Features Can and Cannot Do](/docs/governance/what-ai-features-can-and-cannot-do), [Comparison Directions](/docs/assets/comparison-directions), [Approving Suggested Policies](/docs/correctness-governance/policies-and-rubrics), [Editing Suggested Rubrics](/docs/correctness-governance/policies-and-rubrics), and [Approval History and Reviewer Activity](/docs/governance/approval-history-and-reviewer-activity).
{% example-demo title="Human Approval Boundaries boundary" %}
Reviewer context: Several experts reject unsupported refund exceptions.
Suggested policy: The system proposes a policy that exceptions require approved support.
Approval boundary: The suggestion becomes governed only when a human owner approves the policy and its applicability.
Benchmark interpretation: Runs should cite the approved policy, not the unapproved suggestion that preceded it.
{% /example-demo %}
## Source confidence
Code-backed: Policy approval and activity components expose accountable approval state and history; contributed-artifact types keep expert learning distinct from materialized governed objects. Comparison Directions deliberately use active, archived, and advisory-stale behavior instead of the Policy approval lifecycle.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [What AI Features Can and Cannot Do](/docs/governance/what-ai-features-can-and-cannot-do)
- [Comparison Directions](/docs/assets/comparison-directions)
- [Review Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.policy-versioning
title: Policy Versioning
summary: Preserve policy changes so decisions can be interpreted against the standard used at the time.
kind: reference
product_area: governance
status: stable
updated: 2026-08-23
canonical: /docs/governance/policy-versioning
---
# Policy Versioning
## Definition
A Policy version preserves one governed statement of intended behavior together with its applicability and approval context. The durable Policy identity connects revisions; the version identity tells readers which wording and boundary informed linked Rubrics, Cases, Benchmark Versions, and evaluation results.
## Fields, states, or lifecycle rules
- Editing meaning, scope, precedence, exception handling, or applicability requires a distinguishable version.
- Draft or suggested wording is not an approved Policy version.
- Rubrics linked to a Policy must be reviewed when the Policy meaning changes; versioning the Policy does not silently repair every Rubric.
- Existing Benchmark Versions and Runs retain the Policy version admitted at their evidence boundary.
- A later current Policy does not make an older result false; it makes the result historical under the old standard.
- Activity and approval context explain why a version changed, but do not replace the version identity.
- Policy versions do not authorize downstream deployment or governance decisions.
## Related objects
Use Correctness Governance to inspect the current Policy, related Rubrics, linked Cases, approval context, and activity. Use Versioning and Staleness to route follow-up when changed Policy meaning affects current dataset readiness or future benchmark interpretation.
{% example-demo title="Adding an exception" %}
The current Policy says opened items cannot be returned. Experts confirm an exception for documented safety recalls. The revised Policy creates a new version, and its applicability and linked Rubrics are reviewed. Runs under the earlier Benchmark Version remain evidence of behavior against the old rule. A new Snapshot and Benchmark Version admit the changed standard for later evaluation.
{% /example-demo %}
## Source confidence
Code-backed: Policy types and detail routes expose durable identity, current version context, approval, activity, related Rubrics, and linked Cases. Benchmark pages define how an exact Policy version enters immutable evaluation evidence.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Policies](/docs/object-model/policies)
- [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Versioning and Staleness](/docs/governance/versioning-and-staleness)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.reproducibility
title: Reproducibility
summary: Preserve enough source context to explain and repeat correctness decisions.
kind: reference
product_area: governance
status: stable
updated: 2026-08-23
canonical: /docs/governance/reproducibility
---
# Reproducibility
## Definition
Reproducibility means preserving enough exact identity and observable configuration to explain what was evaluated and to repeat the supported execution path. It does not mean every future execution will produce an identical stochastic output. It means a reader can distinguish changes in candidate, evidence, evaluator, sampling, and runtime metadata instead of attributing every result difference to the model.
## Fields, states, or lifecycle rules
- Preserve Project, Benchmark, Benchmark Version, Dataset Snapshot, and Run identity.
- Preserve the exact saved Harness Version rather than an editable draft or display label.
- Preserve admitted Case, Policy, and Rubric version boundaries through the Benchmark Version.
- Record Run Group, attempt, sampling profile, evaluator set, and visible execution settings.
- Retain run metadata and measured telemetry when captured; missing values remain unknown.
- Record completeness, incomplete Cases, and terminal state beside scores.
- Use canonical evaluation receipts for Improvement Session candidate claims.
- Do not claim private worker activity, hidden reasoning, infrastructure internals, or unavailable traces as reproducibility evidence.
## Related objects
Dataset Snapshots preserve the evidence set. Benchmark Evaluations preserves candidate, Run, settings, results, and available telemetry. Compare and Arena interpret candidates inside compatible evidence boundaries. Improve adds Goal Contract, candidate, and canonical receipt identity when evaluation drives code or Harness changes.
{% example-demo title="Diagnosing a score change" %}
Two Runs use the same Harness Version but report different pass rates. The operator confirms that one Run used Benchmark Version 6 and the other used Version 7, which added source-conflict Cases and a revised grounding Rubric. The version and completeness record explains the movement. The team avoids filing a candidate regression until it compares Runs inside the same evidence boundary.
{% /example-demo %}
## Source confidence
Code-backed: Snapshot, Run detail, and run-metadata surfaces expose the immutable evidence boundary, candidate identity, status, counts, settings, and available metadata needed for supported reproducibility. They do not promise deterministic model output or unrestricted execution traces.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Benchmark Versioning](/docs/governance/benchmark-versioning)
- [Compare Harness Versions](/docs/benchmark-evaluations/compare)
- [Benchmark Evaluations](/docs/benchmark-evaluations)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.reviewer-project-access
title: Reviewer and Project Access
summary: Route reviewers and project participants to the work they are qualified to inspect or approve.
kind: reference
product_area: governance
status: stable
updated: 2026-09-07
canonical: /docs/governance/reviewer-and-project-access
---
# Reviewer and Project Access
## Definition
Reviewer access combines several explicit boundaries. Organization roles are managed in the Admin Console. Project membership determines access to a particular project. Expert Contribution assignment determines which scoped work an expert receives. Artifact approval authority belongs to the owning workflow and must not be inferred from a general role label.
## Fields, states, or lifecycle rules
- A workspace or organization role does not automatically prove access to every Project.
- Project membership does not automatically assign a person to every Expert Contribution.
- Assignment makes work available to the intended expert; it does not pre-approve the expert's response.
- Checkpoint and artifact approval state must be read from the Contribution or governed object itself.
- Removing access can prevent future interaction without erasing already recorded attributable evidence.
- API keys, service scopes, and customer integrations are separate authorization boundaries.
- Exact permission keys and enforcement behavior should be read from current administration and project surfaces, not inferred from persona names.
## Related objects
Use [Admin Console](/docs/admin-console) for organization roles and directory controls, [Project Members](/docs/project-settings/project-members) for project participation, and [Expert Contributions](/docs/expert-contributions) for assignment and expert-runtime state. Use the owning Policy, Rubric, Case, or Contribution page to determine approval.
{% example-demo title="Qualified expert without project access" %}
A specialist is the right reviewer for a source-authority Contribution but cannot open the assigned task. The operator checks organization membership, then Project membership, then Contribution assignment. Granting the missing Project access makes the task reachable; the assignment still controls which work is presented, and the specialist's later Checkpoint decision remains a separate recorded action.
{% /example-demo %}
## Source confidence
Code-backed: Project Permissions, Project Members, user, and workspace types establish project participation and role presentation. Contribution pages define assignment, runtime, and approval state separately; this page does not claim a complete authorization matrix.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Roles and Permissions](/docs/governance/roles-and-permissions)
- [Reviewer Assignments and Statuses](/docs/expert-contributions)
- [Permissions](/docs/reference/permissions)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.roles-permissions
title: Roles and Permissions
summary: Use roles and permissions to route ownership, review, and approval work clearly.
kind: reference
product_area: governance
status: stable
updated: 2026-09-07
canonical: /docs/governance/roles-and-permissions
---
# Roles and Permissions
## Definition
Roles and Permissions covers the access controls visible in the current product source. It is not a generic enterprise RBAC promise.
The main product exposes project permission management for users and groups. The Admin Console exposes workspace role management and permission keys. Orientation pages may describe common personas such as product owner, reviewer, expert, AI engineer, or administrator, but those personas are not exact permission values unless a code-backed source says so.
Use the roles table for organization access claims, and use project permission pages for project-level access claims.
## Fields, states, or lifecycle rules
- Main-product project permissions use user or group subjects and code-backed `ADMIN` or `MEMBER` project roles.
- Admin Console roles are workspace roles with permission keys managed through the admin app.
- Reviewer assignments and review ownership are workflow concepts; they should not be treated as workspace role names.
- Approval actions require accountable ownership; AI suggestions do not approve themselves.
## Related objects
Related objects may include workspace users, workspace groups, workspace roles, permission keys, project permission assignments, reviewer assignments, review sessions, approvals, and the correctness artifacts those users can access.
{% example-demo title="Human approval boundary" %}
A domain expert receives a Contribution and can complete its assigned Tasks. The same person is also a member of one Project, so they can inspect that Project's permitted surfaces. Neither fact makes them an organization administrator or grants authority to approve every governed artifact. Read access, Contribution assignment, artifact approval, and Workspace administration from their owning surfaces.
{% /example-demo %}
## Source confidence
Code-backed: this page is grounded in the main product project permission routes and Admin Console role management source listed in source_refs.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Build policies and rubrics](/docs/operating-manual/build-policies-and-rubrics)
- [Manage Project Members](/docs/project-settings/project-members)
{% /related-card-grid %}
---
id: governance.rubric-versioning
title: Rubric Versioning
summary: Track changes to pass/fail criteria and the evidence they produce.
kind: reference
product_area: governance
status: stable
updated: 2026-08-23
canonical: /docs/governance/rubric-versioning
---
# Rubric Versioning
## Definition
A Rubric version preserves one observable binary evaluation criterion and its relationship to a Policy, applicability boundary, and normative force. The durable Rubric identity connects revisions; the version identity explains which exact criterion produced a historical result.
## Fields, states, or lifecycle rules
- Change the version when criterion meaning, pass/fail threshold, applicability, Policy relationship, or normative force changes.
- Editorial cleanup that genuinely preserves meaning should not be described as a new correctness rule without evidence.
- A Rubric must remain testable as one binary criterion; combining several independent requirements makes failures ambiguous.
- Suggested or contributed wording is not admitted evaluation authority until the owning workflow approves it.
- Existing Runs retain the exact Rubric version they evaluated.
- A new Rubric version can require a new Benchmark Version before it affects future evidence.
- Prefer-level improvement does not cancel a must-level regression.
## Related objects
Use Correctness Governance to inspect criterion text, Policy linkage, applicability, Cases, and approval state. Use Binary Rubrics when authoring or splitting a criterion. Use Benchmark Versioning when the revised Rubric should become part of future evaluation evidence.
{% example-demo title="Splitting an ambiguous criterion" %}
A Rubric currently passes only when an answer cites the controlling source and states uncertainty when no source is available. Reviewers find that the combined criterion produces unclear failures. They retain the original historical version and approve two narrower Rubrics for citation support and missing-source uncertainty. Future Benchmark Versions admit the new criteria; old Runs keep their original result meaning.
{% /example-demo %}
## Source confidence
Code-backed: Rubric types and detail routes expose criterion, version, Policy relationship, applicability, normative force, linked Cases, and evaluation context. Linked authoring and Benchmark pages define approval and admission boundaries.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Rubrics](/docs/object-model/rubrics)
- [Binary Rubrics](/docs/correctness-governance/binary-rubrics)
- [Versioning and Staleness](/docs/governance/versioning-and-staleness)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.ai-feature-boundaries
title: What AI Features Can and Cannot Do
summary: Explain the difference between AI-assisted suggestions and approved correctness infrastructure.
kind: reference
product_area: governance
status: stable
updated: 2026-09-07
canonical: /docs/governance/what-ai-features-can-and-cannot-do
---
# What AI Features Can and Cannot Do
## Definition
AI-assisted features can help prepare correctness work, but they do not own the approval boundary. They may draft, classify, summarize, propose, or organize artifacts; a human still needs to approve governed standards and review context before benchmark evidence depends on them.
Use this page when a reader needs to separate preparation from authority. A fluent suggestion can be useful, but it is not an approved Policy, Rubric, Case-scoped reference output, Benchmark evidence, or downstream customer decision by itself.
> Human approval boundary
>
> AI features can draft, summarize, classify, or suggest; they do not approve policies, rubrics, benchmark evidence, or customer-owned decisions by themselves.
## Fields, states, or lifecycle rules
- AI-assisted output can be draft material, review support, classification help, or summarization.
- AI-assisted output should not be treated as approved standards, benchmark evidence, or customer-owned decisions without human approval.
- Suggested policies, rubrics, and classifications need visible approval or rejection state before they affect governed evidence.
- AI-suggested Comparison Directions are different from suggested standards: they can become active directions immediately, but they still do not approve policies, rubrics, cases, benchmark membership, or review context.
- AI-assistance language does not establish model-provider behavior, data retention, compliance posture, or autonomous approval.
- Source-backed product pages decide exact supported behavior; this page defines the public boundary.
## Related objects
Read this with [Human Approval Boundaries](/docs/governance/human-approval-boundaries), [Comparison Directions](/docs/assets/comparison-directions), [Using Expert Judgment](/docs/concepts/correctness-elicitation), [Using Checkpoints](/docs/expert-contributions/complete-contribution), and [Noisy AI Suggestions](/docs/troubleshooting/noisy-ai-suggestions).
{% example-demo title="What AI Features Can and Cannot Do boundary" %}
Draft suggestion: An AI-assisted workflow proposes a new compatibility policy after reading rejected cases.
Allowed use: The suggestion can help a reviewer start the policy draft.
Not allowed as governed evidence: A benchmark should not cite the policy until an accountable human approves the artifact and its applicability.
Interpretation: The AI feature accelerated preparation; the human approval boundary still controls whether the standard can govern benchmark evidence.
{% /example-demo %}
{% example-demo title="Example: active AI-suggested direction" %}
AI-suggested direction: Teammately suggests a stale-source conflict Comparison Direction after project cases and ontology values show that gap.
Allowed use: The direction can become active immediately and can guide preview examples or future variant generation.
Not allowed as governed evidence: The direction does not approve a policy, rubric, generated case, benchmark membership, or downstream decision. Users still review generated examples and manage the direction normally.
{% /example-demo %}
## Source confidence
Code-backed: generation surfaces can propose Dimension schemas, synthesize candidate Cases, and suggest Comparison Directions; Policy approval and contributed-artifact state establish separate human-governance boundaries. These sources support the product-state distinction here, not claims about model providers, retention, or autonomous authority.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Human Approval Boundaries](/docs/governance/human-approval-boundaries)
- [Comparison Directions](/docs/assets/comparison-directions)
- [Using Expert Judgment](/docs/concepts/correctness-elicitation)
- [Using Checkpoints](/docs/expert-contributions/complete-contribution)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: governance.workspace-administration
title: Workspace Administration
summary: Administer workspace and project boundaries so correctness work has clear ownership and scope.
kind: reference
product_area: governance
status: stable
updated: 2026-09-07
canonical: /docs/governance/workspace-administration
---
# Workspace Administration
## Definition
Workspace Administration covers two code-backed layers:
- Organization administration in the Admin Console, including members, groups, roles, domain control, IP address control, audit log, usage statistics, profile, support settings, integrations, and API keys.
- Project administration in the main product, including project members, groups, and project permission assignments.
Keep these layers separate when writing or operating from the docs. A workspace-level admin control is not the same as a project-specific correctness workflow.
Organization groups belong to the admin layer. Project membership and correctness artifact ownership remain separate concepts.
## Fields, states, or lifecycle rules
- Project permissions in the main product are code-backed for user and group subjects with `ADMIN` or `MEMBER` project roles.
- The Admin Console exposes additional organization-level controls such as roles, domain control, IP address control, integrations, and API keys.
- Do not infer compliance status, retention rules, SSO behavior, billing behavior, rate limits, or deployment options from the existence of an admin screen.
- Approval actions in correctness workflows require accountable ownership; AI suggestions do not approve themselves.
## Related objects
Related objects may include workspace members, groups, workspace roles, permission keys, project permission assignments, domain controls, IP controls, API keys, notification integrations, and the project artifacts those people can access.
{% example-demo title="Human approval boundary" %}
A Workspace administrator manages an organization group in the Admin Console. A Project owner then grants that group access under Project Settings. The group can inspect the Project according to the current Project membership surface, but this does not change organization security controls or create authority over every governed correctness artifact.
{% /example-demo %}
## Source confidence
Code-backed: this page is grounded in the Admin Console navigation and the main product project settings routes listed in source_refs.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Manage Project Members](/docs/project-settings/project-members)
- [Import and prepare cases](/docs/operating-manual/import-and-prepare-cases)
{% /related-card-grid %}
---
id: improve.candidates-frontier
title: Candidates and the Current Frontier
summary: Interpret candidate stages, canonical receipts, constraints, retained Candidate Systems, and the current frontier.
kind: reference
product_area: improve
status: stable
updated: 2026-09-07
canonical: /docs/improve/candidates-and-frontier
---
# Candidates and the Current Frontier
## Definition
An Improve candidate moves through `working`, `focused_evaluated`, `canonical_evaluated`, `retained`, `rejected`, or `incomparable`. Stage describes evidence maturity, not source quality or recency.
## Fields, states, or lifecycle rules
Candidate stage, Harness Version, Benchmark Version, canonical Run, provenance, scores, constraint results, parent branch, and receipts form the durable evidence record. Frontier membership is a retained evidence decision rather than another candidate stage.
## Candidate System requirements
A retained Candidate System identifies an exact saved Harness Version, immutable Benchmark Version, canonical Run, provenance, scores, and constraint results. Focused evaluation can route exploration, but only canonical evaluation against the pinned cohort supports frontier retention.
A rejected candidate remains useful evidence when it reveals a regression or failed hypothesis. An incomparable candidate lacks the common evidence boundary needed for a valid decision. Do not fill that gap with aggregate intuition.
Candidate Exploration evidence is scoped to the session and branch that produced it. It is not automatically the project's global best candidate. The current frontier contains retained candidates supported by the latest valid Goal and receipts, but “current” does not mean newest or highest aggregate score.
## Harness materialization boundary
The active Improve model does not require a separate legacy “Save candidate as Harness” action. A candidate eligible for retention already references an immutable saved Harness Version. Use Assets → Harnesses to inspect that Version or select it directly in a future Benchmark Evaluation launch.
If an external worker returns only a patch, prose, or private claim, the candidate is not yet a Candidate System. Materialize the return through the supported contract so the saved Version and evaluation request become observable.
## Retention checklist
- Exact Harness and Benchmark Versions are present.
- The canonical Run is complete enough for the measurement bindings.
- Hard constraints pass; soft-constraint tradeoffs are visible.
- Evaluation receipts and hashes resolve to the claimed Case cohort.
- Parent branch and source provenance are recorded.
- The candidate is retained for evidence, not merely because it is latest.
{% example-demo title="Example: aggregate winner outside the frontier" %}
A new candidate has the highest overall pass rate but violates a hard required-Policy constraint. Its canonical receipt remains visible, but the candidate is rejected and does not displace two lower-scoring retained candidates that satisfy the confirmed Goal.
{% /example-demo %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Start an Improvement Session](/docs/improve/start-improvement-session)
- [Compare Harness Versions](/docs/benchmark-evaluations/compare)
{% /related-card-grid %}
## Source confidence
Code-backed: the active session contract, archive view, and coordinator evidence define candidate stages, exact identities, canonical receipts, retention, rejection, incomparability, and frontier semantics.
---
id: improve.chronology-trajectories
title: Chronology, Trajectories, and Receipts
summary: Read durable session events, safe narrated work segments, evaluation receipts, external handoffs, and usage evidence.
kind: reference
product_area: improve
status: stable
updated: 2026-09-07
canonical: /docs/improve/chronology-and-trajectories
---
# Chronology, Trajectories, and Receipts
## Definition
Chronology is the durable ordered record of an Improvement Session. Event blocks can represent user or assistant messages, activity, Harness output, usage, Goal changes, input requests, external handoffs, and cancellation. Pagination preserves long sessions without implying that the initially loaded page is complete history.
## Fields, states, or lifecycle rules
- Chronology event types retain their identity and ordering.
- Trajectory segments link only to observable activity.
- Evaluation ledger rows preserve Case counts, completion, receipts, and hashes.
- Worker packages and sessions have independent lifecycle states.
## Safe trajectories
Trajectory segments narrate observable work and link to durable activity references. They may explain that a candidate was prepared, evaluated, retained, or rejected. They exclude hidden reasoning and private worker state. Do not rewrite silence between returned artifacts as a detailed external-worker trajectory.
Benchmark Evaluations does not currently expose Traces / Spans; Improve trajectories are a separate safe session narration surface. They should not be described as raw model reasoning, execution spans, or evaluator authority.
## Evaluation ledger and receipts
The evaluation ledger records exact Case counts, evaluable and incomplete populations, scores, constraint results, and receipt or content hashes. A receipt links a candidate claim to the canonical Benchmark Evaluation that supports it. When the cohort is incomplete, preserve that state in frontier and completion decisions.
Usage entries can identify provider use by model. Missing usage evidence is unknown, not zero. Harness output records candidate-visible results without granting access to private runtime reasoning.
## External handoffs
An external worker package is prepared only after a Goal Contract is confirmed. Its status can be prepared, submitted, expired, or closed. The return contract accepts an immutable Harness Version or a supported evaluation request. Expiration or closure describes the package lifecycle, not whether private work occurred.
## Session lifecycle
Sessions can be draft, active, paused, completing, completed, cancelling, cancelled, or failed. Attention states identify an input or review need. Pausing stops new scheduling while retaining chronology and evidence. Cancelling records a terminal path; it does not erase candidates, receipts, or usage already recorded.
> Observable-state rule
>
> Chronology may report only durable messages, events, returned artifacts, requests, and receipts. Never infer hidden chain of thought or fabricate progress for an external worker.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Start an Improvement Session](/docs/improve/start-improvement-session)
- [Choose Work or Evolve](/docs/improve/work-and-evolve)
{% /related-card-grid %}
## Source confidence
Code-backed: chronology rendering, the session contract, and archive model define event types, safe trajectories, ledger and receipt fields, handoff states, usage, pagination, and lifecycle controls.
---
id: improve.goal-contracts
title: Goal Contracts
summary: Bind an Improvement Session to exact targets, measurable objectives, constraints, and permitted intervention scope.
kind: reference
product_area: improve
status: stable
updated: 2026-09-07
canonical: /docs/improve/goal-contracts
---
# Goal Contracts
## Definition
A Goal Contract is the authority and measurement boundary for one Improvement Session. It turns a natural-language intent into canonical targets, prioritized objectives, protected constraints, and evaluable bindings.
## Fields, states, or lifecycle rules
### Contract contents
The contract identifies exact target IDs and records intent. Objectives carry priorities. Constraints are hard or soft. Measurement bindings name whether evidence should **improve**, **preserve**, or **reduce** a canonical Case, Policy, Rubric, metric, or other admitted reference. Intervention constraints state what work may change. Unresolved items remain explicit until the contract can be confirmed safely.
Every requested outcome needs an observable binding. “Make answers better” is not measurable; “improve the selected grounding Rubric while preserving the selected uncertainty Cases” is. A hard constraint disqualifies a candidate when violated. A soft constraint records a tradeoff that still requires review.
### Revisions and confirmation
Goal Contract revisions can be proposed, confirmed, superseded, or rejected. Keep superseded and rejected revisions as history so candidate activity can be explained against the contract that authorized it. Do not silently edit the meaning of a session after work has begun.
Evolve locks the Goal Contract after the first epoch begins. If new evidence reveals a fundamentally different goal, stop or complete the current session and create an explicit new boundary rather than retrofitting prior epochs.
### Confirmation checklist
- Target Harness, Harness Version, Benchmark Version, and starting Run resolve to exact identities.
- Each objective has a direction and canonical measurement binding.
- Non-regression behavior is represented by preserve bindings or constraints.
- Hard and soft constraints are distinguishable.
- Intervention scope permits the intended code, prompt, retrieval, or configuration work.
- No unresolved item makes evaluation or authorization ambiguous.
> Authority boundary
>
> The Goal Contract authorizes session work; it does not change the Benchmark Version, approve a new correctness standard, or waive human governance of upstream artifacts.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Start an Improvement Session](/docs/improve/start-improvement-session)
- [Choose Work or Evolve](/docs/improve/work-and-evolve)
{% /related-card-grid %}
## Source confidence
Code-backed: the active session contract and commands define canonical targets, objectives, constraints, measurement bindings, intervention scope, revision states, confirmation, and Evolve locking.
---
id: object-model.overview
title: Object model
summary: Understand how project foundations, benchmark artifacts, contributions, evaluations, and improvement sessions connect.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/object-model
---
# Object model
Use this reference when a workflow depends on artifact ownership, identity, version history, or provenance. Teammately keeps reusable project foundations separate from benchmark-scoped evidence so a later reader can reconstruct what the agent knew, what was evaluated, who supplied judgment, and why a candidate was retained.
> Preserve scope and version
>
> An artifact name alone is not sufficient evidence. Preserve its project or benchmark scope, version or snapshot identity, lifecycle state, and provenance whenever it affects evaluation or improvement.
## Definition
A **Project** owns reusable foundations: the Project Agent Brief, Reference blocks, policies, rubrics, coverage facets, Cases, Harnesses, Review Screen configuration, Comparison Directions, and Project Input Schema. These objects can support more than one benchmark.
A **Benchmark** owns a particular correctness boundary. Its workspace selects Cases into Dataset snapshots, organizes benchmark-scoped coverage work, requests Expert Contributions, and creates Benchmark versions. A Benchmark version is the fixed boundary against which a saved Harness version is evaluated.
An **Evaluation Run** records the response, Rubric outcomes, mapping, settings, and metadata produced by one saved Harness Version against one Benchmark Version. It does not currently expose execution trajectories. An **Improvement Session** pins target evidence through a Goal Contract, evaluates candidates, retains canonical receipts and safe narrated session activity, and updates a Current frontier. The frontier is evidence-backed session state, not an unnamed replacement for the saved Harness.
## Decision checkpoint
| Question | Object boundary to preserve | Common mistake |
| --- | --- | --- |
| What did the agent understand? | Project Agent Brief and Reference block generation | Treating the latest source file as the knowledge used by an earlier run |
| What behavior was evaluated? | Case, Dataset snapshot, Benchmark version, and saved Harness version | Describing only a prompt and score |
| What standard applied? | Policy, applicability, Rubric, and provenance | Letting a Rubric stand in for the Policy it checks |
| Who supplied judgment? | Contribution, Task, Checkpoint, expert, and Contributed artifact | Treating an unattributed comment as governed truth |
| Why was a candidate retained? | Improvement Session, Goal Contract, evaluation receipt, and Current frontier | Treating an unsaved draft as the evaluated candidate |
## Fields, states, or lifecycle rules
### Project foundations
- **Project Agent Brief:** Published project understanding used to orient agents.
- **Reference block:** Indexed knowledge with source and generation identity.
- **Policy and Rubric:** Governed meaning and executable judgment criteria, with applicability and provenance.
- **Coverage Facet:** Reusable Dimensions, Project Topics, and Case Construction Patterns that describe important behavior space.
- **Case:** Canonical `content.input`, optional `content.case_materials`, metadata, and optional executable world reference.
- **Harness:** An agent configuration with mutable Draft state and immutable saved versions used for evaluation.
- **Project Input Schema:** The project-level architecture that governs canonical Case input and materials.
### Benchmark artifacts
- **Dataset snapshot:** A reproducible selection and representation of benchmark Cases.
- **Coverage Story:** Benchmark-scoped intent that connects coverage structure to concrete case work.
- **Expert Contribution:** A benchmark-scoped request containing Tasks, context, statuses, and optional Checkpoints.
- **Contributed artifact:** A policy, Rubric, Case, or coverage observation supplied through a Contribution with attributable provenance.
- **Benchmark version:** The fixed evaluation boundary used by Runs and Improvement Sessions.
- **Evaluation Run:** One execution with candidate, benchmark, response, Rubric outcomes, settings, mapping, and metadata identity.
- **Improvement Session:** A goal-directed candidate exploration process with pinned evidence, receipts, trajectories, and frontier state.
{% artifact-map title="How correctness artifacts connect" %}
{% /artifact-map %}
## Static materials and executable worlds
Case materials and executable worlds are intentionally different. `content.case_materials` holds canonical material data or references that can accompany the input. A `world_instance_ref` identifies an executable or queryable environment with a separate capability and lifecycle boundary. Rendering either for inspection does not create another authoring source.
{% example-demo title="From contribution to frontier" %}
A benchmark Contribution asks a procurement specialist to resolve three ambiguous Cases. The expert confirms a Policy and binary Rubric at a Checkpoint. The artifacts are reconciled into Correctness Governance while retaining their Contribution provenance. The Cases enter a Dataset snapshot and the standard enters Benchmark version 6. An Evaluation Run tests Harness version 11 and exposes two failures. An Improvement Session pins those failures, evaluates versions 12 and 13, and retains version 13 in the Current frontier with canonical evaluation receipts.
{% /example-demo %}
## Source confidence
Code-backed: current navigation establishes project and benchmark scopes; canonical case-content code establishes the input and materials boundary; the active Improve session contract establishes Goal Contract, candidate, receipt, and frontier concepts. This page describes public object semantics, not a public API or export schema.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [First correctness loop](/docs/operating-manual/first-correctness-loop)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Run a Benchmark Evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Start an Improvement Session](/docs/improve/start-improvement-session)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [Cases](/docs/assets/cases)
- [Harnesses](/docs/assets/harnesses)
- [Glossary](/docs/reference/glossary)
{% /related-card-grid %}
---
id: object-model.applicability-logic
title: Applicability logic
summary: Explain when a policy or rubric should be used for a case, output, or coverage segment.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/applicability-logic
---
# Applicability logic
## Definition
Applicability logic is the boundary that decides whether a policy or rubric should be considered for a specific case, output, or coverage segment. It prevents a good standard from being applied to the wrong behavior.
Use this reference when a benchmark result is hard to explain because a standard seems relevant in some cases but not others. The question is not only whether the policy is correct; it is whether the policy was eligible to judge that output.
## Fields, states, or lifecycle rules
- Applicability sits between the case/output and the policy/rubric that may judge it.
- Weak applicability makes benchmark failures noisy: a candidate can fail a good rubric on a case where the rule should not have applied.
- Strong applicability names the behavior condition, source context, or case segment that brings the standard into scope.
- Applicability changes can make old benchmark evidence stale because the same output may be judged by a different standard boundary.
- This page explains the public object relationship, not a public rule language, API schema, or export contract.
## Related objects
Applicability logic should be read with [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Cases](/docs/object-model/cases), and [Coverage Dimensions](/docs/object-model/coverage-dimensions). Use the Correctness Governance workflow to write the boundary and troubleshooting when the observed Case set is wrong.
{% example-demo title="Applicability logic boundary" %}
Raw case: A user asks whether a product works with equipment they already own.
Policy: Compatibility claims require explicit source support.
Applicability logic: The policy applies only when the answer recommends, validates, or compares a product for a concrete use context.
Benchmark interpretation: If the case only asks for a product description, the compatibility rubric should not judge it. If the answer claims the product will work with the user's equipment, the rule is in scope.
{% /example-demo %}
## Source confidence
Code-backed: Policy and Rubric types and their detail routes expose applicability fields and the links among governed standards, Cases, and evaluation checks. This page describes those product relationships; it does not define a portable rule language.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Work with Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Fix weak applicability logic](/docs/troubleshooting/weak-applicability-logic)
- [Resolve conflicting correctness evidence](/docs/governance/conflict-resolution)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.benchmarks
title: Benchmarks
summary: Understand a Benchmark as the durable program that owns benchmark-scoped coverage, evidence boundaries, evaluations, and improvement work.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/benchmarks
---
# Benchmarks
## Definition
A Benchmark is the durable project object for one intended evaluation program. It owns benchmark-scoped work across Benchmark Datasets, Coverage Management, Expert Contributions, Benchmark Evaluations, and Improve while its selected Cases, governed standards, and candidate systems evolve.
The current Benchmark Dataset is editable. A Dataset Snapshot freezes selected Case membership, and a Benchmark Version provides the immutable boundary consumed by Runs. A Benchmark is therefore not a Snapshot, Benchmark Version, Run, or score.
## Fields, states, or lifecycle rules
- The Benchmark identity persists across changes to its current Dataset, coverage work, standards, and Harness candidates.
- Benchmark Datasets owns selected Cases and immutable Dataset Snapshots.
- A Benchmark Version fixes the evidence boundary used by a Run.
- Benchmark membership should be shaped by coverage work, not by whichever Cases are easiest to run.
- A Run result is weak if the Benchmark Version and saved Harness Version behind it are unclear.
- This page documents object semantics, not public execution, export, rate-limit, or API guarantees.
## Related objects
Benchmarks should be read with [Cases](/docs/assets/cases), [Policies](/docs/object-model/policies), [Rubrics](/docs/object-model/rubrics), [Coverage Engineering](/docs/coverage-engineering), and [Benchmark Evaluations](/docs/benchmark-evaluations). Use [Run an evaluation](/docs/benchmark-evaluations/run-evaluation) when the next step is execution.
{% example-demo title="Benchmarks boundary" %}
Raw case: A team refreshes coverage after finding unsupported compatibility claims.
Benchmark version: The refreshed version includes new unsupported-claim cases and the approved compatibility rubric.
Run: The candidate is evaluated against that version.
Interpretation: If the score drops, reviewers can see that the benchmark became harder instead of assuming the candidate behavior changed.
{% /example-demo %}
## Source confidence
Code-backed: the Benchmark type and workspace establish durable Benchmark identity; Benchmark Datasets → Snapshots establishes immutable Dataset boundaries; the evaluation-runs route consumes a specific Benchmark Version. The public object definition does not imply an execution or export API.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Benchmarks](/docs/coverage-engineering/benchmarks)
- [Benchmarks and versions](/docs/concepts/benchmarks-and-versions)
- [Benchmark Evaluations](/docs/benchmark-evaluations)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.case-pool
title: Case Pool surface
summary: Understand Case Pool as the Assets view for reusable project Cases, candidates, and explicit Benchmark selection.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/case-pool
---
# Case Pool surface
## Definition
**Case Pool** is a tab under **Assets → Cases**, not a separate Case object type. It shows reusable project Cases and candidate material that can be inspected before explicit selection into a Benchmark Dataset.
Use this reference when deciding whether a Case should stay in the project collection, be prepared further, or be selected for a Benchmark. **Sourcing Tasks** is the adjacent job and audit surface; it is not another Case state.
## Fields, states, or lifecycle rules
- Case Pool can contain useful Cases that are not selected into any Benchmark.
- Candidate and in-use labels describe curation and membership context; they do not create a different persisted Case type.
- Coverage dimensions and ontology values help explain what behavior the pool represents.
- Adding a Case to a current Benchmark Dataset changes mutable membership. A new Dataset Snapshot is required to freeze that change as an evidence boundary.
- This page does not define public upload schemas, export contracts, or retention behavior.
## Related objects
Read Case Pool with [Cases](/docs/object-model/cases), [Candidate and In-Use Cases](/docs/coverage-engineering/candidate-and-in-use-cases), [Coverage Dimensions](/docs/object-model/coverage-dimensions), and [Benchmark Dataset Cases](/docs/benchmark-datasets/cases). Use [Case Pool](/docs/coverage-engineering/case-pool) for operating instructions.
{% example-demo title="Case pool boundary" %}
Raw case: A new support transcript shows a refund exception question that reviewers have not seen before.
Case pool state: The example is a candidate case because it may close an exception-handling coverage gap.
Interpretation: It should not affect benchmark scores until source context, output identity, coverage labels, and benchmark membership are clear.
{% /example-demo %}
## Source confidence
Code-backed: the Assets Cases routes expose Case Pool and Sourcing Tasks tabs, candidate and in-use views, and an explicit add-to-Benchmark action. Case Pool is therefore a curation surface, not automatic Benchmark membership, a separate Case schema, or a public storage contract.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Case Pool](/docs/coverage-engineering/case-pool)
- [Candidate and In-Use Cases](/docs/coverage-engineering/candidate-and-in-use-cases)
- [Cases](/docs/object-model/cases)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.cases
title: Cases
summary: Understand cases as the situations Teammately uses to represent important AI behavior.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/cases
---
# Cases
## Definition
Cases are the behavior situations Teammately uses for review, coverage, and benchmark evidence. A case should preserve the input, source context, metadata, and version boundary needed to understand what behavior is being judged.
Use this reference when a workflow depends on whether an example is reviewable, benchmark-ready, stale, duplicated, or missing the context a reviewer needs.
## Fields, states, or lifecycle rules
- A case is not the same thing as an output; the case holds the situation and context, while the output is the behavior being judged.
- A case can be useful for coverage planning before it is ready for benchmark use.
- Case versions matter when source context, metadata, or expected behavior changes.
- A benchmark result is weak if the failed output cannot be traced back to the exact case version.
- Exact import schema and export behavior belong to source-backed integration pages, not this object reference.
## Related objects
Cases should be read with [Outputs](/docs/object-model/outputs), [Case versions](/docs/governance/case-versioning), [Case pool](/docs/object-model/case-pool), and [Benchmarks](/docs/object-model/benchmarks). Use [Importing cases](/docs/operating-manual/import-and-prepare-cases) when the next step is bringing source examples into Teammately.
{% example-demo title="Cases boundary" %}
Raw record: A transcript includes a customer question, assistant answer, account context, and later human correction.
Case: The reviewable unit preserves the customer question and account context.
Output: The candidate answer is attached separately so reviewers know which behavior is being judged.
Interpretation: If the candidate fails later, the run result can point to the exact case and output rather than to a vague transcript.
{% /example-demo %}
## Source confidence
Code-backed: Case view types and the Cases asset route establish the product object and its presentation; internal Case-content types establish how input and contextual material remain separate from outputs. These implementation sources support object semantics, not a customer-facing serialization contract.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Case model overview](/docs/assets)
- [Case Pool](/docs/coverage-engineering/case-pool)
- [Case pool](/docs/object-model/case-pool)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.coverage-dimensions
title: Coverage dimensions
summary: Organize cases by the behavior axes that matter to product correctness.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/coverage-dimensions
---
# Coverage dimensions
## Definition
Coverage dimensions are the axes used to explain what behavior space a case set represents. A dimension can describe source freshness, request type, risk level, product area, policy boundary, or another classification that matters for review and benchmark interpretation.
Use this reference when a benchmark score is not enough and the team needs to ask which kinds of behavior are represented or missing.
## Fields, states, or lifecycle rules
- Dimensions should describe meaningful behavior axes, not arbitrary tags.
- Ontology values should keep each dimension's labels consistent enough for coverage planning.
- Coverage dimensions can reveal untested segments even when aggregate benchmark scores look strong.
- Changing a dimension schema can change how old benchmark evidence is interpreted.
- This page describes object semantics, not a public schema contract.
## Related objects
Coverage dimensions should be read with [Ontology](/docs/object-model/ontology), [Case pool](/docs/object-model/case-pool), [Benchmarks](/docs/object-model/benchmarks), and [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology).
{% example-demo title="Coverage dimensions boundary" %}
Dimension: Source support level.
Ontology values: Explicit support, implied support, conflicting source, no source.
Interpretation: A compatibility benchmark can show whether failures concentrate in cases where the source does not explicitly support the claim.
{% /example-demo %}
## Source confidence
Code-backed: Dimension and classification types plus the Dimensions and Ontology list and detail routes establish Dimensions, ontology values, origin, examples, and Case classification. Whether a Dimension is meaningful or complete remains a coverage-design judgment.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology)
- [Dimensions and ontology](/docs/concepts/dimensions-and-ontology)
- [Ontology](/docs/object-model/ontology)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.ontology
title: Ontology
summary: Use ontology values to classify cases consistently within each coverage dimension.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/ontology
---
# Ontology
## Definition
Ontology values are the controlled labels used inside a coverage dimension. They keep case classification consistent enough for teams to plan coverage, compare benchmark segments, and explain gaps.
Use this reference when free-form tags are making coverage hard to read or when a benchmark segment needs stable vocabulary before results are trusted. Ontology values remain project-scoped foundations.
## Fields, states, or lifecycle rules
- Ontology values belong to dimensions; they should not drift into unrelated free-form tags.
- Values should be specific enough to separate meaningful behavior, but not so narrow that coverage becomes fragmented.
- Changing ontology values can change segment-level benchmark interpretation.
- Ontology should support coverage decisions, not replace expert judgment or policies.
- This page does not define a public taxonomy API.
## Related objects
Ontology should be read with [Coverage dimensions](/docs/object-model/coverage-dimensions), [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology), and [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema).
{% example-demo title="Ontology boundary" %}
Dimension: Source support level.
Ontology values: Explicit support, implied support, conflicting source, no source.
Interpretation: Segment results are easier to trust because the team knows which support condition each case represents.
{% /example-demo %}
## Source confidence
Code-backed: Dimension types and the Dimension settings, ontology-row, and proposal components establish ontology members as labels owned by a Dimension and used for classification. Vocabulary quality and cross-reviewer consistency require project review.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Dimensions and ontology](/docs/coverage-engineering/dimensions-ontology)
- [Coverage dimensions](/docs/object-model/coverage-dimensions)
- [Generate a dimension schema](/docs/coverage-engineering/generate-dimension-schema)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.outputs
title: Outputs
summary: Distinguish managed Run responses, imported output-only Runs, and Case-scoped reference outputs.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/outputs
---
# Outputs
Understand the outputs Teammately reviews, compares, and turns into correctness evidence.
## Definition
An output is response evidence associated with a Case, but its identity depends on the workflow that produced it. A managed Evaluation Run generates a response from a saved Harness Version. An imported output-only Run maps externally produced responses to immutable Case IDs. A Case may also expose a golden or reference response where the current surface supports it.
Do not collapse these into one generic “accepted output” lifecycle. Imported outputs are not Harness Versions, Run responses are not automatically reference answers, and reviewer comments are not outputs.
## Fields, states, or lifecycle rules
- A managed Run response is bound to the Run, saved Harness Version, Benchmark Version, and Case.
- An imported output-only Run joins external rows to immutable Case IDs and retains imported candidate metadata.
- Output-only Runs can be scored and inspected, but cannot be activated, optimized in Improve, or used as Harness columns in Compare or Arena.
- A golden or reference output is Case-scoped example evidence where exposed; it does not become a global Policy.
- Mapping errors can make evidence misleading even when the external response data is unchanged.
- This page documents object boundaries, not model execution, provider behavior, or public output schemas.
## Related objects
Read outputs with [Cases](/docs/object-model/cases), [Reference and golden outputs](/docs/object-model/reference-and-golden-outputs), [Run an evaluation](/docs/benchmark-evaluations/run-evaluation), and [Map external outputs](/docs/benchmark-evaluations/output-mapping).
{% example-demo title="Target model outputs boundary" %}
Case: A user asks whether a product is compatible with equipment they own.
Candidate output: The assistant says the product will work.
Interpretation: The benchmark failure belongs to the candidate output, but the reviewer still needs the case context and applicable rubric to explain why it failed.
{% /example-demo %}
## Source confidence
Code-backed: Benchmark Dataset output mapping and Evaluation Run detail distinguish imported output-only evidence from managed Run responses; review and Case-content sources preserve the Case boundary. Provider execution and customer serialization are outside this reference.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Run an evaluation](/docs/benchmark-evaluations/run-evaluation)
- [Map external outputs](/docs/benchmark-evaluations/output-mapping)
- [Connect model outputs](/docs/integrations/connect-model-outputs)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}
---
id: object-model.policies
title: Policies
summary: Define policies as reusable statements of what correct AI behavior requires.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/policies
---
# Policies
## Definition
Policies are reusable statements of what correct behavior requires. They preserve expert judgment as a governed standard that can later be tested by applicability logic and rubrics.
Use this reference when a team needs to know whether a rule is a draft suggestion, an approved correctness standard, or a standard whose version may affect benchmark evidence.
## Fields, states, or lifecycle rules
- A policy names the rule; a rubric tests the rule.
- A policy needs applicability logic before the team can know which cases it should judge.
- Policy approval state matters before benchmark use.
- Policy revisions can make older benchmark evidence stale or require comparison notes.
- This page does not define legal, compliance, retention, or external policy-management guarantees.
## Related objects
Policies should be read with [Applicability logic](/docs/object-model/applicability-logic), [Rubrics](/docs/object-model/rubrics), [Policy versions](/docs/governance/policy-versioning), and [Human Approval Boundaries](/docs/governance/human-approval-boundaries). Use [Create a policy](/docs/correctness-governance/policies-and-rubrics) for the operating workflow.
{% example-demo title="Policies boundary" %}
Expert judgment: Answers should not claim compatibility unless source data explicitly supports the claim.
Policy: Compatibility claims require explicit source support.
Interpretation: The policy explains the standard; linked applicability and rubrics decide when and how a specific output is judged.
{% /example-demo %}
## Source confidence
Code-backed: Policy types and list/detail routes expose identity, description, applicability, linked Cases and Rubrics, version facts, and approval state. The product object is a governed correctness standard, not an external legal-policy system.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Policies and Rubrics](/docs/correctness-governance/policies-and-rubrics)
- [Applicability logic](/docs/object-model/applicability-logic)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.reference-outputs
title: Reference and golden outputs
summary: Understand Case-scoped example responses without treating them as generic approval state or candidate behavior.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/reference-and-golden-outputs
---
# Reference and golden outputs
## Definition
A golden or reference output is a Case-scoped example response available on supported review, mapping, or Snapshot surfaces. It can help a reviewer understand intended behavior for that Case, but it is not a global Policy, a saved Harness Version, or a generic customer approval state.
Use this reference only where the product explicitly exposes a golden or reference response. Do not rename every expert answer, reviewed response, or imported candidate output as an “accepted output.”
## Fields, states, or lifecycle rules
- Keep the reference response attached to the Case and evidence context that justifies it.
- A candidate Run response does not become a reference response merely because it passed a Rubric.
- Reviewer comments and Contribution rationale remain attributable review input, not output state.
- A reference-output change can require a new Case, Snapshot, or Benchmark Version boundary when it affects governed interpretation.
- Policies and Rubrics remain the reusable correctness standard; a reference output illustrates one Case.
- This page does not define an export format or public output schema.
## Related objects
Read reference outputs with [Outputs](/docs/object-model/outputs), [Cases](/docs/object-model/cases), [Correctness Governance](/docs/correctness-governance), and [Expert Contributions](/docs/expert-contributions).
{% example-demo title="Case-scoped reference response" %}
A compatibility Case includes a reference response that says the available source does not confirm support and directs the user to an explicit compatibility check. The response illustrates desired behavior for that Case. The reusable requirement still belongs in the applicable Policy and Rubric.
{% /example-demo %}
## Source confidence
Code-backed: output mapping, Snapshot artifacts, linked Policy Cases, and Contribution review surfaces expose golden or reference responses as Case-scoped evidence with explicit review or overwrite actions. The sources do not establish a universal accepted-output lifecycle.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Map external outputs](/docs/benchmark-evaluations/output-mapping)
- [Review prepared Cases](/docs/coverage-management/case-review)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
{% /related-card-grid %}
---
id: object-model.conversation-inputs
title: Represent conversations in Cases
summary: Preserve multi-message context inside canonical Case input without inventing a separate Conversation Case object.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/represent-conversations-in-cases
---
# Represent conversations in Cases
## Definition
A canonical Case input can represent a multi-message interaction when correctness depends on prior turns, accumulated context, or state carried across an exchange. “Conversation” describes the shape of the Case input; it is not a separate first-class object or a public schema guarantee.
Use this reference when review or evaluation depends on exactly which messages were available before the response being judged.
## Fields, states, or lifecycle rules
- Preserve the message order and roles needed to interpret the Case.
- Identify the response being judged separately from prior user or assistant messages.
- Store supporting documents or structured material according to the Project Input Schema instead of flattening everything into message text.
- Removing, reordering, or summarizing material can change the Case and should follow the normal Case version boundary.
- Output mapping joins an external response to the Case ID; it does not repair missing conversational context.
- This page does not define a public conversation serialization contract.
## Related objects
Read this representation with [Cases](/docs/object-model/cases), [Outputs](/docs/object-model/outputs), [Project Input Schema](/docs/project-settings/input-schema), and [Metadata and Context](/docs/reference/metadata-and-context).
{% example-demo title="Plan context carried across turns" %}
A user first identifies an enterprise plan, then asks whether a feature is available. The later response can only be judged correctly if the earlier plan message remains part of the canonical Case input. If that message is removed, create a new Case version rather than trying to restore it during output mapping.
{% /example-demo %}
## Source confidence
Code-backed: Case-content services support structured project input and material fields, while record presentation and Case-view types preserve the context used for review. The sources support conversation-shaped Case input but not a distinct Conversation Case object or public import schema.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Configure the Project Input Schema](/docs/project-settings/input-schema)
- [Import and prepare Cases](/docs/operating-manual/import-and-prepare-cases)
- [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results)
- [Troubleshoot unclear Cases](/docs/troubleshooting/unclear-cases)
{% /related-card-grid %}
---
id: object-model.rubrics
title: Rubrics
summary: Turn policies into binary, reviewable checks that explain pass and fail evidence.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/rubrics
---
# Rubrics
## Definition
Rubrics turn policies into reviewable checks for a specific output. A good rubric makes pass, fail, or incomplete evidence explainable without replacing the policy it tests.
Use this reference when a benchmark failure needs to show exactly which condition a candidate output satisfied or violated.
## Fields, states, or lifecycle rules
- Rubrics should be tied to a policy and an applicability boundary.
- A binary rubric should be narrow enough to judge one output clearly.
- Linked Cases make the intended boundary concrete; they do not replace the Rubric's applicability rule.
- Overlapping rubrics can make failures hard to interpret.
- Rubric versions matter when pass/fail logic changes between benchmark runs.
- This page describes rubric semantics, not a public scoring API.
## Related objects
Rubrics should be read with [Policies](/docs/object-model/policies), [Applicability logic](/docs/object-model/applicability-logic), [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics), and [Overlapping rubrics](/docs/troubleshooting/overlapping-rubrics).
{% example-demo title="Rubrics boundary" %}
Policy: Compatibility claims require explicit source support.
Rubric: Pass only if the answer cites explicit compatibility support or states that compatibility is uncertain.
Interpretation: A candidate output that simply says "yes, it works" fails because the rubric names the missing evidence.
{% /example-demo %}
## Source confidence
Code-backed: Rubric types and list/detail routes expose criteria, Policy and Case links, applicability, and versioned evaluation context. The reference explains product semantics and does not promise a public scoring schema.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Writing Binary Rubrics](/docs/correctness-governance/binary-rubrics)
- [Create a rubric](/docs/correctness-governance/policies-and-rubrics)
- [Policies](/docs/object-model/policies)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.versions-staleness-resolution
title: Versions, staleness, and resolution
summary: Track how correctness objects evolve and how teams resolve conflicting evidence.
kind: reference
product_area: object_model
status: stable
updated: 2026-08-23
canonical: /docs/object-model/versions-staleness-and-resolution
---
# Versions, staleness, and resolution
## Definition
Versions, staleness, and resolution describe how correctness artifacts evolve without making old evidence ambiguous. Cases, Policies, Rubrics, Benchmark Versions, supported Case-scoped reference outputs, and customer-owned review context can change at different times; version boundaries explain which evidence belongs to which state.
Use this reference when a result changed unexpectedly, a policy was revised, a case was refreshed, or reviewers need to know whether older benchmark evidence still applies.
## Fields, states, or lifecycle rules
- Versions preserve what changed and what evidence was produced before the change.
- Staleness means older evidence may no longer reflect the current case, standard, coverage, or candidate boundary.
- Resolution work should name whether the fix belongs to a case, output, policy, rubric, coverage plan, benchmark version, or run metadata.
- Comparisons are weak when artifact versions are hidden.
- This page describes public object semantics, not retention, audit-log completeness, or compliance guarantees.
## Related objects
Versions, staleness, and resolution should be read with [Versioning and Staleness](/docs/governance/versioning-and-staleness), [Benchmark versioning](/docs/governance/benchmark-versioning), [Case versioning](/docs/governance/case-versioning), and [Policy Conflicts and Revisions](/docs/governance/conflict-resolution).
{% example-demo title="Versions, staleness, and resolution boundary" %}
State change: Reviewers revise a compatibility policy after finding unsupported-claim failures.
Benchmark evidence: Runs against the old policy remain interpretable, but they should not be summarized as current evidence without naming the old policy version.
Interpretation: The resolution note explains whether to rerun, revise the benchmark version, or preserve the old result as historical context.
{% /example-demo %}
## Source confidence
Code-backed: Benchmark, Policy, and Rubric types carry version facts; Benchmark Datasets → Snapshots and Policy activity preserve named historical boundaries. Cross-object staleness and conflict resolution are explicit review decisions rather than a universal automatic state.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Versioning and Staleness](/docs/governance/versioning-and-staleness)
- [Conflict Resolution](/docs/governance/conflict-resolution)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: object-model.workspaces-projects-systems
title: Workspaces, projects, and target systems
summary: Model organizational boundaries, product boundaries, and the AI system being governed.
kind: reference
product_area: object_model
status: stable
updated: 2026-09-07
canonical: /docs/object-model/workspaces-projects-and-target-systems
---
# Workspaces, projects, and target systems
## Definition
Workspaces, projects, and target systems define where organizational access, product-specific correctness work, and the AI behavior under evaluation are separated. A workspace groups people and administration; a project holds the cases, standards, coverage, and benchmark evidence for a specific target behavior.
Use this reference when a reader needs to know whether an artifact belongs to an organization boundary, a project boundary, or the target system being evaluated.
## Fields, states, or lifecycle rules
- Workspace boundaries should not be used to infer project-level correctness decisions.
- Project boundaries keep cases, policies, rubrics, coverage, benchmark versions, and review context tied to a specific target behavior.
- Target-system identity matters for benchmark-level run metadata and comparison interpretation.
- Cross-project reuse should not imply cross-project approval.
- This page does not make billing, tenancy, deployment, or compliance claims.
## Related objects
Workspaces, projects, and target systems should be read with [Product map](/docs/getting-oriented/product-map), [Product boundaries](/docs/introduction/product-boundaries), [Permissions](/docs/reference/permissions), and [Run metadata](/docs/benchmark-evaluations/run-metadata).
{% example-demo title="Workspaces, projects, and target systems boundary" %}
Scenario: The same company evaluates a support assistant and an internal policy-search assistant.
Project boundary: Each assistant has its own cases, policies, rubrics, benchmark versions, and benchmark-level run metadata.
Interpretation: A passing benchmark in the support project should not imply the policy-search assistant has approved evidence.
{% /example-demo %}
## Source confidence
Code-backed: Project and workspace types establish organization and Project identity; member settings expose Project participation; benchmark Run presentation identifies the evaluated target boundary. These sources do not establish billing, deployment, or tenancy guarantees.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Workspaces and projects](/docs/concepts/workspaces-projects)
- [Product map](/docs/getting-oriented/product-map)
- [Product boundaries](/docs/introduction/product-boundaries)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: project-settings.general
title: General Project Settings
summary: Manage the project name and Project Memo without confusing descriptive metadata with agent context.
kind: reference
product_area: project_settings
status: stable
updated: 2026-09-07
canonical: /docs/project-settings/general
---
# General Project Settings
## Definition
General settings contain the project **Name** and **Project Memo**. The name identifies the project in Teammately. The memo is project-level descriptive metadata for people working in the project.
> Project Memo is not agent context
>
> The Project Memo is not injected into prompts and does not replace Project Context, Reference Materials, or Review Screens. Put agent-facing operating context in Agent Setup.
Use the memo for a concise human-readable purpose, ownership note, or operating reminder. Avoid secrets and avoid relying on it for behavior that must be reproducible in a Harness or evaluation.
## Fields, states, or lifecycle rules
- Name is the editable project identifier shown to people.
- Project Memo is human-facing descriptive text and has no prompt-injection state.
- Saving either field changes current project metadata without versioning historical benchmark evidence.
## Choose the right surface
| Information | Put it in | Reason |
| --- | --- | --- |
| Human-facing project purpose or ownership note | Project Memo | Describes the project without affecting behavior |
| Domain and operating context for agents | Project Context | Enters the governed agent-facing setup |
| Contribution-specific review behavior | Expert Contribution | Keeps behavior visible in the scoped request |
| Source documents and connected knowledge | Reference Materials | Preserves source identity and indexing state |
| Candidate implementation or prompt logic | Harness | Gives the executable candidate an exact Version |
| Scoring behavior for future Benchmark Versions | Regime | Keeps the evaluation contract versioned and inspectable |
Changing the project name updates how people locate the project; it does not create a new Benchmark Version. Changing the memo likewise does not invalidate a Dataset Snapshot or alter historical Runs. If a memo change represents a real change in benchmark purpose, update the governed coverage, evaluator, and versioned evidence surfaces separately.
{% example-demo title="Example: memo versus context" %}
The memo says that the project is owned by the support automation team and covers policy-grounded replies. Project Context explains the product domain and operating constraints to Teammately agents. A grounding rule lives in the candidate Harness or governed policy, not in the memo.
{% /example-demo %}
{% related-card-grid title="Agent-facing configuration" %}
- [Configure Project Context](/docs/agent-setup/project-context)
- [Manage Harnesses](/docs/assets/harnesses)
{% /related-card-grid %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Configure Project Context](/docs/agent-setup/project-context)
{% /related-card-grid %}
## Source confidence
Code-backed: the active General settings route defines the editable name and Project Memo and explicitly distinguishes the memo from prompt context.
---
id: project-settings.input-schema
title: Project Input Schema
summary: Define the canonical input architecture, case-material fields, and accepted artifact formats for project cases.
kind: reference
product_area: project_settings
status: stable
updated: 2026-09-07
canonical: /docs/project-settings/input-schema
---
# Project Input Schema
## Definition
Project Input Schema is the project-managed contract for future cases. It declares the primary input architecture, optional structured-input schema, named case-material fields, and artifact families or file extensions the project accepts.
The active architectures are **plain text**, **chat**, and **structured**. The schema is a project singleton rather than a versioned benchmark object. When no setting exists, the default accepts one plain-text user message and no case materials or artifacts.
## Fields, states, or lifecycle rules
- `architecture` is `plain_text`, `chat`, or `structured`.
- Structured architecture requires a bounded `structuredInputSchema`.
- `caseMaterialSchema` is a closed, flat object. Each material key has a label, optional description, required flag, type, and any accepted artifact rules.
- Material keys use lowercase letters, digits, and underscores, start with a letter, and remain flat.
- `acceptedArtifacts` declares project-level artifact families. Supported families are image, document, tabular, presentation, source text, and audio.
- A case-material artifact rule must be a subset of the artifact families admitted at the project root.
- Saving a new schema governs future case validation. Operators should inspect existing cases before making a change that would make current content invalid.
## Canonical case content
The primary case payload is `content.input`. Optional supporting values and artifacts live in `content.case_materials`. The product renders `record_content.case_view` so people and execution adapters can inspect the canonical content consistently; that view is a projection rather than an alternate authoring contract.
Static support passed to a Harness uses `case_material_refs`. An executable or queryable environment uses an optional `world_instance_ref`. Do not collapse static documents, images, or values into the world boundary merely because a candidate consumes them during a Run.
{% example-demo title="Example: structured support case" %}
A project selects structured input with `question` and `customer_tier` properties. It declares a required `policy_document` case material that accepts PDF documents and an optional `account_history` tabular material. A case is valid only when its structured input matches the schema and the required document is present in the accepted format.
{% /example-demo %}
## Source confidence
Code-backed: the active settings route and backend validator define the input architectures, closed case-material schema, artifact families, defaults, and canonical case paths. This page explains the product contract without presenting internal handlers as a public API.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Project Settings](/docs/project-settings)
- [Product quickstart](/docs/quickstart)
- [Work with cases](/docs/assets/cases)
- [Design Review Screen](/docs/assets/review-screens)
{% /related-card-grid %}
---
id: project-settings.project-members
title: Project Members
summary: Grant or remove user and group access to a project while keeping task assignment and artifact governance separate.
kind: reference
product_area: project_settings
status: stable
updated: 2026-09-07
canonical: /docs/project-settings/project-members
---
# Project Members
## Definition
Project Members manages which workspace users and groups can access a project. A grant names a principal of type **USER** or **GROUP**; removing the grant removes that project-level access path.
Membership is not the same as selecting an expert for a Contribution, assigning a review task, or approving an artifact. Those workflows add scoped responsibility inside an already accessible project.
Before removing access, check whether the person or group owns active Contribution work or operational follow-up. Reassign that work first; this page does not define what historical records remain visible after access changes.
## Fields, states, or lifecycle rules
- A grant targets a user or group principal.
- Adding a grant creates a project access path.
- Removing a grant removes that current Project access path.
- Effective access can remain through another user, group, or workspace-level path.
## Manage access
1. Open **Project Settings → Project Members**.
2. Review existing principals and whether each grant belongs to a user or group.
3. Add the smallest appropriate principal. Prefer a managed group when access follows a durable team responsibility.
4. Before removing a grant, check active Contributions, reviews, and operational ownership that may need reassignment.
5. Confirm the resulting member list and test the intended product path through ordinary authorization.
Group access follows the group relationship managed by the workspace; it should not be duplicated as many individual grants without an operational reason. A user can also have more than one access path, so removing one grant may not remove all effective access.
> Access boundary
>
> Grant the smallest appropriate project scope. Do not copy credentials or sensitive material into project content as a substitute for access control.
Project access does not grant secret values automatically. Harness secret grants remain explicit. Contribution selection also remains separate: an expert needs the applicable project access path and a scoped Contribution before completing assigned work.
This page makes no retention-period or post-removal visibility guarantee. Historical attribution and record access are governed by the records and permissions the current product exposes at the time they are inspected.
{% related-card-grid title="Related governance" %}
- [Roles and permissions](/docs/governance/roles-and-permissions)
- [Reviewer and project access](/docs/governance/reviewer-and-project-access)
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
{% /related-card-grid %}
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Request an Expert Contribution](/docs/expert-contributions/request-contribution)
- [Complete an Expert Contribution](/docs/expert-contributions/complete-contribution)
{% /related-card-grid %}
## Source confidence
Code-backed: the active members and project-permissions routes expose user and group grants and removal. Workspace-wide role administration remains a separate governance surface.
---
id: project-settings.regime
title: Regime Settings
summary: Inspect and publish the project scoring Regime that governs future Benchmark Versions.
kind: reference
product_area: project_settings
status: stable
updated: 2026-09-07
canonical: /docs/project-settings/regime
---
# Regime Settings
## Definition
Regime Settings define how approved and applicable Rubrics contribute to Case and Benchmark results. The Regime is part of the project contract, but its published version is pinned into later Benchmark Versions so historical evaluation evidence remains interpretable.
The settings surface shows the Project's locked framework, its decision rule, what the framework is suited for, and which scoring controls administrators can configure. A framework is not a Policy or Rubric; it determines how those governed artifacts are interpreted and aggregated for evaluation.
## Fields, states, or lifecycle rules
- The framework is locked for an existing Project. Other framework choices can be available when creating a new Project.
- Depending on the framework contract, administrators can configure Case aggregation, score range, penalty policy, Rubric role assignment, safe custom formulas, and an Importance scale.
- A custom formula is validated and stored as typed configuration; it is not executable code.
- Editing creates unpublished scoring changes. **Publish new Version** creates an immutable Regime Version after validation and concurrency checks.
- New drafts and finalized Benchmark Versions retain the Regime Version they were created or finalized with. Publishing a later version does not rewrite existing plans, Runs, or results.
- Published version history shows prior configurations and can show which open drafts or Benchmark Versions use a version.
- A concurrent publication requires the administrator to review the latest version before publishing retained edits.
## Configure safely
Review the framework decision rule before changing scoring behavior. Confirm that the desired aggregation, score range, penalties, and Importance semantics match the Policies and Rubrics that will be evaluated. Publish only after the resulting version can be explained to someone reading a future Benchmark result.
Do not use Regime Settings to change the meaning of a Policy or Rubric. Change those governed artifacts through Correctness Governance, then create or evaluate the appropriate versioned benchmark evidence.
{% example-demo title="Example: preserving historical scoring" %}
An administrator changes the score range and publishes Regime Version 4. New Benchmark Versions pin Version 4, while an existing finalized Benchmark Version continues to report under Version 3. The administrator can inspect both configurations and their usage without treating the new scoring choice as a rewrite of old results.
{% /example-demo %}
## Source confidence
Code-backed: the Project Settings Regime panel exposes locked framework guidance, scoring configuration, validation, immutable publication, concurrent-publication recovery, and version history. The project API exposes framework discovery, current Regime reads, publication, history, and version usage.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Configure Project Settings](/docs/project-settings)
- [Build Policies and Rubrics](/docs/operating-manual/build-policies-and-rubrics)
- [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation)
{% /related-card-grid %}
---
id: reference.index
title: Reference library
summary: Look up Teammately objects, states, permissions, metadata, schemas, and source-confidence boundaries.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/reference
---
# Reference library
## Definition
The reference library is the lookup layer for object semantics, IDs, permissions, metadata, glossary terms, and source-confidence boundaries. Use it when a workflow depends on exact object meaning rather than the broader operating path.
This page is an index, not a schema contract. Use the stable references below for object meaning; draft schemas and internal implementation details do not become public contracts through this index.
## Fields, states, or lifecycle rules
- Object references explain identity, state, lifecycle, and relationship boundaries.
- Permission references explain user-facing access boundaries only where source-backed.
- Metadata and context references explain how records and cases remain reviewable.
- Source-confidence labels tell readers whether a claim is code-backed, doctrine-backed, docs-backed, or inferred.
- Draft schema pages are not public API or export contracts.
## Related objects
Use [Object model](/docs/object-model) for the main artifact graph, [Permissions](/docs/reference/permissions) for access boundaries, [IDs and keys](/docs/reference/ids) for identity references, and [Glossary](/docs/reference/glossary) for vocabulary.
{% example-demo title="Reference library boundary" %}
Question: A reader asks whether a benchmark run result can be treated as a public export schema.
Reference answer: The object references can explain benchmark, run, and result semantics. They do not create a public export contract unless a source-backed public schema page says so.
Next page: Use [Benchmarks](/docs/object-model/benchmarks), [Run metadata](/docs/benchmark-evaluations/run-metadata), or [Product boundaries](/docs/introduction/product-boundaries), depending on the claim.
{% /example-demo %}
## Source confidence
Docs-backed: the documentation manifest and taxonomy define the published reference groups and canonical page relationships. Each linked reference carries its own product-evidence boundary; this index does not raise a draft or internal schema to stable contract status.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Object model](/docs/object-model)
- [Permissions](/docs/reference/permissions)
- [Glossary](/docs/reference/glossary)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: reference.glossary
title: Glossary
summary: Definitions for current Teammately capabilities, product surfaces, artifacts, and lifecycle states.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/reference/glossary
---
# Glossary
Use these definitions when choosing a product surface, writing an operating instruction, or interpreting evaluation evidence. Capitalization identifies current product capabilities, surfaces, and named artifacts where it matters.
## Definition
The glossary fixes the meaning of current capability, surface, artifact, and lifecycle terms. Capability names describe what Teammately enables; product-surface names describe where people operate; artifact names identify the state that must remain traceable.
### Capabilities
- **Correctness Elicitation:** Turns specialist judgment into attributable standards and cases through structured dialogue, tasks, and checkpoints.
- **Coverage Engineering:** Represents the behavior space that a benchmark must cover through facets, stories, patterns, and concrete Cases.
- **Weave:** Builds case-grounded agent behavior from project knowledge, policies, Rubrics, tools, materials, and worlds.
- **Trialground:** Evaluates saved Harness versions against versioned benchmark boundaries and exposes inspectable evidence.
- **Coevolve:** Improves agents and benchmark understanding together by feeding contribution, coverage, and evaluation evidence into governed iteration.
## Fields, states, or lifecycle rules
### Project foundations
- **Project Agent Brief:** The published project-understanding document in Project Context.
- **Reference Materials:** The Agent Setup surface for connecting and indexing source-backed project knowledge.
- **Reference block:** An indexed unit of project knowledge with source and generation identity.
- **Comparison Direction:** Project Asset that guides meaningful candidate-output differences in comparative expert work; it is not a coverage facet or approved standard.
- **Review Screen:** Project Asset that configures the context and inputs presented to an expert.
- **Policy:** A governed statement of required or prohibited behavior, with applicability and provenance.
- **Rubric:** An executable judgment criterion, commonly binary, that tests applicable behavior against a Policy.
- **Coverage Facet:** Reusable project structure for representing behavior space, including Dimensions, Project Topics, and Case Construction Patterns.
- **Case:** A reusable behavior situation with canonical input, optional case materials, metadata, and an optional executable world reference.
- **Harness:** The agent configuration being built and evaluated. A mutable Draft can be saved as an immutable Harness version.
- **Project Input Schema:** The project-level architecture governing canonical Case input and material fields.
- **Benchmark Run Metadata:** Benchmark-level descriptive context requested when an Evaluation Run is created; it is not a project-level template or version identity.
### Benchmark workspace
- **Benchmark:** A named workspace for a particular correctness boundary and its datasets, coverage, contributions, evaluations, and improvement work.
- **Dataset snapshot:** A reproducible selection and representation of benchmark Cases.
- **Coverage Story:** Benchmark-scoped intent connecting a behavior risk or need to concrete coverage work.
- **Expert Contribution:** A benchmark-scoped request for specialist judgment, containing one or more Tasks and optional Checkpoints.
- **Task:** A bounded unit of work inside a Contribution, such as form input, chat, interview, or Case Review.
- **Checkpoint:** An explicit confirmation boundary inside a Contribution.
- **Contributed artifact:** A policy, Rubric, Case, or coverage observation produced by an expert while retaining Contribution provenance.
- **Benchmark version:** The fixed dataset and correctness boundary used for reproducible evaluation.
- **Evaluation Run:** One execution of a saved Harness Version against a Benchmark Version, with response, Rubric outcomes, settings, mapping, and metadata. Execution trajectories are not currently exposed.
- **Compare:** A symmetric Benchmark Evaluations matrix whose columns are saved Harness Versions and whose rows group governed evidence within one Benchmark Version.
- **Arena:** A comparative evaluation surface inside Benchmark Evaluations.
- **Improvement Session:** A benchmark-scoped process that explores candidate Harness changes against an explicit goal and pinned evidence.
- **Goal Contract:** The Improvement Session definition of target evidence, success criteria, and constraints.
- **Evaluation receipt:** Canonical evidence that a particular candidate was evaluated under a particular boundary.
- **Current frontier:** The set of retained candidates that currently represent the session's best supported tradeoffs.
## Decision checkpoint
| If you mean... | Use... | Do not substitute... |
| --- | --- | --- |
| Reusable source-backed project knowledge | Reference Materials / Reference block | An untracked attachment or the latest file without generation identity |
| Human specialist work for a benchmark | Expert Contribution | A generic approval queue |
| The agent state actually evaluated | Saved Harness version | The mutable Harness Draft |
| A fixed evaluation boundary | Benchmark version | A Run or a score |
| One execution and its evidence | Evaluation Run | The Benchmark itself |
| Goal-directed candidate exploration | Improvement Session | An unversioned list of suggestions |
{% example-demo title="Distinguish benchmark and run" %}
Benchmark Version 4 fixes the selected Cases and governed standards. Harness Version 9 is the candidate. The Evaluation Run is the one execution of Harness Version 9 against Benchmark Version 4. Compare can place Harness Version 9 beside other saved Harness Versions in a symmetric evidence matrix, while an Improvement Session can use failed Case evidence as a pinned target for new candidates.
{% /example-demo %}
## Source confidence
Docs-backed and code-aligned: the product doctrine defines the five capabilities, and current navigation establishes the product-surface names. Use the linked code-backed pages when an exact field, state transition, or route behavior matters.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Product map](/docs/getting-oriented/product-map)
- [Product loop](/docs/product-loop)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
## Related reference pages
{% related-card-grid title="Related reference pages" %}
- [Object model](/docs/object-model)
- [Key objects and relationships](/docs/getting-oriented/key-objects-and-relationships)
- [IDs and identity](/docs/reference/ids)
{% /related-card-grid %}
---
id: reference.ids
title: IDs and keys
summary: Understand the identifiers used across Teammately projects, records, benchmarks, runs, policies, and rubrics.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/reference/ids
---
# IDs and keys
## Definition
IDs and keys identify Teammately objects across projects, records, cases, benchmarks, runs, policies, rubrics, and administration surfaces. They matter because review context and benchmark evidence are only reproducible when the referenced objects are unambiguous.
Use this reference when a note, run, comparison, or troubleshooting path depends on matching the same project, case, standard, benchmark version, or candidate boundary.
## Fields, states, or lifecycle rules
- IDs should preserve which object a page, result, or review note is talking about.
- Keys and identifiers may appear in URLs, metadata, manifests, admin surfaces, or internal source refs.
- A stable ID does not imply that a public API, export format, or integration contract exists.
- Review notes should name object boundaries, not only human-readable titles, when reproducibility matters.
- Do not expose secrets or infer API-key behavior beyond source-backed admin pages.
## Related objects
IDs and keys should be read with [Workspaces and projects](/docs/concepts/workspaces-projects), [Assets](/docs/assets), [Run Metadata](/docs/benchmark-evaluations/run-metadata), and [Permissions](/docs/reference/permissions).
{% example-demo title="IDs and keys boundary" %}
Scenario: A comparison note says the candidate improved but does not identify the benchmark version or candidate run.
ID boundary: The note should identify the benchmark version, run, candidate metadata, and changed artifact if those references are available.
Interpretation: Without those identities, the comparison is weak evidence even if the score is visible.
{% /example-demo %}
## Source confidence
Code-backed: current Project, user, workspace, Benchmark, Policy, and Rubric types show distinct opaque identifiers for those product objects. The reference does not assert that every internal identifier is accepted by a customer-facing API or safe to construct.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Workspaces and projects](/docs/concepts/workspaces-projects)
- [Permissions](/docs/reference/permissions)
- [Assets](/docs/assets)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: reference.metadata-context
title: Metadata and context
summary: Use record context and metadata fields to make Teammately cases easier to filter, review, and analyze.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/reference/metadata-and-context
---
# Metadata and context
## Definition
Metadata and context are the supporting fields that make records and cases understandable after they move through import, review, coverage, and benchmark workflows. Context explains the situation; metadata helps filter, segment, trace, and compare it.
Use this page when a case is technically present but not useful because reviewers cannot see why it belongs in a behavior slice or what source information should be considered.
## Fields, states, or lifecycle rules
- Context should preserve the information a reviewer needs to judge the case.
- Metadata should support filtering, coverage planning, benchmark segmentation, and run interpretation.
- Metadata is not a substitute for output identity or source context.
- Dropped or inconsistent metadata can make coverage look balanced while hiding a missing behavior segment.
- This page does not define a public metadata schema or export format.
## Related objects
Metadata and context should be read with [Cases](/docs/assets/cases), [Importing cases](/docs/operating-manual/import-and-prepare-cases), [Case versions](/docs/governance/case-versioning), and [Coverage Engineering](/docs/coverage-engineering).
{% example-demo title="Metadata and context boundary" %}
Raw record: A support transcript includes a customer request, account tier, candidate answer, and later human correction.
Context: The account tier and correction explain why the answer was unsafe.
Metadata: The case can be segmented as exception request, enterprise tier, and human-corrected source.
Interpretation: If that context or metadata is dropped, review and coverage work can still run but the evidence becomes harder to trust.
{% /example-demo %}
## Source confidence
Code-backed: Cases and record presentation expose contextual fields to reviewers, while Case Pool column mapping and upload-queue types show how imported columns become usable record context. Field availability depends on Project configuration; this is not a universal metadata schema.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Cases](/docs/assets/cases)
- [Importing cases](/docs/operating-manual/import-and-prepare-cases)
- [Connect model outputs](/docs/integrations/connect-model-outputs)
- [Product quickstart](/docs/quickstart)
- [Task index](/docs/operating-manual/task-index)
{% /related-card-grid %}
---
id: reference.permissions
title: Permissions
summary: Understand the user-facing permission boundaries for projects, reviewers, settings, and expert UI access.
kind: reference
product_area: reference
status: stable
updated: 2026-09-07
canonical: /docs/reference/permissions
---
# Permissions
## Definition
Permissions describe the user-facing access boundaries that affect projects, reviewer work, settings, Expert UI access, and organization administration. Use this page to decide which access surface to inspect before diagnosing a blocked workflow.
This reference does not turn role labels into a complete public permission matrix. Exact permission behavior should stay tied to source-backed pages and the admin or project surfaces that expose it.
## Fields, states, or lifecycle rules
- Project access affects cases, standards, coverage, benchmark work, and project settings.
- Reviewer access affects assigned expert work and reviewer-facing surfaces.
- Organization administration affects members, groups, roles, security controls, API keys, and integrations.
- Role labels in docs should be treated as understandable operating labels, not as exhaustive permission contracts.
- Do not infer auth, security, compliance, tenant isolation, or billing guarantees from this reference.
## Related objects
Permissions should be read with [Admin Console](/docs/admin-console), [Workspaces and projects](/docs/concepts/workspaces-projects), [Permissions troubleshooting](/docs/troubleshooting/permissions), and [Reviewer and project access](/docs/governance/reviewer-and-project-access).
{% example-demo title="Permissions boundary" %}
Symptom: A reviewer can sign in but cannot complete assigned case review.
Permission boundary: The issue may be reviewer assignment, project access, Expert UI routing, or missing case context.
Interpretation: Diagnose access and assignment before changing cases, standards, or benchmark evidence.
{% /example-demo %}
## Source confidence
Code-backed: Project, user, and workspace types plus Project Permissions and Members surfaces support the user-facing boundaries described here. They do not form an exhaustive authorization matrix; assignment, organization administration, and object approval remain separate product states.
## Related task pages
{% related-card-grid title="Related task pages" %}
- [Admin Console](/docs/admin-console)
- [Workspaces and projects](/docs/concepts/workspaces-projects)
- [Permissions troubleshooting](/docs/troubleshooting/permissions)
- [Expert UI](/docs/integrations/reviewer-workspace)
- [Product quickstart](/docs/quickstart)
{% /related-card-grid %}