# Candidates and the Current Frontier Generated: 2026-09-13T04:42:42.634Z Source build: local Canonical docs: https://teammately.ai/docs --- id: improve.candidates-frontier title: Candidates and the Current Frontier summary: Interpret candidate stages, canonical receipts, constraints, retained Candidate Systems, and the current frontier. kind: reference product_area: improve status: stable updated: 2026-09-07 canonical: /docs/improve/candidates-and-frontier --- # Candidates and the Current Frontier ## Definition An Improve candidate moves through `working`, `focused_evaluated`, `canonical_evaluated`, `retained`, `rejected`, or `incomparable`. Stage describes evidence maturity, not source quality or recency. ## Fields, states, or lifecycle rules Candidate stage, Harness Version, Benchmark Version, canonical Run, provenance, scores, constraint results, parent branch, and receipts form the durable evidence record. Frontier membership is a retained evidence decision rather than another candidate stage. ## Candidate System requirements A retained Candidate System identifies an exact saved Harness Version, immutable Benchmark Version, canonical Run, provenance, scores, and constraint results. Focused evaluation can route exploration, but only canonical evaluation against the pinned cohort supports frontier retention. A rejected candidate remains useful evidence when it reveals a regression or failed hypothesis. An incomparable candidate lacks the common evidence boundary needed for a valid decision. Do not fill that gap with aggregate intuition. Candidate Exploration evidence is scoped to the session and branch that produced it. It is not automatically the project's global best candidate. The current frontier contains retained candidates supported by the latest valid Goal and receipts, but “current” does not mean newest or highest aggregate score. ## Harness materialization boundary The active Improve model does not require a separate legacy “Save candidate as Harness” action. A candidate eligible for retention already references an immutable saved Harness Version. Use Assets → Harnesses to inspect that Version or select it directly in a future Benchmark Evaluation launch. If an external worker returns only a patch, prose, or private claim, the candidate is not yet a Candidate System. Materialize the return through the supported contract so the saved Version and evaluation request become observable. ## Retention checklist - Exact Harness and Benchmark Versions are present. - The canonical Run is complete enough for the measurement bindings. - Hard constraints pass; soft-constraint tradeoffs are visible. - Evaluation receipts and hashes resolve to the claimed Case cohort. - Parent branch and source provenance are recorded. - The candidate is retained for evidence, not merely because it is latest. {% example-demo title="Example: aggregate winner outside the frontier" %} A new candidate has the highest overall pass rate but violates a hard required-Policy constraint. Its canonical receipt remains visible, but the candidate is rejected and does not displace two lower-scoring retained candidates that satisfy the confirmed Goal. {% /example-demo %} ## Related task pages {% related-card-grid title="Related task pages" %} - [Start an Improvement Session](/docs/improve/start-improvement-session) - [Compare Harness Versions](/docs/benchmark-evaluations/compare) {% /related-card-grid %} ## Source confidence Code-backed: the active session contract, archive view, and coordinator evidence define candidate stages, exact identities, canonical receipts, retention, rejection, incomparability, and frontier semantics. --- id: improve.overview title: Improve summary: Coordinate durable Improvement Sessions from pinned benchmark evidence through Goal Contracts, candidates, evaluations, trajectories, and frontiers. kind: concept product_area: improve status: stable updated: 2026-09-07 canonical: /docs/improve --- # Improve Improve is the benchmark-scoped workspace for coordinating candidate work from benchmark evidence. Each Improvement Session pins the exact starting evidence it uses, including any Benchmark Version, Harness Version, and Run identities. Sessions preserve the goal, authorization, agent and worker activity, candidate Harness Versions, canonical evaluation receipts, and Current frontier. ## Definition A new session begins with **Start from evidence**. The operator selects the target and relevant benchmark evidence, then prepares a **Goal Contract**. The contract binds the free-form objective to canonical target identities, measurement bindings, constraints, unresolved items, and intervention boundaries. The current Improve workspace offers **Coevolve**, **External Agents**, and **External fine-tuning** experiences where enabled. Coevolve supports bounded Work and Evolve behavior; external experiences use explicit handoff or provider boundaries. All retained candidates still rely on observable Harness Versions and canonical Benchmark Evaluations rather than unrecorded claims of progress. ## Decision checkpoint | Need | Session choice | Required evidence | | --- | --- | --- | | Investigate or implement one bounded direction | Work | Confirmed Goal Contract and pinned failures or target evidence | | Explore several competing hypotheses | Evolve | Comparable measurement plan and explicit exploration authorization | | Delegate implementation externally | Worker package | Pinned target, return contract, and observable returned candidate | | Decide which candidates remain viable | Current frontier | Canonical evaluation receipts and goal constraints | | Stop scheduling new work temporarily | Pause | Retained evidence and chronology remain available | | Correct missing benchmark meaning | Contribution or coverage follow-up | Upstream policy, rubric, case, or coverage observation is named | ## Goal Contract and chronology The Goal Contract is the session's authority boundary. Confirm it only after the target and measurement plan are resolvable. Later revisions should remain explicit rather than silently changing what earlier candidates attempted. Chronology records durable session transitions, agent activity, proposed approaches, candidates, evaluations, pauses, resumptions, cancellation, and terminal state. Narrated trajectories expose bounded work segments and can identify unannounced durable activity without pretending to know private external-worker behavior. ## Candidates and current frontier A candidate should identify its Harness version, hypothesis, parent or branch context, and evaluation evidence. The current frontier represents retained candidates supported by the latest valid evidence and Goal Contract. It is not synonymous with the most recent candidate or the highest aggregate score. Evaluation receipts link candidates back to Benchmark Evaluations. If a candidate improves one objective but violates a constraint or regresses a must-level rubric, the frontier should keep that tradeoff visible. ## Human and external-worker boundary Operators can pause, resume, cancel, review, or redirect work through the available session controls. External worker packages can carry the confirmed goal and evidence to Codex, Claude Code, or another worker. Teammately records only returned artifacts and requests that satisfy the return contract; it does not infer private work state. {% example-demo title="Evolve a grounding harness" %} The Goal Contract targets unsupported blends of current and superseded sources without regressing missing-source uncertainty. Evolve explores retrieval filtering, source ranking, and response-construction branches. Each candidate Harness version is evaluated against the pinned benchmark. The current frontier retains the candidates supported by both grounding and uncertainty evidence, while a newly discovered policy conflict becomes an Expert Contribution opportunity. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Start an Improvement Session](/docs/improve/start-improvement-session) - [Write and confirm a Goal Contract](/docs/improve/goal-contracts) - [Choose Work or Evolve](/docs/improve/work-and-evolve) - [Interpret candidates and the current frontier](/docs/improve/candidates-and-frontier) - [Read chronology and trajectories](/docs/improve/chronology-and-trajectories) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) - [Request an Expert Contribution](/docs/expert-contributions/request-contribution) {% /related-card-grid %} ## Related reference pages {% related-card-grid title="Related reference pages" %} - [Coevolve](/docs/concepts/coevolve) - [Harnesses](/docs/assets/harnesses) - [Product boundaries](/docs/introduction/product-boundaries) {% /related-card-grid %} ## Source confidence Code-backed: the active versioned Improve workspace and session contract define starting evidence, Goal Contracts, Work and Evolve behavior, candidates, receipts, chronology, frontiers, and lifecycle controls. Internal coordinator implementation is supporting evidence, not a public API. --- id: improve.goal-contracts title: Goal Contracts summary: Bind an Improvement Session to exact targets, measurable objectives, constraints, and permitted intervention scope. kind: reference product_area: improve status: stable updated: 2026-09-07 canonical: /docs/improve/goal-contracts --- # Goal Contracts ## Definition A Goal Contract is the authority and measurement boundary for one Improvement Session. It turns a natural-language intent into canonical targets, prioritized objectives, protected constraints, and evaluable bindings. ## Fields, states, or lifecycle rules ### Contract contents The contract identifies exact target IDs and records intent. Objectives carry priorities. Constraints are hard or soft. Measurement bindings name whether evidence should **improve**, **preserve**, or **reduce** a canonical Case, Policy, Rubric, metric, or other admitted reference. Intervention constraints state what work may change. Unresolved items remain explicit until the contract can be confirmed safely. Every requested outcome needs an observable binding. “Make answers better” is not measurable; “improve the selected grounding Rubric while preserving the selected uncertainty Cases” is. A hard constraint disqualifies a candidate when violated. A soft constraint records a tradeoff that still requires review. ### Revisions and confirmation Goal Contract revisions can be proposed, confirmed, superseded, or rejected. Keep superseded and rejected revisions as history so candidate activity can be explained against the contract that authorized it. Do not silently edit the meaning of a session after work has begun. Evolve locks the Goal Contract after the first epoch begins. If new evidence reveals a fundamentally different goal, stop or complete the current session and create an explicit new boundary rather than retrofitting prior epochs. ### Confirmation checklist - Target Harness, Harness Version, Benchmark Version, and starting Run resolve to exact identities. - Each objective has a direction and canonical measurement binding. - Non-regression behavior is represented by preserve bindings or constraints. - Hard and soft constraints are distinguishable. - Intervention scope permits the intended code, prompt, retrieval, or configuration work. - No unresolved item makes evaluation or authorization ambiguous. > Authority boundary > > The Goal Contract authorizes session work; it does not change the Benchmark Version, approve a new correctness standard, or waive human governance of upstream artifacts. ## Related task pages {% related-card-grid title="Related task pages" %} - [Start an Improvement Session](/docs/improve/start-improvement-session) - [Choose Work or Evolve](/docs/improve/work-and-evolve) {% /related-card-grid %} ## Source confidence Code-backed: the active session contract and commands define canonical targets, objectives, constraints, measurement bindings, intervention scope, revision states, confirmation, and Evolve locking. --- id: benchmark-evaluations.overview title: Benchmark Evaluations summary: Run and inspect exact Harness Versions against an immutable Benchmark Version through Dashboard, List, Arena, and Compare. kind: concept product_area: benchmark_evaluations status: stable updated: 2026-09-13 canonical: /docs/benchmark-evaluations --- # Benchmark Evaluations Benchmark Evaluations is the version-scoped workspace for executing and comparing candidate systems. The active top-level tabs are **Dashboard**, **List**, **Arena**, and **Compare**. Every managed Run binds an exact saved Harness Version to the immutable Benchmark Version shown in the route. > Evaluation boundary > > Interpret evidence inside its recorded Benchmark Version, Harness Version, Run or Run Group, evaluator set, and metadata. Run counts belong to launches. Additional launches add evidence without rewriting earlier Runs. ## Surfaces and objects Dashboard summarizes progress, leaderboards, rank progression across Runs, and available resource telemetry. List is segmented into **Runs**, **Evaluation results**, and **Traces / Spans**. The results segment summarizes Case outcomes and Policy or Rubric failures. Arena compares candidate pairs across governed metrics. Compare is a symmetric matrix of Harness Versions across selected evidence rows. A Run Group can collect one standard attempt or repeated attempts. A Run records one candidate execution and its per-Case progress. Evaluation results record the admitted Policy and Rubric outcomes. Costs, tokens, and latency are telemetry only when the provider or execution path captured them. > Traces / Spans capability fence > > The List navigation exposes Traces / Spans, but the current benchmark API does not expose evaluation execution traces. Do not claim that trajectories, spans, private reasoning, or tool traces can be inspected from Benchmark Evaluations today. ## Decision checkpoint | Need | Open | Evidence to preserve | | --- | --- | --- | | Configure and launch managed Runs | Evaluation Settings and New evaluation run | Machine, saved Harness Versions, and per-Harness Run counts | | Start candidate execution | Run modal | Exact Harness and Benchmark Versions | | Inspect status and output summaries | List → Runs or Evaluation results | Run Group, attempt, Case counts, incomplete state | | Compare candidate pairs | Arena | Metric family, pair count, only-A, only-B, shared failures | | Compare many candidates by governed rows | Compare | Harness columns and chosen Case or facet row mode | | Admit external reference outputs | Output mapping | Case mapping, attempt assignment, insert/update report | ## Rankings and repeated sampling Dashboard aggregates compatible observed Runs for each saved Harness Version across launches. Average score weights Runs equally. Supported binary views report passed at least once or passed every time over the observed case outcomes. Counts and missing evidence are shown; unequal counts do not prevent comparison. Historical group metrics retain their recorded meanings. Ranking is a routing signal. A candidate can lead overall while failing required Policy or high-impact Rubric evidence. Use Arena or Compare to locate the disagreement and List to confirm completeness before starting Improve work. ## External outputs Uploaded or API-supplied reference outputs create output-only Runs that can be scored and inspected in List. They are not saved Harness Versions and therefore cannot be optimized in Improve or selected as Harness columns in Compare or Arena. {% example-demo title="Example: repeated evaluation without evidence drift" %} A team launches three Runs of Harness Version 8 and one Run of Version 11 against the same Benchmark Version. Both appear with their evidence counts. A later launch of Version 11 adds two Runs to its aggregate evidence without changing either launch group. The team can inspect individual Runs before deciding whether more evidence is useful. {% /example-demo %} ## Related workflows {% related-card-grid title="Related workflows" %} - [Configure evaluation execution](/docs/benchmark-evaluations/execution-settings) - [Run a benchmark evaluation](/docs/benchmark-evaluations/run-evaluation) - [Inspect evaluation results](/docs/benchmark-evaluations/inspect-results) - [Use Arena and rankings](/docs/benchmark-evaluations/arena-and-rankings) - [Compare Harness Versions](/docs/benchmark-evaluations/compare) - [Map external outputs](/docs/benchmark-evaluations/output-mapping) {% /related-card-grid %} ## Source confidence Code-backed: the active version-scoped workspace, settings, Run modal, List segments, Dashboard, Arena, and Compare routes define the current evaluation model and capability fences.