# Teammately CLI

Generated: 2026-11-02T05:39:19.520Z
Source build: local
Canonical docs: https://teammately.ai/docs

<doc id="cli.overview" slug="cli" url="https://teammately.ai/docs/cli.md" title="Teammately CLI" updated="2026-10-31" status="stable">
---
id: cli.overview
title: Teammately CLI
summary: Use terminal commands and local MCP to inspect criteria, submit actual outputs and work with managed Harnesses.
kind: reference
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli
---

# Teammately CLI

## Definition

The `teammately` binary reads scoped benchmark context, packages local source and submits application outputs. It also starts a local read-only product MCP server. Application log ingestion belongs in the [SDKs](/docs/sdks).

## Start here

1. [Install from source](/docs/cli/installation) or use the binary supplied to your team.
2. [Authenticate and select scope](/docs/cli/authentication).
3. Choose [external output submission](/docs/cli/external-outputs), an optional [managed Harness](/docs/cli/managed-harness), or [coding-agent MCP setup](/docs/cli/mcp).

```sh
teammately --version
teammately login
teammately status
teammately eval version-context --kind summary
```

Configure the benchmark scope before the last command. The [command reference](/docs/cli/reference) lists environment variables, flags and exit codes.

## Fields, states, or lifecycle rules

Reads do not submit or activate anything. Write workflows retain local recovery receipts and operate on issued identities. A successful command can return a resource that is still processing or has failed operationally; inspect the returned state.

The CLI manages local files and submission workflows. Product MCP exposes reads. Docs MCP retrieves manuals. These interfaces do not exchange credentials through copied tasks or model-generated tool arguments.

## Related task pages

{% related-card-grid %}
- [Authentication](/docs/cli/authentication)
- [Submit external outputs](/docs/cli/external-outputs)
- [Managed Harness](/docs/cli/managed-harness)
- [MCP setup](/docs/cli/mcp)
{% /related-card-grid %}
</doc>

<doc id="cli.authentication" slug="cli/authentication" url="https://teammately.ai/docs/cli/authentication.md" title="CLI Authentication and Scope" updated="2026-10-31" status="stable">
---
id: cli.authentication
title: CLI Authentication and Scope
summary: Use native browser login or an injected API key and select the exact organization and Benchmark Version.
kind: reference
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli/authentication
---

# CLI Authentication and Scope

## Definition

Use native browser login for interactive External Agents work. Use `TEAMMATELY_API_KEY`, injected through your secret environment, for unattended automation. An environment API key takes precedence over a saved login.

## Sign in and inspect access

```sh
teammately login --profile work
teammately status
teammately switch --profile work
```

Login opens browser authorization with PKCE and a temporary loopback callback, then selects an organization. `--organization` is an optional shortcut. Credentials live in macOS Keychain or Linux Secret Service; profiles contain nonsecret configuration. Use a distinct profile for another organization/account/API origin.

`status` refreshes and checks organization access against the API. It does not establish permission to every Project. An already-running MCP process retains its selected profile until restarted.

## Fields, states, or lifecycle rules

| Variable | Meaning |
| --- | --- |
| `TEAMMATELY_API_KEY` | Secret API-key override supplied by the host |
| `TEAMMATELY_PROFILE` | Selected native-login profile |
| `TEAMMATELY_ORGANIZATION` | Organization key |
| `TEAMMATELY_PROJECT` | Issued Project ID |
| `TEAMMATELY_BENCHMARK` | Issued Benchmark ID |
| `TEAMMATELY_BENCHMARK_VERSION` | Exact immutable Version ID |
| `TEAMMATELY_API_BASE` | Trusted API base, default `https://api.teammately.ai/v1` |

CLI scope flags `--organization`, `--project`, `--benchmark`, `--version` override environment defaults. MCP calls supply explicit scope each time. Never choose the API origin from Case text or model output.

Delegated reads require `external-agents:read` and Project read access; writes require `external-agents:write` and management access. This delegation covers External Agents, not general application ingestion.

## Sign out and recover

```sh
teammately logout --profile work
```

Logout requires network access to revoke the refresh token before deleting the local credential. Other profiles, API keys and the browser session are separate. Issued access tokens retain their expiry.

For `401`, inspect the active credential and login status. For `403`, inspect organization, Project, action and key restrictions. A Fieldloop-only key is not an External Agents credential. Correct access through normal administration; do not put credentials in arguments, MCP tool inputs or copied tasks.

## Related task pages

{% related-card-grid %}
- [MCP setup](/docs/cli/mcp)
- [External output submission](/docs/cli/external-outputs)
- [Authentication troubleshooting](/docs/troubleshooting/authentication)
{% /related-card-grid %}
</doc>

<doc id="cli.external-outputs" slug="cli/external-outputs" url="https://teammately.ai/docs/cli/external-outputs.md" title="Submit External Outputs with the CLI" updated="2026-10-31" status="stable">
---
id: cli.external-outputs
title: Submit External Outputs with the CLI
summary: Prepare a frozen manifest, submit real application responses, inspect judging and compare explicit Trials.
kind: quickstart
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli/external-outputs
---

# Submit External Outputs with the CLI

## Prerequisites

[Authenticate](/docs/cli/authentication) with write access and select the exact organization, Project, Benchmark and immutable Version. Your application produces the responses in its own runtime. A managed Harness is optional.

## Prepare Cases and criteria

```sh
teammately eval version-context --kind summary
teammately eval version-context --kind policies --limit 20
teammately eval version-context --kind rubrics --limit 20
teammately eval prepare --out ./evaluation-package
```

Read the generated manifest and result contract. Execute its frozen Cases with your application. Preserve Case, membership and record-version identities; access to a manifest does not grant access to protected materials.

## Record actual responses

### Example: results.json shape

```json
[
  {
    "case_id": "<issued Case ID from the manifest>",
    "output": {"reply": "<actual application response>"}
  }
]
```

Replace both placeholders with an actual manifest ID and its actual response. Each row has exactly one native JSON `output` or `failure` with `code` and `message`. Optional trajectory and measured latency/usage/cost remain absent when unknown. JSON null and text whitespace can be meaningful outputs.

## Validate, submit and inspect

```sh
teammately eval validate-results \
  --manifest ./evaluation-package/manifest.json --results ./results.json
teammately eval submit-results \
  --manifest ./evaluation-package/manifest.json --results ./results.json \
  --name 'Application candidate' --receipt ./import-receipt.json
```

Missing Cases require explicit `--allow-partial` for both validation and submission. The CLI accepts result files up to 32 MiB and uses batches up to 10 MiB. Keep the manifest, results and receipt secure until recovery is no longer needed.

Use the returned child Run ID, not the group ID:

```sh
teammately run wait --run "$RUN_ID" --timeout 10m
teammately eval results --run "$RUN_ID" --limit 20
teammately run compare --run "$BASELINE_RUN_ID" --target "$RUN_ID"
```

Comparison needs an actual baseline selected before inspecting candidate outcomes. Without one, inspect the first Trial's results without inventing a comparison. Page through results and preserve incompleteness/comparability reasons.

## Recovery and corrections

Retry an interrupted write with the same original files and receipt. Changed content needs a new receipt and operation. Accepted responses are immutable within a Trial; a correction can create another Trial in the same external Run using `--group`, or an intentionally new Run.

If completion froze the responses but judging admission failed, use `teammately eval complete --group "$GROUP_ID" --run "$RUN_ID" --retry`, retaining the original partial policy. Admission success is not completed judging. These operations do not adopt code changes or activate a Harness.

## Related reference pages

{% related-card-grid %}
- [Command reference](/docs/cli/reference)
- [External outputs API](/docs/public-api/evaluation-outputs)
- [Output mapping troubleshooting](/docs/troubleshooting/output-mapping)
{% /related-card-grid %}
</doc>

<doc id="cli.installation" slug="cli/installation" url="https://teammately.ai/docs/cli/installation.md" title="Install the CLI from Source" updated="2026-10-31" status="stable">
---
id: cli.installation
title: Install the CLI from Source
summary: Build the terminal tool using the existing DevTools and Go SDK source-workspace workflow.
kind: quickstart
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli/installation
---

# Install the CLI from Source

## Prerequisites

Use macOS or Linux, Go 1.25.1+, and source checkouts of `devtools` and `teammately-go`. The source build uses the Go SDK module for transport and resources. Browser login additionally needs macOS Keychain or an unlocked Linux Secret Service with `secret-tool`.

## Build with the SDK checkout

From the `devtools` checkout, create a local Go workspace if it has none:

```sh
go work init . /path/to/teammately-go
make build
./bin/teammately --version
./bin/teammately --help
```

If a workspace already exists, use `go work use /path/to/teammately-go` instead of `init`. Keep this local workspace separate from published module dependencies. `make build` writes `bin/teammately`; a source build reports `dev` unless build metadata is supplied.

Put that binary on your terminal's PATH using your team's normal tool installation method. These guides use `teammately` to refer to the selected binary. Use `./bin/teammately` directly when working from the checkout.

## Verify and maintain

Run `teammately --version` and `teammately --help` to confirm the selected binary. Follow [Authentication](/docs/cli/authentication) for organization access; local help does not establish API access.

Update a source installation from the same source/build workflow. `teammately update` is for installer-managed releases and does not update source builds. Restart coding-agent hosts after replacing the binary so their MCP processes use the intended copy.

## Common failures

- Missing SDK module: check that the Go workspace contains the application and SDK checkouts.
- Wrong command version: inspect the binary selected by PATH.
- Linux login cannot store credentials: unlock/configure Secret Service; there is no plaintext-token fallback.
- A Windows environment: this CLI's receipt-locking and credential paths target macOS/Linux.

## Related reference pages

{% related-card-grid %}
- [CLI authentication](/docs/cli/authentication)
- [Command reference](/docs/cli/reference)
- [MCP setup](/docs/cli/mcp)
{% /related-card-grid %}
</doc>

<doc id="cli.managed-harness" slug="cli/managed-harness" url="https://teammately.ai/docs/cli/managed-harness.md" title="Submit a Managed Harness with the CLI" updated="2026-10-31" status="stable">
---
id: cli.managed-harness
title: Submit a Managed Harness with the CLI
summary: Select source files, submit a saved Harness, prepare its runtime and inspect every admitted execution.
kind: quickstart
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli/managed-harness
---

# Submit a Managed Harness with the CLI

## Prerequisites

Use authenticated write access to the selected Benchmark Version. Choose managed execution deliberately; [external output submission](/docs/cli/external-outputs) can keep execution in your own application runtime.

## Select source

Provide `main.py` with `generate_result(request)` for Python 3.12, and declare only selected files in `teammately-harness.json`:

### Example: Source selection manifest

```json
{"files":["main.py"],"dependency_specs":[]}
```

Read `teammately eval instructions --run "$BASELINE_RUN_ID"` when an actual baseline exists. Package at most 64 UTF-8 files, 256 KiB each and 1 MiB total. Symlinks, traversal, hidden/credential paths and cache/dependency directories are rejected. Server validation remains authoritative.

## Submit and prepare

```sh
teammately harness validate --source ./harness
teammately harness submit --source ./harness --name 'Application candidate' \
  --receipt ./source-receipt.json
teammately harness prepare-runtime \
  --harness "$HARNESS_ID" --harness-version "$HARNESS_VERSION_ID" \
  --receipt ./runtime-receipt.json
teammately eval managed-profile
```

Take Harness IDs from submission and inspect preparation state. Its acknowledgement does not certify readiness. Read the current server profile and retain its concurrency token.

## Request execution

```sh
teammately eval run \
  --harness "$HARNESS_ID" --harness-version "$HARNESS_VERSION_ID" \
  --profile-revision "$PROFILE_TOKEN" --receipt ./admission-receipt.json
```

`PROFILE_TOKEN` is the exact server-returned value, passed under the existing flag name. Do not invent it. Use a separate receipt for each stage. Managed policy can admit multiple executions; inspect every returned child Run ID with `run get`, `run wait` and `eval results`.

## Recovery and success

Replay an interrupted stage with the original inputs and its existing receipt. A changed source package is a different operation. Keep saved source, runtime readiness, admission and completed judging separate. Submission does not publish or activate the Harness.

## Related reference pages

{% related-card-grid %}
- [Managed execution API](/docs/public-api/managed-execution)
- [Command reference](/docs/cli/reference)
- [Benchmark Run troubleshooting](/docs/troubleshooting/benchmark-runs)
{% /related-card-grid %}
</doc>

<doc id="cli.mcp" slug="cli/mcp" url="https://teammately.ai/docs/cli/mcp.md" title="Connect a Coding Agent with MCP" updated="2026-10-31" status="stable">
---
id: cli.mcp
title: Connect a Coding Agent with MCP
summary: Configure the local read-only product MCP and distinguish it from CLI submissions and Docs MCP.
kind: quickstart
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli/mcp
---

# Connect a Coding Agent with MCP

## Prerequisites

Use the [CLI binary](/docs/cli/installation), an [authenticated profile](/docs/cli/authentication) and a supported coding-agent host. Product MCP reads Teammately data; Docs MCP retrieves manuals; the CLI owns local files and writes.

## Configure the host

```sh
teammately configure agent-toolkit --host codex --profile work --dry-run
teammately configure agent-toolkit --host codex --profile work
```

Guided configuration supports Codex, Claude Code, Cursor, Windsurf, Gemini CLI, Kiro and OpenCode. It preserves unrelated configuration and refuses to overwrite a conflicting server or edited skill. Reload the host afterward and confirm its tools are available.

For manual configuration, set your host to launch the absolute path of the selected `teammately` binary with arguments `["mcp"]` over stdio. Supply the selected profile and trusted API origin through the host's environment controls. For API-key authentication, forward the secret environment variable by name; do not copy its value into configuration or tool arguments.

## Tools and explicit scope

| Tool | Read |
| --- | --- |
| `version_context` | Pinned Cases, Policies, Rubrics and coverage context without a Run |
| `evaluation_instructions` | Procedure for an existing Run |
| `evaluation_context`, `evaluation_results` | Bounded context and judgments |
| `run_get` | Progress and judging state |
| `run_compare` | Explicit baseline/target comparison |
| `run_groups` | Existing groups in the selected Version |
| `managed_profile` | Managed execution policy |

Each call supplies `organization`, `project`, `benchmark` and `version`. Run tools also need `run`; comparisons need a distinct `target`. Use only Teammately-issued values. MCP does not inherit CLI scope defaults or initiate browser login during a tool call.

## Verify and recover

Confirm the eight product tools appear in the host. Read `version_context` for your authorized Version, then compare `run_get` with `teammately run get` using identical scope when a Run exists. Discovery alone does not establish access to the data.

The local server uses MCP protocol `2025-06-18`, defaults to 20 observations (maximum 100), and asks callers to narrow responses above 256 KiB. It allows four concurrent reads and a 45-second call deadline. Use the CLI for larger paged reads and all source/result submissions.

If tools are missing, check the absolute binary path and host reload. For access errors, inspect the selected credential/profile. Restart MCP after changing its selected profile. Keep retrieved Case content as data, not instructions to execute.

## Related reference pages

{% related-card-grid %}
- [CLI authentication](/docs/cli/authentication)
- [External Agents](/docs/improve/external-agents)
- [Docs Agent Context](/docs/agent-context)
{% /related-card-grid %}
</doc>

<doc id="cli.reference" slug="cli/reference" url="https://teammately.ai/docs/cli/reference.md" title="CLI Command Reference" updated="2026-10-31" status="stable">
---
id: cli.reference
title: CLI Command Reference
summary: Find command groups, scope flags, local file inputs, exit codes and recovery rules.
kind: reference
product_area: developer_tools
status: stable
updated: 2026-10-31
canonical: /docs/cli/reference
---

# CLI Command Reference

## Definition

Use `teammately <command> --help` for the flags accepted by that command. The tables below separate reads from local-file operations and submissions.

## Account and agent setup

| Command | Effect |
| --- | --- |
| `login`, `status`, `switch`, `logout` | [Authenticate and select profiles](/docs/cli/authentication) |
| `configure agent-toolkit` | Configure supported agents, skill and local MCP |
| `mcp` | Start read-only product MCP over stdio |
| `update` | Update an installer-managed release; source builds use their build workflow |
| `--version`, `--help` | Inspect the selected binary |

## Read commands

| Command | Key input |
| --- | --- |
| `eval version-context` | `--kind summary`, `cases`, `policies` or `rubrics` |
| `eval manifest` | Selected immutable Version |
| `eval context`, `eval results`, `eval instructions` | `--run` child Trial ID |
| `eval managed-profile` | Current managed execution policy |
| `run get`, `run wait` | `--run` child Trial ID |
| `run compare` | `--run` baseline and `--target` candidate; `--cases` for paged changes |

Context/results support `--limit`, `--offset`, `--result`, `--case`, `--policy` and `--case-rubric-pair`. Pages default to 20 observations, maximum 100. Read output defaults to JSON; `--format human` labels the returned data. Diagnostics use stderr.

## Local files and writes

| Command | Main flags |
| --- | --- |
| `eval prepare` | `--out` new directory |
| `eval validate-results` | `--manifest`, `--results`, optional `--allow-partial` |
| `eval submit-results` | Same files, `--name`, `--receipt`, optional `--group`/`--allow-partial` |
| `eval complete` | `--group`, `--run`, optional `--allow-partial`/`--retry` |
| `harness validate` | `--source` directory |
| `harness submit` | `--source`, `--name`, `--receipt` |
| `harness prepare-runtime` | `--harness`, `--harness-version`, `--receipt` |
| `eval run` | Saved Harness IDs, `--profile-revision`, `--receipt`, optional `--compute-profile` |

The `--profile-revision` flag carries the exact server-issued concurrency token from `eval managed-profile`. Do not choose a value or treat it as a customer-managed configuration lifecycle.

## Fields, states, or lifecycle rules

Global scope uses the [authentication guide's](/docs/cli/authentication) flags/environment. Request timeout defaults to two minutes and can be set with `--timeout` through ten minutes. A timeout stops waiting, not an admitted job.

| Exit code | Meaning |
| --- | --- |
| `0` | Command succeeded; inspect resource state for operational failure |
| `1` | Request, validation or file failure |
| `2` | Invalid command/configuration |
| `3` | Authorization failure |
| `4` | Timeout, cancellation or exhausted transient reads |

Source submission, runtime preparation, managed admission and external-output submission use local recovery receipts. After interruption, reuse original inputs and that stage's receipt. Changed inputs require a new logical operation and receipt. `eval complete` takes no local receipt flag; it uses the server's recorded completion and explicit failed-admission retry contract.

## Related task pages

{% related-card-grid %}
- [External output submission](/docs/cli/external-outputs)
- [Managed Harness](/docs/cli/managed-harness)
- [MCP setup](/docs/cli/mcp)
{% /related-card-grid %}
</doc>

<doc id="improve.work-evolve" slug="improve/external-agents" url="https://teammately.ai/docs/improve/external-agents.md" title="Work with External Agents" updated="2026-10-30" status="stable">
---
id: improve.work-evolve
title: Work with External Agents
summary: Copy a benchmark-scoped task for your coding agent to implement and evaluate in your real application repository.
kind: task
product_area: improve
status: stable
updated: 2026-10-30
canonical: /docs/improve/external-agents
---

# Work with External Agents

External Agents gives your coding agent a concrete improvement task grounded in the selected Benchmark Version. The agent works in your real repository; Teammately supplies the applicable Cases, Policies, Rubrics and recorded results.

## Prerequisites

- A selectable Benchmark Version.
- Access to the application repository and authorization for the intended changes.
- A coding agent connected through the setup supplied with the task.
- An optional reference Run that genuinely represents the application starting point.

## Steps

1. Open **Harness improvement → External Agents**.
2. Select the Benchmark Version. Changing it clears local criterion and reference-Run selections.
3. Focus on the whole Benchmark or choose up to 20 approved and applicable Policy and Rubric Versions.
4. Optionally select one individual reference Run. No Run is silently selected.
5. Choose the allowed improvement direction: supported practices, prompt only, model only, the same model family, or model configuration only.
6. Copy the task, including setup instructions when needed, into your coding-agent conversation.
7. Have the agent inspect the actual repository, retrieve the pinned criteria and Run context, implement the change and evaluate it.
8. Inspect resulting Runs through ordinary Benchmark Evaluations.

The copied task includes the Project Agent Brief when present. Credentials do not belong in that text.

## Execution choices

A Teammately Harness and source upload are optional. Customer-executed responses are the default path for the agent's real application. Managed Harness execution requires a deliberate choice.

The product data MCP supplies read-only context, instructions, results and comparisons. The CLI owns local files and submission operations. The documentation MCP only retrieves these manuals.

Start with [CLI authentication](/docs/cli/authentication) and [product MCP setup](/docs/cli/mcp), then follow [external output submission](/docs/cli/external-outputs) or the optional [managed Harness workflow](/docs/cli/managed-harness).

## Object and state changes

The composer keeps selections locally and copies text. It creates no hosted agent Session, transcript, durable handoff object, automatic code adoption or Harness activation. Later submitted responses or managed execution create their own ordinary Run records.

## Success criteria

- The task preserves the intended Benchmark Version and criteria.
- A selected reference Run is checked against actual code.
- Implementation and comparison use actual pre-change and post-change behavior.
- Accepted responses and candidate identity remain attributable.

## Common failure modes

- Stopping after tool setup or context retrieval.
- Assuming a Harness upload is mandatory.
- Treating copied text as an executed Session.
- Using a reference Run from unrelated code.
- Claiming private agent work before returned artifacts or results exist.
## Related reference pages

{% related-card-grid title="Related reference pages" %}
- [Harness Improvement](/docs/improve)
- [Import External Evaluation Outputs](/docs/benchmark-evaluations/output-mapping)
- [Evaluation Results](/docs/benchmark-evaluations/results)
- [Project Context](/docs/agent-setup/project-context)
{% /related-card-grid %}

## Related troubleshooting pages

{% related-card-grid title="Related troubleshooting pages" %}
- [Output mapping troubleshooting](/docs/troubleshooting/output-mapping)
- [Permissions troubleshooting](/docs/troubleshooting/permissions)
{% /related-card-grid %}
</doc>