Submit External Outputs with the CLI
Prerequisites
Authenticate with write access and select the exact organization, Project, Benchmark and immutable Version. Your application produces the responses in its own runtime. A managed Harness is optional.
Prepare Cases and criteria
teammately eval version-context --kind summary teammately eval version-context --kind policies --limit 20 teammately eval version-context --kind rubrics --limit 20 teammately eval prepare --out ./evaluation-package
Read the generated manifest and result contract. Execute its frozen Cases with your application. Preserve Case, membership and record-version identities; access to a manifest does not grant access to protected materials.
Record actual responses
[
{
"case_id": "<issued Case ID from the manifest>",
"output": {"reply": "<actual application response>"}
}
]
Replace both placeholders with an actual manifest ID and its actual response. Each row has exactly one native JSON output or failure with code and message. Optional trajectory and measured latency/usage/cost remain absent when unknown. JSON null and text whitespace can be meaningful outputs.
Validate, submit and inspect
teammately eval validate-results \ --manifest ./evaluation-package/manifest.json --results ./results.json teammately eval submit-results \ --manifest ./evaluation-package/manifest.json --results ./results.json \ --name 'Application candidate' --receipt ./import-receipt.json
Missing Cases require explicit --allow-partial for both validation and submission. The CLI accepts result files up to 32 MiB and uses batches up to 10 MiB. Keep the manifest, results and receipt secure until recovery is no longer needed.
Use the returned child Run ID, not the group ID:
teammately run wait --run "$RUN_ID" --timeout 10m teammately eval results --run "$RUN_ID" --limit 20 teammately run compare --run "$BASELINE_RUN_ID" --target "$RUN_ID"
Comparison needs an actual baseline selected before inspecting candidate outcomes. Without one, inspect the first Trial's results without inventing a comparison. Page through results and preserve incompleteness/comparability reasons.
Recovery and corrections
Retry an interrupted write with the same original files and receipt. Changed content needs a new receipt and operation. Accepted responses are immutable within a Trial; a correction can create another Trial in the same external Run using --group, or an intentionally new Run.
If completion froze the responses but judging admission failed, use teammately eval complete --group "$GROUP_ID" --run "$RUN_ID" --retry, retaining the original partial policy. Admission success is not completed judging. These operations do not adopt code changes or activate a Harness.