{"query":"Submit External Evaluation Outputs","corpusVersion":"local","generatedAt":"2026-11-02T06:34:53.715Z","results":[{"blockId":"public-api.evaluation-outputs#submit-external-evaluation-outputs","pageId":"public-api.evaluation-outputs","title":"Submit External Evaluation Outputs","pageTitle":"Submit External Evaluation Outputs","url":"https://teammately.ai/docs/public-api/evaluation-outputs.md","humanUrl":"https://teammately.ai/docs/public-api/evaluation-outputs#submit-external-evaluation-outputs","markdownUrl":"https://teammately.ai/docs/public-api/evaluation-outputs.md","sectionId":"submit-external-evaluation-outputs","kind":"api","productArea":"public_api","score":2818.2952923803173,"reasons":["search_match","title_match","term_match"],"markdown":"# Submit External Evaluation Outputs"},{"blockId":"cli.external-outputs#validate-submit-and-inspect","pageId":"cli.external-outputs","title":"Validate, submit and inspect","pageTitle":"Submit External Outputs with the CLI","url":"https://teammately.ai/docs/cli/external-outputs.md","humanUrl":"https://teammately.ai/docs/cli/external-outputs#validate-submit-and-inspect","markdownUrl":"https://teammately.ai/docs/cli/external-outputs.md","sectionId":"validate-submit-and-inspect","kind":"quickstart","productArea":"developer_tools","score":1285.8223394946372,"reasons":["search_match","term_match"],"markdown":"## Validate, submit and inspect\n\n```sh\nteammately eval validate-results \\\n  --manifest ./evaluation-package/manifest.json --results ./results.json\nteammately eval submit-results \\\n  --manifest ./evaluation-package/manifest.json --results ./results.json \\\n  --name 'Application candidate' --receipt ./import-receipt.json\n```\n\nMissing Cases require explicit `--allow-partial` for both validation and submission. The CLI accepts result files up to 32 MiB and uses batches up to 10 MiB. Keep the manifest, results and receipt secure until recovery is no longer needed.\n\nUse the returned child Run ID, not the group ID:\n\n```sh\nteammately run wait --run \"$RUN_ID\" --timeout 10m\nteammately eval results --run \"$RUN_ID\" --limit 20\nteammately run compare --run \"$BASELINE_RUN_ID\" --target \"$RUN_ID\"\n```\n\nComparison needs an actual baseline selected before inspecting candidate outcomes. Without one, inspect the first Trial's results without inventing a comparison. Page through results and preserve incompleteness/comparability reasons."},{"blockId":"cli.external-outputs#submit-external-outputs-with-the-cli","pageId":"cli.external-outputs","title":"Submit External Outputs with the CLI","pageTitle":"Submit External Outputs with the CLI","url":"https://teammately.ai/docs/cli/external-outputs.md","humanUrl":"https://teammately.ai/docs/cli/external-outputs#submit-external-outputs-with-the-cli","markdownUrl":"https://teammately.ai/docs/cli/external-outputs.md","sectionId":"submit-external-outputs-with-the-cli","kind":"quickstart","productArea":"developer_tools","score":1277.2307517555128,"reasons":["search_match","term_match"],"markdown":"# Submit External Outputs with the CLI"},{"blockId":"public-api.evaluation-outputs#definition","pageId":"public-api.evaluation-outputs","title":"Definition","pageTitle":"Submit External Evaluation Outputs","url":"https://teammately.ai/docs/public-api/evaluation-outputs.md","humanUrl":"https://teammately.ai/docs/public-api/evaluation-outputs#definition","markdownUrl":"https://teammately.ai/docs/public-api/evaluation-outputs.md","sectionId":"definition","kind":"api","productArea":"public_api","score":1242.3902008263433,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Definition\n\nUse the organization/project/Benchmark/Version prefix and retrieve `GET /case-manifest` for exact frozen Cases. Product Runs and Trials retain the API's existing `run-groups` and child `runs` wire identities."},{"blockId":"public-api.evaluation-outputs#boundaries","pageId":"public-api.evaluation-outputs","title":"Boundaries","pageTitle":"Submit External Evaluation Outputs","url":"https://teammately.ai/docs/public-api/evaluation-outputs.md","humanUrl":"https://teammately.ai/docs/public-api/evaluation-outputs#boundaries","markdownUrl":"https://teammately.ai/docs/public-api/evaluation-outputs.md","sectionId":"boundaries","kind":"api","productArea":"public_api","score":1220.534618798084,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Boundaries\n\nManaged and controlled-evaluation Groups reject externally submitted responses. Corrections require a new child Trial. The existing external Group can accept that new child through its registered route; a new Group is also an explicit option. Accepted execution failures remain unavailable response judgments rather than fabricated rubric failures."},{"blockId":"public-api.evaluation-outputs#request-sequence","pageId":"public-api.evaluation-outputs","title":"Request sequence","pageTitle":"Submit External Evaluation Outputs","url":"https://teammately.ai/docs/public-api/evaluation-outputs.md","humanUrl":"https://teammately.ai/docs/public-api/evaluation-outputs#request-sequence","markdownUrl":"https://teammately.ai/docs/public-api/evaluation-outputs.md","sectionId":"request-sequence","kind":"api","productArea":"public_api","score":1190.5027635210497,"reasons":["search_match","page_title_match","term_match","prefix_or_fuzzy_match"],"markdown":"## Request sequence\n\nConfigure `BENCHMARK_URL` with the [quickstart](/docs/public-api/quickstart). Prepare `first-results.json` using the body above, an actual manifest Case ID and its application response."},{"blockId":"benchmark-evaluations.output-mapping#import-external-evaluation-outputs","pageId":"benchmark-evaluations.output-mapping","title":"Import External Evaluation Outputs","pageTitle":"Import External Evaluation Outputs","url":"https://teammately.ai/docs/benchmark-evaluations/output-mapping.md","humanUrl":"https://teammately.ai/docs/benchmark-evaluations/output-mapping#import-external-evaluation-outputs","markdownUrl":"https://teammately.ai/docs/benchmark-evaluations/output-mapping.md","sectionId":"import-external-evaluation-outputs","kind":"task","productArea":"benchmark_evaluations","score":1149.241930656859,"reasons":["search_match","term_match","prefix_or_fuzzy_match"],"markdown":"# Import External Evaluation Outputs\n\nImport externally produced responses when execution belongs to your application. Teammately associates them with the Cases in the selected Benchmark Version and evaluates them against that Version's criteria."},{"blockId":"public-api.evaluation-outputs#example-read-the-manifest-and-submit-first-results","pageId":"public-api.evaluation-outputs","title":"Example: Read the manifest and submit first results","pageTitle":"Submit External Evaluation Outputs","url":"https://teammately.ai/docs/public-api/evaluation-outputs.md","humanUrl":"https://teammately.ai/docs/public-api/evaluation-outputs#example-read-the-manifest-and-submit-first-results","markdownUrl":"https://teammately.ai/docs/public-api/evaluation-outputs.md","sectionId":"example-read-the-manifest-and-submit-first-results","kind":"api","productArea":"public_api","score":1085.5871681201882,"reasons":["search_match","page_title_match","term_match"],"markdown":"### Example: Read the manifest and submit first results\n\n```sh\ncurl --fail-with-body \\\n  --header \"Authorization: Bearer ${TEAMMATELY_API_KEY}\" \\\n  \"${BENCHMARK_URL}/case-manifest\"\n\ncurl --fail-with-body \\\n  --header \"Authorization: Bearer ${TEAMMATELY_API_KEY}\" \\\n  --header 'Content-Type: application/json' \\\n  --header \"Idempotency-Key: ${RESULT_OPERATION_KEY}\" \\\n  --data-binary @first-results.json \\\n  \"${BENCHMARK_URL}/run-groups\"\n```\n\n```python\nimport os\nfrom pathlib import Path\nfrom urllib.parse import quote\nfrom urllib.request import HTTPRedirectHandler, Request, build_opener\n\nclass NoRedirect(HTTPRedirectHandler):\n    def redirect_request(self, req, fp, code, msg, headers, newurl):\n        return None\n\nopener = build_opener(NoRedirect())\napi_key = os.environ[\"TEAMMATELY_API_KEY\"]\n\nurl = os.environ[\"BENCHMARK_URL\"] + \"/case-manifest\"\nheaders = {\"Authorization\": \"Bearer \" + api_key}\nrequest = Request(url, headers=headers, method=\"GET\")\nwith opener.open(request) as response:\n    print(response.read().decode(\"utf-8\"))\n\nurl = os.environ[\"BENCHMARK_URL\"] + \"/run-groups\"\nheaders = {\"Authorization\": \"Bearer \" + api_key}\nheaders[\"Idempotency-Key\"] = os.environ[\"RESULT_OPERATION_KEY\"]\nheaders[\"Content-Type\"] = \"application/json\"\nbody = Path(\"first-results.json\").read_bytes()\nrequest = Request(url, data=body, headers=headers, method=\"POST\")\nwith opener.open(request) as response:\n    print(response.read().decode(\"utf-8\"))\n```\n\n```typescript\nimport { readFile } from \"node:fs/promises\";\n\nconst apiKey = process.env.TEAMMATELY_API_KEY!;\n\n{\n  const url = new URL(process.env.BENCHMARK_URL! + \"/case-manifest\");\n  const response = await fetch(url, {\n    method: \"GET\",\n    redirect: \"error\",\n    headers: {\n      Authorization: `Bearer ${apiKey}`,\n    },\n  });\n  if (!response.ok) throw new Error(`HTTP ${response.status}`);\n  console.log(await response.text());\n}\n\n{\n  const url = new URL(process.env.BENCHMARK_URL! + \"/run-groups\");\n  const response = await fetch(url, {\n    method: \"POST\",\n    redirect: \"error\",\n    headers: {\n      Authorization: `Bearer ${apiKey}`,\n      \"Idempotency-Key\": process.env.RESULT_OPERATION_KEY!,\n      \"Content-Type\": \"application/json\",\n    },\n    body: new Uint8Array(await readFile(\"first-results.json\")).buffer,\n  });\n  if (!response.ok) throw new Error(`HTTP ${response.status}`);\n  console.log(await response.text());\n}\n```\n\n```go\npackage main\n\nimport (\n    \"bytes\"\n    \"context\"\n    \"fmt\"\n    \"io\"\n    \"log\"\n    \"net/http\"\n    \"os\"\n)\n\nfunc main() {\n    if err := run(context.Background()); err != nil { log.Fatal(err) }\n}\n\nfunc run(ctx context.Context) error {\n    client := &http.Client{\n        CheckRedirect: func(req *http.Request, via []*http.Request) error {\n            return http.ErrUseLastResponse\n        },\n    }\n    {\n        target := os.Getenv(\"BENCHMARK_URL\") + \"/case-manifest\"\n        request, err := http.NewRequestWithContext(ctx, \"GET\", target, nil)\n        if err != nil { return err }\n        request.Header.Set(\"Authorization\", \"Bearer \" + os.Getenv(\"TEAMMATELY_API_KEY\"))\n        response, err := client.Do(request)\n        if err != nil { return err }\n        payloadBytes, readErr := io.ReadAll(response.Body)\n        response.Body.Close()\n        if readErr != nil { return readErr }\n        fmt.Println(string(payloadBytes))\n        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf(\"HTTP %d\", response.StatusCode) }\n    }\n    {\n        target := os.Getenv(\"BENCHMARK_URL\") + \"/run-groups\"\n        payload, err := os.ReadFile(\"first-results.json\")\n        if err != nil { return err }\n        request, err := http.NewRequestWithContext(ctx, \"POST\", target, bytes.NewReader(payload))\n        if err != nil { return err }\n        request.Header.Set(\"Authorization\", \"Bearer \" + os.Getenv(\"TEAMMATELY_API_KEY\"))\n        request.Header.Set(\"Idempotency-Key\", os.Getenv(\"RESULT_OPERATION_KEY\"))\n        request.Header.Set(\"Content-Type\", \"application/json\")\n        response, err := client.Do(request)\n        if err != nil { return err }\n        payloadBytes, readErr := io.ReadAll(response.Body)\n        response.Body.Close()\n        if readErr != nil { return readErr }\n        fmt.Println(string(payloadBytes))\n        if response.StatusCode < 200 || response.StatusCode >= 300 { return fmt.Errorf(\"HTTP %d\", response.StatusCode) }\n    }\n    return nil\n}\n```\n\nRetain Group and child Run IDs. Append additional batches under the child's `results` route with a key per logical batch. Recover a lost response with its original bytes and key.\n\nRead import state before completing. Send `{\"allow_partial\":false}` to `POST /run-groups/{groupId}/runs/{runId}/complete`; use `true` only for intentionally missing Cases. A failed evaluation-admission retry uses `retry:true` and retains the original partial policy."}]}