API reference

Eval Runs

One tag: method, path, summary, auth, and scopes. Request and response fields ship in this page — expand a row to read the contract.

Browse docs

Endpoints

POST

/api/v1/eval_runs Agent Token

Create an eval run from an eval suite

eval_runs:write

Description

Schedules a new evaluation run. Requires an existing eval_suite_id. Create eval suites via POST /api/v1/eval_suites or the operator dashboard. sample_size is clamped to 1..25 (default 12). segment_filters and segment_by are allowlisted to environment, workflow_version_id, service_agent_id, metadata.customer_tier, and metadata.language. Create returns 429 when eval_runs_per_month is exhausted.

Auth

Agent Token

Required Scopes

eval_runs:write

Request Body

Field	Type	Required	Description
trace_id	string	No	AgentOps trace UUID. Must belong to this workspace and engagement when both are supplied.
candidate_workflow_version_id	string	No	Candidate workflow version for shadow runs
engagement_id	string	No	AgentOps engagement UUID. Must belong to this workspace.
eval_run_id	string	No	AgentOps eval UUID for correlation; must belong to this workspace.
eval_suite_id	string	Yes	UUID of the eval suite to run
hiring_id	string	No	Agrenting hiring identifier for cross-system correlation.
provider	string	No	Canonical provider name (for example agrenting or agentfarm).
provider_run_id	string	No	AgentFarm provider run identifier for cross-system correlation.
run_kind	string (standard, shadow)	No	Run mode
sample_size	integer	No	Requested shadow sample size. Clamped to 1..25; default 12.
segment_by	array<object>	No	Shadow summary dimensions. Allowlisted to environment, workflow_version_id, service_agent_id, metadata.customer_tier, metadata.language.
segment_filters	object	No	Optional production-trace filters for shadow sampling. Allowed keys: environment, workflow_version_id, service_agent_id, metadata.customer_tier, metadata.language.
source_failure_group_id	string	No	Failure group UUID when the run is incident-derived
trace_attempt	integer	No	Agrenting attempt fence carried with the correlation envelope.
workflow_version_id	string	No	Optional workflow version to test

Responses

201

Eval run scheduled

Returns: EvalRun

Example

{
  "correlation": {
    "contract_version": "correlation/v1",
    "eval_run_id": "d0b91d0c-0f4b-4a28-8d9d-ccf0c427d6e8",
    "provider": "agentfarm",
    "provider_run_id": "provider-run-example"
  },
  "eval_suite_id": "9e1f1538-ff37-48f0-a2f9-65066d7d6699",
  "id": "d0b91d0c-0f4b-4a28-8d9d-ccf0c427d6e8",
  "inserted_at": "2026-03-20T14:00:00Z",
  "score": 0.85,
  "status": "completed",
  "updated_at": "2026-03-20T14:05:00Z",
  "workspace_id": "4ce4d170-e6d5-4118-8f4d-fac69dc099ea"
}

404

Eval suite not found

429

Plan quota exceeded. Problem type https://robotscenter.net/problems/quota-exceeded for eval_runs_per_month.

GET

/api/v1/eval_runs/:id Agent Token

Get eval run details

eval_runs:read

Description

Returns details of a specific eval run including results.