retry evaluation side
Retry only a failed/cancelled side with its exact frozen inputs and pinned production version. Keep completed counterpart evidence and charges. To change inputs, start a new comparison.
/api/v1/organizations/{organizationId}/evals/{evaluationId}/retryAuthorization
ManagementOAuth managementUser-based Clerk OAuth. CLI is a public Authorization Code client using S256 PKCE and browser loopback redirect. Hosted MCP uses resource-bound OAuth. Verify issuer, token type, intended audience/resource, scopes, live membership and current role. Do not forward arbitrary OAuth tokens into Convex session JWT auth. OAuth URLs below are placeholders for deployment configuration.
In: header
Scope: management
Path Parameters
Permanent organizationId; selected organization must be authorized for the current user.
Permanent evaluationId; selected organization must be authorized for the current user.
Header Parameters
Management writes: scoped to OAuth user+client+organization+operation. Store request digest and response for at least 24 hours. Same key/body replays original response, including after completion; different body fails idempotency_conflict. Exception: create_retrieval_key stores a non-secret receipt referencing the created key, never its secret. Same-key/body retries return 409 secret_unavailable with existing key metadata and do not create another key. Evaluation creation must not execute twice.
8 <= length <= 128Request Body
application/json
TypeScript Definitions
Use the request body type in TypeScript.
Response Body
application/json
application/json
application/json
application/json
application/json
application/json
application/json
application/json
application/json
curl -X POST "https://example.com/api/v1/organizations/string/evals/string/retry" \ -H "Idempotency-Key: stringst" \ -H "Content-Type: application/json" \ -d '{ "side": "candidate" }'{ "id": "string", "promptId": "string", "versionId": "string", "version": 0, "status": "pending", "scope": "quick", "totalCases": 0, "datasetSize": 0, "scoredCases": 0, "pendingReviewCases": 0, "passedCases": 0, "failedCases": 0, "passRate": 0, "productionRunId": "string", "createdAt": "2019-08-24T14:15:22Z", "error": "string"}resume evaluation POST
Explicitly resume unfinished work on a paused evaluation using its original frozen inputs. Requires writable organization access and current plan and usage eligibility. Completed cases are not repeated.
review evaluation results POST
Current authorized users and their OAuth clients may review outputs. Record user and OAuth client identity from the verified token. Do not claim a human reviewed agent-submitted decisions. Calibration accepts generated or corrected expectations; scored review marks output correct/incorrect.