review evaluation results

Current authorized users and their OAuth clients may review outputs. Record user and OAuth client identity from the verified token. Do not claim a human reviewed agent-submitted decisions. Calibration accepts generated or corrected expectations; scored review marks output correct/incorrect.

POST/api/v1/organizations/{organizationId}/evals/{evaluationId}/reviews

Authorization

ManagementOAuth management

AuthorizationBearer <token>

User-based Clerk OAuth. CLI is a public Authorization Code client using S256 PKCE and browser loopback redirect. Hosted MCP uses resource-bound OAuth. Verify issuer, token type, intended audience/resource, scopes, live membership and current role. Do not forward arbitrary OAuth tokens into Convex session JWT auth. OAuth URLs below are placeholders for deployment configuration.

In: header

Scope: management

Path Parameters

organizationId*string

Permanent organizationId; selected organization must be authorized for the current user.

evaluationId*string

Permanent evaluationId; selected organization must be authorized for the current user.

Header Parameters

Idempotency-Key*string

Management writes: scoped to OAuth user+client+organization+operation. Store request digest and response for at least 24 hours. Same key/body replays original response, including after completion; different body fails idempotency_conflict. Exception: create_retrieval_key stores a non-secret receipt referencing the created key, never its secret. Same-key/body retries return 409 secret_unavailable with existing key metadata and do not create another key. Evaluation creation must not execute twice.

Length8 <= length <= 128

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Response Body

`application/json`

curl -X POST "https://example.com/api/v1/organizations/string/evals/string/reviews" \  -H "Idempotency-Key: stringst" \  -H "Content-Type: application/json" \  -d '{    "reviews": [      {        "resultId": "string",        "decision": "accept_generated"      }    ]  }'

{  "id": "string",  "promptId": "string",  "versionId": "string",  "version": 0,  "status": "pending",  "scope": "quick",  "totalCases": 0,  "datasetSize": 0,  "scoredCases": 0,  "pendingReviewCases": 0,  "passedCases": 0,  "failedCases": 0,  "passRate": 0,  "productionRunId": "string",  "createdAt": "2019-08-24T14:15:22Z",  "error": "string"}

POST/api/v1/organizations/{organizationId}/evals/{evaluationId}/reviews

Authorization

ManagementOAuth management

AuthorizationBearer <token>

In: header

Scope: management

Path Parameters

organizationId*string

Permanent organizationId; selected organization must be authorized for the current user.

evaluationId*string

Permanent evaluationId; selected organization must be authorized for the current user.

Header Parameters

Idempotency-Key*string

Length8 <= length <= 128

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

reviews*array<>

Items1 <= items <= 100

Response Body

`application/json`

curl -X POST "https://example.com/api/v1/organizations/string/evals/string/reviews" \  -H "Idempotency-Key: stringst" \  -H "Content-Type: application/json" \  -d '{    "reviews": [      {        "resultId": "string",        "decision": "accept_generated"      }    ]  }'

{  "id": "string",  "promptId": "string",  "versionId": "string",  "version": 0,  "status": "pending",  "scope": "quick",  "totalCases": 0,  "datasetSize": 0,  "scoredCases": 0,  "pendingReviewCases": 0,  "passedCases": 0,  "failedCases": 0,  "passRate": 0,  "productionRunId": "string",  "createdAt": "2019-08-24T14:15:22Z",  "error": "string"}

Account and organizations

Members and invitations

Prompts and drafts

Versions and labels

Datasets

Evaluations

Model access

Retrieval keys

Billing

Warnings and observability

200application/json

400application/json

401application/json

403application/json

404application/json

409application/json

422application/json

429application/json

503application/json

Account and organizations

Members and invitations

Prompts and drafts

Versions and labels

Datasets

Evaluations

Model access

Retrieval keys

Billing

Warnings and observability

review evaluation results

200application/json

400application/json

401application/json

403application/json

404application/json

409application/json

422application/json

429application/json

503application/json

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`

`application/json`