list evaluations
Uses the selected organization and the authenticated user’s current permissions. Applies the same setup and archive rules as the dashboard.
/api/v1/organizations/{organizationId}/evalsAuthorization
ManagementOAuth managementUser-based Clerk OAuth. CLI is a public Authorization Code client using S256 PKCE and browser loopback redirect. Hosted MCP uses resource-bound OAuth. Verify issuer, token type, intended audience/resource, scopes, live membership and current role. Do not forward arbitrary OAuth tokens into Convex session JWT auth. OAuth URLs below are placeholders for deployment configuration.
In: header
Scope: management
Path Parameters
Permanent organizationId; selected organization must be authorized for the current user.
Query Parameters
1 <= value <= 10025Value in
- "pending"
- "running"
- "paused"
- "completed"
- "failed"
- "cancelled"
Response Body
application/json
application/json
application/json
application/json
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/api/v1/organizations/string/evals"{ "items": [ { "id": "string", "promptId": "string", "versionId": "string", "version": 0, "status": "pending", "scope": "quick", "totalCases": 0, "datasetSize": 0, "scoredCases": 0, "pendingReviewCases": 0, "passedCases": 0, "failedCases": 0, "passRate": 0, "productionRunId": "string", "createdAt": "2019-08-24T14:15:22Z", "error": "string" } ], "nextCursor": "string"}initialize baseline POST
Freeze the setup draft and selected cases. On completion with all cases scored and no pending review or execution errors, atomically publish immutable version 1 to staging and production. Assertion failures do not block initialization. Retried setup cannot publish a different snapshot accidentally.
start evaluation POST
Pin current production at request acceptance and freeze the latest ordered dataset, saved version/model/settings, and scoring policy. Quick uses first 10 (all available on Free); comprehensive uses all (Pro only). Reuse most recent compatible completed production evidence by default, otherwise evaluate both sides in parallel. rerun_both forces fresh evidence. No model/settings overrides for saved versions. Start requests are idempotent to prevent duplicate charges.