Troubleshooting

Start with the symptom below. When contacting support, include the operation, approximate time, organization and resource IDs, and the request identifier if available. Keep credentials, private prompts, and customer case contents out of the report unless you have a suitable private channel and need to share them.

Prompt setup will not finish

Check that every dataset case has an applicable evaluator and that required references are present. A row-scoped evaluator may leave other cases uncovered. Run all setup cases and resolve generation or evaluator execution errors. Failed assertions can form a completed baseline; execution errors cannot. See initial baseline.

A model or judge cannot run

Inspect the selected provider route, the organization's connection status, and available credits. A model catalog entry is not proof of usable access. Each LLM evaluator needs its own explicit judge model and usable funding. A connected provider key takes precedence over credits for its route. See model access.

Dataset import fails

Check that a JSON file contains an array of cases and that each input is a nonempty string. JSON-formatted fields contain JSON-encoded strings, not nested objects directly in input. For TSV, use input and optional reference columns; use JSON for embedded tabs or newlines.

Inspect file size, row count, duplicates, and remaining dataset capacity. A revision conflict means the shared dataset changed after your read. Reload and review it before retrying. See import formats.

An evaluator gives an unexpected result

For exact matching, inspect whitespace and case settings; punctuation and explanatory text still matter. For JSON schema, inspect the raw output for Markdown fences, invalid JSON, missing fields, and disallowed values. A valid schema result does not prove factual accuracy.

For LLM judges, read the criteria and explanation, confirm the selected judge model, and compare the decision with your own review. Check whether scope includes the intended case and whether All or Any matches the requirement. See evaluator rules.

Results or publication look incomplete

Check scored cases and pending reviews alongside the pass rate. Finish human review and resolve execution errors before treating results as complete. Compare the evaluated dataset and scope with the current dataset.

A publication response with published: false has not moved the label. Read its warnings and decide whether to acknowledge that context. If the label revision changed, inspect the current assignment before resubmitting. See labels.

Prompt retrieval fails

Retrieve saved prompts from https://www.promptlens.io/api/v1/prompts/{promptId}. Archived prompts can still be retrieved with active keys. See retrieval and key rotation.

Login is unavailable

Retry the sign-in flow from the intended client and public PromptLens origin. If OAuth discovery returns 503, retain the request identifier and contact support. Do not work around discovery by choosing an unrelated issuer or pasting another client's token.

Unauthorized or forbidden

A 401 can indicate an absent, expired, revoked, or incorrectly targeted token. Sign in again using the correct client. A 403 can indicate missing scope, removed membership, an insufficient role, or a workspace access restriction. Ask an administrator to resolve the restriction.

API and MCP tokens target different resources, and retrieval keys authorize neither management surface. A local CLI uses credentials belonging to the OS account that signed in; another OS user may not be able to read them.

Credential store unavailable

On macOS, unlock Keychain. On Linux, make sure Secret Service and its session are available. On Windows, use the same Windows account and credential store. Storage does not automatically downgrade to plaintext. If you deliberately choose file storage, pass --credential-store file at login. See CLI setup.

Revision or idempotency conflict

Read current state and review your change before submitting a new write with the correct revision. Keep the same idempotency key and identical request when recovering an unknown outcome; use a new key for a new request.

A repeated key-creation request can return secret_unavailable: the key exists, but its original secret cannot be recovered. Revoke that key explicitly before creating a replacement.

Rate limited

Respect Retry-After. Switching between CLI and MCP does not increase your personal request budget. Distinguish management request limits from evaluation limits and provider capacity; each can require waiting or resolving a different cause.

Evaluation still running after disconnect

Accepted evaluations continue independently. Reconnect and inspect the run IDs or submit explicit cancellation. Ctrl+C in a CLI wait returns control to your terminal without cancelling the work. See cancel and retry.

MCP discovery fails

Check the /mcp endpoint, server enablement in the current conversation, and completion of OAuth. Reconnect if the grant expired or was revoked. Unsupported registration, a wrong token resource, or a proxy that strips required headers can prevent discovery. Follow your client's setup guide and include the request identifier when reporting a failure.

Organization or billing access is blocked

Confirm the selected organization, current membership, and role. A Free organization with more than two members is paused; an administrator can reduce membership or reactivate Pro. Purchasing credits does not remove that restriction.

After checkout, wait for confirmed payment to update the plan or credits. Check that you are viewing the checkout's organization before trying another purchase. Report a persistent mismatch with the payment reference and workspace ID. See billing and credits.