Establish the initial baseline

The initial baseline establishes version 1 for a new prompt. It captures the starting point you will compare future changes against and assigns both staging and production to that version.

Prepare every case

Finish the prompt's messages, input mode, model route, and generation settings. Add a dataset that represents the task. In dashboard setup, configure at least one applicable evaluator for every case and provide references wherever a rule needs them.

For a guided example, use the five ticket-routing cases with an exact-match evaluator. For extraction or support replies, use their schema or judge lesson instead.

Run and inspect

Choose Run all test cases and wait for completion. Read each generated output and evaluator decision. A failed assertion is a completed judgment, so a baseline can finish with a less-than-perfect pass rate. An execution error is unfinished evaluation work and must be resolved before setup can complete.

If a model call fails, inspect provider access, credits, and the error explanation. If an evaluator fails to execute, inspect its configuration and input requirements. Do not rewrite expected outputs merely to hide a real prompt failure.

Once all required setup work completes, the first saved version and its initial label assignments appear. Inspect the score before using the newly assigned production label in a live application.

Initialize from another surface

The management API, CLI, and MCP accept a draft revision, scope, and scoringMethod for initialization. Their walkthroughs use expected, with a reference label for every routing input. Read the current draft revision before submitting and poll the returned evaluation ID until completion.

These fields are not a JSON representation of the dashboard's configured evaluator editor. Follow the CLI walkthrough or management API walkthrough for complete request sequences.