Your first evaluation

Build a prompt that routes a support ticket to billing, technical, or account. These fixed labels make exact matching a useful first evaluator: a reply such as “This is a billing issue” should fail even when its meaning is correct.

Before you start

Sign in, select an organization, and make sure a model route is available through a connected provider or PromptLens Credits. Running the example uses model calls and may incur charges. All tickets below are fictional.

1. Create the prompt

Create a prompt named Ticket router. Select an available model, set temperature to 0, and choose User message as the input mode. Add this system message:

Classify the support ticket into exactly one team:
billing: invoices, charges, refunds, and payment failures.
technical: errors, broken features, and performance problems.
account: signing in, passwords, and profile changes.
Return only billing, technical, or account. No punctuation or explanation.

Each dataset input becomes the final user message. Keep the model route and settings unchanged while establishing your baseline.

Screenshot placeholder — Ticket router editor Show the system message, selected model, temperature, and User message input mode.

2. Add five test cases

Add these rows to the dataset, or import ticket-routing.json. Use Text for both input and reference.

These cases establish a small starting set. Later, add ambiguous tickets and examples of actual failures. The reference is the answer you want, not a model-generated suggestion to accept without review.

3. Configure exact matching

Add an Exact match evaluator named Correct team and apply it to all cases. Enable trimming whitespace and leave ignoring case disabled. This accepts billing with a trailing newline but rejects Billing, billing., and Team: billing.

If you create the evaluator from an individual case, check its scope: a case-specific evaluator must be expanded to cover the other rows. Every setup case needs an applicable evaluator.

Screenshot placeholder — Exact-match settings Show Correct team, whitespace trimming, case sensitivity, and all five cases in scope.

4. Run the baseline

Choose Run all test cases and wait for every case to finish. Inspect the actual output and evaluator result for each row. A failed assertion means the generated answer did not meet the rule; an execution error means the call or evaluation did not finish successfully.

A completed baseline saves version 1 and assigns both staging and production to it. A baseline may contain failed assertions, so inspect the results before using it in an application. If a call errors, resolve the cause and rerun the affected work; an execution error does not establish a completed baseline.

Screenshot placeholder — Initial baseline complete Show actual routing labels, per-case pass/fail results, and the saved version 1 with its initial labels.

5. Make the next improvement

Do not change a reference simply to make a result pass. Decide whether the instructions, expected label, or test coverage needs improvement. Continue with Take your prompt to production to test an ambiguous ticket and publish a later version.

For a different kind of output, use the bug extraction lesson or support-reply judges. Each teaches a check suited to that task.