Compare a candidate model with your current production baseline on the same prompts, test cases, and scoring rules before switching.
What to evaluate
Run the current production route first so the migration is judged against behavior users already rely on.
Include normal traffic, high-risk edge cases, format contracts, and failures previously observed in production.
Review failed outputs alongside provider-reported or estimated cost and latency when those measurements are available.
Document whether to switch, keep the current model, revise the prompt, or route only a narrow workload to the candidate.
PromptLens works best when each model is judged against the same dataset rows and pass criteria.
Freeze the prompt version, temperature, dataset, and scoring method. Run the current model route as the baseline, then run the candidate route against the exact same cases.
Start the review with cases that passed under the current model and fail under the candidate. A lower average cost does not justify a migration if the failures cluster around expensive customer or safety scenarios.
The outcome can be narrower than a full switch. A candidate may be good enough for classification while the current model remains necessary for complex support answers. Preserve that reasoning in the report.
Migration test plan
{
baseline_route: "current-production-model",
candidate_route: "candidate-model",
hold_constant: [
"prompt version",
"temperature",
"test cases",
"scoring method"
],
release_bar: {
critical_regressions: 0,
required_format_pass_rate: 100
}
}Compare the model outputs, score the failures, and share the decision record with the team.
Compare a candidate model with your current production baseline on the same prompts, test cases, and scoring rules before switching.
What to evaluate
Run the current production route first so the migration is judged against behavior users already rely on.
Include normal traffic, high-risk edge cases, format contracts, and failures previously observed in production.
Review failed outputs alongside provider-reported or estimated cost and latency when those measurements are available.
Document whether to switch, keep the current model, revise the prompt, or route only a narrow workload to the candidate.
PromptLens works best when each model is judged against the same dataset rows and pass criteria.
Freeze the prompt version, temperature, dataset, and scoring method. Run the current model route as the baseline, then run the candidate route against the exact same cases.
Start the review with cases that passed under the current model and fail under the candidate. A lower average cost does not justify a migration if the failures cluster around expensive customer or safety scenarios.
The outcome can be narrower than a full switch. A candidate may be good enough for classification while the current model remains necessary for complex support answers. Preserve that reasoning in the report.
Migration test plan
{
baseline_route: "current-production-model",
candidate_route: "candidate-model",
hold_constant: [
"prompt version",
"temperature",
"test cases",
"scoring method"
],
release_bar: {
critical_regressions: 0,
required_format_pass_rate: 100
}
}Compare the model outputs, score the failures, and share the decision record with the team.