Choose the Decision Model Setup to Ship, Backed by Your Data
Each variant is a candidate for production: a pinned model version plus the wording of each question — the instructions, the criteria and the option names. Decision Lab tests the variants on your labeled data and compares their accuracy, stability and cost, so you can decide which one to adopt.
Available on Team and Enterprise plans
How Decision Lab Works
From labeled data to the variant you adopt in three steps
- 01
Build a Dataset
Collect conversation states with a ground-truth label for every question — choice, score or yes/no. Import the rows from CSV, and add more rows later.
- 02
Create Variants (Model + Wording)
Each variant pairs a pinned model version with the instructions, the criteria and the option names it reads. Variants used in an experiment are locked; copy one to try a change.
- 03
Run Single or Statistical Experiments
Compare the variants side by side. A single experiment reports accuracy with its expected range, a confusion matrix, thresholds and example rows; a statistical experiment repeats identical requests to measure stability. The estimated number of calls and cost are shown before you run.
What Your Experiments Tell You
One accuracy number is not enough to choose what to ship. Experiments answer the questions that matter before you adopt a variant.
Stability Under Repeated Requests
The same request is sent several times. See how often the answer stays the same, and which rows flip — the borderline cases whose verdict could change from day to day in production.
Non-Inferiority Against a Baseline You Choose
Pick a baseline variant and a margin, and check that a candidate is at most that much worse — so you can shorten instructions, rename options or move to a new model version with confidence.
Compare Adoption Candidates (Accuracy, Stability, Cost)
See accuracy with the range to expect on new data, stability and cost per 1,000 decisions for every variant side by side, with a recommended variant per question chosen by a fixed rule.
Cost and Latency per Variant
Longer wording costs more tokens and time. See, for each variant, how much the decision model costs per 1,000 decisions and how long it takes (median and slowest 5%).
Frequently Asked Questions
Ready to Improve Your LLM Prompts?
Start testing with statistical rigor today
Enterprise SSO • Data Isolation • Cloud-Native