Skip to main content
The Compare Experiments feature helps you identify meaningful improvements in performance across experiments so you can decide which experiment to move forward to. This enables:
  • Faster iteration : Quickly spot where performance diverges without manual guesswork.
  • Evidence-based decisions: Confirm if improvements are real, significant, or just noise.
  • Understand trade-offs: See if gains come with costs to balance accuracy, speed, & token usage.

​
1. Select Experiments to Compare

Select the experiments you want to analyze and click Compare Experiments to view runs side by side.
  • See outputs, evaluator results, and metadata together for direct comparison.
  • Choose only the columns that matter most and hide the rest.
  • Use Table View for detailed text results or Charting View to visualize evaluator outputs across runs.

​
2. Enable Diff Mode

Turn on Diff Mode to quickly see where runs improved or regressed against a baseline.
  • Select a baseline experiment to measure other runs against.
  • Differences in evaluator results are highlighted, making improvements and failures easy to spot.
  • Aggregated metrics at the top show how each evaluation measure changed relative to the baseline.
This makes it faster to identify meaningful changes without scanning through every result.

​
3. Enable Diff Output Mode

Enable Diff Output Mode to visually compare the textual outputs of experiments.
  • Highlights insertions, deletions, and changes compared to a baseline experiment.
  • Makes it easy to see how model responses evolve across runs.
  • Useful for spotting subtle but important output differences that metrics alone may miss.
This helps you understand not just whether results changed, but exactly how they changed.