Skip to main content
When modifying a prompt in the playground, you can test your new prompt across a dataset of examples to validate that the model is hill climbing in terms of performance across challenging examples, without regressing on core business use cases.

​
Step 1: Set a Dataset

  1. Follow this guide to upload your dataset to Arize AX Create a dataset
  2. Go back to the prompt playground, and choose your dataset from the Select a Dataset dropdown

​
Step 2: Set your Prompt

  1. Load your prompt from the Prompt Hub, using the Select a template from prompt hub dropdown
  2. OR, fill in a new prompt (See more: Create a Prompt)
  3. Include variables from your dataset in the prompt, inside curly braces
Set prompt with variables from dataset

​
Step 3: Add Evaluators

Select Add Evaluator to add evaluators to evaluate outputs generated by this experiment.

​
Add a Code Eval

Write a programmatic evaluator if you’d like to use code to judge your experiment outputs. Learn more here: Code Evaluations

​
Add an LLM as Judge Evaluator

Use an LLM to judge your experiment outputs. Learn more: LLM as a Judge
  1. Select one our Arize eval templates
  2. OR write your own. Make sure to embed variabes from the dataset, so that the evaluator has something to evaluate
  3. Set your eval labels. These are the labels the evaluator will pick from, when judging the output
  4. Set explanations on/off. Explanations are short reasoning blobs that the LLM will generate to explain its reasoning. Also set advanced options.
  5. Click Create Eval once you are done.
Add evaluators to experiment

​
Step 4: Run Experiment

Once you hit Run, the experiment run will trigger. Hit View Experiment to get a detailed view of your experiment run.
View experiment results
Hover over the eval label to see the eval explanation.