When to Use Prompt Optimization
- Improve prompt performance based on specific evaluation metrics
- Eliminate guesswork in prompt engineering
- Automatically generate better prompt versions from test data
Optimize a prompt
Navigate to a test run
Open your workspace and select a completed test run associated with a prompt.
Start prompt optimization
Click the
Optimize prompt button on the run page to open the optimization dialog.The button may be disabled if:
- The test run is still running or queued
- Required evaluators or models are not available
Configure optimization settings
Select your optimization preferences:
- Evaluators to prioritize: Choose which metrics to focus on during optimization
-
Optimization iterations: Set how many improvement cycles to run
Run the optimization
Submit your configuration to start the optimization process. The system will:
- Analyze your current prompt and test results
- Generate improved prompt versions using AI models
- Test new versions against your dataset
-
Iterate based on evaluator feedback
Review optimization results
Once complete, you’ll receive:
- Side-by-side comparison of original and optimized prompts
- Detailed reasoning for each change made
- Performance improvements across your chosen evaluators
-
Suggestions for accepting or modifying the optimized version
Accept or iterate
Review the optimized prompt and choose to:
- Accept: Creates a new prompt version and links it to your runs
-
Discard: Discard the optimized prompt and keep the original
