Prerequisites
- A finished baseline experiment (for example, from Measure a tool’s impact or Compare agents and models).
- Runs in your run balance for recurring launches.
Steps
1
Freeze and tag your benchmark tasks
Choose a stable task suite and avoid editing task prompts unnecessarily so trend comparisons stay apples-to-apples. Tag them (for example,
regression) for quick filtering.2
Run your baseline experiment
Launch the initial experiment to establish your reference pass rates and frictions.
3
Launch again after product changes
After shipping updates to your product, tools, or dependencies, open the baseline experiment and select Launch again (uses latest task and tool versions; select Edit first to swap models).
4
Schedule recurring runs for drift
To catch upstream model updates or external API changes automatically, select Schedule, pick Repeat, time, and Time zone, review the Uses run forecast, and select Create schedule. For team-owned schedules, prefer shared organization providers and API-key headers over personal integrations so teammate edits do not pause the schedule for review.
5
Compare Matrix pass rates across runs
Open your schedule under Automate → Schedules to inspect its Experiments and Runs tabs, and compare Matrix pass rates against earlier runs.
6
Sort frictions by First seen
Open the schedule’s Frictions tab (or All frictions) and sort by First seen to isolate newly introduced problems immediately.
Get notified
Clean scheduled experiments send no completion alert. You receive in-app and email notices (controlled by Experiment updates under Profile → Notifications) only when something fails or needs attention:Interpret the results
From the CLI and the plugin
oqoqo experiments rerun from your pipeline (Run an experiment in CI; pass --versions experiment to keep original task versions).
Next steps
Schedules
Cadence, time zones, pause rules, and run forecasts.
Find and fix frictions
Turn new frictions into verified fixes.
Notifications
Control email and in-app schedule alerts.
Tags
Tag regression tasks and experiments.

