> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Track results over time

> Run a fixed task set again after each change, or on a schedule, to catch regressions and drift from product, model, or dependency changes.

Keep a fixed benchmark running after product updates—or automatically on a schedule—to catch drops in pass rate and newly introduced frictions early.

```mermaid theme={null}
flowchart LR
  B["1. Baseline experiment<br/>(tagged tasks)"] --> R["2. Launch again or<br/>Schedule"] --> C["3. Compare Matrix<br/>pass rates"] --> N["4. Sort Frictions by<br/>First seen"] --> B
```

## Prerequisites

* A finished baseline experiment (for example, from [Measure a tool's impact](https://docs.oqoqo.ai/guides/measure-tool-lift.md) or [Compare agents and models](https://docs.oqoqo.ai/guides/compare-agents-and-models.md)).
* Runs in your [run balance](https://docs.oqoqo.ai/setup/billing.md) for recurring launches.

## Steps

**1. Freeze and tag your benchmark tasks**

Choose a stable task suite and avoid editing task prompts unnecessarily so trend comparisons stay apples-to-apples. [Tag](https://docs.oqoqo.ai/experiments/tags.md) them (for example, `regression`) for quick filtering.

**2. Run your baseline experiment**

[Launch](https://docs.oqoqo.ai/experiments/launch.md) the initial experiment to establish your reference pass rates and frictions.

**3. Launch again after product changes**

After shipping updates to your product, tools, or dependencies, open the baseline experiment and select **Launch again** (uses latest task and tool versions; select **Edit first** to swap models).

**4. Schedule recurring runs for drift**

To catch upstream model updates or external API changes automatically, select [**Schedule**](https://docs.oqoqo.ai/experiments/schedules.md), pick **Repeat**, time, and **Time zone**, review the **Uses** run forecast, and select **Create schedule**. For team-owned schedules, prefer shared organization providers and API-key headers over personal integrations so teammate edits do not pause the schedule for review.

**5. Compare Matrix pass rates across runs**

Open your schedule under **Automate → Schedules** to inspect its **Experiments** and **Runs** tabs, and compare **Matrix** pass rates against earlier runs.

**6. Sort frictions by First seen**

Open the schedule's **Frictions** tab (or **All frictions**) and sort by **First seen** to isolate newly introduced problems immediately.

## Get notified

Clean scheduled experiments send no completion alert. You receive in-app and email notices (controlled by **Experiment updates** under [**Profile → Notifications**](https://docs.oqoqo.ai/setup/notifications.md)) only when something fails or needs attention:

| Notification | Action |
| - | - |
| **Schedule paused** | Fix the listed blocker (missing provider, deleted resource, etc.) and select **Resume**. |
| **Schedule skipping launches** | Previous run is still active after 3 slots; choose a less frequent repeat. |
| **Schedule short of runs** | Top up your run balance before the next scheduled launch. |

## Interpret the results

| Observation | Diagnosis and next step |
| - | - |
| Pass rate drops across tasks right after a release | Product or tool regression—open frictions sorted by **First seen**. |
| Pass rate shifts with no code change on your side | Model or upstream dependency drift—check **Rate by agent** on new frictions. |
| **Treatment use** drops | Agents stopped discovering your tool—check skill descriptions, tool schemas, or agent release notes. |
| New **Machine** or **Task quality** frictions | Fix broken machine packages or test data before trusting the trend. |

## From the CLI and the plugin

| Step | CLI | Plugin (MCP) |
| - | - | - |
| Relaunch an experiment | [`oqoqo experiments rerun`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rerun) | [`rerun_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#rerun_experiment) |
| Preview or create a schedule | [`schedules preview`](https://docs.oqoqo.ai/cli/schedules#oqoqo-schedules-preview), [`create`](https://docs.oqoqo.ai/cli/schedules#oqoqo-schedules-create) | [`preview_schedule`](https://docs.oqoqo.ai/mcp-server/schedules#preview_schedule), [`manage_schedule`](https://docs.oqoqo.ai/mcp-server/schedules#manage_schedule) |
| List scheduled experiments | [`oqoqo experiments list --schedule-id`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-list) | [`list_experiments`](https://docs.oqoqo.ai/mcp-server/experiments#list_experiments) |
| Compare pass rates across scheduled runs | [`oqoqo results analyze summarize_scope --schedule-id`](https://docs.oqoqo.ai/cli/runs#oqoqo-results-analyze) | [`analyze_results`](https://docs.oqoqo.ai/mcp-server/runs-and-results#analyze_results) |
| Find newest frictions | [`oqoqo frictions list --schedule-id --sort new`](https://docs.oqoqo.ai/cli/frictions#oqoqo-frictions-list) | [`list_frictions`](https://docs.oqoqo.ai/mcp-server/frictions#list_frictions) |
| Read schedule alerts | [`oqoqo notifications list`](https://docs.oqoqo.ai/cli/billing-and-notifications#oqoqo-notifications-list) | [`list_notifications`](https://docs.oqoqo.ai/mcp-server/quota-and-notifications#list_notifications) |

```bash theme={null}
oqoqo schedules preview exp_8f2c41d0 --cron "0 9 * * 1-5" --time-zone America/New_York
oqoqo schedules create exp_8f2c41d0 --cron "0 9 * * 1-5" --time-zone America/New_York
```

To gate builds on pass-rate drops in CI (schedules only alert on run or Evals/Insights errors), trigger [`oqoqo experiments rerun`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rerun) from your pipeline ([Run an experiment in CI](https://docs.oqoqo.ai/operate/cli.md#run-an-experiment-in-ci); pass `--versions experiment` to keep original task versions).

## Next steps

> [card] [Schedules](https://docs.oqoqo.ai/experiments/schedules.md)
>
> Cadence, time zones, pause rules, and run forecasts.

> [card] [Find and fix frictions](https://docs.oqoqo.ai/guides/fix-frictions.md)
>
> Turn new frictions into verified fixes.

> [card] [Notifications](https://docs.oqoqo.ai/setup/notifications.md)
>
> Control email and in-app schedule alerts.

> [card] [Tags](https://docs.oqoqo.ai/experiments/tags.md)
>
> Tag regression tasks and experiments.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.