> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Launch an experiment

> Choose the tasks, treatments, agents, and settings for an experiment, then launch it, rename it, launch it again, or cancel it.

Select tasks, treatments, agents, and trial counts, then launch your experiment. Each launch draws runs from your organization's run balance ([Experiments](https://docs.oqoqo.ai/experiments/overview.md)).

## Prerequisites

* At least one [task](https://docs.oqoqo.ai/experiments/tasks.md) (with a rubric if you want Evals pass/fail verdicts).
* At least one [treatment](https://docs.oqoqo.ai/experiments/treatments.md) (every project includes **Raw Agent**).
* Connected [model providers](https://docs.oqoqo.ai/setup/model-providers.md), [Oqoqo AI feature models](https://docs.oqoqo.ai/setup/oqoqo-ai.md), and at least one enabled [agent](https://docs.oqoqo.ai/setup/agents.md).
* Available runs in your organization's [run balance](https://docs.oqoqo.ai/setup/billing.md).

## Start a new experiment

Select **New experiment** in the sidebar (or start from a [Home template](https://docs.oqoqo.ai/experiments/templates.md)) and enter an **Experiment name** (up to 120 characters). Because tasks carry their own files and machines and treatments carry their own tools, you only choose what to combine in the **Configuration** panel.

![New experiment page with the selected tasks and the Configuration panel: Agents, Treatments (Toolsets), Trials, Max Concurrency, Evals, and Insights.](https://mintcdn.com/oqoqo/YO1ysyUx4SPwBwoz/images/screenshots/launch-config-light.png?fit=max\&auto=format\&n=YO1ysyUx4SPwBwoz\&q=85\&s=d54482136e6a4f36493351c50db3b6b7)

## Choose tasks

Select tasks from the list, search by name, or filter with [**Tags**](https://docs.oqoqo.ai/experiments/tags.md) to grab an entire release or feature suite at once. Select **New task** to create one inline.

Each task uses its current version by default, or you can pick an older version on its row. For statistical comparisons, include more than 6 tasks—[Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md) needs at least 6 differing tasks, and tied tasks do not count.

## Choose agents

By default, **New experiment** selects every built-in agent with a recommended model. Select **Clear** to pick specific agents, models, and effort levels. To compare two models or effort levels on the same agent, add the agent again with the second choice.

[Custom agents](https://docs.oqoqo.ai/setup/custom-agents.md) (private beta) start unselected unless you open **New experiment** from a custom agent's page; launching an untested custom agent displays a reminder notice.

## Choose treatments and a baseline

Select one or more treatments under **Treatments (Toolsets)** (**Raw Agent** starts selected). When you select two or more treatments, a **Baseline** selector appears—keep **Raw Agent** or choose any selected treatment as your reference.

Oqoqo stores your chosen baseline with the experiment so you can measure each treatment's [lift](https://docs.oqoqo.ai/experiments/overview.md#baseline-and-lift) in [Read results](https://docs.oqoqo.ai/evaluation/results.md#pass-rate-and-lift).

## Set trials

Set **Trials** between `1` and `20`. Use **2 or more trials** whenever you compare treatments or agents so single-run variance does not skew results.

The footer calculates total runs (`tasks × treatments × agents × trials`, such as **Uses 12 runs**) and links to [**Usage & billing**](https://docs.oqoqo.ai/setup/billing.md) if your balance is short.

## Set Max Concurrency

**Max Concurrency** caps how many agent runs execute in parallel (defaulting to the experiment's run count). Lower it—down to `1` for strictly sequential runs—when tasks share mutable remote state through an MCP server, CLI, or shared test account, or to avoid provider rate-limit spikes.

Max Concurrency does not throttle Evals or Insights. Organization and platform capacity limits can still reduce effective concurrency; email [support@oqoqo.ai](mailto:support@oqoqo.ai) if you need higher capacity.

## Turn on Evals and Insights

| Control | What it does |
| - | - |
| [**Evals**](https://docs.oqoqo.ai/evaluation/evals.md) | Grades each run against its task rubric to produce Pass/Fail verdicts and pass rates. |
| [**Insights**](https://docs.oqoqo.ai/evaluation/insights.md) | Analyzes run trajectories to surface frictions and populate the [Brief](https://docs.oqoqo.ai/evaluation/comparisons.md#brief). |

Both require models set under [Oqoqo AI defaults](https://docs.oqoqo.ai/setup/oqoqo-ai.md). Leave either off at launch and trigger **Run evals** or **Run insights** later on the finished runs.

If your organization uses [**Agent mode**](https://docs.oqoqo.ai/evaluation/agent-mode.md) (private beta) for Evals or Insights, choose **Standard** or **Agent** under **Mode** (below **Mode**, the panel estimates the Agent-mode review count and model). See [Agent mode](https://docs.oqoqo.ai/evaluation/agent-mode.md#standard-and-agent-modes) for how the two modes compare.

## Override models for this experiment

Expand **Advanced** to override the **Evals model**, **Insights model**, or **Experiment insights model** for this experiment alone (**Default** uses your organization settings). Enable **Customize** under a role to override your organization's [custom instructions](https://docs.oqoqo.ai/setup/oqoqo-ai.md#add-custom-instructions).

In CLI and plugin launch plans, pass `projectAiSelections` and `projectAiInstructionOverrides` using role keys `judge` (**Evals model**), `sweep` (**Insights model**), and `consolidation` (**Experiment insights model**).

## Launch

```mermaid theme={null}
flowchart LR
  P["Preflight check"] -- "Missing secrets or quota" --> F["Fix blockers"] --> P
  P -- "Ready" --> L["Launch experiment"] --> W["Watch progress"] --> R["Read Matrix & Comparisons"]
```

> [tab] Dashboard
>
> Verify the run count in the footer and select **Launch**. If a tool needs values or a provider is disconnected, the panel highlights each blocker with a direct fix link.

> [tab] CLI
>
> Validate the plan with `preflight` (spends no runs), launch it, and watch progress:
>
> ```bash theme={null}
> oqoqo experiments preflight --plan-file plan.json
> oqoqo experiments launch --plan-file plan.json
> oqoqo experiments watch exp_8f2c41d0
> ```
>
> Start from the [example plan](https://docs.oqoqo.ai/cli/experiments.md#example) or copy an existing experiment with `clone-plan`. See [`oqoqo experiments launch`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-launch) for all flags.

> [tab] Plugin (MCP)
>
> Ask your AI assistant to launch the experiment. It runs **Check an experiment plan before launch** ([`preflight_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#preflight_experiment)), **Launch an experiment** ([`launch_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#launch_experiment)), and **Get experiment status** ([`get_experiment_status`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_status)). Turn on tool approval in your host so it confirms before spending runs.
>
> ```text theme={null}
> Launch an Oqoqo experiment with the checkout tasks, Raw Agent and our Stripe MCP treatment, Claude Code and Codex, and 2 trials.
> ```

## Rename an experiment

You can rename any active or finished experiment (up to 120 characters) without changing its `exp_` ID, runs, or links.

> [tab] Dashboard
>
> Open the experiment, select its title at the top of the page, edit the name, and press Enter (or select outside the field) to save.

> [tab] CLI
>
> ```bash theme={null}
> oqoqo experiments rename exp_8f2c41d0 "Stripe CLI vs Raw Agent (v2)"
> ```
>
> See [`oqoqo experiments rename`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rename).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to rename the experiment; it calls **Rename an experiment** ([`rename_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#rename_experiment)).

## Launch again

Once an experiment reaches **Completed**, **Needs review**, **Error**, or **Cancelled**, use **Launch again** to rerun the same configuration against the latest task versions and current treatment tools (or select **Edit first** to adjust tasks, agents, or trials before launching). If any task or treatment was deleted, use **Edit first** instead.

> [tab] Dashboard
>
> Open the experiment (or its **⋯** row menu in **All experiments**) and select **Launch again**, resolve any listed blockers, and select **Launch**.

> [tab] CLI
>
> ```bash theme={null}
> oqoqo experiments rerun exp_8f2c41d0
> ```
>
> Pass `--versions experiment` to lock the original task versions, or use [`oqoqo experiments clone-plan`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-clone-plan) to edit the plan before launching ([`oqoqo experiments rerun`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rerun)).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to rerun it; it calls **Launch an experiment again** ([`rerun_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#rerun_experiment)) or **Get an experiment's launch plan** ([`get_experiment_plan`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_plan)) to edit first.

Relaunched experiments become child experiments of the original **parent experiment**. In **All experiments**, child rows show **Parent:** and the parent's name, and their **⋯** menu offers **Open parent experiment**.

![All experiments with two rows named Run again: Stripe CLI and Raw Agent, each showing Parent: Stripe CLI and Raw Agent, 7 create tasks. The first row's menu is open, with Open parent experiment at the top.](https://mintcdn.com/oqoqo/k4Yj05MbUPbdkSQD/images/screenshots/experiments-list-parent-light.png?fit=max\&auto=format\&n=k4Yj05MbUPbdkSQD\&q=85\&s=b681c5e9d59c35f1eee780794cd3c8ed)

## Cancel an experiment

Stop a **Running** experiment anytime to halt unfinished runs while keeping completed runs intact (cancelled runs do not use your run balance).

> [tab] Dashboard
>
> Select **Cancel experiment** on the experiment page (or **Cancel run** in a single run's **⋯** menu) and confirm.

> [tab] CLI
>
> ```bash theme={null}
> oqoqo experiments cancel exp_8f2c41d0
> ```
>
> Pass `--run-id` to cancel one run ([`oqoqo experiments cancel`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-cancel)).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to stop the experiment; it calls **Cancel an experiment or run** ([`cancel_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#cancel_experiment)) after your approval.

## From the CLI and the plugin

| Task | CLI | Plugin (MCP) |
| - | - | - |
| Check a launch plan | [`oqoqo experiments preflight`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-preflight) | [`preflight_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#preflight_experiment) |
| Launch an experiment | [`oqoqo experiments launch`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-launch) | [`launch_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#launch_experiment) |
| Follow progress | [`oqoqo experiments watch`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-watch), [`status`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-status) | [`get_experiment_status`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_status), [`get_launch_status`](https://docs.oqoqo.ai/mcp-server/experiments#get_launch_status) |
| Rename an experiment | [`oqoqo experiments rename`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rename) | [`rename_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#rename_experiment) |
| Relaunch or clone a plan | [`oqoqo experiments rerun`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-rerun), [`clone-plan`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-clone-plan) | [`rerun_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#rerun_experiment), [`get_experiment_plan`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_plan) |
| Cancel an experiment or run | [`oqoqo experiments cancel`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-cancel) | [`cancel_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#cancel_experiment) |

## Next steps

> [card] [Read results](https://docs.oqoqo.ai/evaluation/results.md)
>
> Analyze Matrix pass rates, cost, and runs.

> [card] [Schedules](https://docs.oqoqo.ai/experiments/schedules.md)
>
> Automate recurring experiments on a schedule.

> [card] [Frictions](https://docs.oqoqo.ai/evaluation/frictions.md)
>
> Inspect where agents got stuck and how to fix it.

> [card] [oqoqo experiments](https://docs.oqoqo.ai/cli/experiments.md)
>
> Reference for all experiment commands and plan fields.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.