> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# oqoqo evals and insights

> Preview, start, list, and page Evals and Insights attempts, and compare treatments and agents, from the terminal.

[Evals](https://docs.oqoqo.ai/evaluation/evals.md#what-evals-read) score each finished run against its task's rubric, and [Insights](https://docs.oqoqo.ai/evaluation/insights.md#what-insights-read) read each run's trajectory to find frictions. Each time either runs on an experiment, Oqoqo records an attempt (`eval_` or `insights_`), and earlier attempts stay in the history.

## Statuses

An attempt is `queued`, `running`, `completed`, `completed_with_errors`, `failed`, or `not_run`. Each of its runs has a `status`:

| Status | Meaning |
| - | - |
| `queued`, `running` | No result yet. The run itself can still be going. |
| `completed` | The result is final. |
| `skipped`, `failed` | This attempt gives the run no result. |

For when a pending result becomes final, see [Pending verdicts](https://docs.oqoqo.ai/evaluation/evals.md#pending-verdicts) and [Pending frictions](https://docs.oqoqo.ai/evaluation/insights.md#pending-frictions).

## oqoqo evals preview

Check, without changing anything, which runs Evals can score, which rubrics changed, and whether the **Evals model** is ready.

```bash theme={null}
oqoqo evals preview <experiment-id> [flags]
```

**Plugin (MCP) tool:** [`preview_evals`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#preview_evals)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |

Takes only the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Preview Evals:

```bash theme={null}
oqoqo evals preview exp_8f2c41d0 --output json
```

**Result fields**

| Field | Meaning |
| - | - |
| `ready` | Whether Evals can start now. `modelsReady` and `userProviderReady` say what is missing. |
| `setupUrl` | The dashboard page that fixes a missing model or provider, or `null` |
| `eligibleRunCount` | Runs with a usable rubric and output |
| `runsWithoutRubricsCount` | Runs with no rubric |
| `runsWithInvalidRubricsCount` | Runs whose saved rubric settings must be reviewed and saved |
| `runsNotJudgeableCount` | Runs that were blocked or haven't finished |
| `rubricChanges` | Each task whose rubric changed, with `fromVersion` and `toVersion` |
| `failedRunIds` | Runs whose latest Evals failed, to pass to `--run-ids` |
| `mode` | The modes this experiment can use, as below |

`mode` has these fields:

| Field | Meaning |
| - | - |
| `experimentMode` | The mode the experiment launched with. A trigger without `--mode` uses it. |
| `latestMode` | The mode of the latest Evals attempt that ran |
| `agentAvailable` | Whether your organization can use Agent mode |
| `estimate` | Only while your organization can use Agent mode: the `runs` the evals agent would review, and the `model` it uses |

**Notes**

While your organization can use [Agent mode](https://docs.oqoqo.ai/evaluation/agent-mode.md), the preview prints what Agent mode would use, for example "Agent mode: Oqoqo's evals agent reads the whole run and checks the agent's claims before it scores. Slower, and costs more. Up to 56 agent reviews on Claude Opus 5.5." For an experiment that launched in Agent mode, it also prints "This experiment launched in Agent mode. Evals use it unless you add --mode standard."

## oqoqo evals trigger

Score an experiment's finished runs, for the first time or again, optionally with another model, instructions, mode, or set of runs. The agent doesn't run again.

```bash theme={null}
oqoqo evals trigger <experiment-id> [flags]
```

**Plugin (MCP) tool:** [`trigger_evals`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#trigger_evals)

The CLI doesn't ask for confirmation. In Ask Oqo, you approve each call.

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--mode` | `standard`, `agent` | The experiment's launch mode | How this attempt scores each run. See **Notes**. |
| `--run-ids` | string | Every eligible run | Comma-separated `run_` IDs to score, such as the preview's `failedRunIds`. One ineligible ID fails the whole request. |
| `--project-ai-selections` | string | The organization's **Evals model** | A JSON array with one `{"role":"judge","providerConnectionId":"aic_…","providerModelId":"aim_…","effortLevel":"…"}`. `effortLevel` is optional. |
| `--project-ai-instruction-overrides` | string | The organization setting | A JSON array with one `{"role":"judge","customInstructions":"…"}`. |
| `--attempt-id` | string | A new ID | An ID you choose, so that a retry doesn't repeat the change. It has 1 to 100 characters. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Score an experiment again after you fix a rubric:

```bash theme={null}
oqoqo evals trigger exp_8f2c41d0
```

Score only the runs whose Evals failed last time:

```bash theme={null}
oqoqo evals trigger exp_8f2c41d0 --run-ids run_6b31e09f,run_c27d4a58
```

Score in Agent mode:

```bash theme={null}
oqoqo evals trigger exp_8f2c41d0 --mode agent --output json
```

**Result fields**

| Field | Meaning |
| - | - |
| `evalsId` | The new attempt's `eval_` ID. Read its results with [`oqoqo evals get`](#oqoqo-evals-get). |
| `status`, `mode` | The attempt's status, and the mode it uses: the one you chose, or the experiment's launch mode |
| `pollAfterMs` | How long to wait before you read the results: 60 seconds in Agent mode, 10 seconds in Standard mode, or `null` when the attempt has already finished |
| `replayed` | `true` when a repeated `--attempt-id` returned the earlier attempt |

**Notes**

> [!NOTE]
>
> Agent mode for Evals and Insights is in private beta. Email [support@oqoqo.ai](mailto:support@oqoqo.ai) to enable it for your organization.

Both modes use your **Evals model** and score against the same rubric.

* `standard` scores each run in one pass with your Evals model. It's the fastest and lowest-cost mode.
* `agent` has Oqoqo's evals agent read the whole run and the changed files and check the agent's claims before it scores. It costs more and takes a few minutes per run. Use it for long or complex runs, or when a score depends on checking what the agent did.

Without `--mode`, the attempt uses the experiment's launch mode, not the latest attempt's mode as the dashboard does. To repeat the latest mode, pass the preview's `mode.latestMode`.

If your organization can't use Agent mode, a request for it is refused with `agent_mode_unavailable`: "Agent mode isn't available for your organization. Use Standard mode instead." If you left out `--mode` and the experiment launched in Agent mode, the message is "This experiment launched Evals in Agent mode, which your organization can't use now. Rerun with --mode standard." If the **Evals model** can't run Agent mode, the request is refused with `agent_mode_model_unsupported` and a `setupUrl` to **Oqoqo AI defaults**, for example "Agent mode can't use Claude Pro / Max models. Choose an Evals model from an API-key provider, or use Standard mode."

## oqoqo evals list

List an experiment's Evals attempts, with each one's status, mode, and models.

```bash theme={null}
oqoqo evals list <experiment-id> [flags]
```

**Plugin (MCP) tool:** [`list_evals`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#list_evals)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--limit` | integer | `25` | Results per page, up to 100 (default 25). |
| `--cursor` | string | | The `nextCursor` from the previous page. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

List an experiment's Evals attempts:

```bash theme={null}
oqoqo evals list exp_8f2c41d0
```

**Result fields**

Each attempt has its `eval_` ID, `status`, `mode` (`standard` or `agent`), and `models`. A failed attempt has `error` (`code` and `message`). An Agent mode attempt also has `agentUsage` once its agent has reviewed runs: the `inputTokens`, `cacheReadTokens` (the part your provider read from its prompt cache), and `outputTokens` that the evals agent has used so far. The terminal prints the mode after the status, as in `agent mode · 2.4M input tokens on Claude Opus 5.5 (300K cached)`, and one line for each model, with its provider, model, and effort.

**Notes**

Each attempt records the models it started with, so later changes under [Oqoqo AI defaults](https://docs.oqoqo.ai/setup/oqoqo-ai.md) don't affect it. An attempt from before Oqoqo recorded models shows **Not recorded**.

## oqoqo evals get

Page the per-run results of one Evals attempt, or read one run's full verdict.

```bash theme={null}
oqoqo evals get <experiment-id> [evals-id] [flags]
```

**Plugin (MCP) tool:** [`get_evals_results`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_evals_results)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |
| `[evals-id]` | The attempt's `eval_` ID. Omit it to read the latest attempt, except with `--offset` above 0. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--view` | `summary`, `trial_evidence` | `summary` | `summary`: one row per run. `trial_evidence`: one run's full verdict, with `--trial-id`. |
| `--limit` | integer | `50` | Results per page, up to 200 (default 50). Only in the `summary` view. |
| `--cursor` | string | | The `nextCursor` from the previous page. |
| `--trial-id` | string | | The `run_` ID to read in the `trial_evidence` view. |
| `--offset` | integer | `0` | The byte to start from in the `trial_evidence` view. |
| `--limit-bytes` | integer | `12288` | Bytes per `trial_evidence` page, at most 12,288. |
| `--wait-ms` | integer | | Wait for a change before you return, up to 25,000 ms. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Read the latest attempt:

```bash theme={null}
oqoqo evals get exp_8f2c41d0 --output json
```

Read the next page of the same attempt:

```bash theme={null}
oqoqo evals get exp_8f2c41d0 --cursor <next-cursor> --output json
```

Read the next page of one run's verdict:

```bash theme={null}
oqoqo evals get exp_8f2c41d0 eval_5f1a9c --view trial_evidence --trial-id run_6b31e09f --offset 12288
```

**Result fields**

| Field | Meaning |
| - | - |
| `attempt.evalsId` | The attempt this page reads |
| `attempt.mode`, `attempt.agentUsage` | The attempt's mode, and in Agent mode what its agent has used, as in [`oqoqo evals list`](#oqoqo-evals-list) |
| `status` | Each row's [status](#statuses) |
| `error` | For a failed row, `code` and `message`. The message starts with "Evals could not evaluate this run, so it has no Pass or Fail." and gives the cause and the next step. |
| `outcome` | The run's verdict, with an explanation |
| `evidenceCoverage` | Which steps the **Evals model** read in full, shortened, or left out to fit its context window, and where the changed files came from. `null` means the read wasn't recorded, which doesn't prove a full read. See [What Evals read](https://docs.oqoqo.ai/evaluation/evals.md#what-evals-read). |
| `hasMore`, `nextCursor` | Whether more rows remain, and the cursor for the next page |

**Notes**

A cursor stays on the attempt it came from, even after a newer attempt starts. A page can hold fewer rows than `--limit` because each page also has a size limit. When something was left out of a run's read, the CLI prints a note under its row.

## oqoqo evals compare

See which differences between an experiment's treatments and agents hold up, task by task, from the Evals verdicts on the Matrix. The output starts as [Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md#answers) does: one sentence that answers the experiment's question, then a block for each difference with each side's pass rate, how the tasks went, the result, and the next step. Every comparison follows.

```bash theme={null}
oqoqo evals compare <experiment-id> [evals-id] [flags]
```

**Plugin (MCP) tool:** [`get_evals_comparisons`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_evals_comparisons)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |
| `[evals-id]` | The attempt's `eval_` ID. Omit it to compare the latest attempt. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--kind` | `treatment`, `agent`, `interaction` | Every kind | List only this kind. See **Notes**. |
| `--label` | `clear`, `likely`, `unclear` | Every label | List only this label. See **Notes**. |
| `--limit` | integer | `200` | Results per page, up to 200 (default 200). |
| `--cursor` | string | | The `nextCursor` from the previous page. Send it with the same `--kind` and `--label`. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Compare the latest Evals of a seven-task experiment with two treatments and two agents:

```bash theme={null}
oqoqo evals compare exp_8f2c41d0
```

```text theme={null}
Stripe CLI did better than Raw Agent on all 7 tasks.

Stripe CLI and Raw Agent  cmp_4a0e2c9d
  Stripe CLI   93%   26 of 28 runs
  Raw Agent     0%    0 of 28 runs
  Better on all 7 tasks.
  Clear difference
  Both agents did better with Stripe CLI.

All 6 comparisons (Eval 1 (latest))
  Treatments across all agents
    cmp_4a0e2c9d  Clear difference
      Stripe CLI   93%   26 of 28
      Raw Agent     0%    0 of 28
      Stripe CLI better on all 7
  Treatments with Claude Code · claude-haiku-5-5 · default
    cmp_7b1d3e5f  Likely difference · Run again to be sure · 42 runs: oqoqo experiments clone-plan exp_8f2c41d0 --comparison cmp_7b1d3e5f
      Stripe CLI   86%   12 of 14
      Raw Agent     0%    0 of 14
      Stripe CLI better on all 7
  Treatments with Claude Code · claude-opus-5-5 · default
    cmp_2c8e0a4b  Likely difference · Run again to be sure · 42 runs: oqoqo experiments clone-plan exp_8f2c41d0 --comparison cmp_2c8e0a4b
      Stripe CLI   100%   14 of 14
      Raw Agent      0%    0 of 14
      Stripe CLI better on all 7
  Agents with Raw Agent
    cmp_5e0b8c73  No clear difference
      Claude Code · claude-haiku-5-5 · default   0%   0 of 14
      Claude Code · claude-opus-5-5 · default    0%   0 of 14
      Same on all 7
  Agents with Stripe CLI
    cmp_9f3a6d21  No clear difference
      Claude Code · claude-haiku-5-5 · default    86%   12 of 14
      Claude Code · claude-opus-5-5 · default    100%   14 of 14
      Claude Code · claude-opus-5-5 · default better on 2, same on 5
  Does a treatment help one agent more?
    cmp_1d6f4a98  No clear difference
      Stripe CLI helped Claude Code · claude-opus-5-5 · default more on 2 tasks, the same on 5.

Notes
  With 7 tasks, a result for one agent alone or between agents can be a likely difference at most.
  In 2 of 28 Matrix cells, repeated runs got different results.
```

The sentence comes first, then the block for the one difference, with its `cmp_` ID. Then **All 6 comparisons** lists each comparison under its group, with its `cmp_` ID. A likely difference prints, on its result line, the command that plans the runs to [run it again](https://docs.oqoqo.ai/evaluation/comparisons.md#run-a-comparison-again-to-be-sure). **Notes** come last.

Read the comparisons as JSON:

```bash theme={null}
oqoqo evals compare exp_8f2c41d0 --output json
```

**Result fields**

| Field | Meaning |
| - | - |
| `state` | `ready`, or the first reason that applies: `nothing_to_compare`, `too_many_agents` (one treatment with more than 6 agents), `evals_off`, or `no_verdicts` |
| `message` | The `title` and `description` for the state, or `null` when the state is `ready`. For `no_verdicts`, it says whether more verdicts can still come. |
| `summary` | The sentence that answers the experiment's question over its main comparisons: `text`, and `parts`, where each treatment or agent it names carries its `treatmentKey` or `agentKey`. `null` when the experiment has no main comparison. The same on every page and with every `--kind` and `--label`. |
| `differences` | One block for each clear or likely difference, in the order to read them, as below: the main comparisons first, then any other comparison with a clear difference, up to 5. A comparison with no clear difference is never in it. The same on every page and with every `--kind` and `--label`. |
| `restLine` | One sentence on what `summary` and `differences` leave out, such as "7 other comparisons show a likely difference. See All 40 comparisons." `null` when they leave nothing out. |
| `notes` | One sentence each on what limits the comparisons, such as too few tasks. With an empty state, only the too-few-tasks note stays. |
| `evals` | The attempt: `evalsId`, `sequenceNumber`, `isLatest`, and `status` |
| `running` | `true` while the experiment still has runs to finish. Labels can change until the experiment and its Evals attempt finish. |
| `comparisonCount` | Every comparison in the experiment, whatever `--kind` and `--label` list |
| `comparisons` | The listed comparisons, as below |
| `treatments` | The experiment's treatments, each with `treatmentKey`, `name`, and `isReference` for the baseline |
| `agents` | The agents that the listed comparisons name, each with `agentKey` and `label`. Here an agent is one agent with one model and effort. |
| `taskCount`, `runsEach` | The experiment's tasks, and the trials of each combination |
| `scoredRuns`, `unscoredRuns` | Runs with an Evals verdict, and runs without one |
| `noiseFloor` | The `cases` where a task ran more than once with the same agent and treatment, and how many of them were `disagreeing`. `null` when no task ran more than once. |
| `ceilingTaskCount`, `floorTaskCount` | Tasks that passed in every run, and tasks that failed in every run. These tasks can't show a difference. |
| `agentPairsOmitted` | `true` when the experiment ran more than 6 agents. Agents are then not compared with each other, and there are no `interaction` comparisons. |
| `hasMore`, `nextCursor` | Whether more comparisons remain, and the cursor for the next read |

Each block in `differences` has these fields:

| Field | Meaning |
| - | - |
| `comparisonId` | The `cmp_` ID of the comparison |
| `label` | `clear` or `likely` |
| `name` | The two sides, the side that did better first, and what the comparison holds fixed when it isn't a main comparison, for example "Stripe CLI and Raw Agent, with Claude Code · claude-haiku-5-5 · default" |
| `sides` | Both sides, the side that did better first: `key`, `label`, `passes`, `runs`, and `percent` (a whole number that is 100 only when every run passed, and 0 only when none did). Empty for an `interaction`. |
| `taskLine` | How the tasks went for the side that did better, for example "Better on 14 tasks, worse on 2, same on 26." For an `interaction`, its task sentence. |
| `headline` | The difference as one sentence, for example "Stripe CLI did better than Raw Agent on all 7 tasks." A likely one says "likely did better than". |
| `verdict` | The result: "Clear difference", "Likely difference", "Likely difference · Needs more tasks to be sure", or "No clear difference". Show it on one line with `nextStep`, joined by " · ". |
| `nextStep` | `kind` `run_again` with its `text` ("Run again to be sure · 42 runs") and `runs`, or `kind` `more_tasks` with its `text` ("Launch again with more tasks"). `null` when there is nothing to do, or while the results can still change. |
| `byAgent` | How the agents split, for example "Both agents did better with Stripe CLI." `null` unless the block compares treatments for all agents together and the experiment ran two or more agents. |

Each comparison has these fields:

| Field | Meaning |
| - | - |
| `comparisonId` | The `cmp_` ID. It stays the same when Evals run again. |
| `kind`, `label` | What it compares, and its result |
| `family` | `primary` for a main comparison, or `breakdown`. See [Main comparisons and breakdowns](https://docs.oqoqo.ai/evaluation/comparisons.md#main-comparisons-and-breakdowns). |
| `title`, `scope` | The two sides, and what they share: `All agents`, an agent, or a treatment |
| `headline`, `sides`, `verdict`, `nextStep` | As in a block |
| `sentence` | The headline, both sides' runs, and the verdict in one line |
| `groupLabel` | The group that **All N comparisons** lists it under, for example "Treatments across all agents", "Treatments with Claude Code · claude-haiku-5-5 · default", "Agents with Raw Agent", or "Does a treatment help one agent more?" |
| `taskSplit` | How the tasks went, naming the side that did better, for example "Stripe CLI better on all 7". For an `interaction`, a sentence. |
| `treatmentKey`, `referenceTreatmentKey` | The two treatments of a `treatment` or `interaction` comparison, or the treatment of an `agent` comparison (`treatmentKey` only) |
| `agentKey`, `referenceAgentKey` | The two agents of an `agent` comparison, or the agent of a `treatment` comparison (`agentKey`, which is `null` for all agents together) |
| `arm`, `reference` | The `passes` and `runs` that each side compared. Each task compares the same number of runs on each side, so these counts can be lower than the **Matrix**. An `interaction` has no top-level `arm` and `reference`. |
| `first`, `second` | For an `interaction`, the two agents, each with `agentKey`, `arm`, and `reference` |
| `tasks` | The tasks where `arm` did better (`higher`), worse (`lower`), or the same (`same`). For an `interaction`, `higher` counts the tasks where the treatment helped `first` more than `second`. |
| `pValue`, `adjustedPValue` | The exact two-sided sign test on the tasks that differed, and the same after the Holm correction within the comparison's `family` |
| `runsLeftOut` | Runs of the two sides that the comparison doesn't use: runs with no verdict, runs on a task where the other side has no verdict, and a side's runs beyond the other side's count |
| `confirmation` | For a likely difference, the runs that check it again: `taskCount`, `treatmentKeys`, `agentKeys`, `runsEach` (runs of each combination), and `runs` (the total). `null` for other labels. |
| `confirmationUnavailable` | Why a likely difference has no `confirmation`, as below |

| `confirmationUnavailable` | Meaning |
| - | - |
| `earlier_evals` | The read is of an earlier Evals attempt. A likely difference is run again from the latest Evals. |
| `in_progress` | The experiment or its Evals is still running. |
| `cannot_settle` | Running these tasks again can't settle the difference: too few of them can still show a difference, or it would take more than 10 runs each. Launch the experiment again with more tasks. |

**Notes**

The sentence and the blocks come first. Then the most decisive comparisons come first: a clear difference, then a likely one, then the rest, and within a label the main comparisons before the breakdowns. `--kind` and `--label` only filter which comparisons are listed: labels don't change, and `summary`, `differences`, and `comparisonCount` stay the same.

| `--kind` | Compares |
| - | - |
| `treatment` | A treatment against the baseline, for all agents together and for each agent. Without a baseline, each pair of treatments. With one agent, `agentKey` names it. |
| `agent` | Two agents with the same treatment |
| `interaction` | Whether a treatment helps one agent more than another (**Does a treatment help one agent more?**) |

`--label` is `clear` (**Clear difference**), `likely` (**Likely difference**), or `unclear` (**No clear difference**). `unclear` means that these runs can't tell the two sides apart, not that the sides are the same.

* The main comparisons are checked together, and the breakdowns are checked together and more strictly, because there are more of them. So the same split can be clear for all agents together and likely for one agent. See [How results are decided](https://docs.oqoqo.ai/evaluation/comparisons.md#how-results-are-decided).
* One read lists up to 200 comparisons, and only as many as fit in 20 KiB. Page with `--cursor` while `hasMore` is true. If the comparisons changed since the first read, for example because Evals ran again, the cursor fails with `invalid_cursor`, and you start again without it.
* When a likely difference is run again, a `treatment` comparison keeps its agent, or every agent for all agents together. An `agent` comparison keeps its treatment, and an `interaction` keeps both treatments and both agents. Plan the runs with [`oqoqo experiments clone-plan --comparison`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-clone-plan).
* A difference needs at least 6 tasks where one side did better than the other. Tasks where both did the same don't count. With fewer than 6 tasks, no comparison can show a difference, and `notes` says so.
* An experiment with more than 10,000 Matrix cells fails with `experiment_too_large`.

## oqoqo insights preview

Check whether the **Insights model** and **Experiment insights model** are ready for an experiment. This command changes nothing.

```bash theme={null}
oqoqo insights preview <experiment-id> [flags]
```

**Plugin (MCP) tool:** [`preview_insights`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#preview_insights)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |

Takes only the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Preview Insights:

```bash theme={null}
oqoqo insights preview exp_8f2c41d0
```

**Result fields**

| Field | Meaning |
| - | - |
| `ready` | Whether Insights can start now. `modelsReady` and `userProviderReady` say what is missing. |
| `setupUrl` | The dashboard page that fixes a missing model or provider, or `null` |
| `failedRunIds` | Runs whose latest Insights failed, to pass to `oqoqo insights trigger --run-ids` |
| `mode` | The modes this experiment can use, as below |

`mode` has these fields:

| Field | Meaning |
| - | - |
| `experimentMode` | The mode the experiment launched with. A trigger without `--mode` uses it. |
| `latestMode` | The mode of the latest Insights attempt that ran |
| `agentAvailable` | Whether your organization can use Agent mode |
| `estimate` | Only while your organization can use Agent mode: the `runs` the insights agent would review, the `model` it reviews them with, and `experimentModel` when the **Experiment insights model** differs from the **Insights model** |

**Notes**

While your organization can use [Agent mode](https://docs.oqoqo.ai/evaluation/agent-mode.md), the preview prints what Agent mode would use, for example "Agent mode: Oqoqo's insights agent reads the whole run and checks each friction against what the tools returned. Slower, and costs more. Up to 56 agent reviews on Claude Opus 5.5." For an experiment that launched in Agent mode, it also prints "This experiment launched in Agent mode. Insights use it unless you add --mode standard."

## oqoqo insights trigger

Read an experiment's finished runs for frictions, for the first time or again, optionally with other models, instructions, mode, or runs. Evals verdicts don't change.

```bash theme={null}
oqoqo insights trigger <experiment-id> [flags]
```

**Plugin (MCP) tool:** [`trigger_insights`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#trigger_insights)

The CLI doesn't ask for confirmation. In Ask Oqo, you approve each call.

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--mode` | `standard`, `agent` | The experiment's launch mode | How this attempt reviews each run. See **Notes**. |
| `--run-ids` | string | Every eligible run | Comma-separated `run_` IDs to read, such as the preview's `failedRunIds`. One ineligible ID fails the whole request. |
| `--project-ai-selections` | string | The organization's models | A JSON array of `{"role":"…","providerConnectionId":"aic_…","providerModelId":"aim_…","effortLevel":"…"}`. `role` is `sweep` for the **Insights model** or `consolidation` for the **Experiment insights model**. |
| `--project-ai-instruction-overrides` | string | The organization setting | A JSON array of `{"role":"…","customInstructions":"…"}`, with `role` `sweep` or `consolidation`. A role you omit uses the organization setting. |
| `--attempt-id` | string | A new ID | An ID you choose, so that a retry doesn't repeat the change. It has 1 to 100 characters. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Find frictions again on an experiment:

```bash theme={null}
oqoqo insights trigger exp_8f2c41d0
```

Review in Agent mode:

```bash theme={null}
oqoqo insights trigger exp_8f2c41d0 --mode agent --output json
```

**Result fields**

| Field | Meaning |
| - | - |
| `insightsId` | The new attempt's `insights_` ID |
| `status`, `mode` | The attempt's status, and the mode it uses: the one you chose, or the experiment's launch mode |
| `pollAfterMs` | How long to wait before you read the results: 60 seconds in Agent mode, 10 seconds in Standard mode, or `null` when the attempt has already finished |
| `replayed` | `true` when a repeated `--attempt-id` returned the earlier attempt |

**Notes**

When Insights finish, Oqoqo groups the findings into the project's frictions. Read them with [`oqoqo frictions list`](https://docs.oqoqo.ai/cli/frictions#oqoqo-frictions-list).

Both modes use your **Insights model** and **Experiment insights model**, and report frictions in the same way. [Agent mode](https://docs.oqoqo.ai/evaluation/agent-mode.md) is in private beta.

* `standard` reviews each run in one pass with your Insights model. It's the fastest and lowest-cost mode.
* `agent` has Oqoqo's insights agent read the whole run and check each friction against what the tools returned. It costs more and takes a few minutes per run. Use it for long or complex runs, or when one pass misses frictions.

In Agent mode, the insights agent also does the experiment's grouping of frictions, which Standard mode does in one pass. For each task of each difference in the Brief, it says what a failed run and a passed run did differently ([`oqoqo experiments brief`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-brief)). Without `--mode`, the attempt uses the experiment's launch mode, not the latest attempt's mode as the dashboard does. To repeat the latest mode, pass the preview's `mode.latestMode`.

If your organization can't use Agent mode, a request for it is refused with `agent_mode_unavailable`: "Agent mode isn't available for your organization. Use Standard mode instead." If you left out `--mode` and the experiment launched in Agent mode, the message is "This experiment launched Insights in Agent mode, which your organization can't use now. Rerun with --mode standard." If the **Insights model** or the **Experiment insights model** can't run Agent mode, the request is refused with `agent_mode_model_unsupported` and a `setupUrl` to **Oqoqo AI defaults**. The message names the model, for example "Agent mode can't use Claude Pro / Max models. Choose an Insights model from an API-key provider, or use Standard mode."

## oqoqo insights list

List an experiment's Insights attempts, with each one's status, mode, and models.

```bash theme={null}
oqoqo insights list <experiment-id> [flags]
```

**Plugin (MCP) tool:** [`list_insights`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#list_insights)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--limit` | integer | `25` | Results per page, up to 100 (default 25). |
| `--cursor` | string | | The `nextCursor` from the previous page. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

List an experiment's Insights attempts:

```bash theme={null}
oqoqo insights list exp_8f2c41d0
```

**Result fields**

Each attempt has its `insights_` ID, `status`, `mode` (`standard` or `agent`), and `models`: the **Insights model** and **Experiment insights model** it started with.

A failed attempt has `error` (`code` and `message`). In Agent mode, when the insights agent couldn't group the frictions, `error.message` starts with "Insights could not group this experiment's frictions." and gives the cause and the next step.

An Agent mode attempt also has `agentUsage` once its agent has reviewed runs: the `inputTokens`, `cacheReadTokens` (the part your provider read from its prompt cache), and `outputTokens` that the insights agent has used so far. It includes grouping the frictions and the pairs of runs in the Brief ([`oqoqo experiments brief`](https://docs.oqoqo.ai/cli/experiments#oqoqo-experiments-brief)).

The terminal prints the mode after the status, as in `agent mode · 2.4M input tokens on Claude Opus 5.5 (300K cached)`, and one line for each model. The usage line names a model only when the Insights model and the Experiment insights model are the same.

## oqoqo insights get

Page the per-run results of one Insights attempt, read its report, or read one run's full findings.

```bash theme={null}
oqoqo insights get <experiment-id> [insights-id] [flags]
```

**Plugin (MCP) tool:** [`get_insights_results`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_insights_results)

**Arguments**

| Argument | Description |
| - | - |
| `<experiment-id>` | Required. The experiment's `exp_` ID. |
| `[insights-id]` | The attempt's `insights_` ID. Omit it to read the latest attempt, except with `--offset` above 0. |

**Flags**

| Flag | Type | Default | Description |
| - | - | - | - |
| `--view` | `summary`, `attempt_report`, `trial_evidence` | `summary` | `summary`: one row per run. `attempt_report`: the attempt's report across the experiment. `trial_evidence`: one run's findings, with `--trial-id`. |
| `--limit` | integer | `50` | Results per page, up to 200 (default 50). Only in the `summary` view. |
| `--cursor` | string | | The `nextCursor` from the previous page. |
| `--trial-id` | string | | The `run_` ID to read in the `trial_evidence` view. |
| `--offset` | integer | `0` | The byte to start from in the `attempt_report` and `trial_evidence` views. |
| `--limit-bytes` | integer | `12288` | Bytes per page in the `attempt_report` and `trial_evidence` views, at most 12,288. |
| `--wait-ms` | integer | | Wait for a change before you return, up to 25,000 ms. |

Also takes the [global flags](https://docs.oqoqo.ai/cli/overview.md#global-flags).

**Examples**

Read the latest attempt:

```bash theme={null}
oqoqo insights get exp_8f2c41d0 --output json
```

Read the next page of an attempt report:

```bash theme={null}
oqoqo insights get exp_8f2c41d0 insights_9e04b2 --view attempt_report --offset 12288
```

**Result fields**

| Field | Meaning |
| - | - |
| `attempt.insightsId` | The attempt this page reads |
| `attempt.mode`, `attempt.agentUsage`, `attempt.error` | The attempt's mode, what its agent has used in Agent mode, and its error, as in [`oqoqo insights list`](#oqoqo-insights-list) |
| `status` | Each row's [status](#statuses) |
| `error` | For a failed row, `code` and `message`. The message starts with "Insights could not analyze this run." and gives the cause and the next step. |
| `evidenceCoverage` | How much of the trajectory the **Insights model** read. `null` means the read wasn't recorded. See [What Insights read](https://docs.oqoqo.ai/evaluation/insights.md#what-insights-read). |
| `hasMore`, `nextCursor` | Whether more rows remain, and the cursor for the next page |

**Notes**

A cursor stays on the attempt it came from, even after a newer attempt starts. The `attempt_report` and `trial_evidence` views page with `--offset`, which doesn't name the attempt, so pages after the first need the `insights-id` argument.

## Related

> [card] [Evals](https://docs.oqoqo.ai/evaluation/evals.md)
>
> What Evals read, Agent mode, and pending verdicts.

> [card] [Insights](https://docs.oqoqo.ai/evaluation/insights.md)
>
> What Insights read, the modes, and pending frictions.

> [card] [oqoqo frictions](https://docs.oqoqo.ai/cli/frictions.md)
>
> Read the frictions that Insights found.

> [card] [Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md)
>
> Read the labels and run a likely difference again.

> [card] [oqoqo runs and results](https://docs.oqoqo.ai/cli/runs.md)
>
> Count pending and inconclusive runs with results analyze.

> [card] [Oqoqo AI defaults](https://docs.oqoqo.ai/setup/oqoqo-ai.md)
>
> Choose the Evals and Insights models.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.