> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent mode

> Have an Oqoqo agent check each run in depth for Evals or Insights, see how many runs it will review, and read what it found.

> [!NOTE]
>
> Agent mode for Evals and Insights is in private beta. Email [support@oqoqo.ai](mailto:support@oqoqo.ai) to enable it for your organization.

In **Agent mode**, Oqoqo runs an autonomous review agent on your Evals or Insights model—reading the entire run and calling your model multiple times to verify facts instead of relying on a single condensed pass.

* **Evals Agent mode:** Oqoqo's evals agent inspects the full trace and changed files, verifies claims before scoring, and attaches clickable step and file citations.
* **Insights Agent mode:** Oqoqo's insights agent verifies each friction against tool outputs and adds [**What the runs did differently**](#what-the-runs-did-differently) to **Comparisons**.

Oqoqo enables Agent mode for Evals and Insights independently, so specify Evals, Insights, or both when requesting access.

## Standard and Agent modes

Both modes use the same models and produce the same verdict and friction formats:

| Dimension | Standard | Agent |
| - | - | - |
| **How it reviews** | Single pass; long runs are condensed to fit the context window | Opens the full run and verifies details across multiple calls |
| **Coverage** | May miss details dropped during condensing | Inspects every step and changed file (no condensing) |
| **Model cost** | Lowest (1 call per run) | Higher (many calls to your provider) |
| **Duration** | Fastest | A few minutes per run |
| **Best for** | Short runs or quick iteration | Long or complex runs, claim verification, or deep friction discovery |

Agent mode never consumes extra experiment runs from your Oqoqo balance; your model provider bills the extra model tokens directly ([Runs and billing](https://docs.oqoqo.ai/setup/billing.md)).

## Choose models for Agent mode

Agent mode requires models that support tool calling and have a known context window (for **Evals model**, or both **Insights model** and **Experiment insights model**). Claude Pro / Max models from **Anthropic (Claude Subscription)** cannot run Agent mode; select a model from an API-key provider under [Oqoqo AI defaults](https://docs.oqoqo.ai/setup/oqoqo-ai.md) or use Standard mode.

## Turn on Agent mode

Select **Agent** under **Mode** at launch or when rerunning Evals or Insights.

> [tab] Dashboard
>
> 1. In **New experiment** (or **Run evals** / **Rerun evals** / **Run insights** / **Rerun Insights**), ensure **Evals** or **Insights** is enabled.
> 2. Set **Mode** to **Agent**.

> [tab] CLI
>
> ```bash theme={null}
> oqoqo evals trigger exp_8f2c41d0 --mode agent
> oqoqo insights trigger exp_8f2c41d0 --mode agent
> ```
>
> In a launch plan, set `"evalsMode": "agent"` or `"insightsMode": "agent"`. Without `--mode`, CLI triggers default to the experiment's launch mode rather than the latest attempt's mode (`mode.latestMode` in `preview`) ([`oqoqo evals trigger`](https://docs.oqoqo.ai/cli/evals-and-insights#oqoqo-evals-trigger)).

> [tab] Plugin (MCP)
>
> Ask your AI assistant to run Evals or Insights in Agent mode; it passes `mode: "agent"` to [`trigger_evals`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#trigger_evals) or [`trigger_insights`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#trigger_insights) after your approval.

The **Configuration** panel previews how many runs the agent will review and on which model before you launch.

![New experiment Configuration panel. Evals and Insights are on, each with Mode set to Agent and the line Up to 56 agent reviews on Claude Opus 5.5.](https://mintcdn.com/oqoqo/ff3fQj-pAuWhN5jf/images/screenshots/launch-config-agent-mode-light.png?fit=max\&auto=format\&n=ff3fQj-pAuWhN5jf\&q=85\&s=3079d8499cd1ec07e539f1bf2737fc3c)

## Check what Agent mode uses

Before launching or rerunning, **Mode** estimates the review count and model (such as *"Up to 56 agent reviews on Claude Opus 5.5"*); for Insights, friction clustering and pair comparisons add calls on top of per-run reviews.

![Rerun Insights dialog with Mode set to Agent and the line Up to 56 agent reviews on Claude Opus 5.5, above the Insights and Experiment insights models.](https://mintcdn.com/oqoqo/ff3fQj-pAuWhN5jf/images/screenshots/rerun-insights-agent-mode-light.png?fit=max\&auto=format\&n=ff3fQj-pAuWhN5jf\&q=85\&s=b90e48a529ebf1f44b7eb73ff1337958)

Once running or finished, the attempt menu on **Matrix** (when multiple attempts exist) reports cumulative input and cached tokens (`agentUsage` in CLI and plugin attempt lists; `agentModeEstimates` in preflight and `mode.estimate` in previews).

## Read Evals in Agent mode

Runs graded in Agent mode show an **Agent mode** badge next to **Evals model**. Each criterion lists the evals agent's explanation with direct links to cited run steps, diff lines, and changed files (selecting a file citation jumps directly to **Output**).

![Run Evals tab in Agent mode. A failed criterion shows the evals agent's explanation, a cited step of the run, and two cited changed files.](https://mintcdn.com/oqoqo/Q_WHkFBXOEmr7LQV/images/screenshots/run-evals-agent-mode-light.png?fit=max\&auto=format\&n=Q_WHkFBXOEmr7LQV\&q=85\&s=a2b65bf407bab74d399b82993ffdcd8c)

## Read Insights in Agent mode

Runs analyzed in Agent mode show **Agent mode** next to **Insights model** above **Traces**. Like Standard mode, the insights agent inspects only the trajectory (never changed files, rubrics, or Evals verdicts).

### What the runs did differently

When Insights run in Agent mode, each difference block with a [Brief](https://docs.oqoqo.ai/evaluation/comparisons.md#brief) ends with **What the runs did differently**. For every task where the worse side lost, the insights agent pairs one failed run from the worse side with one passed run from the better side and pinpoints the exact step and quoted lines where their paths diverged.

Once all pairs are analyzed, the Brief updates once more to summarize the cross-task pattern, highlighting one representative task (**Like N other tasks**) and any exceptions (**Went differently**).

![What the runs did differently, 1 of 7 tasks: Create a customer with metadata, Like 6 other tasks, with a line and a quoted step for each run, then Show all 7 tasks.](https://mintcdn.com/oqoqo/w2RdEXAeOkId2YDS/images/screenshots/agent-mode-pairs-light.png?fit=max\&auto=format\&n=w2RdEXAeOkId2YDS\&q=85\&s=54bffe4ec056c368be9f615f02912b3a)

Each row compares one specific pair of runs (when trials disagreed, a subtitle notes *"Raw Agent failed 2 of 3 runs"*). Select **⋯ → Open both runs** to inspect both trajectories side by side.

## When Agent mode fails

If an Agent mode run stops early, the **Evals** tab (for Evals) or CLI and plugin reads (`oqoqo insights get`, `oqoqo insights list`, `get_insights_results`, `list_insights`) report the exact cause—such as rate limits or exceeding context—while preserving any runs that succeeded. See [Agent mode did not return a result](https://docs.oqoqo.ai/reference/troubleshooting.md#agent-mode-did-not-return-a-result) to rerun only failed runs.

## Next steps

> [card] [Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md)
>
> See which differences hold up and inspect paired runs.

> [card] [Frictions](https://docs.oqoqo.ai/evaluation/frictions.md)
>
> Prioritize verified frictions by unrecovered runs.

> [card] [Oqoqo AI defaults](https://docs.oqoqo.ai/setup/oqoqo-ai.md)
>
> Choose tool-capable models for Agent mode.

> [card] [oqoqo evals and insights](https://docs.oqoqo.ai/cli/evals-and-insights.md)
>
> Trigger and inspect Agent mode attempts from the CLI.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.