> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# FAQ

> Find answers about plans and runs, agents and models, results, sharing, Ask Oqo, the CLI, and the plugin.

Can't find your question? Email [support@oqoqo.ai](mailto:support@oqoqo.ai), or see [Troubleshooting](https://docs.oqoqo.ai/reference/troubleshooting.md) for a specific problem.

## About Oqoqo

> [accordion] What is Oqoqo for?
>
> It measures how well AI agents, such as Claude Code, Codex, and Cursor, do real work with your product, and what helps them. You build evals and custom benchmarks, compare agents and models on the same tasks, and find where agents fail. See [What is Oqoqo](https://docs.oqoqo.ai/introduction.md).

> [accordion] Why run an experiment instead of trying an agent on my own machine?
>
> A local try runs the agent once, so it can't tell you how often the agent succeeds. An experiment repeats the same tasks across agents, models, and treatments on fixed machines and files, records every trajectory, and marks each rubric criterion Pass or Fail. See [How Oqoqo works](https://docs.oqoqo.ai/core-concepts.md).

> [accordion] What can I evaluate?
>
> Skills, MCP servers, CLIs, and SDKs, and the docs, APIs, and files they use. Put the tools in a treatment and compare it with **Raw Agent**, the usual baseline, which gives no tools. See [Measure a tool's impact](https://docs.oqoqo.ai/guides/measure-tool-lift.md).

> [accordion] Where do files, machines, and tools attach?
>
> Files and a machine attach to a **task**, tools attach to a **treatment** (toolset), and you choose agents at launch. See [Where files, machines, and tools attach](https://docs.oqoqo.ai/core-concepts.md#where-files-machines-and-tools-attach).

## Plans and runs

> [accordion] Is there a free plan?
>
> Yes, with no credit card required. It resets your organization's runs each month, and unused plan runs don't roll over. See [Pricing](https://oqoqo.ai/pricing) and [Runs and billing](https://docs.oqoqo.ai/setup/billing.md).

> [accordion] How do I get more runs?
>
> Subscribe for more runs each month from **Organization settings → Usage & billing**, or top up at any time for runs that never expire. Larger top-ups cost less per run. For volume discounts, email [support@oqoqo.ai](mailto:support@oqoqo.ai).

> [accordion] How many runs does an experiment use?
>
> One for each combination of task, treatment, agent, and trial. For example, 3 tasks × 2 treatments × 1 agent × 2 trials uses 12 runs, and the **New experiment** footer shows **Uses 12 runs**. See [Launch an experiment](https://docs.oqoqo.ai/experiments/launch.md).

> [accordion] Do failed runs count?
>
> A run that finishes and fails its rubric counts. Errored or cancelled runs don't count against your organization's run balance.

> [accordion] What does a run include?
>
> The cost of running the experiment on Oqoqo, including the cloud machines. Your providers bill model use separately, through your own API keys or subscriptions.

> [accordion] Does every plan include all features?
>
> Yes. Every plan includes unlimited members, projects, experiments, tasks, treatments, and tools, and full access to the dashboard, the CLI, and the plugin.

> [accordion] Can I invite my team?
>
> Yes, at no extra cost. Members share your organization's run balance. See [Invite teammates](https://docs.oqoqo.ai/setup/organization.md#invite-teammates).

## Agents and models

> [accordion] Which agents can I run?
>
> Built-in agents such as Claude Code, Codex, Cursor, Antigravity, and Qwen Code. Each works only with certain providers: [Which providers each agent uses](https://docs.oqoqo.ai/setup/agents.md#which-providers-each-agent-uses) lists them, and each agent's page under **Manage → Agents** shows which of your providers it can use.
>
> Custom agents are in private beta. Email [support@oqoqo.ai](mailto:support@oqoqo.ai) to enable them for your organization, and then you can add any Agent Client Protocol (ACP) agent. See [Custom agents](https://docs.oqoqo.ai/setup/custom-agents.md).

> [accordion] Can I run Claude Code with Claude Pro or Max?
>
> Yes. Claude Code can use **Anthropic (Claude Subscription)**, which you connect under **Profile → Integrations**. See [Which providers each agent uses](https://docs.oqoqo.ai/setup/agents.md#which-providers-each-agent-uses).

> [accordion] Can I use Claude Subscription for Oqoqo AI features?
>
> Yes, for Evals and Insights: connect **Anthropic (Claude Subscription)** under **Profile → Integrations**, then choose a Claude model under **Oqoqo AI features**. Ask Oqo and Agent mode can't use **Anthropic (Claude Subscription)** models. See [Oqoqo AI features](https://docs.oqoqo.ai/setup/oqoqo-ai.md).

> [accordion] Which model providers can I connect?
>
> Organization API keys, gateways such as OpenRouter and Vercel AI Gateway, and OpenAI- or Anthropic-compatible endpoints. Each person can also connect personal subscriptions under **Profile → Integrations**. See [Model providers](https://docs.oqoqo.ai/setup/model-providers.md).

> [accordion] Can I compare models and effort levels?
>
> Yes. Choose more than one model or effort level for an agent at launch, and each choice gets its own **Matrix** column. See [Compare agents and models](https://docs.oqoqo.ai/guides/compare-agents-and-models.md).

> [accordion] What is Max Concurrency?
>
> The limit on how many agent sessions run at once in a launch. It doesn't control Evals or Insights. See [Set Max Concurrency](https://docs.oqoqo.ai/experiments/launch.md#set-max-concurrency).

## Evals, Insights, and results

> [accordion] What is the difference between Evals and Insights?
>
> [Evals](https://docs.oqoqo.ai/evaluation/evals.md) score each run against your rubric, and [Insights](https://docs.oqoqo.ai/evaluation/insights.md) read the trajectory to find frictions. They run independently.

> [accordion] Can I set my own pass or fail criteria?
>
> Yes. Write a rubric of plain-language criteria, and Evals mark each one Pass or Fail. See [Tasks and rubrics](https://docs.oqoqo.ai/experiments/tasks.md).

> [accordion] Do Evals read every step?
>
> Not always. Evals and Insights read as much of the trajectory as fits the model's context window (Evals also read the changed files), and the result says when something was left out. On older runs, a missing note doesn't mean that the whole trajectory was read. See [What Evals read](https://docs.oqoqo.ai/evaluation/evals.md#what-evals-read) and [What Insights read](https://docs.oqoqo.ai/evaluation/insights.md#what-insights-read).

> [accordion] What is Agent mode?
>
> An Evals mode in which Oqoqo's evals agent reads the full run and changed files and checks the agent's claims before it scores. It costs more and takes a few minutes per run. Agent mode is in private beta. Email [support@oqoqo.ai](mailto:support@oqoqo.ai) to enable it for your organization. See [Agent mode](https://docs.oqoqo.ai/evaluation/agent-mode.md).

> [accordion] Can I change the rubric after a run?
>
> Yes. Edit the rubric and select **Rerun evals** on the experiment to rescore the same recorded work. If the task itself changed, launch a new experiment. See [Run Evals again](https://docs.oqoqo.ai/evaluation/evals.md#run-evals-again).

> [accordion] How do I read lift?
>
> On **Matrix**, with the **Evals** metric, compare each treatment's cells with the baseline's (chosen at launch) for the same agent and task. See [Pass rate and lift](https://docs.oqoqo.ai/evaluation/results.md#pass-rate-and-lift), and check whether a difference holds up on [Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md).

> [accordion] What is the Brief?
>
> One written sentence about each difference on **Comparisons** that holds up, with the frictions seen with the side that did worse. Oqoqo counts every number from Comparisons and Frictions, and the **Experiment insights model** writes only the words. The Brief never names a cause, and it needs Insights. See [Brief](https://docs.oqoqo.ai/evaluation/brief.md).

> [accordion] Can I see the full trace behind a result?
>
> Yes. Open a run: **Traces** shows the **Timeline** and transcript, **Output** shows the final answer and changed files, **Evals** shows each criterion's verdict, and the **Run profile** rail shows treatment use and where the time went. See [Run details](https://docs.oqoqo.ai/evaluation/traces.md).

> [accordion] Can I compare runs side by side?
>
> Yes. On the **Runs** tab, select two or three runs and select **Compare**. See [Compare runs side by side](https://docs.oqoqo.ai/evaluation/results.md#compare-runs-side-by-side).

> [accordion] Can I export my results?
>
> Yes. A run's **Download** menu saves the trajectory, the output files, or the changes as a diff. The CLI and the plugin read runs, run profiles, trajectories, Evals, Insights, and frictions, and `oqoqo data query` returns rows for your own analysis. See [Run details](https://docs.oqoqo.ai/evaluation/traces.md).

> [accordion] Can I share results with people outside my organization?
>
> Yes. Anyone with a public experiment link can view the results without an account. For a teammate, invite them to the organization instead. See [Share results](https://docs.oqoqo.ai/evaluation/share.md).

## Experiments

> [accordion] Can I start from a template?
>
> Yes. **Home** has templates such as **Getting started**, **Stripe interface evals**, **Latest model shootout**, and **Skill lift**. See [Templates](https://docs.oqoqo.ai/experiments/templates.md).

> [accordion] Can I run an experiment again?
>
> Yes. On a finished experiment, select **Launch again** to run the same tasks, treatments, and agents, or **Edit first** to change something before you launch. See [Launch again](https://docs.oqoqo.ai/experiments/launch.md#launch-again).

> [accordion] Can I run experiments on a schedule?
>
> Yes. Select **Schedule** on a finished experiment, or open **Automate → Schedules**, and set the repeat and time zone. Each launch is a new experiment that uses runs. See [Schedules](https://docs.oqoqo.ai/experiments/schedules.md).

> [accordion] Can I cancel a running experiment?
>
> Yes. Select **Cancel experiment** while it runs. Unfinished runs stop, completed runs stay available, and cancelled runs don't count against your run balance. See [Cancel an experiment](https://docs.oqoqo.ai/experiments/launch.md#cancel-an-experiment).

> [accordion] Can I import skills or files from GitHub?
>
> Yes, from public `github.com` repositories, but not private ones. Use **From GitHub** to import, and **Check for updates** later. See [Tools](https://docs.oqoqo.ai/experiments/tools.md).

> [accordion] Can I organize tasks, tools, and experiments?
>
> Yes. Tag tasks, files, machines, tools, treatments, and experiments, then filter any list with **Tags**. Tags don't change a run. See [Tags](https://docs.oqoqo.ai/experiments/tags.md).

## Ask Oqo, the plugin, and the CLI

> [accordion] What is Ask Oqo?
>
> The in-app assistant for your project: it can create tasks, launch experiments, and explain results. Give it a model from **OpenAI (ChatGPT Subscription)**, **Nous Portal**, or an organization provider under **Organization settings → Oqoqo AI features**. It can't use **Anthropic (Claude Subscription)**. See [Ask Oqo](https://docs.oqoqo.ai/operate/ask-oqo.md).

> [accordion] Can my AI assistant operate Oqoqo?
>
> Yes. With the Oqoqo plugin, your AI assistant (such as Codex, Claude Code, Cursor, or GitHub Copilot) can create tasks and treatments, launch and watch experiments, and read results. Launches use runs, so [turn on tool approval](https://docs.oqoqo.ai/operate/plugin.md#turn-on-tool-approval) in your AI assistant's settings to have it ask you first. See [Install the Oqoqo plugin](https://docs.oqoqo.ai/operate/plugin.md).

> [accordion] Why can't I enter an API key from the CLI or the plugin?
>
> You enter secrets, such as OAuth sign-ins, API keys, and secret headers, only in the dashboard. The CLI, the plugin, and Ask Oqo give you a link to the exact page instead. To list every link, run `oqoqo setup links` or ask for **Get secure setup links** (`get_setup_links`).

> [accordion] Can I use Oqoqo in CI/CD?
>
> Yes: launch and watch experiments from the [CLI](https://docs.oqoqo.ai/operate/cli.md) in your own automation, signed in with a personal API key in `OQOQO_API_KEY`. There is no built-in CI pipeline gate. See [Run an experiment in CI](https://docs.oqoqo.ai/operate/cli.md#run-an-experiment-in-ci).

> [accordion] Where do I manage projects and account settings?
>
> Switch [projects](https://docs.oqoqo.ai/setup/projects.md) in the header, use **Organization settings** for [shared setup](https://docs.oqoqo.ai/setup/organization.md), and use **Profile** for [integrations, API keys, and notifications](https://docs.oqoqo.ai/setup/profile.md). The [Dashboard map](https://docs.oqoqo.ai/reference/dashboard.md) shows where everything is.

## Related

> [card] [Troubleshooting](https://docs.oqoqo.ai/reference/troubleshooting.md)
>
> Fix a specific problem.

> [card] [Quickstart](https://docs.oqoqo.ai/quickstart.md)
>
> Launch your first experiment.

> [card] [Runs and billing](https://docs.oqoqo.ai/setup/billing.md)
>
> See what a run is and how to get more.

> [card] [Glossary](https://docs.oqoqo.ai/reference/glossary.md)
>
> Look up a product term.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.