> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oqoqo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Plugin (MCP) skills

> The four skills in the Oqoqo Plugin: when your AI assistant uses each one, what it does, and which Oqoqo tools it calls.

The [Oqoqo Plugin](https://docs.oqoqo.ai/operate/plugin.md) gives your [AI assistant](https://docs.oqoqo.ai/reference/glossary.md#ai-assistant) four skills. A skill is a set of written instructions that the AI assistant reads when your request matches it. The skills change nothing themselves: they tell the AI assistant which [Oqoqo tools](https://docs.oqoqo.ai/mcp-server/tools.md) to call, in which order, and how to read what comes back.

| Skill | Use it to |
| - | - |
| [`product-eval-authoring`](#product-eval-authoring) | Build a complete eval set from a product's docs: tasks, a machine, files, tools, and treatments. |
| [`product-eval-tasks`](#product-eval-tasks) | Write tasks and fair rubrics from a product's docs, or review an existing rubric. |
| [`product-eval-setup`](#product-eval-setup) | Set up the machine, files, treatments, and test data for a clean baseline and valid parallel runs. |
| [`product-eval-results`](#product-eval-results) | Launch or rerun an experiment, follow it to the end, and read verdicts, frictions, and comparisons. |

Your AI assistant picks a skill from your request. To make sure it uses one, name the skill in your request, for example "Use product-eval-results to compare the treatments in my latest experiment."

## What every skill does

* **Stays in one project.** The AI assistant calls [`get_project_context`](https://docs.oqoqo.ai/mcp-server/identity-and-projects#get_project_context) first and passes the same `project_id` on every call. When your organization has more than one project and you don't name one, it says which project it will use and continues. Name the project in your request to be sure.
* **Stays in one organization.** Every change sends `expected_organization_id`. If someone signs in to another organization while the AI assistant works, the change fails with `organization_scope_changed` and changes nothing ([Organization check](https://docs.oqoqo.ai/mcp-server/overview.md#organization-check)).
* **Leaves secrets to you.** The AI assistant declares each key and OAuth connection by name and gives you the dashboard link where you enter it ([Secrets stay in the dashboard](https://docs.oqoqo.ai/mcp-server/overview.md#secrets-stay-in-the-dashboard)).
* **Keeps working.** When the AI assistant has a question, it says what it will assume, asks, and continues. Reply right away if you want a change.
* **Reads the product docs.** For anything about how Oqoqo works, the skills read `https://docs.oqoqo.ai/llms.txt` and then the pages they need.

## product-eval-authoring

Builds a complete eval set in an Oqoqo project from a product's documentation. Your AI assistant uses it when you point at a product and ask for an eval set, or ask what you should evaluate about your product.

**What it does**

**1. Sets the scope**

It covers the area of the product you name. If you name none, it covers the product broadly, across its documented user jobs.

**2. Reads the product's docs**

It reads an index (such as `llms.txt`) and five or six pages, and it looks for the product's MCP server, CLI, SDK, and published skills on the product's own domain.

**3. Picks task kinds**

It writes at least one task of each kind the product supports: integration code, live interaction (only when the product has an account you can sign up for), and knowledge.

**4. Posts a plan**

Before it creates anything, it posts the scope, each task and its kind, the machine, files, tools, and treatments, and the runs for one trial and for two trials (tasks × treatments × trials, for each agent). Then it continues.

**5. Writes the resources**

It checks the write schema, reads the catalog, and creates the machine, files, tools, treatments, and tasks, in that order, because each one points at the one before it.

**6. Hands it over**

It runs preflight and gives you the link for each key or connection you still have to enter.

**Tools it calls:** [`get_project_context`](https://docs.oqoqo.ai/mcp-server/identity-and-projects#get_project_context), [`inspect_catalog`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#inspect_catalog), [`inspect_project_ai`](https://docs.oqoqo.ai/mcp-server/agents-and-setup#inspect_project_ai), [`author_resource`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#author_resource), [`bulk_author_resources`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#bulk_author_resources), [`create_upload`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#create_upload), [`complete_upload`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#complete_upload), [`get_upload`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#get_upload), [`manage_resource`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#manage_resource), [`preflight_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#preflight_experiment), [`get_setup_links`](https://docs.oqoqo.ai/mcp-server/agents-and-setup#get_setup_links). With the CLI, it uses [`oqoqo resources apply`](https://docs.oqoqo.ai/cli/catalog#oqoqo-resources-apply), [`oqoqo resources bulk-apply`](https://docs.oqoqo.ai/cli/catalog#oqoqo-resources-bulk-apply), and [`oqoqo uploads create`](https://docs.oqoqo.ai/cli/catalog#oqoqo-uploads-create).

**What you get:** a finished eval set in your project, and a list of what is left for you:

1. Enter the keys and connections at the links it gives you.
2. Turn on the agents you want under **Manage → Agents**.
3. Run preflight again.
4. Launch with at least two trials.

**Example prompt:** "Build an Oqoqo eval set for the Acme API from [https://docs.acme.dev](https://docs.acme.dev) in my Payments project."

**Notes**

* It can create MCP servers, CLIs, and SDKs as tools, but it can't sign in to them or enter their keys.
* Preflight doesn't check a CLI or SDK variable that isn't marked required, so the AI assistant lists each one with the page where you enter it.
* It uses `product-eval-tasks` to write the tasks, `product-eval-setup` for the machine and data, and `product-eval-results` when you ask it to launch.

## product-eval-tasks

Turns a product's documentation into tasks with fair rubrics. Your AI assistant uses it when you point at a product's docs, README, or skills repository and ask for tasks, or when you ask it to review or fix a rubric.

**What it does**

* Starts from the example prompts that the product's own docs publish, because they are the work the product claims to support.
* Lists the product's surface areas and gives each one at least one task. The number of tasks comes from the product, not from a target or from your run balance.
* Puts one cheap task first that only reports what the agent can see, so you know the tools reached the run.
* Writes each prompt the way a user would ask, and doesn't name the tool, the steps, or the file to change.
* Builds each task so that an agent with your tools and an agent without them can get different results.
* Writes two to five criteria for each rubric. Each criterion checks one thing that the judge can see in the trajectory or the workspace, and it doesn't require one function, endpoint, or command name.

**Tools it calls:** none. It writes the tasks and rubrics, and `product-eval-authoring` saves them to your project.

**What you get:** tasks ordered from simple to complex, each with its kind, what it needs (files, a machine, an account), and a rubric. It suggests that you run one task on one agent first and read its trajectory.

**Example prompts:** "Write eval tasks for our CLI from its README." · "Review the rubric on task\_08. Is it fair?"

**Notes**

* The judge reads only the trajectory and the files left at the end, never the product's own account. A live interaction task therefore asks the agent for a short written report, and the rubric grades that report.
* Rubric rules are on [Tasks](https://docs.oqoqo.ai/experiments/tasks.md).

## product-eval-setup

Builds the machine, files, treatments, and test data that an experiment runs on. Your AI assistant uses it when you set up an experiment's environment, ask what belongs on a machine or in a treatment, or seed a product account with test data.

**What it does**

* Keeps the product off the machine. The machine attaches to the task, so the baseline gets it too. Every part of the product goes into a treatment as a skill, MCP server, CLI, or SDK.
* Gives each tool its credential in one of three ways: the agent creates its own account during the run, a person enters a key or connects OAuth in the dashboard, or the tool needs none. It marks each key a task can't run without as required, so that preflight blocks the launch until the key is set.
* Plans for runs that happen at the same time. It prefers tasks that only read from a shared account, and gives each task that writes its own target.
* Seeds data so that a shortcut gives the wrong answer, then checks the data after it lands.
* Makes the files and the data agree, for example stack traces that point at real lines in the task's files.

**Tools it calls:** [`get_project_context`](https://docs.oqoqo.ai/mcp-server/identity-and-projects#get_project_context), [`get_setup_links`](https://docs.oqoqo.ai/mcp-server/agents-and-setup#get_setup_links) (`product-eval-authoring` calls [`author_resource`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#author_resource) and [`bulk_author_resources`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#bulk_author_resources) to save them).

**What you get:** a summary that says how each tool gets its credential, and each key you still have to enter, the page where you enter it, and the task it blocks.

**Example prompts:** "Set up a machine for these Next.js tasks." · "Should the Acme CLI be on the machine or in a treatment?"

**Notes**

* It suggests one cheap run on a task that only reports what the agent can see, before you launch the whole experiment.
* Machines are on [Machines and files](https://docs.oqoqo.ai/experiments/machines-and-files.md), and treatments on [Treatments (Toolsets)](https://docs.oqoqo.ai/experiments/treatments.md).

## product-eval-results

Launches an experiment, follows it to the end, and turns its results into conclusions that hold up. Your AI assistant uses it when you ask to launch or rerun an experiment, check on one, compare treatments, explain why runs passed or failed, find out whether agents used a tool, find frictions, or count anything across runs.

**What it does**

* **Launches once.** It says the run count (tasks × treatments × agents × trials) before it launches, suggests at least two trials, runs [`preflight_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#preflight_experiment), and launches with an ID it keeps, so that a retry doesn't spend the runs again.
* **Follows the experiment.** It waits in [`get_experiment_status`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_status) until the experiment finishes, then reads the final runs. It says which Evals and Insights results are still pending.
* **Reads the Brief first.** For "What did the experiment show?", it says the [Brief](https://docs.oqoqo.ai/evaluation/comparisons.md#brief)'s sentence and each result as written, and lists the frictions seen with the side that did worse, with both sides' counts.
* **Compares fairly.** Before it calls one treatment better, it checks [Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md), the runs behind each pass rate, whether the agents used the tool, and each task.
* **Runs a likely difference again once**, after you say to go ahead, and says the run count first.
* **Explains a run with evidence.** It starts from the verdict and reads only the steps of the trajectory that it needs. When a criterion asked for the wrong thing, it suggests a rubric fix and **Rerun evals** on the same recorded work.
* **Schedules.** It previews a schedule and tells you the next launches and the runs in the next 30 days before it creates one.

**Tools it calls**

| Question | Tools |
| - | - |
| Project and catalog context | [`get_project_context`](https://docs.oqoqo.ai/mcp-server/identity-and-projects#get_project_context), [`inspect_catalog`](https://docs.oqoqo.ai/mcp-server/catalog-and-uploads#inspect_catalog), [`list_experiments`](https://docs.oqoqo.ai/mcp-server/experiments#list_experiments) |
| Launch or rerun | [`preflight_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#preflight_experiment), [`launch_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#launch_experiment), [`rerun_experiment`](https://docs.oqoqo.ai/mcp-server/experiments#rerun_experiment), [`get_experiment_plan`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_plan) |
| Follow it | [`get_launch_status`](https://docs.oqoqo.ai/mcp-server/experiments#get_launch_status), [`get_experiment_status`](https://docs.oqoqo.ai/mcp-server/experiments#get_experiment_status), [`list_runs`](https://docs.oqoqo.ai/mcp-server/runs-and-results#list_runs) |
| What it showed | [`get_experiment_brief`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_experiment_brief), [`get_evals_comparisons`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_evals_comparisons), [`analyze_results`](https://docs.oqoqo.ai/mcp-server/runs-and-results#analyze_results) |
| Why a run passed or failed | [`get_evals_results`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_evals_results), [`get_run_trajectory`](https://docs.oqoqo.ai/mcp-server/runs-and-results#get_run_trajectory), [`get_run_transcript`](https://docs.oqoqo.ai/mcp-server/runs-and-results#get_run_transcript), [`get_run_output`](https://docs.oqoqo.ai/mcp-server/runs-and-results#get_run_output) |
| Tool use and frictions | [`get_run_profile`](https://docs.oqoqo.ai/mcp-server/runs-and-results#get_run_profile), [`list_frictions`](https://docs.oqoqo.ai/mcp-server/frictions#list_frictions), [`get_friction`](https://docs.oqoqo.ai/mcp-server/frictions#get_friction), [`get_insights_results`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#get_insights_results) |
| Counts | [`inspect_data_model`](https://docs.oqoqo.ai/mcp-server/tags-and-data#inspect_data_model), [`query_data`](https://docs.oqoqo.ai/mcp-server/tags-and-data#query_data) |
| Rubric fixes | [`preview_evals`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#preview_evals), [`trigger_evals`](https://docs.oqoqo.ai/mcp-server/evals-and-insights#trigger_evals) |
| Schedules | [`preview_schedule`](https://docs.oqoqo.ai/mcp-server/schedules#preview_schedule), [`manage_schedule`](https://docs.oqoqo.ai/mcp-server/schedules#manage_schedule), [`list_schedules`](https://docs.oqoqo.ai/mcp-server/schedules#list_schedules) |

**What you get:** a report in which every number comes from a tool result, with the public IDs of the experiment, runs, and trials it cites. It says what is still pending, what was too small to decide, and what to read next.

**Example prompts:** "Launch my Stripe evals experiment again with two trials." · "Did Stripe CLI do better than Raw Agent in exp\_8f2c41d0, and did the agents use it?"

**Notes**

* A launch has no approval step on the server. [Turn on tool approval](https://docs.oqoqo.ai/operate/plugin.md#turn-on-tool-approval) in your host so that you approve each launch.

## Install, update, or remove the skills

The skills come with the plugin, and they update with it. Install, update, and remove steps for each host are on [Install the Oqoqo Plugin](https://docs.oqoqo.ai/operate/plugin.md). To use the skills with a direct MCP connection or the CLI, see [Add the skills without the plugin](https://docs.oqoqo.ai/operate/plugin.md#add-the-skills-without-the-plugin).

Each skill's full text is a `SKILL.md` file in the public [Oqoqo-Inc/agent-plugin](https://github.com/Oqoqo-Inc/agent-plugin) repository.

## Related

> [card] [Install the Oqoqo Plugin](https://docs.oqoqo.ai/operate/plugin.md)
>
> Install the plugin in your host and sign in.

> [card] [Plugin (MCP) tools](https://docs.oqoqo.ai/mcp-server/tools.md)
>
> Index of all 83 tools by area and matching CLI commands.

> [card] [Tasks and rubrics](https://docs.oqoqo.ai/experiments/tasks.md)
>
> Write tasks and rubrics in the dashboard.

> [card] [Comparisons](https://docs.oqoqo.ai/evaluation/comparisons.md)
>
> Whether a difference holds up, and the Brief.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.