> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# What Loop can do

> Query logs, diagnose traces, write scorers, run experiments, and build dashboards through natural language, with example prompts for every task.

Loop can find a failure in your logs, write a scorer for it, and open that scorer for review in a single turn. It combines tools for playgrounds, custom views, dataset snapshots, and the workspace with the [Braintrust MCP](/docs/integrations/developer-tools/mcp) server's read and write tools.

Explore its capabilities and example prompts by task: investigate data, build objects, run evaluations and automations, or get help.

## Investigate

Ask questions about your data, from a single trace up to every project in your organization.

<h3 id="explore-your-data">
  <Icon icon="search" /> Explore your data
</h3>

Loop queries logs, experiments, and datasets with SQL and inspects their schemas. For large or complex traces, it can read, search, and list sandbox files or run commands. Open any trace it resolves or links to in the workspace beside the thread.

Loop can also:

* **Hand off to the SQL sandbox.** Loop can write and run a query in the [**<Icon icon="asterisk" /> SQL sandbox**](https://www.braintrust.dev/app/~/sql), then [refine or fix it](/docs/reference/sql#loop) on request. The query stays there for you to edit.
* **Start from the logs table.** Loop can [turn a question into a SQL filter](/docs/observe/filter#ask-questions-with-loop) and [find traces similar](/docs/observe/filter#ask-questions-with-loop) to your selection.
* **Resolve identities.** Loop resolves `span_attributes.created_by_user_id` to organization member names and emails, and `span_attributes.created_by_api_key_id` to API key and service token names, so you can ask about people and keys by name instead of by ID.

<Note>
  Fields you define in `metadata`, including ones named like `metadata.user_id`, describe your own application's users. Loop doesn't read them as Braintrust members.
</Note>

Example prompts:

<AccordionGroup>
  <Accordion title="Summarize logs and suggest next steps">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Look through my logs, explain what you find, and recommend the best next steps for my project.
    ```
  </Accordion>

  <Accordion title="Break down cost by model">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me the cost of traces named X over the last week, broken down by model. Include total, average, p50, p95, and p99 cost.
    ```
  </Accordion>

  <Accordion title="Graph caching and usage metrics">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Find the metrics related to prompt caching and LLM usage, then graph cache-hit rate and the number of LLM calls per trace.
    ```
  </Accordion>

  <Accordion title="Hand off a query to the SQL sandbox">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Write that query into the SQL sandbox and run it there so I can keep refining it.
    ```
  </Accordion>

  <Accordion title="Attribute traces to teammates">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Which teammates logged the most traces this week, and what did each one cost?
    ```
  </Accordion>
</AccordionGroup>

<h3 id="analyze-traces">
  <Icon icon="sparkles" /> Analyze traces
</h3>

From the trace view, Loop can diagnose a failure or summarize work across LLM and tool calls. See [Debug a trace](/docs/observe/debug-traces).

To ask about a trace in a Loop thread, paste its permalink or ID. Loop resolves it and loads its spans.

Example prompts:

<AccordionGroup>
  <Accordion title="Understand what happened">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Summarize this trace. What was the total latency, and where was the time spent?
    ```
  </Accordion>

  <Accordion title="Find related failures">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Are there other traces with similar errors? Show me what they have in common.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="find-patterns">
  <Icon icon="scan-search" /> Find patterns
</h3>

Loop searches logs and raw traces for recurring problems and trends, even without a scorer measuring them, and records them as [patterns](/docs/observe/patterns).

Each pattern includes supporting evidence. Loop updates an existing pattern when it finds related evidence, rather than creating a duplicate.

Example prompts:

<AccordionGroup>
  <Accordion title="Record a failure you suspect">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    I think our tool calls are failing more often since last week. Dig into it, and if you find something, record it as a pattern.
    ```
  </Accordion>

  <Accordion title="Turn an investigation into a tracked pattern">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Look through this project's recent logs, find a recurring failure, and record it as a pattern with the supporting traces.
    ```
  </Accordion>

  <Accordion title="Ask how a pattern was found">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    How did you derive this pattern, and what would make it more accurate?
    ```
  </Accordion>
</AccordionGroup>

<Note>
  Loop can also find patterns on a schedule, without being asked. See [Enable Patterns](/docs/observe/patterns/enable).
</Note>

<h3 id="analyze-projects">
  <Icon icon="house" /> Analyze projects
</h3>

Loop can investigate a whole project or compare projects across your organization.

For a single project, open <Icon icon="blend" /> **Loop agent** at the bottom right of its **<Icon icon="house" /> Overview** page and enter a prompt. Loop can inspect the project's logs, recent experiments, datasets, prompts, and score progress chart.

Example prompts:

<AccordionGroup>
  <Accordion title="Track a score over time">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What's the trend in my accuracy score over the last 7 days? Which scorer has been improving the most?
    ```
  </Accordion>

  <Accordion title="Find the experiments that moved">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Which experiments had the biggest score changes?
    ```
  </Accordion>

  <Accordion title="Check recent activity">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me recent logs with errors, and tell me what datasets were created recently.
    ```
  </Accordion>

  <Accordion title="Watch token usage">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    How has my project's token usage changed over time?
    ```
  </Accordion>
</AccordionGroup>

To compare projects, select <Icon icon="blend" /> **Loop agent** from the organization-level [projects list](https://www.braintrust.dev/app). Ask about project metrics, usage, or performance trends.

Example prompts:

<AccordionGroup>
  <Accordion title="Rank projects by usage">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Which projects have the highest token usage, and which have the most experiments?
    ```
  </Accordion>

  <Accordion title="Compare performance across projects">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Compare latency metrics across my top 5 projects.
    ```
  </Accordion>

  <Accordion title="Spot trends in volume">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me trends in log volume across projects, and which projects have declining usage.
    ```
  </Accordion>

  <Accordion title="Audit recent projects">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What projects were created in the last month?
    ```
  </Accordion>
</AccordionGroup>

## Build

Turn findings into saved Braintrust objects. Loop asks for approval before creating or changing them.

<h3 id="author-prompts-and-evaluators">
  <Icon icon="triangle" /> Author prompts and evaluators
</h3>

Loop can suggest, create, and save evaluators based on your application's behavior. A [scorer](/docs/evaluate/write-scorers) returns a number, and a classifier returns a label. New and updated evaluators open in the workspace for review.

Loop can also create and update prompts, including [revising playground prompts from your annotations](/docs/evaluate/playgrounds#annotate-outputs) on individual outputs.

Example prompts:

<AccordionGroup>
  <Accordion title="Suggest scorers for my project">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Suggest useful scorers for my project.
    ```
  </Accordion>

  <Accordion title="Write a scorer for known errors">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Write a scorer that detects the errors I just identified, then create and test it on a few traces.
    ```
  </Accordion>

  <Accordion title="Create a helpfulness judge">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Create an LLM-as-a-judge scorer for helpfulness based on these logs.
    ```
  </Accordion>

  <Accordion title="Set up online scoring">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Set up online scoring with the scorer I just created, and verify it runs automatically on new traces.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="curate-datasets">
  <Icon icon="database" /> Curate datasets
</h3>

Loop can [generate datasets](/docs/annotate/datasets/create#generate-with-loop) from log patterns and edit existing rows. Edits open in a read-only workspace table that refreshes as Loop works. Select a row to inspect its trace inline.

Loop can also manage dataset versions and the environments that point at them:

* **Snapshots.** Save a [snapshot](/docs/annotate/datasets/manage#save-snapshots) before editing, or restore one to undo edits. Loop can list snapshots, their environment tags, and the current head version. Restoring uses the dataset UI's confirmation flow and can save the current state first.
* **Environments.** Point an [environment](/docs/deploy/environments), such as `production` or `staging`, at a snapshot or clear its mapping. Loop can list your organization's environment slugs and create missing environments.

<Warning>
  Restoring a dataset to a snapshot replaces its current rows, and retagging an environment changes which dataset state your applications resolve. Review what Loop proposes before you approve either action.
</Warning>

Example prompts:

<AccordionGroup>
  <Accordion title="Generate a dataset from errors">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Generate a dataset from logs with errors.
    ```
  </Accordion>

  <Accordion title="Add traces to a dataset">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Add these traces to my regression dataset and set the expected output.
    ```
  </Accordion>

  <Accordion title="Snapshot before editing">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Snapshot this dataset before you change anything, then remove the rows that no longer reflect our current behavior.
    ```
  </Accordion>

  <Accordion title="Review and restore a version">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What snapshots exist for this dataset, and which one is production pointing at? Restore the one from before yesterday's edits.
    ```
  </Accordion>

  <Accordion title="Promote a dataset version">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Point the staging environment at the snapshot you just saved, and tell me which environments exist first.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="build-views-and-dashboards">
  <Icon icon="chart-no-axes-column" /> Build views and dashboards
</h3>

Loop can create and edit [custom views](/docs/annotate/custom-views) for traces and dataset rows, and [build dashboard charts](/docs/observe/dashboards/build-charts). Open new or updated dashboards in the workspace to review them beside the thread.

Example prompts:

<AccordionGroup>
  <Accordion title="Suggest trace visualizations">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What are some useful trace visualizations you can make for me? List the options and I'll tell you which ones to create as custom trace views.
    ```
  </Accordion>

  <Accordion title="Build a custom dataset view">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Build a custom view for this dataset that renders the input and expected output side by side.
    ```
  </Accordion>

  <Accordion title="Build a daily cost dashboard">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Create a dashboard for daily cost analysis.
    ```
  </Accordion>

  <Accordion title="Investigate an error spike">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    This topic or error spike looks interesting. Show me which customers and interactions are affected, inspect representative traces, and explain the likely root cause.
    ```
  </Accordion>

  <Accordion title="Share a dashboard with your team">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Turn this Loop analysis into a shareable dashboard my team and executives can use to understand the result and monitor it over time.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="configure-topics">
  <Icon icon="pentagon" /> Configure Topics
</h3>

Give Loop a goal to configure the [Topics](/docs/observe/topics) pipeline:

* **Check the preprocessor.** Loop tests your project's effective default [preprocessor](/docs/observe/topics/enable#write-a-custom-preprocessor) on real traces for usable conversation text. If it returns nothing, Loop writes and tests a custom preprocessor for your trace shape and can make it the project default.
* **Confirm the scope.** Loop looks for a grouping key, such as `metadata.conversation_id`, and recommends analyzing a span, trace, or group of related traces.
* **Validate the facets.** Loop tests a facet on up to ten representative traces and shows results inline for your feedback. Reuse that trace set as you refine the prompt. For larger sets, Loop runs an eval.
* **Evaluate candidates.** Loop compares candidate facet prompts in evals before saving them.
* **Enable the automation.** Loop creates the facets, enables the Topics automation, and offers to process a historical window.

<Note>
  Changing the project default preprocessor affects every function that uses it, including built-in Topics facets. Loop asks for approval before changing it.
</Note>

<Note>
  Rewinding a Topics automation processes historical traces and draws from your monthly [model credits](/docs/plans-and-limits#model-credits). Loop estimates the volume and asks for approval before it rewinds.
</Note>

Example prompts:

<AccordionGroup>
  <Accordion title="Set up Topics from scratch">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Set up Topics for my project.
    ```
  </Accordion>

  <Accordion title="Configure Topics with a custom preprocessor">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Help me configure Topics with a custom preprocessor. My traces don't store conversation text on LLM spans.
    ```
  </Accordion>

  <Accordion title="Discover error categories">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    I want to discover the kinds of errors my users hit. Build a facet for that, validate it on real traces, and enable it.
    ```
  </Accordion>
</AccordionGroup>

## Run and automate

Test changes with evaluations, then schedule recurring checks or alerts.

<h3 id="run-evaluations">
  <Icon icon="beaker" /> Run evaluations
</h3>

Loop runs evaluations two ways:

* **In a playground or experiment.** Loop can edit the task prompt or code, add or modify scorers, and re-run the evaluation to show the effect of a change.
* **As a new experiment.** Loop can run an experiment from a dataset, from rows it supplies inline, or from a prior experiment, using a saved or inline task and any scorers you choose. When a prior experiment supplies the data, its outputs become the expected values.

Evaluation runs open in the workspace's **Experiments** panel, where you can compare progress and scores without leaving the chat.

Example prompts:

<AccordionGroup>
  <Accordion title="Build a dataset and run an experiment">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Build a dataset from the traces where my agent gave a wrong answer, then run an experiment on it with my current prompt and scorers, and show me which cases still fail.
    ```
  </Accordion>

  <Accordion title="Evaluate an agent decision">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Evaluate whether this agent made the right decision, but only use the relevant messages, reasoning, and tool calls from these very large traces.
    ```
  </Accordion>

  <Accordion title="Compare a cheaper model">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Evaluate whether we can switch this workflow to a cheaper candidate model. Compare quality, cost, and latency, and tell me whether it is safe to roll out.
    ```
  </Accordion>
</AccordionGroup>

<Note>
  Loop runs evaluations in Braintrust rather than by writing an `Eval()` script. To keep evaluations in your own repository and run them in CI, see [Run evaluations in code](/docs/evaluate/run-in-code).
</Note>

<h3 id="manage-automations-and-alerts">
  <Icon icon="radio" /> Manage automations and alerts
</h3>

Turn an investigation into an alert or a scheduled check:

* **Alerts.** Create an [alert](/docs/observe/alerts) for matching logs, aggregates crossing a threshold over a recent window, or environment updates. Loop generates the filter from your description.
* **Automations.** Create a [scheduled Loop job](/docs/loop/automations) with its own instructions and schedule. Loop can also list existing online scoring rules, alerts, exports, retention policies, and Topics automations with their full configurations.
* **Status.** Pause an automation or make it active again.
* **Updates.** Rewrite an alert or Loop automation under its existing name. Scheduled exports and retention policies are inspect-only.

<Warning>
  Rewriting an automation replaces its entire configuration. Ask Loop to read the current configuration first, then review the proposed replacement before approving it.
</Warning>

You can also edit any automation directly in **<Icon icon="settings-2" /> Settings** > [**<Icon icon="radio" /> Automations**](https://www.braintrust.dev/app/~/configuration/automations).

Example prompts:

<AccordionGroup>
  <Accordion title="Alert on the errors you just found">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Create an alert that notifies me when the error you just found shows up in my logs again.
    ```
  </Accordion>

  <Accordion title="Alert on a quality regression">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Alert me when the average score for this scorer drops below 0.8 over any six hour window.
    ```
  </Accordion>

  <Accordion title="Alert on cost growth">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Watch my daily spend and alert me when it goes more than 20% above last week's average.
    ```
  </Accordion>

  <Accordion title="Audit what's configured">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What automations are configured in this project, and which ones are paused?
    ```
  </Accordion>

  <Accordion title="Work out why a scheduled run found nothing">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me the configuration for my pattern discovery automation, then explain why yesterday's run didn't record anything.
    ```
  </Accordion>
</AccordionGroup>

## Get help

Find documentation and objects, or contact support through Loop.

<h3 id="search-docs-and-find-objects">
  <Icon icon="book-open" /> Search docs and find objects
</h3>

Loop can search the Braintrust documentation, resolve object names and URLs, list recent objects, summarize an experiment's metrics against a baseline, and generate shareable links to objects it creates.

Example prompts:

<AccordionGroup>
  <Accordion title="Look up how to set up human review">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    How do I set up human review?
    ```
  </Accordion>

  <Accordion title="Compare experiments">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Summarize my latest experiment compared to the previous one.
    ```
  </Accordion>

  <Accordion title="Get a shareable link">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Give me a shareable link to the dashboard you just made.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="request-support">
  <Icon icon="life-buoy" /> Request support
</h3>

Ask Loop for help, or type `/` and choose the support command, to file a ticket with Braintrust. Loop guides you through the request and includes relevant context from your page and conversation.

<h2 id="next-steps">
  Next steps
</h2>

* [Run Loop on a schedule](/docs/loop/automations) to turn any of this into recurring work.
* [Enable Patterns](/docs/observe/patterns/enable) for automatic discovery of recurring problems.
* [Manage threads and limits](/docs/loop/manage) for sandbox behavior and cleanup.
* [What Loop can't do](/docs/loop#what-loop-cant-do) for the boundaries it works inside.