> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Run Loop on a schedule
> Configure a Loop automation to investigate your project on a schedule, control what it's allowed to change, and send results to Slack or a webhook.
Use Loop automations to investigate project data on a schedule and optionally send results to Slack or a webhook. For example:
* Review cost or latency on a schedule.
* Check whether quality is degrading.
* Summarize recent errors.
* Review logs using the filters from a dashboard.
* Send a Slack or webhook digest after each run.
[Enable Patterns](/docs/observe/patterns/enable) to discover recurring problems and trends your scorers may miss, with supporting traces and suggested next steps. The discovery instruction and write permissions are preconfigured for you.
## Create a Loop automation
Go to ** Settings** > [** Automations**](https://www.braintrust.dev/app/~/configuration/automations), click **+ Automation**, and select **Loop**.
Enter a **Name** to identify the automation in Settings and an optional **Description**, then configure:
* **Instruction:** Describe what to inspect, how deeply to investigate, what to return, and when to stop. Use these techniques to make runs more consistent:
* **Pin the time range.** Use the automation's query window to keep consecutive runs comparable.
* **Specify the output.** Define the format and columns so recurring digests are consistent.
* **Define exclusions.** State what Loop should exclude or avoid inferring, such as guessing names or treating explicitly requested behavior as a failure.
See [Loop automation recipes](#loop-automation-recipes) for complete examples you can adapt.
* **Frequency:** Choose **Daily**, **Weekly**, **Interval**, or **Custom (cron)**. Daily and weekly schedules include a run time and show the next scheduled run. Custom cron uses five fields: minute, hour, day of month, month, and day of week. Select a timezone beside the cron expression. Schedule previews use the schedule's timezone.
* **Agent configuration:** Choose the model, reasoning effort, and whether to use Braintrust's built-in models or your own [AI providers](/docs/admin/ai-providers). Built-in models use your monthly [model credits](/docs/plans-and-limits#model-credits). Other models bill to their providers. Cost depends on the model and how much data each run inspects, so keep instructions focused on a narrow slice of traces. See [Models and providers](/docs/loop/manage#models-and-providers) and [automatic pausing](#pause-or-delete-an-automation).
* **Default query range:** Choose 3 hours, 6 hours, 12 hours, 1 day, 3 days, 7 days, or a custom range. Each run inspects the full range, not only traces created since the previous run.
In an interactive thread, actions that change something pause for your approval. A scheduled run has nobody to ask, so it relies on its **Write tool permissions** instead.
* Read-only tools never require approval, so a run can always inspect project data such as traces, existing patterns, experiments, and datasets.
* Automations created from the [** Patterns**](https://www.braintrust.dev/app/~/patterns) page can create and update pattern records by default. A pattern record can include trace evidence, monitor chart configurations, and recommendations for measuring the pattern later. Creating scorers, facets, or other project objects requires the corresponding write tools.
* To let a run create or modify other project objects, add the corresponding tools. The dropdown lists what each one permits.
Enabling a tool lets scheduled runs make that kind of change without asking. Choose only the tools the automation needs.
Allowing a run to create or update online scoring rules can start work that continues after the run ends. Online scoring rules score production traffic continuously and keep costing money until you stop them.
Objects a run creates are not linked back to any pattern it recorded. If a pattern's monitor chart measures a scorer or facet the run created, deleting that object stops new data from reaching the chart.
Destinations are optional. Every run's results are available in Braintrust on the automation's [**Past runs**](#review-past-runs) tab, and any patterns it records appear in the [** Patterns**](https://www.braintrust.dev/app/~/patterns) list. Add a destination when you also want each run pushed to your team.
Click **+ Destination** in the configuration panel, then choose **Send to webhook** or **Send to Slack**.
* **Slack.** Select a channel to post a text digest. This requires a [connected Slack workspace](/docs/admin/organizations#enable-slack-integration). Every message opens with the project name in bold on its own line. Use the destination's **Formatting prompt** to tell Loop how to format the rest of the message.
* **Webhook.** Enter a URL to send run results to an external service. Use the **Formatting prompt** to describe the JSON payload Loop should produce.
If no Slack workspace is connected, the destination menu depends on your organization permissions:
* **[Owners](/docs/admin/access-control) or a custom permission group with Manage settings:** Click **Connect to Slack** to connect a workspace in a new tab. When you return, the workspace list refreshes and the menu offers **Send to Slack**.
* **Without those permissions:** The menu shows **Ask an org admin to connect Slack**.
Destinations belong to automation runs. Patterns that Loop records during an interactive chat appear in the Patterns list but do not trigger Slack or webhook destinations.
Click **Create** to save the automation.
To run a saved automation immediately, open it and click **Run now**. Braintrust queues the run and shows a link to the Loop thread when it's ready.
## Loop automation recipes
Adapt these recipes to review cost and quality, summarize errors, or discover patterns. Each includes a schedule, query range, write permissions, and an instruction you can customize for your project.
Posts a cost breakdown to Slack every Monday morning.
| Field | Value |
| -------------------------- | ------------------------------------- |
| **Frequency** | Weekly, Mondays at 9:00 AM |
| **Default query range** | 7 days |
| **Write tool permissions** | None. The run only reads and reports. |
| **Destinations** | Send to Slack |
```text wrap Instruction theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
Create a weekly estimated-cost report for this project.
Periods:
- Use the automation's seven-day window as the current period.
- Compare it with the immediately preceding seven days.
- State both UTC date ranges in the report.
Cost by user:
- Query project logs separately for each period.
- Group by `span_attributes.created_by_user_id` and sum `metrics.estimated_cost`.
- Keep null creator IDs as `Unattributed`, and never guess a name.
Return only:
1. A Markdown table with columns `User`, `Past 7 days`, `Prior 7 days`, `Change`.
2. One sentence summarizing the total week-over-week change.
3. A short paragraph on which work drove the spend, based on the highest-cost
traces in the window. Describe the tasks, not the people.
```
Compares existing quality scores with the previous day and investigates possible regressions. This example uses `scores.factuality`, with higher values indicating better quality. Replace it with a score your project already records and adjust the direction if lower values are better.
| Field | Value |
| -------------------------- | ------------------------------------- |
| **Frequency** | Daily |
| **Default query range** | 1 day |
| **Write tool permissions** | None. The run only reads and reports. |
| **Destinations** | Send to Slack |
```text wrap Instruction theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
Review factuality in this project's logs for signs of degrading quality.
Compare:
- Use the automation's one-day window and the immediately preceding day.
- State both UTC date ranges. For each period, report the average of
`scores.factuality` and the number of scored rows. Higher scores are better.
- Exclude missing scores from the average and report how many rows lack scores.
- If either period has no scores, report that the comparison is unavailable.
Investigate:
- If the average falls, inspect up to 10 low-scoring traces from the current
period and up to 10 scored traces from the previous period.
- Check whether the tasks or traffic mix changed. Distinguish observed score
changes from possible causes, and do not treat a small sample as proof.
- Use existing scores only. Do not run scorers or create scoring rules.
Return a comparison table, a short summary of the evidence and its limitations,
and up to three representative trace links with suggested follow-up checks.
```
Summarizes errors in the same subset of logs you monitor on a [dashboard](/docs/observe/dashboards). Replace `metadata.environment = 'production'` with the filter conditions from your dashboard. Include the conditions in the instruction so the run has the scope it needs.
| Field | Value |
| -------------------------- | ------------------------------------- |
| **Frequency** | Daily |
| **Default query range** | 1 day |
| **Write tool permissions** | None. The run only reads and reports. |
| **Destinations** | Send to Slack |
```text wrap Instruction theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
Summarize recent errors for the production environment in this project.
Scope:
- Use the automation's one-day window and state the UTC date range.
- Select traces matching `metadata.environment = 'production'` and inspect
their spans for recorded errors. If the field is unavailable, report that
the scope cannot be applied instead of broadening the search.
Investigate:
- Group similar recorded errors and count distinct affected root traces in
each group. Do not count repeated errors in one trace as separate incidents.
- Inspect up to three representative traces from each of the five largest
groups. Check whether the operation recovered later in the trace.
- Distinguish recorded errors from confirmed failures visible to the end user.
If the outcome is unclear, say so. Do not guess a root cause.
Return a table with each error group, its affected trace count, and a
representative trace link, followed by suggested follow-up checks.
If there are no recorded errors in scope, say so rather than inventing groups.
```
Narrows pattern discovery to one concern, using an existing [facet](/docs/observe/topics/custom-facets) to choose which traces to investigate. Run this alongside the default **Pattern discovery** automation when a specific area deserves its own cadence.
| Field | Value |
| -------------------------- | -------------------------- |
| **Frequency** | Daily |
| **Default query range** | 7 days |
| **Write tool permissions** | Create and update patterns |
| **Destinations** | Optional |
```text wrap Instruction theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
Analyze recent project traces to find recurring failures within the existing
"Primary failure mode" facet's topic classifications.
Use the facet and its topic classifications only to select candidates. They are
noisy, and are not evidence by themselves.
Workflow:
1. Use the automation window.
2. Collect a sample of 100-200 facet occurrences, spread across the available
topic classifications rather than only the newest or largest cluster.
3. Cap the sample at two occurrences per root trace, and deduplicate repeated
facet runs and copies of the same underlying conversation.
4. For each candidate, inspect the compact context around the occurrence: the
user request, agent actions, tool results, and outcome.
5. Validate every proposed pattern against raw trace evidence. Reject any
hypothesis explained by explicit user direction, a permission or environment
block, or intentional fail-fast behavior.
Record a pattern only when it describes a specific recurring behavior, is
supported by at least three independent conversations, has a meaningful impact
and an actionable response, and includes short quotes and relevant span IDs.
```
## Review past runs
Each run creates a read-only thread recording the work Loop did. To find it, open the automation, select the **Past runs** tab, and select a run.
These threads stay with their automation and don't appear in the thread list on the [** Loop**](https://www.braintrust.dev/app/~/loop) page. They also can't be deleted the way an interactive thread can.
To ask follow-up questions from a run, select **Continue**. This forks the read-only transcript into a private, editable thread, which is the fastest way to work out why a run behaved the way it did.
## Pause or delete an automation
Pause an automation to stop its scheduled runs while keeping its configuration and history. Do this when you're tuning an instruction, controlling spend, or investigating a run that behaved unexpectedly. When you're ready, you can re-enable the automation.
To pause, open the automation and turn off **Active**. The rule then shows as **Paused** in ** Settings** > [** Automations**](https://www.braintrust.dev/app/~/configuration/automations). Turn **Active** back on to resume the schedule.
An automation configured to use built-in models does not switch to your own AI providers when its model credits are exhausted.
An automation also shows as **Paused** automatically when Braintrust detects at read time that it cannot run. This happens when your model credits are depleted, the selected built-in model is unavailable, or built-in models are disabled for your organization. The automation list marks it with a warning indicator, and opening the automation shows a banner with the reason and a link to the relevant settings.
To remove an automation entirely, delete the rule from ** Settings** > [** Automations**](https://www.braintrust.dev/app/~/configuration/automations).
Deleting an automation is permanent. If you might want the automation again, pause it instead.
Neither pausing nor deleting an automation affects the objects its runs already created. Patterns, scorers, facets, and other objects a run produced remain in your project, and any online scoring rules a run created keep running until you stop them separately.
## Next steps
* [Enable Patterns](/docs/observe/patterns/enable) for scheduled pattern discovery.
* [Set up alerts](/docs/observe/alerts) for conditions you can already define, without an agent run.
* [What Loop can do](/docs/loop/capabilities) to decide what's worth automating.