MCP or CLI?
MCP or CLI?
- MCP: Best when your AI tool can connect to Braintrust but has no authenticated shell, which is common in chat applications. It also fits when you want an assistant to reason over your Braintrust data and take several connected actions in one conversation, without installing and maintaining a CLI in its execution environment.
btCLI: Best for repeatable work in scripts, CI, local files, and shell pipelines, where you want deterministic commands instead of an assistant’s judgment. Coding agents with shell access can call those commands too.
Connect your client
The server is remote, so there is nothing to install or deploy. Point your client at your MCP endpoint and authenticate with OAuth or an API key.Claude Code
Claude Code
Install Claude Code
Set your API key
BRAINTRUST_API_KEY environment variable with your API key:Add the Braintrust MCP server
Claude Desktop
Claude Desktop
Install Claude Desktop
Add the Braintrust MCP server
- Name:
Braintrust - URL:
https://api.braintrust.dev/mcp
Codex (OpenAI)
Codex (OpenAI)
Install Codex
Set your API key
BRAINTRUST_API_KEY environment variable with your API key:Add the Braintrust MCP server
~/.codex/config.toml and add the Braintrust MCP server configuration:BRAINTRUST_API_KEY environment variable.Verify the setup
/mcp command to verify Braintrust is installed and accessible.Cursor
Cursor
Install Cursor
Add the Braintrust MCP server
.cursor/mcp.json:YOUR_BRAINTRUST_API_KEY with your actual API key.Cursor also supports OAuth authentication. If you omit the headers field, Cursor will prompt you to authenticate via OAuth when you first use the server.VS Code
VS Code
Install VS Code
Install an AI assistant extension
- GitHub Copilot
- Continue
- Other MCP-compatible extensions
Add the Braintrust MCP server
-
Workspace settings - Create or edit
.vscode/mcp.jsonin your project: -
User settings - Add to your VS Code user settings (
Cmd+,/Ctrl+,→ Search for “mcp”):
YOUR_BRAINTRUST_API_KEY with your actual API key.VSCode also supports OAuth authentication. If you omit the headers field, VSCode will prompt you to authenticate via OAuth when you first use the server.Restart VS Code
Cmd+R / Ctrl+R) or restart VS Code to apply the configuration.Devin Desktop
Devin Desktop
Install Devin Desktop
Add the Braintrust MCP server
~/.codeium/windsurf/mcp_config.json and add the Braintrust server:YOUR_BRAINTRUST_API_KEY with your actual API key.Restart Devin Desktop
Gemini CLI
Gemini CLI
Install Gemini CLI
Set your API key
BRAINTRUST_API_KEY environment variable with your API key:Add the Braintrust MCP server
~/.gemini/settings.json and add the Braintrust MCP server configuration:YOUR_BRAINTRUST_API_KEY with your actual API key.Verify the setup
/mcp command to confirm the Braintrust server is connected.Antigravity
Antigravity
Install Antigravity
Open the MCP configuration
mcp_config.json (located at ~/.gemini/config/mcp_config.json).Add the Braintrust MCP server
mcp_config.json:YOUR_BRAINTRUST_API_KEY with your actual API key.Refresh the server list
Zed
Zed
Install Zed
Add the Braintrust MCP server
Cmd+, on macOS / Ctrl+, on Windows/Linux) and add the Braintrust server under context_servers:YOUR_BRAINTRUST_API_KEY with your actual API key.If you omit the headers field, Zed prompts you to authenticate via OAuth when you first use the server.Verify the setup
Amp
Amp
Install Amp
Add the Braintrust MCP server
~/.config/amp/settings.json and add the Braintrust server under amp.mcpServers:YOUR_BRAINTRUST_API_KEY with your actual API key.Verify the setup
amp mcp list to confirm the Braintrust server is connected.OpenCode
OpenCode
Install OpenCode
Add the Braintrust MCP server
YOUR_BRAINTRUST_API_KEY with your actual API key.Restart OpenCode
Warp
Warp
Install Warp
Add the Braintrust MCP server
- Name:
Braintrust - URL:
https://api.braintrust.dev/mcp - Header:
Authorization: Bearer YOUR_BRAINTRUST_API_KEY
YOUR_BRAINTRUST_API_KEY with your actual API key.Verify the setup
Other clients
Other clients
What the MCP can do
Your assistant works with the data and objects in your Braintrust organization, and it chains several tools in one turn. Suppose a support chatbot starts making claims about your product that aren’t true. In a single conversation, your assistant can:- Query recent logs to find examples of the behavior.
- Write an evaluator that detects the unsupported claims, and test it against those traces before saving it.
- Add the failing cases to a regression dataset, with the corrected responses as the expected output.
- Run an eval comparing the current prompt against a proposed fix.
Install the Braintrust SDK
Before you can query anything, your application has to send traces to Braintrust. Your assistant handles that setup: it detects your programming language and frameworks, installs the appropriate SDK, and configures auto-instrumentation. Once complete, it runs your app, verifies traces are being logged, and provides a permalink to view them in Braintrust. Example prompts:Set up tracing from scratch
Set up tracing from scratch
Instrument an existing integration
Instrument an existing integration
Confirm traces are landing
Confirm traces are landing
docs://sdk-install.
Explore your data
Your assistant can answer questions about what your application actually did. It queries logs, experiments, and datasets with SQL, discovers the fields and value distributions in a data source before writing a query, and pulls aggregated metrics for an experiment with or without a baseline to compare against. Because it reads the same data the UI shows, you can investigate a production issue without switching to a browser. When a query returns more than 1 MB,sql_query returns a signed URL to the full result instead of inline rows, so your assistant can work with production-scale results without filling its context window. See Tools for how to control that behavior.
Example prompts:
Find recent errors
Find recent errors
See what fields your data has
See what fields your data has
Break down cost
Break down cost
Compare an experiment to its baseline
Compare an experiment to its baseline
sql_query, infer_schema, summarize_experiment.
Find and share objects
Most Braintrust tools take an object ID, so your assistant finds the right project, experiment, or dataset by name and translates between names and IDs on its own. It usually does this as a step inside a larger request rather than as something you ask for. When you want to hand a result to someone else, it produces a direct link to the object. Example prompts:List recent work in a project
List recent work in a project
Identify a Braintrust URL
Identify a Braintrust URL
list_recent_objects, resolve_object, generate_permalink.
Configure Topics
Topics preprocesses traces into text, extracts facets from that text, and clusters the results to show what your users actually do. Your assistant builds that pipeline for you: it writes a preprocessor that matches your trace shape, tests it against real traces before saving, defines the facets to extract, and enables the automation that keeps it running. It can also rewind an automation over historical traffic.Set up Topics from scratch
Set up Topics from scratch
Write a preprocessor for an unusual trace shape
Write a preprocessor for an unusual trace shape
Process historical traces
Process historical traces
braintrust/topics-workflow skill first, so it validates each stage before saving.
Tools: create_preprocessor, test_preprocessor_on_trace, create_facet, test_facet_on_trace, enable_topics_automation, set_topics_automation, rewind_topics_automation.
Build monitor views
Monitor views collect the charts you check regularly. Your assistant previews a chart against real project logs so you can see it before anything is saved, then puts it into a new or existing view. It can also read back what a view already contains and edit charts in place, one at a time or in bulk. Example prompts:Preview a chart before saving it
Preview a chart before saving it
Create a cost dashboard
Create a cost dashboard
Add a chart to an existing view
Add a chart to an existing view
generate_monitor_chart, list_monitoring_views, get_monitoring_view, create_monitoring_view, update_monitoring_view.
Manage automations and alerts
Automations watch your data so you don’t have to. Your assistant inspects what a project already runs, including online scoring rules, exports, and retention policies, then creates what’s missing: an alert on individual matching logs, an alert on an aggregate threshold over a recent window, an alert on environment updates, or a Loop job that analyzes recent traffic on a schedule. It can also pause and resume any of them. Example prompts:Audit what's configured
Audit what's configured
Alert on an error rate
Alert on an error rate
Schedule recurring analysis
Schedule recurring analysis
Pause an automation
Pause an automation
list_automations, set_automation_status, create_log_alert, create_environment_update_alert, create_threshold_alert, create_scheduled_loop_job.
Author prompts and evaluators
Prompts and evaluators are versioned objects that your application, experiments, and online scoring all share. Your assistant drafts one from what it found in your logs, runs it against real traces to see how it behaves before anything is saved, and saves it as a new version when you’re satisfied. It can then attach an evaluator to an online scoring rule so it scores production logs continuously. Example prompts:Write a scorer for errors you found
Write a scorer for errors you found
Create an LLM-as-a-judge
Create an LLM-as-a-judge
Apply a scorer to production logs
Apply a scorer to production logs
Save a prompt as a new version
Save a prompt as a new version
create_prompt, create_evaluator, test_evaluator, update_online_scoring_rule.
Run evals and edit datasets
Evals, and the datasets that feed them, are how you measure whether a change helps. Your assistant curates dataset rows from the failures it finds in your logs, then runs an experiment against them using a saved or inline task and whichever scorers you want. A prior experiment can supply the input data, in which case its outputs become the expected values. Example prompts:Build a regression dataset from failures
Build a regression dataset from failures
Compare two prompts
Compare two prompts
Re-run last week's inputs
Re-run last week's inputs
run_eval, edit_dataset_rows.
run_eval creates an experiment and can execute your code or call AI providers, so it incurs compute and model usage.Manage project settings
Project settings hold the defaults that other functions inherit. Your assistant reads a project’s typed settings, including which preprocessor facets and other project functions fall back to, and changes that default when you want a new one to apply everywhere. Example prompts:Check the default preprocessor
Check the default preprocessor
Change the default
Change the default
get_project_settings, set_project_default_preprocessor.
Search docs and load skills
Your assistant grounds its answers in Braintrust documentation rather than guesswork, so it can explain a concept or find the right guide without leaving your editor. For multi-step work, it loads a skill first: a workflow guide that tells it the order to do things in and what to validate at each stage. Example prompts:Ask a product question
Ask a product question
Clarify a concept
Clarify a concept
Follow a guided workflow
Follow a guided workflow
search_docs, load_braintrust_skill.
Reference
Endpoints
The MCP endpoint is your Braintrust API URL with/mcp appended, which depends on your organization’s data plane region:
Authentication
The Braintrust MCP server supports two authentication methods:-
OAuth
Clients that support OAuth-based MCP authentication connect without an API key. The server implements OAuth 2.0 with dynamic client registration and publishes its metadata at
/.well-known/oauth-authorization-serveron the same host, so a client can register itself. The first time you use the server, your client opens a Braintrust authorization page where you approve access. -
API key
Clients that don’t support OAuth, along with programmatic clients, send a Braintrust API key as a bearer token on every request:
Tools
Every tool the server exposes, in the order of the capabilities above.sql_query uploads it to object storage and returns an overflow envelope instead of inline rows. The envelope includes an overflow_url (a signed URL to the JSON result), a byte_length, a row_count (when available), and an instructions field describing how to retrieve the full result. Set return_url: true to request a URL even when the result is below the threshold, which is useful when you want to download or save results without putting them in model context. Field values in the result are truncated to preview_length characters (1024 by default). Set preview_length: -1 to include untruncated field values.
Skills
Skills are workflow guides your assistant loads withload_braintrust_skill and then follows. Where a tool reference tells your assistant what a tool does, a skill tells it the order to do things in, what to validate at each stage, and when to ask you for input.
braintrust/topics-workflow- Configure, evaluate, and improve the Topics pipeline, covering preprocessors, facets, scope, and Topics automations.braintrust/evaluator-workflow- Create, test, refine, deploy, and rewind evaluators, and apply them to production logs with an online scoring rule.braintrust/automations-workflow- Set up, validate, and manage alerts and scheduled Loop jobs, including threshold-triggered work, Slack and webhook delivery, and refining existing automations.
braintrust/topics-workflow to be loaded first, so your assistant validates the preprocessor and facets against real traces before it saves anything or enables an automation. Loading a skill is read-only and costs one tool call.
Resources
MCP resources provide contextual documentation that AI assistants can read to perform tasks more effectively.docs://sdk-install- Step-by-step guidance for installing the Braintrust SDK into a project, setting up tracing, configuring auto-instrumentation, and running your first eval.docs://sql- Documentation for thesql_querytool, including syntax, available fields, and examples.docs://url-formats- Reference for Braintrust URL patterns, used by theresolve_objecttool.docs://experiments- Background on Braintrust experiments and how to create them.
docs://sdk-install has companion resources for Python, TypeScript, Go, Java, Ruby, and C#. Your assistant reads the one matching your project automatically.
Troubleshooting
Invalid client errors: Verify the URL is exactlyhttps://api.braintrust.dev/mcp (no trailing slash).
Connection timeouts:
Check internet connection. Corporate networks may need to allowlist api.braintrust.dev and *.braintrust.dev.
MCP server not appearing:
Restart your AI tool and verify JSON configuration syntax.
Server URL errors on a self-hosted deployment:
The MCP server derives its own address from the forwarding headers your ingress sets. If it reports that it could not determine the server URL, set the MCP_SERVER_URL environment variable on your data plane to your API URL.