> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sureops.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tempo

> Connect Tempo for distributed trace search and TraceQL queries during diagnosis

The Tempo integration gives sureops agents access to your Tempo instance for distributed trace search, individual trace retrieval, and TraceQL metric queries — used during diagnosis to identify where in a request path a failure or latency spike originates.

<Note>
  Unlike Prometheus and Loki, Tempo **should be connected even if Grafana is already configured**. The Grafana MCP server does not expose Tempo's full TraceQL surface (trace search, get-trace, and traceql-metrics). Connect Tempo directly to give agents the full distributed tracing capability.
</Note>

***

## What this integration enables

* **Trace search** — agents search for traces matching service name, span attributes, error status, and time range
* **Individual trace retrieval** — agents pull the complete trace waterfall for a specific trace ID to identify the failing span
* **TraceQL metric queries** — agents use Tempo's `traceql_metrics` API to aggregate trace data into rate, error, and duration metrics
* **Multi-tenant support** — if you run a multi-tenant Tempo, configure the Tenant ID to scope queries to the correct org

***

## Prerequisites

* A running Tempo instance reachable from your network
* The `tempo-mcp` server running and reachable from sureops

***

## Setup

<Steps>
  <Step title="Stand up tempo-mcp">
    Deploy `tempo-mcp` pointed at your Tempo instance. If you're using the sureops demo environment, `tempo-mcp` is already provisioned in-cluster — you can skip this step.

    For your own environment, deploy `tempo-mcp` with your Tempo base URL. The sureops MCP setup guide has Helm and Docker Compose examples.
  </Step>

  <Step title="Enter MCP endpoint and token">
    In sureops, go to **Settings → Integrations** and click **Connect** next to Tempo.

    Fill in:

    * **Tempo MCP Endpoint** — the URL of your running `tempo-mcp` server (e.g. `https://tempo-mcp.internal.company.com/mcp`)
    * **MCP Connection Token** — the bearer token used to authenticate sureops to your `tempo-mcp` deployment
    * **Tenant ID** — optional; only required for multi-tenant Tempo deployments. Uses the `X-Scope-OrgID` header. Defaults to `single-tenant`.

    Click **Verify connection** to confirm sureops can reach the MCP server.
  </Step>
</Steps>

***

## Required credentials

| Field                | Required | Description                                                            |
| -------------------- | -------- | ---------------------------------------------------------------------- |
| Tempo MCP Endpoint   | Yes      | URL of your running `tempo-mcp` server                                 |
| MCP Connection Token | Yes      | Bearer token for authenticating sureops to your `tempo-mcp` deployment |
| Tenant ID            | No       | Optional Tempo `X-Scope-OrgID` for multi-tenant deployments            |

***

## Common errors

| Error                  | Fix                                                                                                                                                                |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| "401 Unauthorized"     | The MCP connection token is incorrect or the `tempo-mcp` server doesn't recognize it. Regenerate the token and update both the server config and sureops.          |
| "No traces found"      | Your TraceQL query returned no results. Confirm that your services are emitting traces with `resource.service.name` set and that they are being ingested by Tempo. |
| "Connection timed out" | Your `tempo-mcp` server is not reachable from sureops. Check network accessibility and port configuration.                                                         |

***

## Official documentation

* [Tempo documentation](https://grafana.com/docs/tempo/latest/)
* [TraceQL reference](https://grafana.com/docs/tempo/latest/traceql/)