> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sureops.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy identity & recovery cooldown

> Configure the service account sureops uses to execute remediation actions, and set minimum wait times between recovery attempts

When sureops executes a resolution action — restarting a deployment, triggering a rollback, scaling a resource — it needs an identity with permission to do so in your infrastructure. That identity is called the **deploy identity**. The recovery cooldown controls how long sureops waits before attempting another automated recovery on the same service.

***

## What is a deploy identity?

The deploy identity is a service account or credential set that sureops uses when executing approved remediation actions. It is separate from the identity used to read observability data — deploy identity has write access, so it is more sensitive and is configured explicitly per environment.

Examples of what the deploy identity does:

* Restarts a Kubernetes deployment (`kubectl rollout restart`)
* Triggers a rollback via your GitOps operator
* Scales a deployment up or down
* Executes a runbook action against your infrastructure

Without a configured deploy identity, sureops can diagnose and recommend — but it cannot execute fixes autonomously. Resolution actions will remain in a **pending** state until a human performs them manually or until an identity is configured.

***

## Prerequisites

Before configuring a deploy identity:

* You need a service account in your infrastructure with the minimum permissions required for your resolution actions (see [Principle of least privilege](#principle-of-least-privilege) below)
* Your Admin must have access to **Settings → Environments → \[environment] → Deploy Identity**

***

## Configuring a deploy identity

Deploy identity is configured per environment. Each environment can have its own identity with its own permission scope.

<Steps>
  <Step title="Create a service account in your infrastructure">
    Create a dedicated service account for sureops in your infrastructure (Kubernetes service account, cloud IAM role, etc.). Do not reuse a developer account or a human account — service accounts are easier to audit, rotate, and revoke independently.

    Grant it only the permissions it needs for your resolution actions. See [Principle of least privilege](#principle-of-least-privilege).
  </Step>

  <Step title="Open Settings → Environments → Deploy Identity">
    In sureops, navigate to **Settings → Environments**, select the target environment, and open the **Deploy Identity** tab.
  </Step>

  <Step title="Enter the credentials">
    The credential format depends on how the environment is connected:

    * **Kubernetes environments** — provide a kubeconfig or service account token scoped to the target cluster and namespace
    * **Cloud provider environments** — provide an IAM role ARN or access key with deploy permissions
    * **Custom environments** — provide whatever credentials your MCP integration requires for write access

    Credentials are stored encrypted and are never exposed in logs or the audit trail (only the identity name and last-used time are visible).
  </Step>

  <Step title="Test the identity">
    Click **Test connection** to verify sureops can authenticate and reach your infrastructure with the deploy identity. sureops performs a read-only preflight check — it does not execute any changes during the test.
  </Step>

  <Step title="Save">
    Click **Save**. The deploy identity is active for new resolution actions in this environment from this point forward.
  </Step>
</Steps>

***

## Principle of least privilege

The deploy identity should have only the permissions it needs — nothing more. A good starting set for a Kubernetes-managed environment:

| Permission                        | Why                                 |
| --------------------------------- | ----------------------------------- |
| `deployments.apps/update`         | Restart or scale deployments        |
| `deployments.apps/rollback`       | Roll back to a previous revision    |
| `pods/delete`                     | Evict stuck or crashlooping pods    |
| `horizontalpodautoscalers/update` | Adjust HPA min/max during incidents |

Avoid granting cluster-admin or wildcard permissions. If a resolution action requires a permission you haven't granted, sureops will report the specific error — you can expand permissions incrementally.

***

## Recovery cooldown

The recovery cooldown is a minimum wait time between automated recovery attempts for the same service. It prevents sureops from rapidly retrying a failed remediation in a tight loop, which could worsen an incident (for example, repeated pod restarts that exhaust restart limits).

### Default behavior

By default, sureops will not attempt a second automated recovery on the same service within **10 minutes** of the first attempt. If the first attempt fails or verification fails, the incident is escalated for human review rather than retried immediately.

### Configuring cooldown

Cooldown is configured per environment in **Settings → Environments → \[environment] → Recovery Cooldown**:

| Setting          | Default     | Description                                                     |
| ---------------- | ----------- | --------------------------------------------------------------- |
| Minimum cooldown | 10 minutes  | Minimum wait between recovery attempts for the same service     |
| Maximum retries  | 2           | Number of automated retry attempts before escalating to a human |
| Cooldown scope   | Per-service | Whether cooldown is tracked per service or per incident         |

<Note>
  Setting the minimum cooldown to zero disables the protection. This is not recommended for production environments — only consider it for development environments where rapid retries are acceptable.
</Note>

### Why cooldown matters

Without a cooldown, an automated resolution loop can cause more harm than good:

* A restart attempt that fails may trigger a CrashLoopBackOff — repeated restarts exhaust Kubernetes restart limits and delay the eventual human intervention
* Rapid retries can generate noise in your observability stack, making it harder to distinguish recovery from continued failure
* Some fixes (like a database migration) need time to propagate before a verification check is meaningful

The cooldown gives your infrastructure time to stabilize between attempts and ensures a human sees the situation before it spirals.

***

## Rotating credentials

Service account credentials should be rotated periodically. To rotate:

1. Generate new credentials in your infrastructure
2. Navigate to **Settings → Environments → \[environment] → Deploy Identity**
3. Replace the stored credentials with the new ones
4. Click **Test connection** to verify
5. Click **Save**

Immediately after saving, revoke the old credentials in your infrastructure. There is a brief overlap window during which both credentials are valid — keep this window as short as possible.

***

## Revoking access

To revoke sureops's deploy access to an environment:

1. Navigate to **Settings → Environments → \[environment] → Deploy Identity**
2. Click **Remove deploy identity**
3. Revoke the service account credentials in your infrastructure

After removal, resolution actions in that environment stay in **pending** state for manual execution. Diagnostic and read-only actions continue to work normally.

***

## Audit trail

Every action taken by the deploy identity is logged in the incident timeline and the audit log:

* What command was executed
* Which deploy identity was used (by name — credentials are never logged)
* When it ran
* Whether it succeeded or failed, with the error message if applicable

Admins and Owners can view the full audit trail in **Settings → Audit**.

***

## Related

* [Roles & Capabilities](/next/administration/roles-and-capabilities) — Admin role required to configure deploy identity
* [Notifications](/next/administration/notifications) — get alerted when a recovery attempt fails or times out