Self Diagnostics - Raindrop

What is Self Diagnostics

Self Diagnostics allows your AI agents to proactively report their own issues back to your team. Agents are exponentially smarter than before. They often understand their own failures better than we do. Imagine your agent keeps failing to install a package, or can’t find something a user is asking for in your docs. With Self Diagnostics, it can just tell you.

Default signal categories

Out of the box, the agent will detect and report four categories of issues:

Category	What it catches
missing_context	Critical information, credentials, or access is missing and the user cannot provide it
repeatedly_broken_tool	A tool has failed or not returned the expected response after multiple attempts.
capability_gap	The task requires a tool, permission, or capability the agent doesn’t have
complete_task_failure	The agent was unable to accomplish the task despite making genuine attempts

You can also define your own custom categories — see Customization below.

Enabling Self Diagnostics

Self Diagnostics is enabled through the SDK when you instrument your agent. Add selfDiagnostics: { enabled: true } to your wrap() call:

const { generateText, streamText } = raindrop.wrap(ai, {
  context: { userId: "user_123", eventName: "support-agent" },
  selfDiagnostics: {
    enabled: true,
  },
});

For full SDK integration details, see:

Customization

You can define your own signal categories and add domain-specific guidance. When you provide custom signals, they replace the defaults entirely:

selfDiagnostics: {
  enabled: true,
  // Copy the SDK defaults, then customize as needed:
  signals: {
    missing_context: {
      description:
        "You cannot complete the task because critical information, credentials, or access is missing and the user cannot provide it. " +
        "Do NOT report this for normal clarifying questions — only when you are blocked.",
      sentiment: "NEGATIVE",
    },
    repeatedly_broken_tool: {
      description:
          "A tool has failed or not returned the expected response on multiple distinct attempts in this conversation, preventing task completion. " +
          "A single tool error is NOT enough — the tool must be persistently broken or aberrantly behaving across retries.",
      sentiment: "NEGATIVE",
    },
    capability_gap: {
      description:
        "The task requires a tool, permission, or capability that you do not have. " +
        "For example, the user asks you to perform an action but no suitable tool exists, or you lack the necessary access. " +
        "Do NOT report this if you simply need more information from the user — only when the gap is in your own capabilities.",
      sentiment: "NEGATIVE",
    },
    complete_task_failure: {
      description:
        "You were unable to accomplish what the user asked despite making genuine attempts. This might be things like, you genuinely do not have the capabilities the user is asking for. You have tried but run into a persistent bug in the environment etc. " +
        "This is NOT a refusal or policy block — you tried and failed to deliver the result.",
      sentiment: "NEGATIVE",
    },
  },
}

The SDK auto-generates the tool prompt from your signal definitions.

Viewing Self Diagnostics in the dashboard

Self Diagnostics reports appear in a dedicated Self Diagnostics tab on the Signals page. From here you can:

Track issue rates over time with signal charts
Drill into individual incidents to see the agent’s description of what went wrong
Set up alerts to get notified when problems spike

How it relates to Signals

Self Diagnostics signals are a special type of Signal. While standard signals are detected by Raindrop’s classifiers or tracked manually via the SDK, Self Diagnostics signals come directly from the agent itself. They appear with signal_type: "agent" and are grouped separately so you can distinguish agent-reported issues from other signal types.

Other ways to report Self Diagnostics

Beyond wrap(), the SDK offers two additional approaches for more advanced use cases:

createSelfDiagnosticsTool() — Create a standalone self-diagnostics tool for agents that don’t use wrap(). See the TypeScript SDK docs.
selfDiagnose() — Manually report a diagnostic from your own code, no LLM tool call needed. See the TypeScript SDK docs.

​What is Self Diagnostics

​Default signal categories

​Enabling Self Diagnostics

​Customization

​Viewing Self Diagnostics in the dashboard

​How it relates to Signals

​Other ways to report Self Diagnostics