Self Diagnostics

Self Diagnostics lets your AI agents proactively report their own issues back to your team. Agents often understand their own failures better than we do: if your agent keeps failing to install a package or can’t find something a user is asking for in your docs, it can just tell you, and the report shows up alongside your other signals.

How it works

When you enable Self Diagnostics, the SDK gives your agent a reporting tool and prompt-engineers its description from your category definitions. Whenever the agent hits one of those situations, it reports it, and Raindrop records it as a signal with signal_type: "agent", grouped separately from classifier- and SDK-tracked signals so agent-reported issues don’t get lost in the mix. See Default signal categories for what it catches out of the box.

Using Self Diagnostics

Enable it

Self Diagnostics is enabled through the SDK when you instrument your agent. Add selfDiagnostics: { enabled: true } to your wrap() call:

const { generateText, streamText } = raindrop.wrap(ai, {
  context: { userId: "user_123", eventName: "support-agent" },
  selfDiagnostics: {
    enabled: true,
  },
});

For full SDK integration details, see:

Customize the categories

You can define your own signal categories and add domain-specific guidance. When you provide custom signals, they replace the defaults entirely:

selfDiagnostics: {
  enabled: true,
  // Copy the SDK defaults, then customize as needed:
  signals: {
    missing_context: {
      description:
        "You cannot complete the task because critical information, credentials, or access is missing and the user cannot provide it. " +
        "Do NOT report this for normal clarifying questions — only when you are blocked.",
      sentiment: "NEGATIVE",
    },
    repeatedly_broken_tool: {
      description:
          "A tool has failed or not returned the expected response on multiple distinct attempts in this conversation, preventing task completion. " +
          "A single tool error is NOT enough — the tool must be persistently broken or aberrantly behaving across retries.",
      sentiment: "NEGATIVE",
    },
    capability_gap: {
      description:
        "The task requires a tool, permission, or capability that you do not have. " +
        "For example, the user asks you to perform an action but no suitable tool exists, or you lack the necessary access. " +
        "Do NOT report this if you simply need more information from the user — only when the gap is in your own capabilities.",
      sentiment: "NEGATIVE",
    },
    complete_task_failure: {
      description:
        "You were unable to accomplish what the user asked despite making genuine attempts. This might be things like, you genuinely do not have the capabilities the user is asking for. You have tried but run into a persistent bug in the environment etc. " +
        "This is NOT a refusal or policy block — you tried and failed to deliver the result.",
      sentiment: "NEGATIVE",
    },
  },
}

The SDK auto-generates the tool prompt from your signal definitions.

Report without `wrap()`

Beyond wrap(), the SDK offers two additional approaches for more advanced use cases:

createSelfDiagnosticsTool() — a standalone self-diagnostics tool for agents that don’t use wrap(). See the TypeScript SDK docs.
selfDiagnose() — manually report a diagnostic from your own code, no LLM tool call needed. See the TypeScript SDK docs.

View reports in the dashboard

Self Diagnostics reports appear in a dedicated Self Diagnostics tab on the Signals page. From there you can:

Track issue rates over time with signal charts
Drill into individual incidents to see the agent’s description of what went wrong
Set up alerts to get notified when problems spike

Reference

Default signal categories

Out of the box, the agent detects and reports four categories of issues:

Category	What it catches
missing_context	Critical information, credentials, or access is missing and the user cannot provide it
repeatedly_broken_tool	A tool has failed or not returned the expected response after multiple attempts
capability_gap	The task requires a tool, permission, or capability the agent doesn’t have
complete_task_failure	The agent was unable to accomplish the task despite making genuine attempts

Providing custom signals replaces these defaults entirely (see Customize the categories).

Troubleshooting

Reports aren’t showing up. Confirm selfDiagnostics.enabled is true in your wrap() config and that your agent has run since. Reports land under the Self Diagnostics tab on the Signals page, not the main Signals list. Custom categories replaced the defaults. That’s expected: custom signals replace the four defaults entirely. Re-add any defaults you still want alongside your custom ones.

Getting Started

SDK

Integrations

Platform

Organization

Cookbooks

Workshop

Query API

MCP

Security

Support

How it works

Using Self Diagnostics

Enable it

Customize the categories

Report without `wrap()`

View reports in the dashboard

Reference

Default signal categories

Troubleshooting

​How it works

​Using Self Diagnostics

​Enable it

​Customize the categories

​Report without wrap()

​View reports in the dashboard

​Reference

​Default signal categories

​Troubleshooting

​Related

How it works

Using Self Diagnostics

Enable it

Customize the categories

Report without `wrap()`

View reports in the dashboard

Reference

Default signal categories

Troubleshooting

Related