Skip to content
Raindrop
Chat with us

An agent that investigates
your AI agents.

Ask what broke. Get the evidence. Turn the answer into a signal, a scheduled brief, or your next fix. In Slack, in Raindrop, and through MCP.

Start buildingUse it with your coding agent
Checkout confirmations after a timeoutProduction

Why are customers getting order confirmations when the order lookup fails?

I found 3 related events with the same failure. lookup_order times out after 5 seconds, then checkout-v2 confirms an order ID the tool never returned.

Across the last 7 days, this pattern appears in 43 conversations from 18 users. Most days were quiet, then matches spiked to 29 yesterday.

Unsupported order confirmationsSep 9–15 · daily matches
3 related eventsExamples from the 43 matches

The fix: only confirm an order after a successful lookup returns its ID. On timeout, retry the lookup or tell the customer the status is unverified.

Ask a question...
Product preview · sample conversations
From a question to an investigation to an action.See what Triage can do

Understanding production alongside teams at

VercelSpeakClayFramerAngelListTolan
Investigate

Ask the teammate
who’s read the traces.

Tag @raindrop in Slack, open Triage in the web app, or ask through MCP. Bring a customer complaint, a screenshot, an issue, or just a question.

Triage digs through the conversations, finds the pattern, and brings back the evidence. Ask “who else is affected?” or “didn’t we fix this?” and keep the investigation going.

  • Root causes with the original traces attached
  • Impact across users, tools, and releases
  • Context that carries through follow-up questions
Thread agent-quality
Jamie10:42 AM

@Raindrop why is checkout-v2 failing?

3 replies
RaindropAPP10:44 AM

I found 43 conversations where lookup_order timed out after 5 seconds. Checkout-v2 then confirmed the order using an ID the tool never returned.

The failure is in the timeout path: the agent continues to confirmation with no verified order.

View 3 example traces ↗

Jamie10:45 AM

Is this happening on the baseline too?

RaindropAPP10:45 AM

Yes, but less often. The fabricated order ID signal rose from 0.4% to 3.8% in checkout-v2.

Reply…
Example Slack conversation
Automate

Found a failure?
Teach Raindrop to catch it.

Ask Triage to create a signal from what it finds. It drafts the detector and pulls in real examples so you can review what should count, refine it in conversation, and start tracking.

Score new events, backfill recent history, and set up a Slack alert when the behavior crosses a threshold. Your investigation becomes an ongoing check.

Explore signals

New Signal: Fabricated order ID

  1. 1

    The assistant confirms an order and includes a specific order ID.

  2. 2

    The order lookup failed or timed out, and no successful tool result contains that ID.

  3. 3

    Do not match when the assistant explains the failure or asks the user to try again.

43 matched events ready to review. Create signal
Signal policy · Example draft
Scheduled briefs

The questions you ask
every week. Already answered.

“Every Monday, summarize the biggest issues enterprise customers had.” Give Triage the question once. Agent Briefs run a fresh investigation on your schedule and deliver the findings to Slack.

Review tool errors each morning, keep an eye on a rollout, or discover what people are using your agent for. Choose the prompt, cadence, timezone, and channel or DM.

  • Fresh findings with links to the conversations
  • Edit, pause, or run a brief whenever you need
  • Follow up in the thread to dig deeper

Briefs

New brief
Enterprise qualityActive

Investigate the top issues affecting enterprise customers. Include what changed and the supporting traces.

Every Monday at 9:00 AMCreated by Jamie
Last run Sep 14, 9:00 AM PDT
Next run Sep 21, 9:00 AM PDT # agent-quality
Daily tool errorsActive

Review yesterday’s tool failures. Group recurring errors and identify the conversations most affected.

Every day at 8:00 AMCreated by Jamie
Last run Sep 15, 8:00 AM PDT
Next run Sep 16, 8:00 AM PDT # engineering
Briefs · Example schedules
01 / Agent Briefs

Your recurring question.

A custom investigation on a schedule. What changed for enterprise customers? How is the rollout going?

02 / Daily Digest

Yesterday, in perspective.

A daily summary of issues and wins, with the details in a Slack thread when you want to dig in.

03 / Signal Alerts

A behavior worth watching.

Get notified on a signal match or when affected events or users cross your threshold.

Triage through MCP

Give your coding agent
a production expert.

Claude Code, Codex, Cursor, and Devin can call Triage as a tool. Your coding agent asks the question; Triage investigates your production data and returns the context for a fix.

Raindrop MCP

› Investigate checkout-v2 failures from the last 24 hours.

●
ask_agent_question
{ "message": "Why is checkout-v2 failing over the last 24 hours?" }
●
get_agent_progress

Done. Found 43 conversations with order lookup timeouts followed by fabricated confirmations. Supporting traces returned.

› Create a signal for this. I’ll review the examples.

●
start_signal_session
{ "project": "default",
  "intent": "Detect fabricated order IDs after a failed lookup." }

The draft is ready to review. Confirm which events should match, then create the signal.

MCP conversation excerpt · Example data

Call the investigator.

Use the Triage agent itself through MCP, alongside direct tools for traces, issues, signals, datasets, and experiments.

Automate the follow-through.

Configure your coding agent to check the top issues each night, create or refine signals, and prepare a fix with tests and a pull request.

Pick up the conversation.

Start investigating in Slack and let your coding agent continue from the findings. The traces and context come with it.

Connect Raindrop MCP Build an automated fix workflow
Compare and verify

“Did the change
actually help?”

Ask Triage to compare a new model, prompt, or feature flag against your baseline. It uses the traffic you’ve already logged and the signals that matter to your team.

Preview the comparison, inspect the conversations behind a regression, and save the experiment. Do it from Slack, the web, or your coding agent through MCP.

Explore experiments
checkout-v1vscheckout-v2

Experiment

View
Users
1,200
1,100
−8%
Events
2,400
2,100
−13%
Negative Signals
5.2%
9.4%
+81%
Positive Signals
62%
58%
−6%

Compared to checkout-v1, users are down 8%, events are down 13%; negative signals are up 4.2pp, positive signals are down 4.0pp.

Experiment preview · Illustrative data

One investigation.
Wherever you work.

Use Triage in the Raindrop web app, invite it into your team’s Slack channels, or connect your coding agent through MCP. It works with the traces you already send to Raindrop.

Get started with Triage
“We have Triage running a Monday digest of our top issues and new patterns. It surfaces things we would’ve missed otherwise and points us to the relevant troublesome conversations.”

Tuan Ahn Pham AI Engineer at Speak

A few more details.

What can I ask Triage to do?
Investigate a customer complaint, explain an issue, find patterns across conversations, measure user impact, or check a rollout. It can also create and refine signals, set up recurring briefs and signal alerts, and preview or save experiments. Follow up in the same conversation to keep the context.
How do scheduled briefs differ from the daily digest?
Agent Briefs run your custom investigation on a schedule, using fresh production data each time. The Daily Digest is a built-in summary of the previous day’s issues and wins. Signal Alerts notify you when a signal matches or crosses a threshold. Use each for a different kind of update.
Can my coding agent use Triage through MCP?
Yes. Connect Raindrop MCP to Claude Code, Codex, Cursor, Devin, or another compatible client. Your agent can call Triage to investigate, continue a conversation, and use the evidence in a fix. MCP also exposes tools for signals, issues, datasets, and experiments. See the setup instructions.
Can I automate signal creation and fixes?
Yes. Ask Triage to turn a failure pattern into a signal, then review the examples and refine what it should catch. You can also configure your coding agent to call Raindrop on a schedule, investigate issues, create signals, and prepare fixes. Triage supplies production context; your coding agent edits the code, runs tests, and opens the pull request.
Does Triage run my agent to compare a change?
Experiments compare traffic already logged to Raindrop. Ask Triage to compare models, prompts, feature flags, or time periods against the signals you care about. It can preview a comparison and save the experiment. Experiments do not execute or replay your agent.
How do I get started?
Connect your agent’s traces to Raindrop, then open Triage in the web app, install the Slack integration, or connect MCP. Triage is available on every plan, with plan-specific usage limits. See pricing or the Triage documentation.

Build with the evidence

Find the next failure.
Verify the next fix.

Start building Book a demo

Start free. No credit card required.