Introducing Change Reports

As agents take on more complex tasks and companies trust them with more consequential work, there are more ways for things to go wrong and the stakes for failure are higher.

At Raindrop, we're building multiple layers of reliability into the agent development process. Issue Detection finds unexpected failures in production. With Simulations, you can test proposed changes before deployment to catch potential regressions.

Today, we're adding another layer: Change Reports.

When you add a tool, switch models, or update a prompt, you need to know how your agent behaves after the change. Is the agent using the new tool successfully? Is it following the updated instructions? Have new failures appeared?

With Change Reports, you get a Slack report comparing your agent's behavior before and after a change, highlighting relevant activity, successes, and potential issues.

One team deployed a fix to prevent agent runs from ending without a response. Change Reports found two production runs where the agent still returned no text, revealing that the fix hadn't fully resolved the problem.

How it works

1. Connect your GitHub repository. Raindrop tracks changes to your agent's code, including new tools, model switches, and prompt updates.
2. Enable Change Reports. Choose which agents to monitor and where to receive reports: your project's Slack channel, or to the PR author's DMs.
3. Ship your change. Raindrop compares your agent's behavior before and after deployment, analyzing production activity for successes and potential issues.
4. Get reports in Slack. Raindrop sends reports soon after the merge and 24 hours later, highlighting what happened after your change.

Change Reports are available to all Pro and Enterprise users. Get started.