> ## Documentation Index > Fetch the complete documentation index at: https://raindrop.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Simulations > Test changes to your agent against real production events before you merge. Simulations replays your agent's real production events against a change to its code, prompts, tools, or model, then compares each new run with what happened in production. Raindrop runs your agent in an isolated sandbox called a World and simulates the tools and services it calls. Use Simulations to see whether a change fixes what you meant to fix, and what else it changes, before you merge. Simulations is in public beta. To set it up for the first time, follow [Set up Simulations](https://raindrop.ai/docs/guides/simulations-setup.md). To run simulations from Claude Code, Cursor, or another coding agent, see [Run simulations from your coding agent](#run-simulations-from-your-coding-agent). ![A Simulations review of a pull request that switches a trip planner agent to a faster model, flagged High severity](https://mintcdn.com/dawn-a6c57108/GUO875mXuv5wVSPD/images/simulations/review-regression-light.webp?fit=max\&auto=format\&n=GUO875mXuv5wVSPD\&q=85\&s=9bd960f70351842437add02f41a718a1) ## How Simulations works ### The pieces * **Repository:** a GitHub repository you connect. Raindrop reads it to find your agents. * **Agent:** an AI agent Raindrop finds in your code. Each agent gets its own page in **Simulations**. * **Events:** the agent's production events in Raindrop. Raindrop matches each agent to the event names it sends, and replays those events. * **World:** a sandboxed copy of your agent and its environment. Raindrop builds a World per agent, then updates it for each pull request it reviews. * **Replay:** one production event run again against a change. A replay records the new conversation, tool calls, and trace. * **Review:** the result of a set of replays. A review compares each replay with production and flags the changes that matter. ### What runs for real and what is simulated | Part of your agent | In a simulation | | - | - | | Your code, prompts, model, and agent loop | Run for real, from your repository | | Model calls | Run for real | | Tools and external services | Simulated: Raindrop answers each call with a response grounded in your production traces and typechecks it in TypeScript Worlds | | Production credentials and services | Never reached: the World runs in an isolated sandbox | Because the World answers tool calls instead of replaying a fixed trace, your agent can take a different path from production and still get consistent results. ### What a review compares A pull request review runs the pull request's head commit against recent production events and compares each run with what happened in production. It does not rerun your base branch. Each review replays 10 events by default. ## Roles and permissions * **Owners** and **Admins** connect GitHub, choose repositories, build Worlds, and change when simulations run. * **Members** can read every agent, World, review, and replay in the organization. See [Roles](https://raindrop.ai/docs/platform/roles.md) for the full permission list. ## Using Simulations ### Set up an agent Setup takes four steps in **Simulations**: connect GitHub, find your agents, choose an agent, and build its World. The first build takes hours, so allow about a day. [Set up Simulations](https://raindrop.ai/docs/guides/simulations-setup.md) walks through each step. ### Run simulations on a pull request Simulations runs automatically when you open a pull request that changes an agent. Raindrop posts a comment on the pull request and links to the full review in Raindrop. To start or rerun a simulation by hand, check the box in the Raindrop comment, or comment on the pull request: ```text theme={null} @raindrop simulate ``` To run one agent, name it: ```text theme={null} @raindrop simulate trip-planner ``` ### Read a review Open the agent in **Simulations**, then open the pull request from the **Pull requests** tab. The review shows: * **Overview:** what the change does, what the agent did, and the before and after counts of tool-call failures and tool calls per trace. * **Simulation faithfulness:** how confident Raindrop is that the World reproduced production for these events. Hover it to read the reason. * **Flags:** each behavior change worth reviewing, with a link to the run that shows it. * **Before and After PR:** each event side by side, with the conversation, details, and trace tree for both runs. ![Before and after runs of the same event: production books a refundable flight, and the pull request books a nonrefundable one](https://mintcdn.com/dawn-a6c57108/GUO875mXuv5wVSPD/images/simulations/review-detail-light.webp?fit=max\&auto=format\&n=GUO875mXuv5wVSPD\&q=85\&s=3ba2c07c95eff3bc79e49c1ca4d61199) ### Try a change in the Playground The **Playground** tab tests a change without a pull request. Edit the prompts and model the World exposes, or edit up to 10 source files. Then add test cases **From events** or **From dataset**. Each case starts from a real event's user message, and you can edit that message to try a new question. Select up to 10 cases and select **Run selected**, or **Run all**. Each result appears next to what production did. Raindrop can notify you in the browser or by Slack DM when a run finishes. ![The Playground with the trip planner's system prompt open](https://mintcdn.com/dawn-a6c57108/GUO875mXuv5wVSPD/images/simulations/playground-light.webp?fit=max\&auto=format\&n=GUO875mXuv5wVSPD\&q=85\&s=22cfc67c19aa14854c92f165e41d42fa) ### Choose when simulations run Open **Simulation settings** on an agent's **Pull requests** tab and pick one: * **Automatically on PRs:** Raindrop runs simulations when a pull request changes the agent. This is the default. * **Only when I run them:** Raindrop runs nothing on its own. Start runs from the Playground, the pull request comment, or `@raindrop simulate`. ### Run simulations from your coding agent The [Raindrop MCP server](https://raindrop.ai/docs/mcp/overview.md) brings Simulations into Claude Code, Cursor, Codex, or any MCP client. Your coding agent can change a prompt or the model, replay real production events, and show you each new run next to production, all without leaving the editor or opening a pull request. Add the server once. In Claude Code: ```bash theme={null} claude mcp add raindrop --transport http https://mcp.raindrop.ai/mcp ``` For other editors, see [Connect your editor](https://raindrop.ai/docs/mcp/overview.md#connect-your-editor). Then ask in plain language: ```text theme={null} Replay my last three trip-planner events from this week with the faster model, and show me what changed. ``` ```text theme={null} Did the latest simulation review on my open pull request flag anything? Summarize the flags and link the runs. ``` Your coding agent finds the World, picks the events, starts a saved Playground run, waits for it to finish, and shows the results in chat. Every run also appears in the agent's **Playground** tab, so your team can open it in Raindrop. A run from MCP changes prompts and the model only and replays 1 to 10 events from the last month. To edit source files, use the Playground. Starting and stopping runs needs a signed-in user or a personal API key, and over OAuth it needs the `write:simulations` scope. See [Simulation tools](#simulation-tools) for every tool. ## Reference ### Requirements | Requirement | Detail | | - | - | | Language | TypeScript, JavaScript, or Python. Python agents replay saved events; live replay is not available for Python yet. | | Source | A GitHub repository connected through the Raindrop GitHub App | | Events | At least 100 events in the project, and at least one event from the last 7 days matched to the agent with its input recorded | ### Good events for Simulations The World learns your agent's tools and data from its traces. Two habits make it faithful: * **Record tool calls in your traces.** A World cannot simulate a tool it never saw called. * **Send prompts unredacted.** A redacted system prompt weakens the World. Redact personal data with [PII redaction](https://raindrop.ai/docs/security/pii-redaction.md) instead. ### Setup states | State | Meaning | | - | - | | Setting Up | Raindrop is preparing the World build | | Building | Raindrop is building the World from your code and events | | Testing | Raindrop is replaying real events to verify the World | | Ready | The World is ready for pull request reviews and Playground runs | ### Review results | Result | Meaning | | - | - | | High severity | At least one flag shows a regression worth fixing before merge | | Medium severity | A flag shows a behavior change worth checking | | No findings | The replays found no behavior change worth flagging | | Incomplete | No replay finished in a way Raindrop could compare with production | | Could not review | The review failed before it could compare the runs | ### Pull request commands | Comment | Effect | | - | - | | `@raindrop simulate` | Runs every agent the pull request changes | | `@raindrop simulate ` | Runs one agent, named by its slug | ### Simulation tools | Tool | What it does | | - | - | | `list_simulation_worlds` | Lists the organization's Worlds and their agents | | `get_simulation_world` | Reads the prompts and model a run can change on one World | | `start_simulation_run` | Replays 1 to 10 events with changed prompts or model, as a saved Playground run | | `get_simulation_run` | Waits up to 15 seconds for progress on a run | | `show_simulation_run` | Shows a run's results in chat | | `list_simulation_runs` | Lists a World's runs, newest first | | `stop_simulation_run` | Stops a run's unfinished replays | | `list_simulation_reviews` | Lists pull request reviews, newest first | | `get_simulation_review` | Reads one review with its replays and flags | | `get_replay_progress` | Waits up to 15 seconds for one replay to finish | | `get_simulation_settings` | Reads when simulations run and how many events each review replays | | `update_simulation_settings` | Changes when simulations run and how many events each review replays. API key connections only | ## Troubleshooting **I cannot pick my agent.** Raindrop has not matched any events to it yet. Select **Add Raindrop** on the agent, paste the prompt into your coding agent, and deploy the change. See [Requirements](#requirements). **Build World says to select a capture with recorded input.** Raindrop found no matched event with recorded input in the current project from the last 7 days. Check that you are in the project the agent sends events to, that the agent has sent events this week, and that those events include the agent's input. Then select **Build World** again. See [Requirements](#requirements). **My agent shows "Not supported yet."** Simulations builds TypeScript, JavaScript, and Python agents. Agents in other languages cannot build a World yet. **The World has said "Building" for hours.** The first build takes hours, and Raindrop verifies the World before it is ready. Allow about a day. Raindrop can notify you by Slack DM or in the browser when it is ready. See [Setup states](#setup-states). **My pull request has no simulation.** Simulations runs on pull requests that change an agent, and only when settings allow automatic runs. Comment `@raindrop simulate` to start one. See [Choose when simulations run](#choose-when-simulations-run). ## Related * [Set up Simulations](https://raindrop.ai/docs/guides/simulations-setup.md) * [Change Reports](https://raindrop.ai/docs/platform/change-reports.md) * [Events](https://raindrop.ai/docs/platform/events.md) * [MCP tools](https://raindrop.ai/docs/mcp/overview.md#simulations) * [Roles](https://raindrop.ai/docs/platform/roles.md) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.