← Resources
TalkYouTube
How To Test AI Agents With Simulations
Raindrop cofounder Ben Hylak joined Hamel Husain's AI Evals course to explain simulations: replaying real production traces against a change to see what it breaks. He covers why cached tool results tip the agent off that it's in a simulation, how to rebuild the world around the agent, why five to fifty well-chosen simulations beat thousands of random ones, and why evals are turning into simple functions over traces.
