Raindrop Logo
Raindrop Logo
Customer Logo

How Unstuck AI monitors their learning agents with Raindrop

"Raindrop has become table stakes for building AI products. We need to understand how our agents perform in production to ensure they're working well for our millions of users."
Sid Bendre
Sid Bendre
CTO

Unstuck AI, an academic copilot developed by Oleve, is one of the most popular AI tools in EdTech with over 2 million users, and part of Oleve's edtech products, a portfolio that's generating $6mm+ in ARR with a team of just 5 people.

Students rely on Unstuck's learning agent to generate study notes from lecture recordings, summaries of videos, create flashcards and even quizzes. As the team rapidly expanded tool use capabilities, they needed an effective way to monitor changes and quickly identify issues.

"Building an agent-native platform, Raindrop is a core part of our feedback loop. We need to monitor agent improvements as soon as we ship them to see if they're working for users. This is core to the UX of the product, so it's critical that we have a very strong pulse on it."

Sid Bendre
Sid Bendre
CTO

Highlights

Caught Tool Use Issues

Quickly identified and resolved critical issues with their summarization tool

Use Notifications to Prioritize

Real-time notifications flagged urgent issues like incomplete lecture notes or missing math explanations

Issue Notifications

Raindrop sends Unstuck daily alerts highlighting user wins and surfacing emerging issues. When users started receiving incomplete summaries and lecture notes, these notifications allowed the team to quickly prioritize fixes.

"Raindrop has been super high leverage for us. Every day when I'm on my way to the office, I check the notification to see what issues users have been facing and that impacts what I prioritize."

Sid Bendre
Sid Bendre
CTO

Monitoring Agentic Tool-Use

As soon as Unstuck's AI engineering team pushed their fixes for the summarization issues, they were immediately able to jump into the Raindrop dashboard and see if the issue incidence rate started decreasing.

"We had just pushed out a major change for the summarization issue, and then we were able to go into Raindrop, filter User Frustration by the summarization tool, and immediately get feedback on whether our changes were impactful"

Sid Bendre
Sid Bendre
CTO

Feature Prioritization

Raindrop's notifications don't just flag issues, they highlight wins too. This lets the Unstuck team quickly see which features users love and identify unexpected ways people engage with the product.

"Raindrop has been invaluable in helping us assess demand for a feature over another. It directly led to us launching flashcards and quizzes!"

Sid Bendre
Sid Bendre
CTO

Beyond Unstuck

Unstuck is part of Oleve's suite of AI-driven consumer products. Raindrop has become so essential to Oleve's AI engineering workflow that the team now integrates it from day one in every new product they build, just as they did with Unstuck.