Skip to main content
Raindrop automatically instruments LangChain Deep Agents workflows. Pass the callback handler once and Raindrop captures every LLM call, tool invocation, chain step, and agent action — with full OpenTelemetry trace nesting linked to the resulting dashboard event. Available for both TypeScript (@raindrop-ai/deep-agents) and Python (raindrop-deep-agents).

Installation

Quick Start

What Gets Traced

The Deep Agents integration automatically captures:
  • LLM calls — model name, input messages, output text, prompt + completion tokens, finish reason
  • Tool calls — tool name (write_todos, read_file, write_file, edit_file, ls, glob, grep, execute, task), input arguments, output, duration, errors as nested TOOL_CALL spans linked to the parent event (via interaction.track_tool())
  • Chains — root chain input (the user’s question) and output (the final assistant reply) captured as the canonical event, with aggregated model + token usage merged in from all child LLM calls
  • Agent actions — tool selection events captured during the agent loop
  • Errors — exception type and message captured for both LLM and tool failures
  • Tags & metadata — anything passed via LangChain config={"tags": [...], "metadata": {...}} is forwarded to event properties
  • Extended token categories — cached tokens (ai.usage.cached_tokens) and reasoning tokens (ai.usage.thoughts_tokens) when available from the provider (e.g. OpenAI, Anthropic)
  • Finish reason — captured as ai.finish_reason in event properties ("stop", "length", "tool_calls", etc.)
  • Multi-modal messages — typed-parts content ([{type: "text", text: "..."}, ...]) is extracted cleanly; text parts are concatenated, non-text parts (images, audio) are skipped
All operations are linked with parent-child relationships, so you can see the full execution tree in the Raindrop dashboard’s Trace tab.

Configuration

Projects

Route events to a specific project by passing its slug as projectId (project_id in Python):
This sets the X-Raindrop-Project-Id header on every event. Omit it (or pass "default") to use your org’s default Production project, which is the existing behavior. Single-project orgs need nothing new.

Multiple projects in one process

Available in raindrop-ai>=0.0.56. When one service runs several Deep Agents that should report to different projects, create one RaindropDeepAgents wrapper per project. Each wrapper owns its own raindrop.Raindrop client — the handler is a plain LangChain callback with no shared module-level state — so the two route independently:
Each wrapper owns its configuration and delivery pipeline, so concurrent requests handled by different agents route independently. To share a single client across wrappers (or with the module-level API), construct a raindrop.Raindrop yourself and pass it via client=:
Distributed tracing uses a single process-global OpenTelemetry pipeline, so auto-instrumented trace spans follow whichever client initialized tracing first. The per-wrapper routing above applies to the flat events shipped by the callback handler (begin/track_ai/flush/identify/track_signal), which is the data the dashboard’s event views rely on. For fully independent tracing across projects, run one project per process.

Tool Calls

When your agent uses tools, each tool execution appears as a nested TOOL_CALL span under the root event with the tool name, input arguments, output, and duration. Tool errors include the exception type and message.
The tool call appears in the event’s toolCalls array and as a span in the Trace tab.

Tags and Metadata

Tags and metadata passed via LangChain config are forwarded to Raindrop event properties:

Identify Users

Associate user metadata with events for richer filtering and trajectories:

Track Signals

Track positive / negative feedback or edits attached to a specific event. The TypeScript client exposes lastEventId, the event_id of the most recently finalized root event, so you can attach feedback to the agent invocation you just ran without polling:

Async Usage

The handler inherits from LangChain’s BaseCallbackHandler and supports both sync and async invocations:

Debug Mode

Enable verbose logging when you need to diagnose extraction or shipping issues:
In Python, debug=True raises the raindrop_deep_agents logger to DEBUG level. The setting is process-global (it mutates the named module logger) — once any instance enables it, subsequent instances will also see DEBUG output.

Flushing and Shutdown

Always call shutdown() before your process exits — flush() only sends queued data without waiting for the worker thread to drain:

LangGraph Internals

Deep Agents is built on LangGraph. The handler keeps the outer LangGraph chain as the canonical root (so OpenTelemetry association context is set before any span exists) and filters internal LangGraph nodes (__start__, __end__, ChannelWrite:*, ChannelRead:*, Branch:*, dunder names) so they don’t pollute the dashboard. LLM callbacks that LangGraph fires multiple times with the same runId are deduplicated automatically.

Python: wrapt compatibility

This section applies to the Python integration only. The TypeScript package does not depend on wrapt.
Works with both wrapt < 2.0 and wrapt >= 2.0. The upstream LangChain OpenTelemetry instrumentor (and ~15 sibling instrumentors) still call wrap_function_wrapper(module="...") with the legacy module= kwarg that wrapt 2.0 renamed to target=. Without a workaround the instrumentor silently fails to activate and traces never land — tracked at traceloop/openllmetry#4009 (and a parallel Arize-ai/openinference#2996 for a sibling instrumentor family). On import, the package installs a small backwards-compat shim (raindrop_deep_agents/_wrapt_compat.py) that translates module=target=, so both wrapt versions work transparently — no constraint imposed on your environment. The shim becomes pointless once upstream ships a fix.

Known Limitations

  • Beta status — API surface may change in minor releases.
  • Deep Agents version — Requires deepagents >= 0.5.0.
  • Streaming — Token-by-token streaming events are not captured individually; only the final aggregated response per LLM call is tracked.
  • Subagent isolation — Each subagent invoked via the task tool fires its own callback chain. Events are correctly grouped by convo_id / convoId, but the parent–child trace relationship across subagents is best-effort.
  • OTel context propagation (Python) — The LangChain instrumentor sometimes does not propagate traceloop.association.properties.user_id baggage to every span produced inside a LangGraph callback context. The dashboard still shows the trace under the correct event because the SDK records trace_id on the root event’s properties, but filtering the global Traces page by user_id may not surface every relevant span. Tracked upstream at traceloop/openllmetry#2271.
  • Concurrent invocations on a shared handler — A single handler instance keeps one in-flight association / span map at a time. Running multiple agent.invoke(...) calls concurrently with the same handler (e.g. Promise.all([...]) in TS or asyncio.gather(...) in Python) will let later invocations overwrite the earlier one’s interaction. Workaround: instantiate one Raindrop client per concurrent request — sequential invocations on a shared instance are fully supported.
  • Long chain inputs/outputs are truncated — Chain-level input and output captured on the root event are truncated to ~8 KB to stay within the SDK’s payload limits. Per-LLM child events still carry full prompts; only the canonical-row summary is shortened.
  • Python SDK feature surface — The TypeScript client exposes additional low-level APIs (events.patch, events.finish, events.addAttachments, events.setProperties). The Python SDK uses module-level functions; equivalent capability is exposed as top-level methods (identify(), track_signal()) on the RaindropDeepAgents class.