# Dunetrace

> Dunetrace is the open-source runtime agent reliability platform for production AI agents. It combines structural detection (34 deterministic detectors, 31 of them runnable in-path, for tool loops, retry storms, runaway iteration, and silent failures), semantic evaluation (7 LLM-based evaluators for hallucination, task completion, and cross-turn frustration), and runtime prevention (policies that stop, redirect, or downgrade a run while it is still executing). It closes the loop with native root-cause analysis and automated fixes. Apache 2.0, self-hostable, with a managed Cloud tier available.

Dunetrace is self-hosted via Docker Compose in one command, with a managed Cloud tier available. The Python SDK (`pip install dunetrace`) and TypeScript SDK (`npm install dunetrace`) instrument agents with two lines of code and add sub-500μs overhead. A zero-code path accepts OpenTelemetry traces over OTLP/HTTP. An MCP server (`pip install dunetrace-mcp`) lets Claude Code, Cursor, and other editors query agent health, signals, and run timelines directly; it is read-only by default, and the nine tools that write (policy and custom-detector lifecycle, resolve_issue, trigger_explain) are only registered when DUNETRACE_MCP_READONLY=false.

License: Apache 2.0. Source: https://github.com/dunetrace/dunetrace

## Platform

Five pillars, from raw capture to prevention:

1. Sessions & Events — every run, tool call, and LLM exchange, the raw data everything else is built on.
2. Structural Detection — 34 deterministic detectors, 31 of them in-path, sub-500μs overhead, always on, no configuration. 30 are live; 4 ship in shadow mode pending calibration.
3. Semantic Evaluation — 7 LLM-based evaluators, post-completion, adaptive sampling, powered by DeepEval.
4. Runtime Prevention — policies that stop a tool call, override a model, inject a prompt, require approval, or escalate to a human while the run is still executing. HMAC-signed, deterministic, sub-millisecond enforcement. This is the primary differentiator: Dunetrace prevents failures, it does not just report them.
5. Root Cause & Fix — native root-cause analysis from structured trace evidence with no third-party tracer required, auto-applied policy fixes, and draft GitHub PRs for prompt or code fixes (always human-reviewed).

## Detectors

Structural detectors (Tier 1, 34 deterministic, 31 of them runnable in-path), grouped by the failure they catch:

- Runaway loops & cost: Tool Loop, Tool Thrashing, Retry Storm, Runaway Iteration, Step Count Inflation, Context Bloat, Cost Spike, Reasoning Stall.
- Silent failures: Premature Termination, Unread Tool Error, Empty LLM Response, First Step Failure, LLM Truncation Loop, Silent Truncation, Goal Abandonment.
- Fabrication & bad grounding: Tool Argument Fabrication, RAG Empty Retrieval, Excessive Retrieval, Tool Avoidance, Oversized Tool Arguments, Cascading Tool Failure.
- Injection & memory poisoning: Prompt Injection Signal, Retrieved Content Injection, Memory Poisoning, Ungrounded Destination.
- Multi-agent breakdowns: Handoff Context Loss, Agent Handoff Failure, Delegation Loop.
- Model degradation & latency: Model Fallback Drift, Slow Step, Session Latency.

Semantic evaluators (Tier 2, 7 LLM-based, adaptive sampling): Hallucination, Task Completion, User Frustration, Confusion Loop, Task Understanding Failure, Sycophancy Signal, Off-Topic Drift. Sampling covers 100% of flagged runs, 100% of critical agents, 20% of retrieval runs, and a 5% baseline. Cost is tracked and quota-enforced.

Detector packs (opt-in, per-org modules): Voice is live with 9 detectors (Transcription Confidence Drop, Silence Timeout, Turn-Taking Collision, Latency-Induced Hangup, Audio Quality Degradation, Speaker Confusion, Barge-In Failure, TTS Truncation, VAD False Trigger). Coding, Multi-Agent, RAG/Search, Browser, Transactional, MCP-Native, Long-Running, and Streaming packs are planned.

## Runtime prevention and approvals

Policies fire on structural signals during a run. Actions include: stop the tool call, override the model, inject a prompt, require approval, escalate to a human, stop TTS, and inject a recovery prompt. Enforcement is HMAC-signed, deterministic, and sub-millisecond. Approval workflows let an agent pause execution and wait for human approval, delivered via Slack, and a runtime policy can require approval before high-value tool calls. Runs can be grouped into conversations via conversation_id for cross-run semantic evaluation.

## Docs

- [Quick start](https://www.dunetrace.com/docs/quickstart.html): Two paths. Fastest is the `dunetrace-setup` coding-agent skill (`npx skills add dunetrace/dunetrace-skills --skill dunetrace-setup`), which instruments the repo for you. Manual path: clone, `docker compose up -d`, install the SDK, add two lines, open the dashboard at localhost:3000.
- [Architecture](https://www.dunetrace.com/docs/architecture.html): Services for ingest, customer API, detection, dashboard, and PostgreSQL. Structural detection runs in-path with sub-500μs SDK overhead.
- [MCP server](https://www.dunetrace.com/docs/mcp-server.html): `pip install dunetrace-mcp`. 31 tools plus 8 documentation resources. Read tools: list_agents, summarize_agent, get_agent_health, get_agent_patterns, get_agent_token_stats, get_agent_runs, get_run_detail, compare_runs, get_agent_signals, get_signal_detail, search_signals (alias list_all_signals), get_failure_pattern_detail, get_fix_status, list_agent_fixes, list_agent_issues, get_issue, search_issues, list_policies, list_custom_detectors, list_voice_calls, get_call_detail, get_instrumentation_guide. Nine tools write and are withheld unless DUNETRACE_MCP_READONLY=false: trigger_explain, resolve_issue, create_policy, toggle_policy, delete_policy, create_custom_detector, activate_custom_detector, pause_custom_detector, delete_custom_detector — create_policy and toggle_policy affect live agent runs, since a stop policy terminates real runs once the SDK next pulls policies. Supports Claude Code, Cursor, and SSE clients.
- [Coding-agent skills](https://github.com/dunetrace/dunetrace-skills): `npx skills add dunetrace/dunetrace-skills --skill dunetrace-setup`. Apache 2.0, separate repo so skills version independently of the platform. One skill today, `dunetrace-setup`: it instruments a repository with Dunetrace, debugs an integration producing no events, or extends one to a new code path. Invoke with /dunetrace-setup in Claude Code and Cursor, $dunetrace-setup in Codex (Codex uses $, not /). It stops and asks when a repository has more than one LLM entry point, requires explicit approval before modifying any file, never reads or prints an API key value, and reports afterwards which detectors the new event stream can actually trigger. Per-framework reference files are generated from the platform's integration guides and the detector counts from the live detector set, so neither can drift. Distinct from the MCP server: the skill writes instrumentation into your repo, the MCP server reads telemetry out of a running deployment.
- [Integrations](https://www.dunetrace.com/docs/integrations.html): Framework SDKs plus Slack, Linear, a generic HMAC-signed webhook (which is how PagerDuty or any other endpoint is reached), and a GitHub App for PR automation. There is no email destination. Alert delivery destinations are exactly slack, webhook and linear. Langfuse, LangSmith, and Braintrust are optional pull integrations: their evaluation results flow into Dunetrace. None of them are required for root-cause analysis.

## Instrument

- [LangChain / LangGraph](https://www.dunetrace.com/docs/integrate-langchain.html): `pip install 'dunetrace[langchain]'`. Pass `DunetraceCallbackHandler` in the `callbacks` list. Works alongside a Langfuse callback in the same list.
- [CrewAI](https://www.dunetrace.com/docs/integrate-crewai.html): `DunetraceCrewCallback` hooks into the CrewAI global hook system. Call `cb.install()` before `crew.kickoff()`, wrap with `dt.run()`.
- [AutoGen](https://www.dunetrace.com/docs/integrate-autogen.html): `DunetraceAutoGenObserver` wraps autogen-agentchat >= 0.4. Use `observer.wrap_client()` per agent model client and `async with observer.run()` around team execution.
- [Haystack](https://www.dunetrace.com/docs/integrate-haystack.html): `DunetraceHaystackTracer` implements the Haystack `Tracer` protocol. One `enable_tracing()` call covers every pipeline run.
- [Mistral](https://www.dunetrace.com/docs/integrate-mistral.html): `pip install 'dunetrace[mistral]'`. Auto-instruments chat/embeddings/FIM including Azure- and GCP-hosted clients; Bedrock-hosted models are covered by the botocore patcher. Also runs Tier 2 semantic evaluation and Dunetrace's own LLM features on Mistral (`SEMANTIC_LLM_PROVIDER=mistral`, `API_LLM_PROVIDER=mistral`) with no cross-provider fallback, for EU data residency.
- [Custom Python agent](https://www.dunetrace.com/docs/integrate-python.html): `@dt.trace` / `@dt.tool` decorators, `@dt.agent()`, ASGI/WSGI middleware, `dt.run()` context manager, or the OTel receiver.
- [TypeScript agent](https://www.dunetrace.com/docs/integrate-typescript.html): `npm install dunetrace`. Auto-instrument with `dt.wrapOpenAI()` or `dt.wrapAnthropic()`. Manual `dt.tool()` and `dt.trace()` decorators. Voice hooks (transcriptionReceived / ttsGenerated / voiceActivityDetected / turnTaking / recordingMetadata), plus OpenTelemetry export (send events to Datadog/Grafana/Honeycomb) and an OTel receiver that ingests existing gen_ai.* spans.
- [Langdock](https://www.dunetrace.com/docs/integrate-langdock.html): Zero-code monitoring. Point Langdock's tracing cloud URL at `POST /v1/otlp/traces` on the Dunetrace ingest service. All structural detectors activate with no code changes.
- [Langfuse](https://www.dunetrace.com/docs/integrate-langfuse.html): Optional pull integration. Pass both `DunetraceCallbackHandler` and `LangfuseCallbackHandler` in the same `callbacks` list to land your Langfuse evaluation results in the Dunetrace dashboard. Langfuse is not required; Dunetrace performs native root-cause analysis from its own trace evidence.

## Deployment and privacy

Free and open source under Apache 2.0, self-hosted via Docker Compose in one command; a managed Cloud tier is available. Privacy is handled through the option to self-host so data never leaves your own infrastructure, plus encryption in transit and at rest.

## Careers

- [Careers](https://www.dunetrace.com/careers.html): Dunetrace is hiring a Technical Co-Founder — a hands-on founding role owning technical vision, architecture, and the core product. Berlin-based, remote-first within Germany; candidates should be based in or willing to relocate to Germany. Co-founder equity with vesting plus base salary. Apply by emailing a CV and LinkedIn profile to vikas@dunetrace.com.