What is LLM/agent observability?

Capturing what an AI agent did in production. That means the prompts, model calls, tool calls, delays, costs, and outputs. With that record you can debug failures, measure quality, and catch regressions. Many of these tools emit or ingest OpenTelemetry, so the data fits the monitoring you already have.

Which observability tools are open source and self-hostable?

Langfuse, OpenLLMetry, Helicone, Opik, and W&B Weave are open source. Arize Phoenix is source-available under the Elastic License 2.0. LangSmith is proprietary but can run self-hosted or BYOC. Datadog LLM Observability is SaaS.

Top AI Agent Observability & LLM Tracing Tools (2026)

When an agent misbehaves in production, you need to see what it did. That means the prompts, tool calls, and outputs, turn by turn. Tracing tools capture that, often over OpenTelemetry, with evals attached. This list covers the tools teams use to watch agents in prod.

1
Langfuse
An open-source (MIT, except ee folders) LLM engineering platform. Use it to monitor, evaluate, and debug AI apps. You can self-host it, and it plugs in to OpenTelemetry.
License:
MIT (except ee)
Self-host:
Yes
2
Arize Phoenix
A source-available AI observability platform under the Elastic License 2.0. Use it for experiments, evals, and debugging. It is built on OpenTelemetry, via OpenInference. You can self-host it.
OTel-based:
Yes
Self-host:
Yes
3
OpenLLMetry
An open-source (Apache-2.0) set of OpenTelemetry extensions from Traceloop. They instrument LLM providers and vector databases. The output is standard OTel data.
OTel-based:
Yes
License:
Apache-2.0
4
Helicone
An open-source (Apache-2.0) platform for LLM observability, plus an AI gateway. You can self-host it via Docker Compose or Helm.
License:
Apache-2.0
Self-host:
Yes
5
LangSmith
A proprietary platform from LangChain. It traces and watches agents and LLM apps. It has support for OpenTelemetry. It can run as managed cloud, BYOC, or self-hosted.
OTel support:
Yes
Self-host:
Yes (incl. BYOC)
6
Opik
An open-source (Apache-2.0) platform by Comet for LLM observability, evals, and agent tracing. It is fully self-hostable. It has support for OpenTelemetry alongside its own SDKs.
License:
Apache-2.0
Self-host:
Yes
7
Datadog LLM Observability
A proprietary Datadog platform. Use it to monitor, debug, and evaluate LLM-powered apps and agents. It supports the OTel GenAI semantic conventions out of the box.
OTel:
GenAI conventions
Self-host:
No (SaaS)
8
W&B Weave
An open-source (Apache-2.0) toolkit from Weights & Biases for building generative-AI apps. It logs and evaluates LLM inputs, outputs, and traces.
License:
Apache-2.0
Focus:
Tracing + evals

Reviewed quarterly. Durable facts only: what it is, license, where it runs. Stars, pricing, and benchmark scores are left off. They rot too fast to stay right on a static page. Each row carries the date it was last checked against the source.

Common questions

What is LLM/agent observability?: Capturing what an AI agent did in production. That means the prompts, model calls, tool calls, delays, costs, and outputs. With that record you can debug failures, measure quality, and catch regressions. Many of these tools emit or ingest OpenTelemetry, so the data fits the monitoring you already have.
Which observability tools are open source and self-hostable?: Langfuse, OpenLLMetry, Helicone, Opik, and W&B Weave are open source. Arize Phoenix is source-available under the Elastic License 2.0. LangSmith is proprietary but can run self-hosted or BYOC. Datadog LLM Observability is SaaS.