AI Observability - Monitor LLM Cost, Tokens, Latency

Use AI Observability to see cost, token usage, latency, errors, time to first token (TTFT), and tool calls for your LLM and agent workloads. SigNoz reads these values from the OpenTelemetry GenAI attributes on your spans, and it calculates cost from your model prices.

AI Observability Overview tab with the model, provider, environment, and service_name filters, six stat cards, and the Cost & tokens panels — *The AI Observability Overview dashboard*

Prerequisites

An LLM or agent application that sends traces to SigNoz with OpenTelemetry GenAI attributes. For setup steps for OpenAI, Anthropic, LiteLLM, CrewAI, and other frameworks, see LLM Observability. For the attributes that each panel needs, see Telemetry Requirements.
A price for each model, for the cost panels. SigNoz Cloud includes default prices for common models. In self-hosted SigNoz, add a price for each model in Model pricing.
If you run self-hosted SigNoz with a custom collector configuration, add the signozspanmapper and signozllmpricing processors to the traces pipeline. See Add the AI observability processors.

Key capabilities

To open AI Observability, select More > AI Observability in the side navigation.

Overview: a dashboard with stat cards and four sections: Cost & tokens, Per-trace usage, LLM latency & errors, and Tool calls. Filter it by model, provider, environment, and service. Each panel has an info icon that explains what it shows. You cannot edit or clone this dashboard. To create an alert from a panel, open the panel menu and select Create Alerts.
Explorer: query your LLM and agent traces, including trace totals such as trace.total_tokens.
Model pricing: set the price for each model.
Attribute Mapping: map other attribute names to the GenAI names.

Next steps

Telemetry Requirements: check the attributes that each panel and filter needs.
LLM Observability integrations: instrument OpenAI, Anthropic, LiteLLM, CrewAI, and other frameworks.
Dashboards: build your own LLM panels.

Get Help

If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack. If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.