This dashboard provides a comprehensive view of the Langflow service using trace data. It is built on the gen_ai.* OpenTelemetry span attributes exported by Langflow's built-in Traceloop tracer and focuses on LLM usage: token consumption (input, output, and total), per-model breakdown, LLM call latency, tool calls, agent and flow runs, and errors.
Dashboard Preview
Dashboard JSON
Recommended. Uses the V2 dashboard schema and needs SigNoz v0.135.0 or newer.
Import it in SigNoz with Dashboards → + New dashboard → Import JSON. Import guide
What This Dashboard Monitors
This dashboard tracks critical performance and cost metrics for your Langflow service using OpenTelemetry trace data to help you:
- Optimize LLM Cost: Break down input, output, and total token consumption per model to understand cost drivers and track usage trends over time.
- Track Model Usage: Compare call volumes and token consumption across every model in use to guide model selection.
- Monitor LLM Latency: Watch p50, p95, and p99 latency for LLM calls to surface slow responses and regressions.
- Understand Agent Activity: See how many agent and flow runs execute over time and how long they take.
- Track Tool Usage: Identify which tools your agents call most frequently.
- Catch Errors Early: Surface the count of spans with errors so you can detect incidents immediately.
- Inspect Recent Calls: Drill into the most recent LLM calls for quick debugging.
Panels Included
Usage Summary (Top Row)
| Panel | Type | What It Shows |
|---|---|---|
| Total LLM Tokens | Value | Sum of gen_ai.usage.total_tokens across all spans in the selected window |
| LLM Calls | Value | Count of spans where gen_ai.request.model exists, showing total LLM invocations |
| Input Tokens | Value | Sum of gen_ai.usage.input_tokens across all LLM calls |
| Output Tokens | Value | Sum of gen_ai.usage.output_tokens across all LLM calls |
| Avg Tokens / Call | Value | Average of gen_ai.usage.total_tokens per LLM call |
| Agent / Flow Runs | Value | Count of invoke_agent LangGraph spans, representing agent and flow executions |
| Tool Calls | Value | Count of spans whose name matches execute_tool%, representing tool invocations |
| Errors | Value | Count of spans with hasError = true; highlights failures in the selected window |
Token & Model Usage
- Token Usage Over Time by Model: Time-series graph of total token consumption grouped by
gen_ai.request.model, showing which models drive usage over time. - Input vs Output Tokens Over Time: Time-series graph comparing input and output token consumption to understand the balance between prompt and completion sizes.
- LLM Calls Over Time by Model: Time-series graph of LLM invocation counts grouped by
gen_ai.request.model, revealing model adoption and call-volume trends. - Per-Model Usage Breakdown: Table of call count, input tokens, output tokens, total tokens, and average duration per
gen_ai.request.modelfor per-model cost and performance tracking. - Tokens by Model: Pie chart of total token consumption per model, showing the proportion of usage across models at a glance.
- Response Finish Reasons: Donut chart of LLM responses grouped by finish reason (for example,
stopversustool_call), showing how often calls complete normally versus trigger a tool call.
Latency
- LLM Call Latency (p50 / p95 / p99): Time-series graph of p50, p95, and p99 latency for LLM calls to surface tail latency and regressions.
Agent & Tool Activity
- Tool Calls by Tool: Bar chart of tool-invocation counts grouped by span name, revealing which tools your agents rely on most.
- Agent / Flow Runs Over Time: Time-series graph of
invoke_agent LangGraphspan counts, showing agent and flow execution volume over time. - Agent Run Latency (p95): Time-series graph of p95 latency for agent and flow runs to catch slow executions.
Recent Activity
- Recent LLM Calls: List of the most recent spans where
gen_ai.request.modelexists, for quick inspection and debugging of individual LLM calls.
