LiteLLM Observability & Monitoring with OpenTelemetry

Use no-code auto-instrumentation for a quick setup. You do not change your application code.

Step 1: Install these packages in your Python environment.

pip install \
  opentelemetry-api \
  opentelemetry-distro \
  opentelemetry-exporter-otlp \
  httpx \
  opentelemetry-instrumentation-httpx \
  litellm

Step 2: Install the automatic instrumentation libraries:

opentelemetry-bootstrap --action=install

Step 3: Instrument your LiteLLM SDK application

Set litellm.callbacks = ["otel"] to turn on LiteLLM SDK instrumentation. Set it before your first LiteLLM call:

from litellm import litellm
 
litellm.callbacks = ["otel"]

LiteLLM then sends traces, logs, and metrics for all SDK calls in your application.

Step 4: Run an example

from litellm import completion, litellm
 
litellm.callbacks = ["otel"]
 
response = completion(
  model="openai/gpt-4o",
  messages=[{ "content": "What is SigNoz","role": "user"}]
)
 
print(response)

LiteLLM supports many model providers. This example uses OpenAI. Before you run it, set the OPENAI_API_KEY environment variable.

Step 5: Run your application with auto-instrumentation

OTEL_RESOURCE_ATTRIBUTES="service.name=<service_name>" \
OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443" \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>" \
OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
OTEL_TRACES_EXPORTER=otlp \
OTEL_METRICS_EXPORTER=otlp \
OTEL_LOGS_EXPORTER=otlp \
OTEL_PYTHON_LOG_CORRELATION=true \
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=openai \
opentelemetry-instrument <your_run_command>

OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=openai turns off the OpenAI instrumentor. This prevents conflicts with the built-in LiteLLM instrumentation.

Verify these values:

• <service_name>: The name of your service.
• <region>: Your SigNoz Cloud region.
• <your-ingestion-key>: Your SigNoz ingestion key.
• <your_run_command>: The command that starts your application, for example python main.py.

Code-based instrumentation gives you control over the telemetry configuration. Use it when you need custom resource attributes or sampling, or when you already run OpenTelemetry instrumentation.

Step 1: Install these packages in your Python environment.

pip install \
  opentelemetry-api \
  opentelemetry-sdk \
  opentelemetry-exporter-otlp \
  opentelemetry-instrumentation-httpx \
  opentelemetry-instrumentation-system-metrics \
  litellm

Step 2: Import these modules in your Python application:

Traces:

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

Logs:

from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
from opentelemetry._logs import set_logger_provider
import logging

Metrics:

from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry import metrics
from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor

Step 3: Configure the OpenTelemetry tracer provider to send traces to SigNoz Cloud:

from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry import trace
import os
 
resource = Resource.create({"service.name": "<service_name>"})
provider = TracerProvider(resource=resource)
span_exporter = OTLPSpanExporter(
    endpoint= os.getenv("OTEL_EXPORTER_TRACES_ENDPOINT"),
    headers={"signoz-ingestion-key": os.getenv("SIGNOZ_INGESTION_KEY")},
)
processor = BatchSpanProcessor(span_exporter)
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

Verify these values:

• <service_name>: The name of your service.
• OTEL_EXPORTER_TRACES_ENDPOINT: The SigNoz Cloud trace endpoint for your region, https://ingest.<region>.signoz.cloud:443/v1/traces.
• SIGNOZ_INGESTION_KEY: Your SigNoz ingestion key.

Step 4: Configure logs:

import logging
from opentelemetry.sdk.resources import Resource
from opentelemetry._logs import set_logger_provider
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
import os
 
resource = Resource.create({"service.name": "<service_name>"})
logger_provider = LoggerProvider(resource=resource)
set_logger_provider(logger_provider)
 
otlp_log_exporter = OTLPLogExporter(
    endpoint= os.getenv("OTEL_EXPORTER_LOGS_ENDPOINT"),
    headers={"signoz-ingestion-key": os.getenv("SIGNOZ_INGESTION_KEY")},
)
logger_provider.add_log_record_processor(
    BatchLogRecordProcessor(otlp_log_exporter)
)
# Attach OTel logging handler to root logger
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
logging.basicConfig(level=logging.INFO, handlers=[handler])
 
logger = logging.getLogger(__name__)

Verify these values:

• <service_name>: The name of your service.
• OTEL_EXPORTER_LOGS_ENDPOINT: The SigNoz Cloud logs endpoint for your region, https://ingest.<region>.signoz.cloud:443/v1/logs.
• SIGNOZ_INGESTION_KEY: Your SigNoz ingestion key.

Step 5: Configure metrics:

from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry import metrics
from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
import os
 
resource = Resource.create({"service.name": "<service_name>"})
metric_exporter = OTLPMetricExporter(
    endpoint= os.getenv("OTEL_EXPORTER_METRICS_ENDPOINT"),
    headers={"signoz-ingestion-key": os.getenv("SIGNOZ_INGESTION_KEY")},
)
reader = PeriodicExportingMetricReader(metric_exporter)
metric_provider = MeterProvider(metric_readers=[reader], resource=resource)
metrics.set_meter_provider(metric_provider)
 
meter = metrics.get_meter(__name__)
 
# turn on out-of-the-box metrics
SystemMetricsInstrumentor().instrument()
HTTPXClientInstrumentor().instrument()

Verify these values:

• <service_name>: The name of your service.
• OTEL_EXPORTER_METRICS_ENDPOINT: The SigNoz Cloud metrics endpoint for your region, https://ingest.<region>.signoz.cloud:443/v1/metrics.
• SIGNOZ_INGESTION_KEY: Your SigNoz ingestion key.

SystemMetricsInstrumentor sends system metrics such as CPU and memory use. HTTPXClientInstrumentor sends metrics for outbound HTTP requests, such as request duration. To add custom metrics, see Python Custom Metrics.

Step 6: Instrument your LiteLLM application

Set litellm.callbacks = ["otel"] to turn on LiteLLM SDK instrumentation. Set it before your first LiteLLM call:

from litellm import litellm
 
litellm.callbacks = ["otel"]

LiteLLM then sends traces, logs, and metrics for all SDK calls in your application.

Step 7: Run an example

from litellm import completion, litellm
 
litellm.callbacks = ["otel"]
 
response = completion(
  model="openai/gpt-4o",
  messages=[{ "content": "What is SigNoz","role": "user"}]
)
 
print(response)

LiteLLM supports many model providers. This example uses OpenAI. Before you run it, set the OPENAI_API_KEY environment variable.

The signoz callback sends traces over OTLP/HTTP, but no logs or metrics.

Step 1: Install the LiteLLM Proxy and the OpenTelemetry packages in your Python environment.

pip install 'litellm[proxy]>=1.105.0' \
  opentelemetry-api \
  opentelemetry-sdk \
  opentelemetry-exporter-otlp

If you run the LiteLLM Docker image, use tag v1.105.0 or later. The image already includes these packages.

Step 2: Add the signoz callback to config.yaml:

litellm_settings:
  callbacks: ["signoz"]

Step 3: Set the following environment variables:

export SIGNOZ_INGESTION_ENDPOINT="https://ingest.<region>.signoz.cloud:443"
export SIGNOZ_INGESTION_KEY="<your-ingestion-key>"
export OTEL_SERVICE_NAME="<service_name>"

Verify these values:

• <region>: Your SigNoz Cloud region. LiteLLM adds /v1/traces to the endpoint.
• <your-ingestion-key>: Your SigNoz ingestion key.
• <service_name>: The service name for LiteLLM traces in SigNoz. If you do not set it, LiteLLM uses litellm.

Step 4: Start the proxy server with the configuration file:

litellm --config config.yaml

If you do not set SIGNOZ_INGESTION_ENDPOINT, the proxy logs SIGNOZ_INGESTION_ENDPOINT not found in environment variables and serves requests without traces.

Step 5: Send a test trace:

curl "http://localhost:4000/health/services?service=signoz" \
  -H "Authorization: Bearer <your-litellm-master-key>"

The proxy makes a mock LLM request and returns {"status":"success","message":"Mock LLM request made - check signoz."}. This response only shows that LiteLLM loaded the callback and created the trace. LiteLLM returns it even when SigNoz rejects the ingestion key. Open Traces in SigNoz and make sure that a trace for your service name appears.

Add the SigNoz callback from the Admin UI

If you manage the proxy from the LiteLLM Admin UI, you can add the callback there instead of Steps 2 and 3.

1. In the LiteLLM Admin UI, go to Settings > Logging & Alerts.
2. On the Logging Callbacks tab, click Add Callback.

3. In Callback, select SigNoz.
4. In SigNoz Ingestion Endpoint, enter https://ingest.<region>.signoz.cloud:443.
5. In SigNoz Ingestion Key (optional), enter your SigNoz ingestion key. For Self-Hosted SigNoz, leave it empty.
6. Click Add Callback.

The proxy starts to send traces after you add the callback. If you edit the endpoint or key later, restart the proxy. The proxy keeps the old values until it restarts.

To send the traces of each LiteLLM team or virtual key to a different SigNoz destination, see the LiteLLM SigNoz integration docs. Per-team and per-key settings are a LiteLLM Enterprise feature.

Step 1: Install these packages in your Python environment.

pip install opentelemetry-api \
  opentelemetry-sdk \
  opentelemetry-exporter-otlp \
  'litellm[proxy]'

Step 2: Add the otel callback to config.yaml:

litellm_settings:
  callbacks: ['otel']

Step 3: Set the following environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443"
export OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
export OTEL_TRACES_EXPORTER="otlp"
export OTEL_METRICS_EXPORTER="otlp"
export OTEL_LOGS_EXPORTER="otlp"

Verify these values:

• <region>: Your SigNoz Cloud region.
• <your-ingestion-key>: Your SigNoz ingestion key.

Step 4: Start the proxy server with the configuration file:

litellm --config config.yaml

The proxy then sends a trace to SigNoz for each request.