Send Metrics from Python application

Library instrumentation automatically collects metrics from supported frameworks like Flask, Django, and FastAPI. Choose your framework below to see the specific run command:

opentelemetry-instrument flask run --no-reload

opentelemetry-instrument python app.py

OpenTelemetry supports six metric instrument types. Here's a complete example demonstrating all of them:

Synchronous Instruments

Synchronous instruments are called inline with your application code.

Counter - A value that only increases (e.g., total requests, bytes sent):

counter = meter.create_counter(
    name="requests.total",
    unit="1",
    description="Total number of requests"
)

# Record a value
counter.add(1, {"endpoint": "/api/users"})

UpDownCounter - A value that can increase or decrease (e.g., active connections, queue size):

updown_counter = meter.create_up_down_counter(
    name="connections.active",
    unit="1",
    description="Number of active connections"
)

# Increment
updown_counter.add(1)

# Decrement
updown_counter.add(-1)

Histogram - A distribution of values (e.g., request latency, response sizes):

histogram = meter.create_histogram(
    name="request.duration",
    unit="ms",
    description="Request duration in milliseconds"
)

# Record a value
histogram.record(45.5, {"endpoint": "/api/users"})

Asynchronous Instruments

Asynchronous instruments use callback functions that are invoked during metric collection.

ObservableCounter - Observes a monotonically increasing value (e.g., CPU time, page faults):

from typing import Iterable
from opentelemetry.metrics import CallbackOptions, Observation

def observable_counter_callback(options: CallbackOptions) -> Iterable[Observation]:
    # Return current cumulative value
    yield Observation(get_total_cpu_time(), {"cpu": "cpu0"})

observable_counter = meter.create_observable_counter(
    name="system.cpu.time",
    callbacks=[observable_counter_callback],
    unit="s",
    description="CPU time"
)

ObservableUpDownCounter - Observes a value that can go up or down (e.g., memory usage, thread count):

def observable_updown_callback(options: CallbackOptions) -> Iterable[Observation]:
    yield Observation(get_current_memory_usage(), {})

observable_updown = meter.create_observable_up_down_counter(
    name="process.memory.usage",
    callbacks=[observable_updown_callback],
    unit="By",
    description="Memory usage in bytes"
)

ObservableGauge - Observes a point-in-time value (e.g., temperature, current CPU utilization):

def observable_gauge_callback(options: CallbackOptions) -> Iterable[Observation]:
    yield Observation(get_current_temperature(), {"location": "server-room"})

gauge = meter.create_observable_gauge(
    name="temperature.current",
    callbacks=[observable_gauge_callback],
    unit="Cel",
    description="Current temperature"
)

To collect system metrics (CPU, memory, disk, network) and Python runtime metrics (garbage collection, thread counts), use the SystemMetricsInstrumentor.

Install

pip install opentelemetry-instrumentation-system-metrics

Usage

Add the instrumentor to your application code:

from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor

# Start collecting system and runtime metrics
SystemMetricsInstrumentor().instrument()

# Your application code here...

Then run with opentelemetry-instrument:

opentelemetry-instrument python app.py

Exported Metrics

When you call SystemMetricsInstrumentor().instrument(), it automatically exports 35+ metrics including:

System Metrics:

• system.cpu.time - CPU time spent in different modes (user, system, idle)
• system.cpu.utilization - CPU utilization percentage
• system.memory.usage - Memory usage in bytes
• system.memory.utilization - Memory utilization as a ratio
• system.disk.io - Disk I/O in bytes (read/write)
• system.network.io - Network I/O in bytes (transmit/receive)

Process Metrics:

• process.cpu.time - Process CPU time (user/system)
• process.memory.usage - Process memory usage (resident set size)
• process.thread.count - Process thread count

CPython Runtime Metrics:

• cpython.gc.collections - Garbage collection count by generation
• cpython.gc.collected_objects - Objects collected by GC

See the OpenTelemetry semantic conventions for the complete list:

• System metrics
• Process metrics
• CPython runtime metrics

Metrics not appearing?

1. Check Environment Variables: Ensure OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_PROTOCOL are set correctly:

   • Endpoint: https://ingest.<region>.signoz.cloud:443
   • Protocol: http/protobuf

2. Check Exporter Protocol: This guide uses http/protobuf. Ensure OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf is set.

3. Check Console Errors: The OpenTelemetry SDK prints errors to stderr by default. Check your application logs for any connection refused or authentication errors.

4. Resource Attributes: Ensure OTEL_SERVICE_NAME is set. This helps you filter metrics by service in SigNoz.

5. Verify Package Installation: Ensure all required packages are installed:

   pip list | grep opentelemetry
   

Authentication errors

If you see errors like "Unauthorized" or "403 Forbidden":

• Verify your ingestion key is correct in OTEL_EXPORTER_OTLP_HEADERS
• Ensure the header format is exactly: signoz-ingestion-key=<your-key> (no extra spaces)
• Check that your ingestion key is active in the SigNoz Cloud dashboard

"Connection Refused" errors

• If running locally and sending to SigNoz Cloud, check your internet connection and firewall.
• If sending to a self-hosted collector, ensure the collector is running and listening on port 4317 (gRPC) or 4318 (HTTP).

SSL/TLS errors

If you encounter SSL certificate errors:

• Ensure you're using https:// in the endpoint URL
• For self-hosted setups with self-signed certificates, you may need to configure the exporter to trust your CA

What is the OpenTelemetry Collector?

Think of the OTel Collector as a middleman between your app and SigNoz. Instead of your application sending data directly to SigNoz, it sends everything to the Collector first, which then forwards it along.

Why use it?

• Cleaning up data — Filter out noisy metrics you don't care about, or remove sensitive info before it leaves your servers.
• Keeping your app lightweight — Let the Collector handle batching, retries, and compression instead of your application code.
• Adding context automatically — The Collector can tag your data with useful info like which Kubernetes pod or cloud region it came from.
• Future flexibility — Want to send data to multiple backends later? The Collector makes that easy without changing your app.

See Switch from direct export to Collector for step-by-step instructions to convert your setup.

For more details, see Why use the OpenTelemetry Collector? and the Collector configuration guide.