Getting Started with OpenTelemetry and Python

OpenTelemetry is a vendor-agnostic instrumentation library under CNCF — the organization hosting projects that are the backbone of modern-day cloud-native systems. Over the years, OTel (as OpenTelemetry is conveniently known) has been adopted widely by the developer community and is being used by organisations of all sizes.

Teams are increasingly using it to standardise telemetry (traces, metrics, and logs) data collection and management across languages, frameworks, and environments. In this practical guide, we’ll walk through everything you need to know to use OpenTelemetry in Python application — how it works, how to instrument Python applications, and how to send telemetry to an OpenTelemetry-compatible backend like SigNoz.

What is OpenTelemetry?

OpenTelemetry emerged as a single project after the merging of OpenCensus (from Google) and OpenTracing (from Uber) into a single project. It seeks to unify the fragmented observability ecosystem and provide users with a vendor-agnostic pipeline to any compatible observability platform. This would enable telemetry signals to become a built-in feature of cloud-native software applications.

OpenTelemetry supports all popular programming languages and has a vast number of Python libraries to integrate into your applications. Python features robust integration with OpenTelemetry because of its immense popularity and use in all industries. Let’s understand how OTel works in more detail.

How Does OpenTelemetry Instrumentation Work with Python?

Instrumentation means adding code to your application to make it emit telemetry data that any OpenTelemetry-compatible backend can process and visualize. You can instrument Python applications in two ways:

• Automatic instrumentation uses dedicated libraries to generate telemetry without requiring code changes, significantly reducing the effort required to instrument applications. However, some libraries do not work with auto-instrumentation, and they might not provide the required level of detail based on your needs.
• Manual instrumentation involves adding telemetry collection code directly to your codebase, enabling more control over the data being collected. This comes at the cost of requiring more developer effort compared to automatic instrumentation.

OpenTelemetry’s Building Blocks: Spans & Traces

A span represents a single unit of work executed by an observable application. A span could be anything, from a local function call, a database query, to a third-party API request to process a payment. A trace represents the entire journey of a request across all your application services, and is composed of multiple spans.

Since spans carry the ID of the traces they belong to, visualization platforms can logically group these spans, and sequentially order them. This concept, known as distributed tracing, ensures that operations across complex and distributed systems can be tracked, cross-referenced and viewed in one pane with ease.

Automatic instrumentation simplifies distributed tracing by injecting trace context into outbound requests, gRPC calls, and producer-consumer messages. You can read more about trace context propagation in this article. The following shows a requests HTTP call carrying the traceparent header:

{
    'User-Agent': 'python-requests/2.26.0',
    'Accept-Encoding': 'gzip, deflate',
    'Accept': '*/*',
    'Connection': 'keep-alive',
    'traceparent': '00-d66349998f87d62441140346b2a0b500-febf8549a13f975e-01'
}

OTel maintains specifications for all components, including the data emitted by instrumented applications. This helps ensure an efficient and reliable observability data pipeline between your instrumented application and the observability platform.

Now that you have established a fundamental understanding of OpenTelemetry, let’s set up the software required to instrument a Python application.

Implementing OpenTelemetry in Python

We’ll be using SigNoz as the observability backend of choice for receiving and visualizing Python application telemetry in a single pane.

Prerequisites for Application Instrumentation

There are the main prerequisites to getting started:

• A SigNoz Cloud Account
• Python 3.10 for the sample application (Download the latest version here)
• A locally running MongoDB instance (find detailed instructions below)

Step 1: Setting Up SigNoz

As discussed previously, we will send the application’s emitted telemetry to SigNoz. SigNoz is an OpenTelemetry-native APM that is well-suited for visualizing OpenTelemetry data.

SigNoz Cloud is the easiest way to run SigNoz. Sign up for a free account and get 30 days of unlimited access to all features.

You can also install and self-host SigNoz yourself since it is open-source. With 24,000+ GitHub stars, open-source SigNoz is loved by developers. Find the instructions to self-host SigNoz.

Step 2: Installing MongoDB Locally

You can skip this step if you have already set up a MongoDB instance on your machine.

The easiest way to run MongoDB locally is to set it up using Docker. Run the following command to start MongoDB:

docker run --rm --name my-mongo -it -dp 27017:27017 mongo:latest

Alternatively, if you’d like to install it directly on your machine, you can find instructions based on your OS:

• macOS
• Linux
• Windows
• Ubuntu

Step 3: Preparing the Sample Python Application

We have prepared a sample Flask application Github repository to get you up-and-running quickly. It is a CRUD TODO application to help you get familiar with OpenTelemetry concepts, and its application and OpenTelemetry dependencies are listed in requirements.txt.

Access the code locally by running:

git clone https://github.com/SigNoz/sample-flask-app.git
cd sample-flask-app

Next, we will create and activate a Python virtual environment. Setting up virtual environments is a recommended practice to avoid dependency conflicts between different Python projects on your system.

# create a virtual environment with `.venv` folder name
python3 -m venv .venv
# activate the virtual environment
source .venv/bin/activate
 
# install all required dependencies
python -m pip install -r requirements.txt

If it hangs while installing the grpcio dependency for opentelemetry-exporter-otlp, follow below steps as suggested in this stackoverflow link.

• pip3 install --upgrade pip
• python3 -m pip install --upgrade setuptools
• pip3 install --no-cache-dir --force-reinstall -Iv grpcio

Verify whether the application is now accessible by running

python3 app.py

You will be greeted by the following UI upon visiting http://localhost:5002/. With this, we have completed the initial application setup.

Step 4: Instrumenting the Python Application

Now, we need to install the required OTel instrumentation libraries to generate telemetry from our application, and set up environment variables so SigNoz can capture and visualize this data.

Before proceeding, stop the application by pressing ctrl + c in your terminal window.

The opentelemetry-bootstrap command installs the corresponding instrumentation libraries for libraries present in the current environment.

opentelemetry-bootstrap --action=install

We’re almost done. In the last step, we just need to configure a few environment variables for the OTLP exporters. The run command will look like this:

OTEL_RESOURCE_ATTRIBUTES=service.name=sample-flask-app \
OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443" \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<SIGNOZ-INGESTION-KEY>" \
OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
opentelemetry-instrument python3 app.py

You can get SigNoz ingestion key and region from your SigNoz cloud account from Settings --> Ingestion.

After running the command, play around with the application at http://localhost:5002/ to generate telemetry data. You can add a few TODOs, mark them as completed, or delete them. Open Services in SigNoz and select sample-flask-app to see its request rate, error rate, and latency.

If latency or errors rise, open Traces and filter by the service and the same time range. Select a slow or failed request and inspect its span waterfall or flamegraph to see how much time was spent in database queries and external calls. This helps you narrow down which operation to investigate.

Troubleshooting

If your service does not appear in SigNoz, first check whether instrumentation is creating spans. Stop the application, then run it with the console trace exporter:

OTEL_SERVICE_NAME=sample-flask-app \
OTEL_TRACES_EXPORTER=console \
OTEL_METRICS_EXPORTER=none \
opentelemetry-instrument python3 app.py

Interact with the application again and look for spans in the terminal. This command sends traces to the console instead of SigNoz. If spans appear, check the endpoint, protocol, ingestion key, and network access in your original command. If they do not, check that the instrumentation packages are installed in the active virtual environment and that you have generated requests.

If spans still do not appear and you are running Flask in debug mode, its reloader may be interfering with instrumentation. Keep debug mode enabled and disable the reloader by setting use_reloader=False:

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=5002, debug=True, use_reloader=False)

If you face any problems in instrumenting with OpenTelemetry, refer OpenTelemetry Python docs.

Going Further with Python and OpenTelemetry

Congratulations- you now have hands-on experience using OpenTelemetry in Python! As you continue exploring observability, the following resources will help you dive deeper into logs, frameworks and more advanced instrumentation techniques.

Configuring the SDK and instrumentation in code

So far, opentelemetry-instrument has configured the SDK and enabled instrumentation for us. If you need custom exporters or instrumentation hooks, you can configure these in code instead. This still uses the library instrumentors to capture Flask requests, MongoDB queries, and outgoing HTTP calls.

In the same virtual environment, install the instrumentors at the version matching the sample’s OpenTelemetry dependencies. These versions require pkg_resources, so keep setuptools below version 81:

python -m pip install opentelemetry-instrumentation-flask==0.48b0 opentelemetry-instrumentation-pymongo==0.48b0 opentelemetry-instrumentation-requests==0.48b0 "setuptools<81"

Create otel_setup.py in the sample application's root directory:

from opentelemetry import metrics, trace
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.pymongo import PymongoInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
 
 
def initialize_opentelemetry():
    provider = TracerProvider()
    provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
    trace.set_tracer_provider(provider)
 
    reader = PeriodicExportingMetricReader(OTLPMetricExporter())
    metrics.set_meter_provider(MeterProvider(metric_readers=[reader]))
 
    FlaskInstrumentor().instrument()
    PymongoInstrumentor().instrument()
    RequestsInstrumentor().instrument()

Call this function once at the beginning of app.py, before its existing imports create the Flask app, MongoDB client, or meter:

from otel_setup import initialize_opentelemetry
 
initialize_opentelemetry()

The providers collect traces and metrics, and the batch processor and periodic reader export them over OTLP. Start the application with the same service name and connection settings from Step 4, using python3 app.py in place of opentelemetry-instrument python3 app.py.

Generating and Exporting Logs

The setup above configures traces and metrics. To collect application logs alongside them, add logging statements and configure OpenTelemetry log export using the Python logging guide.

Capturing Custom Traces for Application-Specific Flows

Automatic instrumentation captures common library operations, but it does not describe application-specific work such as validating an order or processing a payment.

Developers can use custom spans to track specific operations and add custom attributes to capture detailed context. The following code snippet shows an example with dynamic attribute values:

from random import choice
import time
 
from opentelemetry import trace
 
tracer = trace.get_tracer(__name__)
 
 
def validate_data(data: dict) -> bool:
	time.sleep(0.1)  # Simulate a delay in processing
	return choice([True, False])
 
with tracer.start_as_current_span("validate_data") as span:
    data = {"sample": "data"}
    
    span.set_attribute("data.length", len(data))
    is_valid = validate_data(data)
    span.set_attribute("data.validated", is_valid)

For more examples of child spans, attributes, and events, see the Python manual instrumentation guide.

Implementing Custom Metrics

Custom metrics derive performance KPIs and measure business-specific health signals. Automatic instrumentation does not measure these values as they require deep insight into an application’s business context, and must be measured manually.

We have used a custom meter to monitor the number of active TODOs in our sample application. The relevant code snippet is as follows:

from flask import Flask
from opentelemetry.metrics import get_meter_provider
 
app = Flask(__name__)
 
# create global meter provider and up-down counter
meter = get_meter_provider().get_meter("sample-flask-app", "0.1.2") 
todo_counter = meter.create_up_down_counter("todo_count")
 
 
@app.route("/action", methods=['POST'])
def action():
    #Adding a Task
    name=request.values.get("name")
    desc=request.values.get("desc")
    date=request.values.get("date")
    pr=request.values.get("pr")
    todos.insert({ "name":name, "desc":desc, "date":date, "pr":pr, "done":"no"})
    todo_counter.add(1) # increment the counter to signal new TODO
    return redirect("/")
 
@app.route("/remove")
def remove():
    #Deleting a Task with various references
    key=request.values.get("_id")
    todos.remove({"_id":ObjectId(key)})
    todo_counter.add(-1) # decrement the counter to signal deleted TODO
    return redirect("/")

For an overview of the instruments and more Python examples, see our OpenTelemetry metrics guide.

Instrumenting FastAPI and Django Applications

We used Flask for this demo because of its ease-of-use and familiarity within the Python developer community. We have a similar blog post covering instrumentation for Django, and an in-depth guide for FastAPI covering advanced techniques like sampling and data retention.

Instrumenting Falcon applications

For a Falcon application, install its dependencies first, then install the Falcon instrumentation alongside the OpenTelemetry distro and OTLP exporter:

python -m pip install opentelemetry-distro opentelemetry-exporter-otlp opentelemetry-instrumentation-falcon
opentelemetry-bootstrap --action=install

The Falcon instrumentation supports WSGI applications served by Gunicorn. For a Falcon ASGI application, use OpenTelemetry's ASGI middleware instead.

Use the same exporter settings as in Step 4, with your Falcon service name and Gunicorn command:

OTEL_RESOURCE_ATTRIBUTES=service.name=sample-falcon-app \
OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443" \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<SIGNOZ-INGESTION-KEY>" \
OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
opentelemetry-instrument gunicorn -b 0.0.0.0:8001 src.app:app

Replace src.app:app with your application's module and WSGI app object, and use your SigNoz region and ingestion key. Run without --reload to avoid reloader-related instrumentation issues.

Send requests to an endpoint in your application to generate traces. If you are using the sample Falcon application, visit http://localhost:8001/api/v1/test. Open Services in SigNoz and select sample-falcon-app to inspect the requests and their traces.