An overview of Context Propagation in OpenTelemetry

Last Updated: November 03, 20267 min read

To effectively manage modern applications, you need to understand how they work on the inside. Distributed tracing is the key to this, providing a detailed picture of a request's journey across every service. OpenTelemetry has emerged as the industry-standard framework for implementing tracing and achieving true observability in complex, distributed systems.

This article explains how context propagation keeps a request's operations connected, how OpenTelemetry passes that context between services, and when you need to propagate it manually.

The Fundamentals of Context Propagation

Context propagation is fundamental to OpenTelemetry's tracing. It's the mechanism for transmitting essential trace and span contexts across services in a distributed application, ensuring every operation is linked to its origin.

Trace Context

Trace context carries the information needed to connect operations as a request moves across services. The trace ID identifies the entire trace, while the sending span's ID lets the receiving service link its work to the caller.

Span Context

In OpenTelemetry, span context is the part of a span that propagators pass between services. It includes:

Span ID: A unique identifier for each span within the trace.
Trace ID: The shared identifier that connects spans in the same trace.
Trace flags: Information such as whether the trace is sampled.
Trace state: Optional tracing-system-specific information.

The span itself records the operation's start and end times. Application-specific values, such as a customer tier, can travel alongside trace context as baggage.

Let's follow how OpenTelemetry creates spans and propagates their context through a request.

The Journey of a Trace from Instrumentation to Export Explained

1. Instrumentation

The first step is instrumentation, which involves integrating the OpenTelemetry SDK into your application. This can be done with auto-instrumentation libraries for many popular frameworks and languages, often requiring little to no code changes or manual instrumentation depending on the amount of telemetry data you want to collect. Instrumentation is the gateway for OpenTelemetry to collect telemetry data, as it captures specific events and operations as spans within a trace.

2. Context Creation

When instrumentation starts a span, it uses the current context to find its parent. A child span keeps its parent's trace ID and gets a new span ID. If there is no valid parent context, it starts a new trace.

3. Propagation Mechanisms

With the context created, it needs to be transmitted between services. This is where propagation mechanisms come in. OpenTelemetry supports various methods for this, discussed in the next section.

For text-based protocols like HTTP, this process is managed by a Trace Context Text Map Propagator. This component is responsible for injecting the trace context into the headers of an outgoing request and extracting it on the receiving end. Text map is a simple key-value store where both keys and values are strings. In the context of web services, HTTP headers are a common example of a text map.

The choice of propagation mechanism depends on your application's architecture. These mechanisms are the conduits through which the trace and span context travel as a request flows through your system.

4. Custom Span Creation and Data Collection

As the context is propagated, developers can create custom spans to represent individual units of work within the application. These spans are linked together by the propagated context, forming a complete trace. Within each span, OpenTelemetry collects a wealth of telemetry data, including timing information, events, and other details that are crucial for understanding system behavior and performance.

5. Exporting Telemetry Data

The final step is exporting telemetry data. OpenTelemetry sends the collected data to a backend system, such as a tracing database or an observability platform like SigNoz. This is where the data is analyzed and visualized, allowing you to troubleshoot issues, optimize performance, and gain a deeper understanding of your application.

The Varied Landscape of Context Propagation Mechanisms

1. HTTP Headers

HTTP headers are commonly employed for context propagation in web-based applications. They offer a straightforward way to carry trace and span context between HTTP requests and responses. Two essential HTTP headers are used for this purpose:

traceparent: This header contains the trace and span IDs, ensuring that context is consistently transferred along with each HTTP request.
tracestate: This header carries optional tracing-system-specific information. Application values such as customer tier travel separately in the baggage header.

2. gRPC Metadata

In systems utilizing gRPC for communication, gRPC metadata is the chosen mechanism for context propagation. Similar to HTTP headers, gRPC metadata includes trace and span context information. OpenTelemetry seamlessly integrates with gRPC, enabling the straightforward propagation of context within gRPC-based microservices.

3. Message Queues

In message-driven architectures, where systems communicate through message queues like Apache Kafka or RabbitMQ, context propagation is achieved by embedding trace context within messages. Each message carries trace and span IDs, allowing for the correlation of events and the tracing of messages as they traverse the messaging system.

4. Custom Propagation

In certain scenarios, you might find it necessary to implement custom context propagation mechanisms tailored to your specific requirements. OpenTelemetry's flexibility allows you to design custom approaches for transmitting context information that aligns with your application's unique needs.

These mechanisms are vital for maintaining visibility and traceability in distributed systems, enhancing observability and troubleshooting capabilities across services and components.

Manual Context Propagation: Taking Control

For a messaging integration without automatic propagation, you can pass context with the message yourself. The following simplified Python example shows Service A sending a message to Service B through RabbitMQ.

# Service A: Sending a message with trace context
from opentelemetry import trace
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
 
carrier = {}
# Create a span for the current operation in Service A
with trace.get_tracer(__name__).start_as_current_span("Operation in Service A") as span:
    # Write the current context into the carrier.
    # A TextMapPropagator works with any dict-like object as its Carrier by default.
    TraceContextTextMapPropagator().inject(carrier)
 
    # Attach trace context to the message
    message = {
        "content": "Hello, Service B!",
        "trace_context": carrier
    }
 
    # Publish the message to RabbitMQ
    rabbitmq_publisher.publish(message)

In Service B, upon receiving the message, we extract the trace context and continue the trace:

# Service B: Receiving and extracting trace context from the message
from opentelemetry import trace
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
 
def process_message(message):
    # Extract the trace context from the received message
    carrier = message.get("trace_context", {})
    ctx = TraceContextTextMapPropagator().extract(carrier=carrier)
 
    # Create a new span in Service B using the extracted trace context
    with trace.get_tracer(__name__).start_as_current_span("Operation in Service B", context=ctx):
        # Your code for processing the message here
        pass

Service A injects its current span's context into the message. Service B extracts it and creates a child span with the same trace ID. This keeps the sending and processing operations connected even though they run in separate services.

Conclusion

Context propagation lets you follow a request within a service and across service boundaries. OpenTelemetry instrumentation handles it for supported libraries. Where that support is missing, injecting and extracting context manually keeps the work connected to the original trace.

Once you have collected data with OpenTelemetry, you can use a backend like SigNoz to store and analyze your OpenTelemetry data. In SigNoz, you can inspect the connected spans to see where a request spent time.

*Traces and spans as visualised in SigNoz*

Getting started with SigNoz

SigNoz Cloud is the easiest way to run SigNoz. Sign up for a free account and get 30 days of unlimited access to all features.

You can also install and self-host SigNoz yourself since it is open-source. With 24,000+ GitHub stars, open-source SigNoz is loved by developers. Find the instructions to self-host SigNoz.

For more on how traces are generated and connected across services, see our guide to OpenTelemetry tracing. Learn how spans in distributed tracing capture individual units of work, or compare options in our roundup of distributed tracing tools.

For a comparison of OpenTelemetry and Jaeger and how they work together, see OpenTelemetry vs Jaeger.

An Open-Source OpenTelemetry APM