How to add manual instrumentation in Python

Manual instrumentation gives you fine-grained control when automatic instrumentation alone cannot express important business operations. Use it to capture steps that matter for debugging, to attach business-specific attributes, or to guarantee that failures surface with the right context in SigNoz.

Prerequisites

Finish the core Python setup in the OpenTelemetry Python instrumentation guide so exporters, tracer provider, and propagators are configured.

Install the API and SDK packages:

pip install opentelemetry-api opentelemetry-sdk

Tested with Python 3.11 and OpenTelemetry Python SDK v1.27.0.

Step 1. Create manual spans

Initialize a tracer and wrap important work inside custom spans:

manual_span.py

from opentelemetry import trace

tracer = trace.get_tracer("order-service")

def process_order(order_id: str):
    with tracer.start_as_current_span("process-order") as span:
        span.set_attribute("order.id", order_id)
        span.set_attribute("order.status", "processing")

        # Business logic here
        # Child spans created in called functions will be linked automatically
        validate_order(order_id)

    # Span ends automatically when exiting the 'with' block

For nested operations, create child spans that link to the parent:

nested_spans.py

def process_order(order_id: str):
    with tracer.start_as_current_span("process-order") as parent:
        parent.set_attribute("order.id", order_id)

        # Nested span tracks a sub-operation
        with tracer.start_as_current_span("validate-inventory") as child:
            child.set_attribute("warehouse.id", "WH-001")
            check_inventory(order_id)

Tips:

Reuse tracer instances instead of creating a new one for each request.
Start spans with descriptive names that match business steps (checkout, fetch-user, etc.).
Use with blocks or decorators to ensure spans always end.

Using decorators

For functions where the span should cover the entire execution:

decorator_span.py

@tracer.start_as_current_span("do_work")
def do_work():
    print("doing some work...")
    # Span is created on function entry and ends on exit

Step 2. Add attributes and events

Attributes show up as key-value pairs in SigNoz so you can filter and aggregate spans. Events capture notable moments inside a span.

attributes.py

from opentelemetry import trace

def handle_payment(amount: float, currency: str = "USD"):
    span = trace.get_current_span()

    span.set_attribute("payment.amount", amount)
    span.set_attribute("payment.currency", currency)
    span.set_attribute("payment.method", "credit_card")

    # Events mark notable moments within the span
    span.add_event("payment.validated")

    # Process payment...

    span.add_event("payment.processed", {
        "status": "success",
        "transaction.id": "txn_123456"
    })

For semantic attributes, use the conventions package:

pip install opentelemetry-semantic-conventions

semantic_attributes.py

from opentelemetry import trace
from opentelemetry.semconv.attributes.http_attributes import HTTP_REQUEST_METHOD
from opentelemetry.semconv.attributes.url_attributes import URL_FULL

span = trace.get_current_span()
span.set_attribute(HTTP_REQUEST_METHOD, "GET")
span.set_attribute(URL_FULL, "https://api.example.com/users")

Keep attribute keys consistent (use semantic conventions when possible).
Use events to mark retries, cache hits/misses, queue waits, and similar milestones.

Step 3. Record errors

Flag failures on the span so they are easy to query in SigNoz.

error_handling.py

from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode

def risky_operation():
    span = trace.get_current_span()

    try:
        result = do_something_risky()
        span.set_status(Status(StatusCode.OK))
        return result
    except Exception as ex:
        span.record_exception(ex)
        span.set_status(Status(StatusCode.ERROR, str(ex)))
        raise

record_exception attaches stack trace and message details.
Setting status to StatusCode.ERROR surfaces the span in SigNoz error views and alerts.
Re-raise the exception so calling code can respond appropriately.

Step 4. Add span links (optional)

Links connect spans that are causally related but not in a parent-child relationship:

span_links.py

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

# First operation
with tracer.start_as_current_span("span-1"):
    ctx = trace.get_current_span().get_span_context()
    link_from_span_1 = trace.Link(ctx)

# Second operation linked to the first
with tracer.start_as_current_span("span-2", links=[link_from_span_1]):
    # span-2 is causally associated with span-1, but not a child
    pass

Validate

Trigger the code paths that emit manual spans.
In SigNoz Traces, filter by service.name or your span name.
Open a trace and verify attributes, events, and error status.
Filter traces with hasError in [true] in the Trace Explorer to confirm failures show up with recorded exceptions.

Troubleshooting

Why don't I see my custom spans in SigNoz?

Confirm the tracer provider is initialized before your application code runs.
Check sampler configuration. Using ratio-based sampling in dev may drop most manual spans.
Verify traffic hits the functions where you inserted spans.

Why are child spans missing even though I create them?

Ensure you use start_as_current_span which automatically sets the parent context.
If using start_span directly, you need to manually manage context propagation.
Check that the parent span hasn't ended before child spans are created.

Why don't attributes or events appear on the span?

Attribute values must be strings, booleans, numbers, or lists of these types.
Call set_attribute or add_event before the span ends. Post-end mutations are ignored.
When using with blocks, add attributes inside the block before it exits.

Spans not connected across async operations?

For async code, use trace.get_current_span() within the async function.
Consider using contextvars or explicitly passing span context for complex async flows.

Next steps

Send logs from Python using auto-instrumentation
Return to the Python instrumentation guide for framework-specific setup