Manual instrumentation gives you fine-grained control when automatic instrumentation alone cannot express important business operations. Use it to capture steps that matter for debugging, to attach business-specific attributes, or to guarantee that failures surface with the right context in SigNoz.
Prerequisites
- Finish the core Go setup in the OpenTelemetry Go instrumentation guide so exporters, tracer provider, and propagators are configured.
- Propagate
context.Contextthrough handlers and business logic. Manual spans derive from these contexts so traces stay connected. - Tested with Go 1.25.1 and OpenTelemetry Go SDK v1.38.0.
Step 1. Create manual spans
Use the SDK tracer to wrap important work inside custom spans:
manual-span.go
import (
"context"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
)
func processOrder(ctx context.Context, orderID string) error {
tracer := otel.Tracer("order-service")
ctx, span := tracer.Start(ctx, "process-order")
defer span.End()
span.SetAttributes(
attribute.String("order.id", orderID),
attribute.String("order.status", "processing"),
)
// Use ctx for downstream work so child spans stay linked.
return nil
}
Tips:
- Reuse tracer instances instead of creating a new one for each request.
- Start spans with descriptive names that match business steps (
checkout,fetch-user, etc.). - Always end the span, preferably with
defer.
Step 2. Add attributes and events
Attributes show up as key-value pairs in SigNoz so you can filter and aggregate spans. Events capture notable moments inside a span.
attributes.go
import (
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
)
func handlePayment(ctx context.Context, amount float64) {
span := trace.SpanFromContext(ctx)
span.SetAttributes(
attribute.Float64("payment.amount", amount),
attribute.String("payment.currency", "USD"),
attribute.String("payment.method", "credit_card"),
)
span.AddEvent("payment.processed", trace.WithAttributes(
attribute.String("status", "success"),
attribute.String("transaction.id", "txn_123456"),
))
}
- Keep attribute keys consistent (use semantic conventions when possible).
- Use events to mark retries, cache hits/misses, queue waits, and similar milestones.
Step 3. Record errors
Flag failures on the span so they are easy to query in SigNoz.
error-handling.go
import (
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/trace"
)
func riskyOperation(ctx context.Context) error {
span := trace.SpanFromContext(ctx)
err := doSomethingRisky()
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
return err
}
span.SetStatus(codes.Ok, "")
return nil
}
RecordErrorattaches stack and message details.- Setting status to
codes.Errorsurfaces the span in SigNoz error views and alerts. - Continue propagating the error upstream so calling code can respond.
Validate
- Trigger the code paths that emit manual spans.
- In SigNoz Traces, filter by
service.nameor your span name. - Open a trace and verify attributes, events, and error status.
- Check the Errors view to confirm failures show up with recorded exceptions.
Troubleshooting
Why don't I see my custom spans in SigNoz?
- Confirm
initTracer(and automatic instrumentation) runs before your application handlers. - Check sampler configuration. Using
TraceIDRatioBased(0.01)in dev may drop most manual spans. - Verify traffic hits the functions where you inserted spans.
Why are child spans missing even though I create them?
- Ensure you pass the
ctxreturned bytracer.Start(ortrace.SpanFromContext) into downstream functions. - HTTP/database client calls must receive the propagated context as well, otherwise traces break across boundaries.
Why don't attributes or events appear on the span?
- Attribute keys must be strings and values must use the correct helper (
attribute.String,attribute.Float64, etc.). - Call
span.AddEventorspan.SetAttributesbeforespan.End(). Post-end mutations are ignored. - Avoid reusing finished spans—always create a new span for each invocation.
Next steps
- Send logs from your Go application using popular logging libraries: Logrus, Zap, or Zerolog
- Correlate traces with logs to accelerate triage across signals
- Add instrumentation to common Go frameworks via the Go instrumentation library catalog