Migrate traces from ELK Stack

Migrating distributed traces involves replacing the proprietary Elastic APM agents within your applications with standard OpenTelemetry SDKs and configuring them to send trace data to SigNoz.

Replacing Elastic APM Agents

• Core Task: The fundamental step is to remove the Elastic APM agent dependencies and initialization code from your applications and replace them with the equivalent OpenTelemetry SDKs and instrumentation libraries for the specific programming languages.
• Instrumentation Approach:
  • Automatic Instrumentation: OpenTelemetry offers automatic instrumentation capabilities for many popular frameworks and libraries, similar to Elastic APM's approach. Examples include the OTel Java agent for Java, auto-instrumentations-node for Node.js and various SDKs for other languages. This often provides the quickest path to getting traces with minimal code changes.
  • Manual Instrumentation: For custom code sections or frameworks not covered by auto-instrumentation, use the OpenTelemetry SDK's Tracer API to manually create and manage spans. This involves obtaining a Tracer, starting spans, setting attributes, recording events, and ensuring spans are ended correctly. Elastic APM's concepts of "transactions" generally map to root spans in OpenTelemetry. Here's how you can manually instrument your NodeJS application.

Migration Steps

1. Remove and Replace Dependencies

• Uninstall Elastic APM libraries
• Install OpenTelemetry SDK, API, auto-instrumentation, and exporters for your language

2. Configure OpenTelemetry

• Create configuration file to initialize OpenTelemetry SDK
• Set up exporters and automatic instrumentation
• Configure service name and telemetry destination

3. Update Code

• Replace APM tracing calls with OpenTelemetry equivalents
• Update HTTP middleware/handlers to use OpenTelemetry versions
• Ensure proper context propagation between spans

4. Deploy and Validate

• Set environment variables (OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT)
• Update application startup to load OpenTelemetry first
• Test that telemetry data flows correctly to your observability platform

Elastic OpenTelemetry Bridge for Go

For applications where a migration is too risky, Elastic provides a bridge library for Go applications. This bridge allows the OTel Go SDK to coexist with the Elastic APM Go agent. When configured, spans created using the OTel SDK are captured and exported via the existing Elastic APM agent pipeline. This enables an atomic migration.

• When configured, spans created using the OTel SDK are captured and exported via the existing Elastic APM agent pipeline.
  **Steps to follow: **
  1. Add the bridge configuration
  2. Incrementally replace Elastic APM instrumentation with OTel SDK instrumentation, deploying changes piece by piece. Traces continue to flow via the Elastic agent.
  3. Once all instrumentation is migrated, remove the Elastic APM agent and the bridge, and configure the OTel SDK to export directly or via a standard OTel Collector.
• While this specific bridge is for Go, the concept of gradual migration or dual instrumentation might be possible in other languages using community tools or custom solutions.

Verification

• Generate traffic/requests in the newly instrumented application. OpenTelemetry SDKs often buffer data, so ensure sufficient activity occurs.
• Navigate to the SigNoz UI. Check the "Services" list for your application.
• Explore the "Traces" tab, applying filters as needed, to confirm traces are being received.
• Examine individual traces using flamegraphs and Gantt charts to ensure spans are correctly nested and timed.
• Check the Service Map for dependencies.

The migration of tracing is often the most involved part of moving from ELK to SigNoz because it directly impacts application code and dependencies. Unlike metrics or logs, where changes can sometimes be confined to agent or collector configurations, trace migration necessitates re-instrumentation using OpenTelemetry SDKs.This requires developer effort, careful testing, and a solid understanding of OpenTelemetry concepts like spans, context propagation, and resource attributes. Strategies like phased rollouts or bridging mechanisms (where available) are essential to manage the inherent risks.

Signoz Traces Features

• Trace Explorer: Search, filter, and analyze distributed traces
• Service Map: Visualize dependencies between services
• Exceptions Monitoring: Track and analyze application exceptions