Migrate Metrics from ELK Stack

Overview

This guide walks you through migrating metrics from the ELK Stack (specifically Metricbeat) to SigNoz. You will:

1. Check your Metricbeat modules and configuration
2. Set up metric collection with OpenTelemetry
3. Map Metricbeat modules to OpenTelemetry receivers
4. Validate metrics are flowing correctly to SigNoz

Key Differences: ELK vs SigNoz

SigNoz uses the OpenTelemetry Collector for metric collection, which offers a wide range of receivers to replace Metricbeat modules.

Prerequisites

Before starting, ensure you have:

• A SigNoz account (Cloud) or a running SigNoz instance (Self-Hosted)
• Access to your Metricbeat configuration (metricbeat.yml, modules.d/)
• Administrative access to deploy the OpenTelemetry Collector

Step 1: Assess Your Current Metrics

Before migrating, inventory what you're collecting with Metricbeat.

Identify Active Modules

Review your metricbeat.yml and modules.d/ directory to see which modules are enabled. Common modules include:

• system (CPU, memory, disk)
• docker (Container stats)
• kubernetes (Kubelet stats)
• prometheus (Scraping endpoints)
• nginx, redis, mysql, etc.

Categorize Your Metrics

Group your metrics by source type:

Step 2: Set Up the OpenTelemetry Collector

Install the OpenTelemetry Collector if you haven't already:

1. Install the OpenTelemetry Collector in your environment.

2. Configure the OTLP exporter for metrics

Step 3: Migrate Each Metric Source

Work through each module from your inventory. For a complete list of supported receivers, see Send Metrics to SigNoz.

From System Metrics

Replace the Metricbeat system module with the hostmetrics receiver. See Host Metrics for detailed configuration.

Metricbeat system:

- module: system
  period: 10s
  metricsets: ["cpu", "memory", "network", "filesystem"]

OTel hostmetrics:

receivers:
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
      memory:
      network:
      filesystem:
      load:
      disk:

Enable it in the pipeline:

service:
  pipelines:
    metrics:
      receivers: [otlp, hostmetrics]
      processors: [batch, resourcedetection]
      exporters: [otlp]

From Docker Metrics

Replace the Metricbeat docker module with the docker_stats receiver. See Docker Container Metrics for detailed configuration.

Metricbeat docker:

- module: docker
  period: 10s
  hosts: ["unix:///var/run/docker.sock"]

OTel docker_stats:

receivers:
  docker_stats:
    endpoint: "unix:///var/run/docker.sock"
    collection_interval: 10s
    timeout: 5s
    metrics:
      container.cpu.utilization:
        enabled: true

From Kubernetes Metrics

For Kubernetes environments, we recommend using the SigNoz K8s Infra Helm chart.

When installed, it automatically collects all necessary metrics (node, pod, container, volume) without requiring manual receiver configuration.

See Install K8s Infra for setup instructions.

From Prometheus Endpoints

If you used Metricbeat's prometheus module to scrape endpoints, use the OTel prometheus receiver. See Prometheus Metrics for detailed configuration.

Metricbeat prometheus:

- module: prometheus
  period: 10s
  hosts: ["localhost:9090"]
  metrics_path: /metrics

OTel prometheus:

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'my-app'
          scrape_interval: 10s
          static_configs:
            - targets: ['localhost:9090']

From Application Custom Metrics

If you were sending custom metrics using Elastic APM agents, migrate to OpenTelemetry Metrics SDKs.

1. Initialize the MeterProvider in your application.
2. Create instruments (Counters, Gauges, Histograms).
3. Record values.

See Send Metrics to SigNoz for language-specific guides.

From Cloud Integrations

If you were using Metricbeat modules for cloud providers (AWS, Azure, GCP), use SigNoz's native cloud integrations or the OpenTelemetry Collector's cloud receivers.

• AWS: AWS Cloud Integrations
• Azure: Azure Monitoring
• GCP: GCP Monitoring

From Synthetics

If you were using Elastic Uptime (Heartbeat), migrate to SigNoz HTTP Host Metrics.

SigNoz supports monitoring HTTP endpoints for availability and latency. See Monitor HTTP Endpoints.

Step 4: Configure Processors

To replicate Metricbeat's metadata enrichment (like add_host_metadata, add_kubernetes_metadata), configure OTel processors.

Resource Detection

Replaces add_host_metadata and add_cloud_metadata.

processors:
  resourcedetection:
    detectors: [env, system, ec2, gcp, azure]
    timeout: 2s

Validate

Verify metrics are flowing correctly by comparing against your source inventory.

Check Metrics Are Arriving

1. In SigNoz, navigate to Metrics in the left sidebar.
2. Use the List View to browse available metrics.
3. Search for metrics like system_cpu_time, container_cpu_usage_seconds, etc.

Next Steps

Once your metrics are flowing to SigNoz:

• Create dashboards to visualize metrics
• Set up alerts based on your metrics
• Migrate traces for end-to-end observability