Docker Swarm Collection Agent - Install

This guide walks you through installing the OpenTelemetry Collector contrib image on your Docker Swarm cluster. The collector is deployed as a global service (one replica per node), collecting host metrics, container metrics, container logs, and forwarding application traces to SigNoz.

Prerequisites

Docker Engine 20.10+ installed and running
Docker Swarm initialized (docker swarm init if not already)
SigNoz Cloud account or self-hosted SigNoz instance
Ports 4317, 4318 available on each Swarm node

Step 1: Create Collector Configuration

Create a config.yaml file with the following content. This config is tailored for SigNoz usage: it collects host metrics, container metrics, container logs, and accepts OTLP data from instrumented applications.

config.yaml

receivers:
  # OTLP: accept traces, metrics, and logs from instrumented apps (SDKs, agents)
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  # Host metrics: CPU, disk, memory, network, etc. (root_path for container)
  hostmetrics:
    collection_interval: 60s
    root_path: /hostfs
    scrapers:
      cpu: {}
      disk: {}
      load: {}
      filesystem: {}
      memory: {}
      network: {}
      paging: {}
      process:
        mute_process_name_error: true
        mute_process_exe_error: true
        mute_process_io_error: true
        mute_process_user_error: true
      processes: {}
  # Container metrics from Docker daemon
  docker_stats:
    endpoint: unix:///var/run/docker.sock
    collection_interval: 10s
    timeout: 20s
  # Container logs: Docker JSON log files
  filelog:
    include: [/var/lib/docker/containers/*/*-json.log]
    start_at: end
    include_file_name: false
    include_file_path: true
    operators:
      - id: container-parser
        type: container
        format: docker
        add_metadata_from_filepath: false

processors:
  batch:
    send_batch_size: 1000
    timeout: 10s
  resourcedetection:
    detectors: [env, system, docker]
    timeout: 2s
    system:
      hostname_sources: [os]
  memory_limiter:
    check_interval: 5s
    limit_mib: 4000
    spike_limit_mib: 800

exporters:
  otlp:
    endpoint: 'ingest.<region>.signoz.cloud:443'
    tls:
      insecure: false
    headers:
      'signoz-ingestion-key': '<your-ingestion-key>'

extensions:
  health_check:
    endpoint: 0.0.0.0:13133
  pprof:
    endpoint: 0.0.0.0:1777
  zpages:
    endpoint: 0.0.0.0:55679

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlp]
    metrics:
      receivers: [otlp, hostmetrics, docker_stats]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlp]
    logs:
      receivers: [otlp, filelog]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlp]

Verify the following values:

<region>: Your SigNoz Cloud region
<your-ingestion-key>: Your SigNoz Cloud ingestion key

Step 2: Deploy the Collector Service

Deploy the collector as a global service so one replica runs on each Swarm node. Each replica collects host metrics, container metrics, and logs from its local node.

Create a docker-stack.yaml file:

docker-stack.yaml

services:
  signoz-collection-agent:
    image: otel/opentelemetry-collector-contrib:0.139.0
    user: "0:0"
    hostname: "{{.Node.Hostname}}"
    environment:
      - OTEL_RESOURCE_ATTRIBUTES=deployment.environment=<your-environment>
    configs:
      - source: otelcol_config
        target: /etc/otelcol-contrib/config.yaml
        mode: 0444
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /var/lib/docker/containers
        target: /var/lib/docker/containers
        read_only: true
      - type: bind
        source: /
        target: /hostfs
        read_only: true
    ports:
      - target: 4317
        published: 4317
        protocol: tcp
        mode: host
      - target: 4318
        published: 4318
        protocol: tcp
        mode: host
    deploy:
      mode: global
      restart_policy:
        condition: any

configs:
  otelcol_config:
    file: ./config.yaml

Deploy the stack:

docker stack deploy -c docker-stack.yaml signoz

To update the config, edit config.yaml and redeploy with the same command. Docker Swarm automatically creates a new config version and updates the running tasks.

First, create a Swarm config from your config.yaml:

docker config create otelcol_config ./config.yaml

Then deploy the service:

docker service create \
  --name signoz-collection-agent \
  --mode global \
  --detach \
  --restart-condition any \
  --user 0:0 \
  --hostname '{{.Node.Hostname}}' \
  --env OTEL_RESOURCE_ATTRIBUTES=deployment.environment=<your-environment> \
  --config source=otelcol_config,target=/etc/otelcol-contrib/config.yaml,mode=0444 \
  --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \
  --mount type=bind,source=/var/lib/docker/containers,target=/var/lib/docker/containers,readonly \
  --mount type=bind,source=/,target=/hostfs,readonly \
  --publish target=4317,published=4317,protocol=tcp,mode=host \
  --publish target=4318,published=4318,protocol=tcp,mode=host \
  otel/opentelemetry-collector-contrib:0.139.0

To update the config later, create a new config (for example otelcol_config_v2), update the service to use it, and remove the old config.

Optional: Attach to an overlay network

If your application services send OTLP over an overlay network, attach the collector to that network by adding --network your_overlay_network to the command above.

Verify the following values:

<your-environment>: The environment name for the deployment

Step 3: Validate

Check that the service is running:

docker service ls | grep signoz-collection-agent
docker service ps signoz-collection-agent

View collector logs:

docker service logs signoz-collection-agent

You should see log lines indicating the receivers started successfully:

... Everything is ready. Begin running and processing data.

In SigNoz, navigate to Infrastructure Monitoring → Hosts and verify your Swarm nodes appear. Check Logs Explorer to confirm container logs are flowing.

Troubleshooting

Service tasks failing on some nodes

Bind mounts require the source path to exist on each node. Ensure /var/run/docker.sock, /var/lib/docker/containers, and / exist on all Swarm nodes. If a node lacks these paths (for example, a Windows worker), add a placement constraint to exclude it:

docker service update signoz-collection-agent \
  --constraint-add 'node.platform.os==linux'

Permission denied on Docker socket

If the collector logs show permission errors accessing /var/run/docker.sock, ensure the service runs as root. The example uses --user 0:0. Alternatively, add the Docker group:

docker service create ... --group-add $(getent group docker | cut -d: -f3) ...

No host metrics appearing

Verify the host filesystem is mounted correctly. Inspect a running task:

docker inspect $(docker ps -q -f name=signoz-collection-agent) | grep -A5 hostfs

Ensure the root_path: /hostfs setting in the hostmetrics receiver matches the mount target.

Configuration errors

Check the collector logs for configuration parsing errors:

docker service logs signoz-collection-agent 2>&1 | head -50

Common causes:

Invalid YAML syntax in config.yaml
Missing or incorrect SigNoz ingestion key
Port conflicts with other services on 4317 or 4318

Updating the config

Docker Stack: Edit config.yaml and redeploy:

docker stack deploy -c docker-stack.yaml signoz

docker service create: Create a new config and update the service manually:

Create a new config: docker config create otelcol_config_v2 ./config.yaml
Update the service: docker service update --config-rm otelcol_config --config-add source=otelcol_config_v2,target=/etc/otelcol-contrib/config.yaml,mode=0444 signoz-collection-agent
Remove the old config: docker config rm otelcol_config