Overview
Docker writes container output to JSON log files on the host and daemon events to journald or syslog. This guide covers collecting both with an OpenTelemetry Collector.
Prerequisites
- Docker Engine 20.10+ installed and running
- Administrative access to the Docker host
- OpenTelemetry Collector installed
Setup
Step 1. Update Collector configuration
Two approaches for collecting container logs, pick one:
- filelog: tails all container log files under a single glob. Simpler setup. Requires only read access to
/var/lib/docker/containers. - receiver_creator + docker_observer: spawns one receiver per container by watching the Docker socket. Sets
container.nameandcontainer.image.nameautomatically with no regex.
Get your Collector's container ID so you can exclude its own logs:
docker inspect opentelemetry-collector --format='{{.Id}}'
Add to your config.yaml, replacing <OTEL_COLLECTOR_CONTAINER_ID> with that ID:
receivers:
filelog/docker:
include: [/var/lib/docker/containers/*/*-json.log]
exclude:
- /var/lib/docker/containers/<OTEL_COLLECTOR_CONTAINER_ID>/*
poll_interval: 200ms
start_at: end
include_file_name: false
include_file_path: true
operators:
- id: container-parser
type: container
format: docker
add_metadata_from_filepath: false
- id: extract-container-id
type: regex_parser
parse_from: attributes["log.file.path"]
regex: '/var/lib/docker/containers/(?P<container_id>[a-f0-9]{64})/'
on_error: send
- id: move-container-id
type: move
from: attributes.container_id
to: resource["container.id"]
if: '"container_id" in attributes'
- id: remove-filepath
type: remove
field: attributes["log.file.path"]
- id: extract-service-name
type: regex_parser
parse_from: body
regex: '"service\.name":\s*"(?P<svc>[^"]+)"'
if: 'body matches "service\\.name"'
on_error: send
- id: set-service-name
type: move
from: attributes.svc
to: resource["service.name"]
if: '"svc" in attributes'
processors:
memory_limiter:
check_interval: 1s
limit_mib: 4000
spike_limit_mib: 800
resourcedetection:
detectors: [env, system]
timeout: 5s
system:
hostname_sources: [os]
transform/docker-meta:
log_statements:
- context: resource
statements:
- set(attributes["container.runtime"], "docker")
batch:
send_batch_size: 1024
send_batch_max_size: 2048
timeout: 10s
service:
pipelines:
logs:
receivers: [filelog/docker]
processors: [memory_limiter, resourcedetection, transform/docker-meta, batch]
exporters: [otlp]
What each operator does:
- exclude: Stops the Collector from ingesting its own logs.
- include_file_path: Keeps
log.file.pathin the entry so the next operator can read the container ID. - extract-container-id / move-container-id: Reads the 64-char container ID from the file path and sets it as
resource["container.id"]. - remove-filepath: Drops
log.file.pathafter extraction. - extract-service-name / set-service-name: Runs only when
bodymatchesservice.name. Pulls the value intoresource["service.name"]and skips all other logs. - resourcedetection: Adds
host.nameand OS info to every log entry. - transform/docker-meta: Sets
container.runtime = "docker"on the resource for runtime filtering in SigNoz.
format: docker matches Docker's JSON log driver. The container operator also supports cri-o and containerd.
receiver_creator watches the Docker socket via docker_observer and spawns one dedicated filelog receiver per container. Docker metadata populates container.name and container.image.name directly, no regex or manual ID lookup.
Add to your config.yaml:
extensions:
docker_observer:
endpoint: unix:///var/run/docker.sock
excluded_images:
- 'otel/opentelemetry-collector-contrib*'
timeout: 5s
cache_sync_interval: 60m
receivers:
receiver_creator/docker-logs:
watch_observers: [docker_observer]
receivers:
filelog/per-container:
rule: type == "container"
resource_attributes:
container.id: '`container_id`'
container.name: '`name`'
container.image.name: '`image`'
config:
include:
- /var/lib/docker/containers/`container_id`/`container_id`-json.log
include_file_path: false
include_file_name: false
start_at: end
poll_interval: 200ms
operators:
- id: container-parser
type: container
format: docker
add_metadata_from_filepath: false
- id: extract-service-name
type: regex_parser
parse_from: body
regex: '"service\.name":\s*"(?P<svc>[^"]+)"'
if: 'body matches "service\\.name"'
on_error: send
- id: set-service-name
type: move
from: attributes.svc
to: resource["service.name"]
if: '"svc" in attributes'
processors:
memory_limiter:
check_interval: 1s
limit_mib: 4000
spike_limit_mib: 800
resourcedetection:
detectors: [env, system]
timeout: 5s
system:
hostname_sources: [os]
transform/docker-meta:
log_statements:
- context: resource
statements:
- set(attributes["container.runtime"], "docker")
batch:
send_batch_size: 1024
send_batch_max_size: 2048
timeout: 10s
service:
extensions: [docker_observer]
pipelines:
logs:
receivers: [receiver_creator/docker-logs]
processors: [memory_limiter, resourcedetection, transform/docker-meta, batch]
exporters: [otlp]
What each key part does:
excluded_images: Drops the Collector container by image name glob; no container ID lookup needed.docker_observer: Watches the Docker socket and fires a discovery event per running container.receiver_creator: Spawns a dedicatedfilelogreceiver per discovered container, targeting its exact log file.resource_attributes: Setscontainer.nameandcontainer.image.namedirectly from Docker metadata.
Step 2. Start the Collector
docker stop opentelemetry-collector
docker rm opentelemetry-collector
docker run --name opentelemetry-collector \
--restart unless-stopped \
--detach \
--user 0:0 \
--pid host \
--network host \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /proc:/hostfs/proc:ro \
-v /sys:/hostfs/sys:ro \
-v /:/hostfs:ro \
-v "$(pwd)/config.yaml":/etc/otelcol-contrib/config.yaml \
-e HOST_PROC=/hostfs/proc \
-e HOST_SYS=/hostfs/sys \
-e HOST_ROOT=/hostfs \
otel/opentelemetry-collector-contrib:latest
Create a docker-compose.yaml in the same directory as your config.yaml:
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:latest
container_name: otel-collector-signoz
restart: unless-stopped
user: "0:0"
volumes:
- ./config.yaml:/etc/otelcol-contrib/config.yaml
- /:/hostfs:ro
- /proc:/hostfs/proc:ro
- /sys:/hostfs/sys:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- HOST_PROC=/hostfs/proc
- HOST_SYS=/hostfs/sys
- HOST_ROOT=/hostfs
command: ["--config=/etc/otelcol-contrib/config.yaml"]
logging:
options:
max-size: 128m
docker compose up -d
Validate
Generate some test logs:
docker run --rm alpine sh -c 'for i in $(seq 1 10); do echo "Test log entry $i"; sleep 1; done'
Open Logs Explorer in SigNoz. The test log lines from the Alpine container appear within seconds.
Docker daemon logs are separate from container stdout/stderr. On Linux, Docker writes them to journald or syslog depending on your distribution.
Run on the Docker host to check which applies:
sudo journalctl -u docker.service -n 20
If you see output like this, your daemon uses journald:
-- Journal begins at Mon 2025-01-13 08:00:01 UTC. --
Apr 23 12:01:05 myhost dockerd[1234]: time="2025-04-23T12:01:05Z" level=info msg="Container started" container=abc123
Apr 23 12:01:06 myhost dockerd[1234]: time="2025-04-23T12:01:06Z" level=info msg="attach stdout" container=abc123
If the command returns no output or an error, your daemon uses syslog instead. Use the matching tab below.
Step 1. Add the journald receiver
Append the journald receiver to your existing config.yaml:
receivers:
journald/docker-daemon:
root_path: /hostfs
journalctl_path: /usr/bin/journalctl
start_at: end
units:
- docker.service
service:
pipelines:
logs:
receivers: [otlp, journald/docker-daemon]
processors: [batch]
exporters: [otlp]
root_path tells the receiver to chroot into /hostfs (the mounted host filesystem) before running journalctl. journalctl_path resolves inside that chroot, so it points to the host's journalctl binary — the container image needs none. Update this path if journalctl lives elsewhere on your host (for example, /bin/journalctl on some Debian systems).
The journal directory defaults to /run/log/journal relative to root_path. On systems with persistent journal storage, add directory: /var/log/journal to the receiver config.
Step 2. Restart the Collector with journal access
The receiver needs two Linux capabilities to read the host journal. Stop the existing Collector and restart it with the capabilities added:
docker stop opentelemetry-collector
docker rm opentelemetry-collector
docker run --name opentelemetry-collector \
--restart unless-stopped \
--detach \
--user 0:0 \
--pid host \
--network host \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /proc:/hostfs/proc:ro \
-v /sys:/hostfs/sys:ro \
-v /:/hostfs:ro \
-v "$(pwd)/config.yaml":/etc/otelcol-contrib/config.yaml \
--cap-add CAP_DAC_READ_SEARCH \
--cap-add CAP_SYS_PTRACE \
-e HOST_PROC=/hostfs/proc \
-e HOST_SYS=/hostfs/sys \
-e HOST_ROOT=/hostfs \
otel/opentelemetry-collector-contrib:latest
Validate
Open Logs Explorer in SigNoz and search for _SYSTEMD_UNIT = docker.service or filter the message body for dockerd. Docker daemon startup messages, warnings, and runtime errors appear.
For advanced configuration, see Collecting systemd logs (journald) and the journald receiver documentation.
Step 1. Add the filelog receiver for syslog files
Append the filelog receiver to your existing config.yaml to tail the syslog file directly:
receivers:
filelog/docker-daemon-syslog:
include:
- /var/log/syslog
- /var/log/messages
start_at: end
include_file_name: false
service:
pipelines:
logs:
receivers: [otlp, filelog/docker-daemon-syslog]
processors: [batch]
exporters: [otlp]
List both paths so the config works across distributions; the receiver skips paths that don't exist.
Without filtering, every syslog line goes to SigNoz, including entries from unrelated processes. On a busy host, this raises ingestion costs. Add a filter operator to keep only Docker lines:
operators:
- type: filter
expr: 'not (body matches "dockerd")'
Step 2. Restart the Collector with syslog access
Stop the existing Collector and restart it with /var/log mounted read-only:
docker stop opentelemetry-collector
docker rm opentelemetry-collector
docker run --name opentelemetry-collector \
--restart unless-stopped \
--detach \
--user 0:0 \
--pid host \
--network host \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /var/log:/var/log:ro \
-v /proc:/hostfs/proc:ro \
-v /sys:/hostfs/sys:ro \
-v /:/hostfs:ro \
-v "$(pwd)/config.yaml":/etc/otelcol-contrib/config.yaml \
-e HOST_PROC=/hostfs/proc \
-e HOST_SYS=/hostfs/sys \
-e HOST_ROOT=/hostfs \
otel/opentelemetry-collector-contrib:latest
Validate
Open Logs Explorer in SigNoz and filter the message body for dockerd. Docker daemon events appear mixed with other syslog entries.
For forwarding syslog over the network instead of tailing files directly, see Collecting Syslogs via OpenTelemetry Collector.
Next Steps
- Parse log fields:
dockerdwrites structured log format (level=info msg="..."). Use the Logs Pipeline to extract fields likelevel,msg, andcontainerinto attributes for filtering and alerting. - Set up alerts: Create log-based alerts to catch errors from
dockerd. - Explore dashboards: View container metrics alongside these logs with the Docker metrics dashboard.
Troubleshooting
No container logs appearing
- Check that Docker container log files exist and are readable:
ls -la /var/lib/docker/containers/
- Verify the Collector container can read the mounted log directory:
docker run --rm \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
alpine ls /var/lib/docker/containers/
- Check the Collector logs for errors:
docker logs opentelemetry-collector | grep -i error
No Docker daemon logs appearing
- Confirm where Docker stores daemon logs on your host:
sudo journalctl -u docker.service -n 20
sudo tail -n 20 /var/log/syslog
sudo tail -n 20 /var/log/messages
For the
journaldreceiver: verify the Collector can accessjournalctland the journal directory. See Collecting systemd logs (journald) for container requirements.For the syslog path: check your syslog forwarder configuration in Collecting Syslogs via OpenTelemetry Collector.
Get Help
If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack.
If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.
