With SigNoz, you can easily collect and analyze system logs generated by your operating system. On Linux and Unix, these system logs are typically represented in the Syslog format.
This document shows how to set up rsyslog to forward logs to the OpenTelemetry (OTel) Collector using the syslog receiver, so you can parse, query, and monitor logs with minimal effort.
Prerequisite
- Unix based Operating System
On most modern Linux setups, journald is the default logging system, and it doesn't emit classic syslog output unless explicitly forwarded. If you're running Kubernetes or Linux nodes, checkout our Systemd Logs documentation.
Setup
Step 1: Add OTel Collector Binary
Add the OpenTelemetry Collector binary to your VM by following this guide.
Step 2: Configure Syslog Receiver in OTel Collector
Add the syslog receiver to the config.yaml of the OTel Collector:
receivers:
syslog:
tcp:
listen_address: "0.0.0.0:54527"
protocol: rfc3164
location: UTC
operators:
- type: move
from: attributes.message
to: body
...
Here, we collect logs and move messages from attributes to body using operators. Read more about operators here.
For additional configurations for the syslog receiver, check here.
Step 3: Update Pipeline in OTel Collector
Modify the pipeline inside config.yaml to include the syslog receiver:
service:
...
logs:
receivers: [otlp, syslog]
processors: [batch]
exporters: [otlp]
Step 4: Restart OTel Collector
Restart the OTel Collector to apply the new changes.
Step 5: Modify rsyslog.conf
Run the following command to edit the rsyslog.conf file:
sudo vim /etc/rsyslog.conf
Add the following lines at the end:
template(
name="UTCTraditionalForwardFormat"
type="string"
string="<%PRI%>%TIMESTAMP:::date-utc% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
)
*.* action(type="omfwd" target="0.0.0.0" port="54527" protocol="tcp" template="UTCTraditionalForwardFormat")
For production use cases, configure retries and queues:
*.* action(type="omfwd" target="0.0.0.0" port="54527" protocol="tcp"
action.resumeRetryCount="10"
queue.type="linkedList" queue.size="10000" template="UTCTraditionalForwardFormat")
Step 6: Restart rsyslog Service
Restart the rsyslog service:
sudo systemctl restart rsyslog.service
Check the status:
sudo systemctl status rsyslog.service
If there are no errors, logs will be visible in the SigNoz UI.
Step 1: Update docker-compose.yaml
Modify the docker-compose.yaml file in deploy/docker to expose port 54527:
...
otel-collector:
image: signoz/signoz-otel-collector:0.88.11
command: ["--config=/etc/otel-collector-config.yaml"]
volumes:
- ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
ports:
- "54527:54527"
...
Step 2: Configure Syslog Receiver in OTel Collector
Add the syslog receiver to otel-collector-config.yaml:
receivers:
syslog:
tcp:
listen_address: "0.0.0.0:54527"
protocol: rfc3164
location: UTC
operators:
- type: move
from: attributes.message
to: body
...
Step 3: Update Pipeline in OTel Collector
Modify the pipeline to include the syslog receiver:
service:
...
logs:
receivers: [otlp, syslog]
processors: [batch]
exporters: [clickhouselogsexporter]
Step 4: Restart OTel Collector Container
Restart the OTel Collector container to apply the changes.
Step 5: Modify rsyslog.conf
Edit the rsyslog.conf file:
sudo vim /etc/rsyslog.conf
Add the following lines:
template(
name="UTCTraditionalForwardFormat"
type="string"
string="<%PRI%>%TIMESTAMP:::date-utc% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
)
*.* action(type="omfwd" target="0.0.0.0" port="54527" protocol="tcp" template="UTCTraditionalForwardFormat")
For production use, add retries and queues:
*.* action(type="omfwd" target="0.0.0.0" port="54527" protocol="tcp"
action.resumeRetryCount="10"
queue.type="linkedList" queue.size="10000" template="UTCTraditionalForwardFormat")
Step 6: Restart rsyslog Service
Restart the rsyslog service:
sudo systemctl restart rsyslog.service
Check the status:
sudo systemctl status rsyslog.service
If there are no errors, logs will be visible in the SigNoz UI.
Add Resource Context
System logs by default don’t include resource context which includes information about the host such as host name, IP Address, container name etc. You can use the OpenTelemetry Collector’s resourcedetection processor to enrich your system logs with this information in the Resource field of your logs.
Related Resources
- Send your SystemD logs to SigNoz
- Send your Windows Event logs to SigNoz
