FluentD to SigNoz

If you use FluentD to collect logs in your stack, this tutorial will help you send logs from FluentD to SigNoz.

SigNoz uses the OpenTelemetry collector to receive logs, which supports the fluentforward protocol. You can forward your logs from your FluentD agent to the OpenTelemetry collector.

Collect Logs Using FluentD in SigNoz Cloud

Add OpenTelemetry Collector Binary
Follow this guide to add the OpenTelemetry collector binary to your VM.
Configure FluentForward Receiver
Add the following to your config.yaml:
```
receivers:
  fluentforward:
    endpoint: 0.0.0.0:24224
```

You can change the port if needed. Learn more about the fluentforward receiver here.

Modify the Service Section

Update your config.yaml:

service:
    ...
    logs:
        receivers: [otlp, fluentforward]
        processors: [batch]
        exporters: [otlp]

Update FluentD Configuration
Add the following to your FluentD configuration to forward logs to the OpenTelemetry collector:
```
<match <directive>>
  @type forward
  send_timeout 60s
  recover_wait 10s
  hard_timeout 60s

  <server>
    name myserver1
    host localhost
    port 24224
  </server>
</match>
```
Replace <directive> with your directive name. If FluentD is not running on the host, adjust the host value accordingly.
Restart Services
Restart FluentD and the OpenTelemetry collector binary. Logs should now appear in SigNoz.

Transform Logs to OpenTelemetry Model

Use processors in OpenTelemetry to transform logs. Example:

processors:
  logstransform:
    operators:
      - type: trace_parser
        trace_id:
          parse_from: attributes.trace_id
        span_id:
          parse_from: attributes.span_id
      - type: remove
        field: attributes.trace_id
      - type: remove
        field: attributes.span_id

Collect Logs Using FluentD in Self-Hosted SigNoz

Configure FluentForward Receiver Add the following to your otel-collector-config.yaml (inside deploy/docker):
```
receivers:
  fluentforward:
    endpoint: 0.0.0.0:24224
```

You can change the port if needed. Learn more about the fluentforward receiver here.

Update Exporter and Pipeline

Uncomment the exporter and pipeline for logs in otel-collector-config.yaml and make the following changes:

exporters:
  ...
  clickhouselogsexporter:
    dsn: tcp://clickhouse:9000/
    timeout: 5s
    sending_queue:
      queue_size: 100
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s

service:
  ...
  logs:
    receivers: [otlp, fluentforward]
    processors: [batch]
    exporters: [clickhouselogsexporter]

Expose Collector Port
Modify docker-compose-core.yaml to expose the port for the OpenTelemetry collector:
```
otel-collector:
  ...
  ports:
    - "24224:24224"
```

Update FluentD Configuration Update your FluentD configuration:

<source>
  @type sample
  sample [{"message": "my log data", "source": "myhost"}, {"message": "my log data 1", "source": "myhost1"}]
  tag sample
  rate 10000
</source>

<match sample>
  @type forward
  send_timeout 60s
  recover_wait 10s
  hard_timeout 60s

  <server>
    name myserver1
    host <otel-collector-host>
    port 24224
  </server>
</match>

Replace <otel-collector-host> with the host where the OpenTelemetry collector is running. For troubleshooting, check here.

Restart Services
Restart FluentD and SigNoz. Logs should now appear in SigNoz.

Transform Logs to OpenTelemetry Model

Use processors in OpenTelemetry to transform logs. Example:

processors:
  logstransform:
    operators:
      - type: trace_parser
        trace_id:
          parse_from: attributes.trace_id
        span_id:
          parse_from: attributes.span_id
      - type: remove
        field: attributes.trace_id
      - type: remove
        field: attributes.span_id

FluentD to SigNoz

Collect Logs Using FluentD in SigNoz Cloud

Collect Logs Using FluentD in Self-Hosted SigNoz

Was this page helpful?