If you use FluentBit to collect logs in your stack, this tutorial will guide you on how to send logs from FluentBit to SigNoz.
At SigNoz, we use the OpenTelemetry Collector to receive logs, which supports the FluentForward protocol. You can forward logs from your FluentBit agent to the OpenTelemetry Collector using this protocol.
Collect Logs Using FluentBit in SigNoz Cloud
Add OpenTelemetry Collector binary to your VM Follow this guide.
Add FluentForward receiver to your
config.yaml:receivers: fluentforward: endpoint: 0.0.0.0:24224Here, port 24224 is used for listening to the FluentForward protocol. You can change it to a port of your choice. Learn more about the FluentForward receiver here.
Modify your
config.yamlto include the receiver:service: ... pipelines: logs: receivers: [otlp, fluentforward] processors: [batch] exporters: [otlp]Update FluentBit configuration to forward logs to the OpenTelemetry Collector:
[OUTPUT] Name forward Match * Host localhost Port 24224- This configuration forwards logs to the OpenTelemetry Collector listening on port 24224.
- If FluentBit is not running on the same host, replace
localhostwith the appropriate host value.
Restart FluentBit and OpenTelemetry Collector.
Use processors to transform logs:
To properly transform your existing log model into the OpenTelemetry log model, use OpenTelemetry processors:
processors: logstransform: operators: - type: trace_parser trace_id: parse_from: attributes.trace_id span_id: parse_from: attributes.span_id - type: remove field: attributes.trace_id - type: remove field: attributes.span_id
Collect Logs Using FluentBit in Self-Hosted SigNoz
Add FluentForward receiver to
otel-collector-config.yaml:receivers: fluentforward: endpoint: 0.0.0.0:24224Port 24224 is used for the FluentForward protocol. You can change it to a preferred port. Learn more about the FluentForward receiver here.
Update the logs pipeline in
otel-collector-config.yaml:service: ... pipelines: logs: receivers: [otlp, fluentforward] processors: [batch] exporters: [clickhouselogsexporter]Expose the port in
docker-compose.yaml:otel-collector: ... ports: - "24224:24224"Update FluentBit configuration:
[INPUT] Name dummy Tag dummy.log Dummy {"message": "mylog", "trace_id": "0000000000000000f4dbb3edd765f620", "span_id": "43222c2d51a7abe3"} [OUTPUT] Name forward Match * Host <otel-collector-host> Port 24224Replace
<otel-collector-host>with the host where the OpenTelemetry Collector is running. For more info, check the troubleshooting guide.Restart FluentBit and SigNoz.
Use processors to transform logs:
To transform your existing log model into the OpenTelemetry log model, use processors:
processors: logstransform: operators: - type: trace_parser trace_id: parse_from: attributes.trace_id span_id: parse_from: attributes.span_id - type: remove field: attributes.trace_id - type: remove field: attributes.span_id