Guide to drop logs

Overview

The filter processor in the OpenTelemetry Collector allows you to drop logs based on body, log level (aka severity text), or other attributes. This is useful if you want to exclude certain logs from being sent to SigNoz.

The filter processor is configured in the processors::filter section of the config.yaml file.

logs:
  receivers: [otlp]
  processors: [filter/drop_logs_by_severity_text, batch]
  exporters: [otlp]

Drop Logs

1. Drop logs by log level / severity text

processors:
  filter/drop_logs_by_level:
    logs:
      log_record:
        - 'IsMatch(severity_text, "(?i)\b(DEBUG)\b")'

2. Drop logs by body regex

processors:
  filter/drop_logs_by_body_regex:
    logs:
      log_record:
        - 'IsMatch(body, ".*password.*")'

3. Drop logs by resource attributes (like service.name, host.name, k8s.pod.name, etc.)

processors:
  filter/drop_logs_by_label_values:
    logs:
      log_record:
        - resource.attributes["k8s.pod.name"] == "test-pod"

4. Drop logs by resource attributes regex

processors:
  filter/drop_logs_by_label_values_regex:
    logs:
      log_record:
        - IsMatch(resource.attributes["k8s.pod.name"], "test-pod-.*")

5. Drop logs by logs attributes (like remote_addr, user_agent.name, etc.)

processors:
  filter/drop_logs_by_label_values:
    logs:
      log_record:
        - attributes["user_agent.name"] == "Safari"

6. Drop logs by logs attributes regex

processors:
  filter/drop_logs_by_label_values_regex:
    logs:
      log_record:
        - IsMatch(attributes["http.method"], "GET|POST")

7. Exclude/Include/Filter logs in Kubernetes environments

For Kubernetes environments, you can also configure log exclusion/inclusion using the k8s-infra chart configuration. This provides a simpler way to exclude or include logs based on namespaces, pods, or containers without writing custom OpenTelemetry processors.

See our Kubernetes pod logs collection guide for detailed configuration examples including:

• Excluding logs from specific namespaces, pods, or containers
• Including only logs from specific namespaces, pods, or containers
• Using filter operators for expression-based filtering

Refer to the OpenTelemetry documentation for more details on how to configure the filter processor.

Limit Log Body Size

If you're concerned about logs that are too large causing issues or potential abuse, you can control the size of log bodies using the transform processor with the truncate_all function.

This is useful to:

• Prevent accidental collection of oversized logs
• Avoid abuse scenarios where extremely large log entries could impact system performance
• Reduce storage costs and improve query performance

Configuration

Add the transform processor to your OpenTelemetry Collector configuration to truncate log bodies to a maximum size:

processors:
  transform/truncate_logs:
    log_statements:
      - context: log
        statements:
          - truncate_all(body, 4096)  # Limit log body to 4096 characters

In this example, log bodies will be truncated to 4096 characters. You can adjust this value based on your needs.

logs:
  receivers: [otlp]
  processors: [transform/truncate_logs, batch]
  exporters: [otlp]

For more information about the transform processor and the truncate_all function, refer to the SigNoz documentation.

Related Guides

• Remove Resource Attributes - Remove unwanted resource attributes from your logs

Get Help

If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack.

If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.