Redis Metrics and Logs

Overview

This integration helps you to monitor key Redis metrics and logs, view them with an out-of-the-box dashboard, and parse Redis logs for better querying and aggregation.

Prerequisites

Before you begin, ensure you have:

  1. A Redis server running version 3.0 or newer
  2. An OpenTelemetry (OTEL) Collector with access to the Redis server
    • Install the OTEL Collector(v0.88.0 or newer) if not done already
    • Ensure you can provide config files and set environment variables for the collector
    • Ensure the OTEL collector has access to the Redis server for metrics collection and can read the Redis log file for logs collection

Collecting Redis Metrics

Step 1: Create the Collector Config File

Create a file named redis-metrics-collection-config.yaml with the following content:

receivers:
  redis:
    # The hostname and port of the Redis instance, separated by a colon.
    endpoint: ${env:REDIS_ENDPOINT}
    # The frequency at which to collect metrics from the Redis instance.
    collection_interval: 60s
    # # The password used to access the Redis instance; must match the password specified in the requirepass server configuration option.
    password: ${env:REDIS_PASSWORD}
    # # Defines the network to use for connecting to the server. Valid Values are `tcp` or `Unix`
    # transport: tcp
    # tls:
    #   insecure: false
    #   ca_file: /etc/ssl/certs/ca-certificates.crt
    #   cert_file: /etc/ssl/certs/redis.crt
    #   key_file: /etc/ssl/certs/redis.key
    metrics:
      redis.maxmemory:
        enabled: true
      redis.cmd.latency:
        enabled: true

processors:
  # enriches the data with additional host information
  # see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor#resource-detection-processor
  resourcedetection/system:
    # add additional detectors if needed
    detectors: ["system"]
    system:
      hostname_sources: ["os"]

exporters:
  # export to SigNoz cloud
  otlp/redis:
    endpoint: "${env:OTLP_DESTINATION_ENDPOINT}"
    tls:
      insecure: false
    headers:
      "signoz-access-token": "${env:SIGNOZ_INGESTION_KEY}"

  # export to local collector
  # otlp/redis:
  #   endpoint: "localhost:4317"
  #   tls:
  #     insecure: true

service:
  pipelines:
    metrics/redis:
      receivers: [redis]
      # note: remove this processor if the collector host is not running on the same host as the redis instance
      processors: [resourcedetection/system]
      exporters: [otlp/redis]

Step 2: Set Environment Variables

Set the following environment variables:


# redis endpoint reachable from the otel collector"
export REDIS_ENDPOINT="localhost:6379"

# password used to access the Redis instance.
# must match the password specified in the requirepass server configuration option.
# can be left empty if the redis server is not configured to require a password.
export REDIS_PASSWORD=""

# region specific SigNoz cloud ingestion endpoint
export OTLP_DESTINATION_ENDPOINT="ingest.{REGION}.signoz.cloud:443"

# your SigNoz ingestion key
export SIGNOZ_INGESTION_KEY="signoz-ingestion-key"

You can find more details about ingestion keys and Regions here

Step 3: Use the Collector Config File

Add the following flag to your collector run command:

--config redis-metrics-collection-config.yaml

Note: The collector can use multiple config files by specifying multiple --config flags.

Collecting Redis Logs

Step 1: Create the Collector Config File

Create a file named redis-logs-collection-config.yaml with the following content:

receivers:
  filelog/redis:
    include: ["${env:REDIS_LOG_FILE}"]
    operators:
      # Parse default redis log format
      # pid:role timestamp log_level message
      - type: regex_parser
        if: body matches '^(?P<pid>\\d+):(?P<role>\\w+) (?P<ts>\\d{2} \\w+ \\d{4} \\d{2}:\\d{2}:\\d{2}\\.\\d+) (?P<log_level>[.\\-*#]) (?P<message>.*)$'
        parse_from: body
        regex: '^(?P<pid>\d+):(?P<role>\w+) (?P<ts>\d{2} \w+ \d{4} \d{2}:\d{2}:\d{2}\.\d+) (?P<log_level>[.\-*#]) (?P<message>.*)$'
        timestamp:
          parse_from: attributes.ts
          layout: '02 Jan 2006 15:04:05.000'
          layout_type: gotime
        severity:
          parse_from: attributes.log_level
          overwrite_text: true
          mapping:
            debug: '.'
            info:
              - '-'
              - '*'
            warn: '#'
        on_error: send
      - type: move
        if: attributes.message != nil
        from: attributes.message
        to: body
      - type: remove
        if: attributes.log_level != nil
        field: attributes.log_level
      - type: remove
        if: attributes.ts != nil
        field: attributes.ts
      - type: add
        field: attributes.source
        value: redis

processors:
  batch:
    send_batch_size: 10000
    send_batch_max_size: 11000
    timeout: 10s

exporters:
  # export to SigNoz cloud
  otlp/redis-logs:
    endpoint: "${env:OTLP_DESTINATION_ENDPOINT}"
    tls:
      insecure: false
    headers:
      "signoz-access-token": "${env:SIGNOZ_INGESTION_KEY}"

  # export to local collector
  # otlp/redis-logs:
  #   endpoint: "localhost:4317"
  #   tls:
  #     insecure: true


service:
  pipelines:
    logs/redis:
      receivers: [filelog/redis]
      processors: [batch]
      exporters: [otlp/redis-logs]

Step 2: Set Environment Variables

Set the following environment variables:


# path of Redis server log file. must be accessible by the otel collector
# typically found in /usr/local/var/log/redis on macOS
# log file location can also be found in the output of `redis-cli CONFIG GET : *`
export REDIS_LOG_FILE=/var/log/redis/redis-server.log

# region specific SigNoz cloud ingestion endpoint
export OTLP_DESTINATION_ENDPOINT="ingest.{REGION}.signoz.cloud:443"

# your SigNoz ingestion key
export SIGNOZ_INGESTION_KEY="signoz-ingestion-key"

You can find more details about ingestion keys and Regions here

Step 3: Use the Collector Config File

Add the following flag to your collector run command:

--config redis-logs-collection-config.yaml

Note: The collector can use multiple config files by specifying multiple --config flags in the collector run command.

Connect Redis

Once you're done with setting up Redis for collecting metrics and logs, head over to the intergrations tab in SigNoz and search for the Redis integration.

Search for Redis in Integrations tab
Search for Redis in Integrations tab

Click on the Connect Redis Button, and select I have already configured, this will start listening for data from your Redis instance. To stop this, you can select the Remove from SigNoz button.

Connect Redis
Connect Redis

 

Listening for data from Redis
Listening for data from Redis

Redis dashboard

Once SigNoz has started listening to your Redis data, head over to the Dashboards tab and search for Redis, this will show you a newly created dashboard which shows different Redis metrics.

Dashboard for monitoring Redis Metrics
Dashboard for monitoring Redis Metrics

Dashboard asset

You can also manually create the above Redis Dashboard by importing the JSON file available here. To learn how to create Dashboards, checkout this documentation.

Data Collected

When you switch to the Data Collected tab of your Redis Integrations, it shows you details about the different logs attributes and the metrics types that you can monitor for your Redis instance. The tables below gives you a list of the different logs attributes and metrics available.

Log attributes and metrics details for Redis
Log attributes and metrics details for Redis

Redis log attributes

  • Name: The name of the log attribute.
  • Path: The specific location or attribute within a log entry where the corresponding data can be found.
  • Type: The data type of the log attribute.
NamePathType
Process IDattributes.pidstring
Process Roleattributes.rolestring
Timestamptimestamptimestamp
Severity Textseverity_textstring
Severity Numberseverity_numbernumber

Redis metrics

  • Name: The name of the metric.
  • Type: The type of the metric (e.g., Sum, Gauge).
  • Unit: The unit of measurement for the metric.
  • Description: A brief description of what the metric represents.
NameTypeUnitDescription
redis_commands_processedSumnumberTotal number of commands processed by the server
redis_cpu_timeSumsecondsSystem CPU consumed by the Redis server in seconds since server start
redis_keys_expiredSumnumberTotal number of key expiration events
redis_db_expiresGaugenumberNumber of keyspace keys with an expiration
redis_commandsGaugeops/sNumber of commands processed per second
redis_replication_offsetGaugeBytesThe server's current replication offset
redis_net_inputSumBytesThe total number of bytes read from the network
redis_clients_connectedSumnumberNumber of client connections (excluding connections from replicas)
redis_keys_evictedSumnumberNumber of evicted keys due to maxmemory limit
redis_maxmemoryGaugeBytesThe value of the maxmemory configuration directive
redis_clients_max_input_bufferGaugeBytesBiggest input buffer among current client connections
redis_cmd_latencyGaugesecondsCommand execution latency
redis_memory_luaGaugeBytesNumber of bytes used by the Lua engine
redis_replication_backlog_first_byte_offsetGaugeBytesThe master offset of the replication backlog buffer
redis_keyspace_hitsSumnumberNumber of successful lookup of keys in the main dictionary
redis_clients_blockedSumnumberNumber of clients pending on a blocking call
redis_connections_rejectedSumnumberNumber of connections rejected because of maxclients limit
redis_latest_forkGaugemicrosecondsDuration of the latest fork operation in microseconds
redis_clients_max_output_bufferGaugeBytesLongest output list among current client connections
redis_slaves_connectedSumnumberNumber of connected replicas
redis_db_keysGaugenumberNumber of keyspace keys
redis_keyspace_missesSumnumberNumber of failed lookup of keys in the main dictionary
redis_uptimeSumsecondsNumber of seconds since Redis server start
redis_memory_usedGaugeBytesTotal number of bytes allocated by Redis using its allocator
redis_net_outputSumBytesThe total number of bytes written to the network
redis_connections_receivedSumnumberTotal number of connections accepted by the server
redis_rdb_changes_since_last_saveSumnumberNumber of changes since the last dump
redis_memory_rssGaugeBytesNumber of bytes that Redis allocated as seen by the operating system
redis_db_avg_ttlGaugemillisecondsAverage keyspace keys TTL
redis_memory_peakGaugeBytesPeak memory consumed by Redis (in bytes)
redis_memory_fragmentation_ratioGaugenumberRatio between used_memory_rss and used_memory

Was this page helpful?