Send CloudWatch logs to SigNoz

Overview

This guide walks you through forwarding AWS CloudWatch logs to SigNoz using the OpenTelemetry (OTel) Collector's awscloudwatch receiver. By the end, you'll have CloudWatch logs flowing into SigNoz where you can search, filter, and correlate them with traces and metrics.

Prerequisites

Before you begin, make sure you have:

• SigNoz account: A SigNoz Cloud account or a running self-hosted SigNoz instance
• AWS account: Access to AWS with permissions to create IAM policies
• CloudWatch log groups: At least one log group with logs you want to forward
• VM or server: A machine (Linux recommended) where you can run the OTel Collector

Step 1: Install the OpenTelemetry Collector

First, install the OpenTelemetry Collector on the machine that will pull logs from CloudWatch. Follow the installation guide for your environment: Install OpenTelemetry Collection Agent

Step 2: Configure AWS credentials

The OTel Collector needs AWS credentials to access CloudWatch. Create or update the credentials file at ~/.aws/credentials on the machine running the collector:

[default]
aws_access_key_id=<YOUR_AWS_ACCESS_KEY_ID>
aws_secret_access_key=<YOUR_AWS_SECRET_ACCESS_KEY>

Replace the placeholders:

• <YOUR_AWS_ACCESS_KEY_ID>: Your AWS access key ID
• <YOUR_AWS_SECRET_ACCESS_KEY>: Your AWS secret access key

[my-cloudwatch-profile]
aws_access_key_id=<YOUR_AWS_ACCESS_KEY_ID>
aws_secret_access_key=<YOUR_AWS_SECRET_ACCESS_KEY>

Step 3: Create the required IAM policy

The AWS credentials you use need permission to read CloudWatch logs. Create an IAM policy with the following permissions and attach it to the user or role associated with your credentials:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CloudWatchLogsReadAccess",
            "Effect": "Allow",
            "Action": [
                "logs:DescribeLogGroups",
                "logs:FilterLogEvents"
            ],
            "Resource": "arn:aws:logs:<REGION>:<AWS_ACCOUNT_ID>:log-group:*"
        }
    ]
}

Replace the placeholders:

• <REGION>: Your AWS region (e.g., us-east-1, eu-west-1), or use * to allow all regions
• <AWS_ACCOUNT_ID>: Your 12-digit AWS account ID (find it in the AWS Console under your account dropdown)

Step 4: Configure the awscloudwatch receiver

Add the awscloudwatch receiver to your OTel Collector configuration file (config.yaml). This receiver polls CloudWatch at regular intervals and forwards logs to SigNoz.

Choose one of the following configurations based on your needs:

Option A: Autodiscover log groups

This configuration automatically discovers and collects logs from log groups matching a prefix:

receivers:
  awscloudwatch:
    region: us-east-1
    logs:
      poll_interval: 1m
      groups:
        autodiscover:
          limit: 100
          prefix: /aws/lambda/

Configuration breakdown:

• region: The AWS region where your log groups are located
• poll_interval: How often to check for new logs (e.g., 1m for every minute, 5m for every 5 minutes)
• limit: Maximum number of log groups to discover (up to 100)
• prefix: Only collect log groups whose names start with this prefix (e.g., /aws/lambda/ for Lambda function logs)

Option B: Specify log groups by name

This configuration explicitly lists which log groups to collect:

receivers:
  awscloudwatch:
    region: us-west-1
    profile: my-cloudwatch-profile  # Optional: use if you have a named AWS profile
    logs:
      poll_interval: 5m
      groups:
        named:
          /aws/eks/my-cluster/cluster:
          /aws/rds/instance/my-database/postgresql:

Configuration breakdown:

• profile: (Optional) The AWS credentials profile to use from ~/.aws/credentials
• named: List specific log group names to collect (each name ends with a colon)

Step 5: Configure the logs pipeline

Enable the awscloudwatch receiver in your OTel Collector pipeline. Add or update the logs pipeline in the service.pipelines section of your config.yaml:

service:
  pipelines:
    logs:
      receivers: [otlp, awscloudwatch]
      processors: [batch]
      exporters: [otlp]

The highlighted line shows the awscloudwatch receiver added to the logs pipeline. If you have existing receivers like otlp, keep them in the array.

exporters:
  otlp:
    endpoint: ingest.<REGION>.signoz.cloud:443
    tls:
      insecure: false
    headers:
      signoz-ingestion-key: <SIGNOZ_INGESTION_KEY>

Step 6: Restart the collector

After updating the configuration, restart the OTel Collector to apply changes:

# If running as a systemd service
sudo systemctl restart otelcol-contrib

# If running directly
# Stop the current process (Ctrl+C) and restart with:
./otelcol-contrib --config config.yaml

Validate

After a few minutes (based on your poll_interval), verify that logs are appearing in SigNoz:

1. Open your SigNoz instance and navigate to Logs → Logs Explorer
2. Look for logs with attributes matching your CloudWatch log groups
3. You should see logs with fields like:
   • aws.cloudwatch.log_group_name
   • aws.cloudwatch.log_stream_name

Next steps

Now that CloudWatch logs are flowing into SigNoz, explore these related features:

• Search and filter logs - Learn to query your CloudWatch logs in SigNoz
• Create log-based alerts - Set up alerts based on log patterns
• Correlate logs with traces - Connect logs to distributed traces for debugging
• AWS monitoring overview - Explore other AWS integrations for metrics and traces