ECS EC2 Collection Agent - Install

This guide walks you through installing the daemon service to collect metrics and logs from your ECS infrastructure.

Prerequisites Check

Before starting, verify you have:

ECS cluster with EC2 or External launch type
AWS CLI configured with appropriate permissions
SigNoz Cloud account or self-hosted SigNoz instance
Access to AWS Parameter Store and CloudFormation

Installation Steps

Step 1: Daemon Service Template

The CloudFormation template defines the infrastructure needed for the daemon service, including container specifications, IAM roles, and network settings.

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  ClusterName:
    Type: String
    Description: Enter the name of your ECS cluster from which you want to collect metrics
  CreateIAMRoles:
    Type: String
    Default: 'False'
    AllowedValues:
      - 'True'
      - 'False'
    Description: Whether to create default IAM roles
    ConstraintDescription: must specify True or False.
  TaskRoleArn:
    Type: String
    Default: Default
    Description: Enter the role arn you want to use as the ecs task role
  ExecutionRoleArn:
    Type: String
    Default: Default
    Description: Enter the role arn you want to use as the ecs execution role
  command:
    Type: String
    Description: Using the right command to choose the config file you want to config your signoz-collector
    Default: Default
  SigNozConfigPath:
    Type: String
    Description: Enter the name of your SigNoz config file to fetch from SSM Parameter Store
    Default: Default
Conditions:
  CreateRoles: !Equals
    - !Ref CreateIAMRoles
    - 'True'
  DefaultTaskRole: !Equals
    - !Ref TaskRoleArn
    - Default
  DefaultExecutionRole: !Equals
    - !Ref ExecutionRoleArn
    - Default
Resources:
  ECSTaskDefinition:
    Type: 'AWS::ECS::TaskDefinition'
    Properties:
      Family: ecs-otel-daemon-service
      TaskRoleArn: !If
        - CreateRoles
        - !GetAtt
          - ECSTaskRole
          - Arn
        - !If
          - DefaultTaskRole
          - !Sub 'arn:aws:iam::${AWS::AccountId}:role/OtelECSTaskRole'
          - !Ref TaskRoleArn
      ExecutionRoleArn: !If
        - CreateRoles
        - !GetAtt
          - ECSExecutionRole
          - Arn
        - !If
          - DefaultExecutionRole
          - !Sub 'arn:aws:iam::${AWS::AccountId}:role/OtelECSExecutionRole'
          - !Ref ExecutionRoleArn
      NetworkMode: host
      ContainerDefinitions:
        - Name: signoz-collector
          Image: 'otel/opentelemetry-collector-contrib:0.109.0'
          User: 0
          MountPoints:
            - ReadOnly: true
              ContainerPath: /rootfs/proc
              SourceVolume: proc
            - ReadOnly: true
              ContainerPath: /rootfs/dev
              SourceVolume: dev
            - ReadOnly: true
              ContainerPath: /sys/fs/cgroup
              SourceVolume: al2_cgroup
            - ReadOnly: true
              ContainerPath: /cgroup
              SourceVolume: al1_cgroup
            - ReadOnly: true
              ContainerPath: /rootfs/sys/fs/cgroup
              SourceVolume: al2_cgroup
            - ReadOnly: true
              ContainerPath: /rootfs/cgroup
              SourceVolume: al1_cgroup
            - ReadOnly: true
              ContainerPath: /rootfs/boot/efi
              SourceVolume: boot
            - ReadOnly: true
              ContainerPath: /var/lib/docker/containers/
              SourceVolume: docker-logs
            - ReadOnly: true
              ContainerPath: /var/run/docker.sock
              SourceVolume: docker-sock
          PortMappings:
            - ContainerPort: 2255
              HostPort: 2255
              Protocol: tcp
            - ContainerPort: 4317
              HostPort: 4317
              Protocol: tcp
            - ContainerPort: 4318
              HostPort: 4318
              Protocol: tcp
          Command: !Split [ " ", !Ref command ]
          Secrets:
            - Name: SIGNOZ_CONFIG_CONTENT
              ValueFrom: !Ref SigNozConfigPath
          HealthCheck:
            Command:
              - CMD-SHELL
              - wget -qO- http://localhost:13133/ || exit 1
            Interval: 5
            Retries: 2
            Timeout: 3
          LogConfiguration:
            LogDriver: json-file
      RequiresCompatibilities:
        - EC2
      Volumes:
        - Name: proc
          Host:
            SourcePath: /proc
        - Name: dev
          Host:
            SourcePath: /dev
        - Name: al1_cgroup
          Host:
            SourcePath: /cgroup
        - Name: al2_cgroup
          Host:
            SourcePath: /sys/fs/cgroup
        - Name: boot
          Host:
            SourcePath: /boot/efi
        - Name: docker-logs
          Host:
            SourcePath: /var/lib/docker/containers/
        - Name: docker-sock
          Host:
            SourcePath: /var/run/docker.sock
      Cpu: '1000'
      Memory: '1024'
  ECSDaemonService:
    Type: 'AWS::ECS::Service'
    Properties:
      TaskDefinition: !Ref ECSTaskDefinition
      Cluster: !Ref ClusterName
      LaunchType: EC2
      SchedulingStrategy: DAEMON
      ServiceName: otel-daemon-service
  ECSTaskRole:
    Type: 'AWS::IAM::Role'
    Condition: CreateRoles
    Properties:
      Description: Allows ECS tasks to call AWS services on your behalf.
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: ''
            Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy'
      RoleName: OtelECSTaskRole
  ECSExecutionRole:
    Type: 'AWS::IAM::Role'
    Condition: CreateRoles
    Properties:
      Description: >-
        Allows ECS container agent makes calls to the Amazon ECS API on your
        behalf.
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: ''
            Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy'
        - 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
        - 'arn:aws:iam::aws:policy/AmazonSSMReadOnlyAccess'
      RoleName: OtelECSExecutionRole

Step 2: Create OpenTelemetry Collector Configuration

The OpenTelemetry Collector configuration defines what data to collect and where to send it.

2a. OtelCollector Configuration Template

Copy the following Collector template to otelcol-daemon.yaml file
Replace SIGNOZ_INGESTION_KEY with your SigNoz Cloud ingestion key
Replace {region} with your SigNoz Cloud region

extensions:
  health_check:
    endpoint: 0.0.0.0:13133

receivers:
  # Standard OTLP receiver for traces, metrics, and logs from instrumented applications
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

  # Collects detailed ECS container metrics from the ECS Task Metadata Endpoint.
  awsecscontainermetrics:
    collection_interval: 20s

  # Collects host-level metrics from the underlying EC2 instance.
  hostmetrics:
    collection_interval: 30s
    root_path: /rootfs
    scrapers:
      cpu:
      disk:
      load:
      memory:
      network:
      filesystem:
        metrics:
          system.filesystem.usage:
            enabled: true
        exclude_mount_points:
          match_type: regexp
          mount_points:
            - /rootfs/boot/*

  # Auto-discovers and tails container logs directly from the Docker json-log files on the host.
  filelog:
    include: [/var/lib/docker/containers/*/*-json.log]
    start_at: end
    include_file_name: false
    include_file_path: true
    operators:
    - id: container-parser
      type: container
      
processors:
  # Detects resource attributes from the environment (EC2, ECS, etc.)
  # This is the replacement for the k8sattributes processor.
  resourcedetection:
    detectors: [env, ecs, ec2]
    override: false

  # Batches telemetry to optimize network traffic.
  batch:
    timeout: 10s
    send_batch_size: 1000

exporters:
  # Exports telemetry to your backend (e.g., SigNoz)
  # Uses environment variables for portability, which you'll set in your ECS task definition.
  otlp:
    endpoint: "ingest.{region}.signoz.cloud:443"
    tls:
      insecure: false
    headers:
      signoz-access-token: "<SIGNOZ_INGESTION_KEY>"
  logging:
    verbosity: basic

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

    metrics:
      receivers: [otlp, hostmetrics, awsecscontainermetrics]
      processors: [resourcedetection, batch]
      exporters: [otlp, logging]

    logs:
      receivers: [otlp, filelog]
      processors: [resourcedetection, batch]
      exporters: [otlp, logging]

2b. Store Configuration in AWS Parameter Store

Navigate to AWS Systems Manager > Parameter Store in the AWS Console
Click Create parameter
Set parameter name: /ecs/signoz/otelcol-daemon.yaml
Choose type: String
Copy the entire contents of your updated otelcol-daemon.yaml file
Paste the content into the Value field
Click Create parameter

Step 3: Set Environment Variables

Configure the necessary environment variables for the CloudFormation deployment:

export CLUSTER_NAME=<YOUR-ECS-CLUSTER-NAME>
export REGION=<YOUR-ECS-REGION>  
export COMMAND=--config=env:SIGNOZ_CONFIG_CONTENT
export SIGNOZ_CONFIG_PATH=/ecs/signoz/otelcol-daemon.yaml

Replace the placeholders:

<YOUR-ECS-CLUSTER-NAME>: Name of your ECS cluster (e.g., my-production-cluster)
<YOUR-ECS-REGION>: AWS region where your cluster runs (e.g., us-east-1)

Step 4: Deploy the Daemon Service

Create the CloudFormation stack that will deploy your daemon service:

aws cloudformation create-stack \
    --stack-name AOCECS-daemon-${CLUSTER_NAME}-${REGION} \
    --template-body file://daemon-template.yaml \
    --parameters ParameterKey=ClusterName,ParameterValue=${CLUSTER_NAME} \
                 ParameterKey=CreateIAMRoles,ParameterValue=True \
                 ParameterKey=command,ParameterValue=${COMMAND} \
                 ParameterKey=SigNozConfigPath,ParameterValue=${SIGNOZ_CONFIG_PATH} \
    --capabilities CAPABILITY_NAMED_IAM \
    --region ${REGION}

Step 5: Verify Installation

Check that the daemon service deployed successfully:

# Check CloudFormation stack status
aws cloudformation describe-stacks \
    --stack-name AOCECS-daemon-${CLUSTER_NAME}-${REGION} \
    --region ${REGION} \
    --query 'Stacks[0].StackStatus'

# List running tasks to see the daemon service
aws ecs list-tasks \
    --cluster ${CLUSTER_NAME} \
    --region ${REGION}

Expected outputs:

Stack status should be CREATE_COMPLETE
You should see task ARNs for the daemon service containers

Step 6: Check Service Health

Verify the daemon service containers are running properly:

# Get detailed information about running tasks
aws ecs describe-tasks \
    --cluster ${CLUSTER_NAME} \
    --tasks $(aws ecs list-tasks --cluster ${CLUSTER_NAME} --query 'taskArns[0]' --output text) \
    --region ${REGION}

Look for:

Task Status: Should be RUNNING
Health Status: Should be HEALTHY
Container Status: All containers should be RUNNING

Troubleshooting Installation

Common Issues

Stack Creation Failed:

Check IAM permissions for CloudFormation, ECS, and Parameter Store
Verify the cluster name and region are correct
Ensure the Parameter Store parameter exists and has correct content

Tasks Not Starting:

Check ECS cluster has available capacity
Verify the container image can be pulled
Review CloudWatch logs for the failed tasks

Parameter Store Access Issues:

Ensure the IAM role has ssm:GetParameter permissions
Verify the parameter name matches exactly: /ecs/signoz/otelcol-daemon.yaml
Check the parameter is in the same region as your ECS cluster

Clean Up (If Needed)

To remove the daemon service installation:

aws cloudformation delete-stack \
    --stack-name AOCECS-daemon-${CLUSTER_NAME}-${REGION} \
    --region ${REGION}

This will remove all resources created by the CloudFormation template, including the ECS service, task definition, and IAM roles.

Next Steps

Last updated: August 13, 2025

Edit on GitHub

Was this page helpful?

Overview

Configure

On this page