Prerequisites

• An ECS cluster running with at least one task definition
• ECS cluster of launch type EC2 or External
• SigNoz Cloud account

Select the type of ECS cluster you are running: EC2 or External.

wget https://github.com/SigNoz/benchmark/raw/main/ecs/ec2/daemon-template.yaml

Step 2: Create SigNoz OtelCollector Config

• Navigate to AWS Parameter Store and create a new parameter named /ecs/signoz/otelcol-daemon.yaml.

• Download the otelcol-daemon YAML configuration file:

  wget https://github.com/SigNoz/benchmark/raw/main/ecs/otelcol-daemon.yaml
  

• Update {region} and SIGNOZ_INGESTION_KEY values in your YAML configuration file and copy the updated content of the otelcol-sidecar.yaml file and paste it in the value field of the /ecs/signoz/otelcol-daemon.yaml parameter that you created.

You will be able to get {region} and SIGNOZ_INGESTION_KEY values in your SigNoz Cloud account under Settings --> Ingestion Settings.

Step 3: Create Daemon Service

Now configure the daemon service you created in Step 1 with the OpenTelemetry Collector (OTelCollector) configuration you prepared and stored in AWS Parameter Store in Step 2. The goal is to ensure the daemon service is properly set up to collect, process, and export your ECS infrastructure metrics and logs to SigNoz Cloud.

As a first step to configure the daemon service, you need to set the environment variable by running the command (using your AWS CLI) below:

export CLUSTER_NAME=<YOUR-ECS-CLUSTER-NAME>
export REGION=<YOUR-ECS-REGION>
export COMMAND=--config=env:SIGNOZ_CONFIG_CONTENT
export SIGNOZ_CONFIG_PATH=/ecs/signoz/otelcol-daemon.yaml

Where, <YOUR-ECS-CLUSTER-NAME> - Name of your ECS cluster. For example, my-test-cluster <YOUR-ECS-REGION> - Region in which your ECS cluster is running. For example, us-east-1

Now with the environment variables set, you proceed to run an AWS CloudFormation command to create the stack that represents your daemon service. This command uses the aws cloudformation create-stack for creating a new stack.

Now run the following command (using your AWS CLI) to create the daemon service:

aws cloudformation create-stack --stack-name AOCECS-daemon-${CLUSTER_NAME}-${REGION} \
    --template-body file://daemon-template.yaml \
    --parameters ParameterKey=ClusterName,ParameterValue=${CLUSTER_NAME} \
    ParameterKey=CreateIAMRoles,ParameterValue=True \
		ParameterKey=command,ParameterValue=${COMMAND} \
		ParameterKey=SigNozConfigPath,ParameterValue=${SIGNOZ_CONFIG_PATH} \
    --capabilities CAPABILITY_NAMED_IAM \
    --region ${REGION}

Step 4: Verify Daemon Service

To verify that the daemon service is running, you can run the following command:

aws ecs list-tasks --cluster ${CLUSTER_NAME} --region ${REGION}

You should see the task ARN of the daemon service in the output.

Step 5: Verify Data in SigNoz

• instance-metrics.json
• hostmetrics-with-variable.json

(Optional) Step 6: Clean Up

In a cloud environment where resources are billed based on usage, cleaning up resources is crucial. This step involves removing the daemon service and any associated resources that were created during the setup process to collect and forward metrics and logs from your ECS infrastructure to SigNoz. To clean up the daemon service, you can run the following command:

aws cloudformation delete-stack --stack-name AOCECS-daemon-${CLUSTER_NAME}-${REGION} --region ${REGION}

Send Data from Applications

Step 1: Add OpenTelemetry Instrumentation to your Application

To add OpenTelemetry instrumentation to your application, you can follow the docs here.

Step 2: Add Entrypoint to your Application Container

We need to add an entrypoint to the application container to set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable to the endpoint of the daemon service. Such that the application container can send data to the daemon container running in the same host.

{
      ...,
      "entryPoint": [
        "sh",
        "-c",
        "export OTEL_EXPORTER_OTLP_ENDPOINT=\"http://$(curl http://169.254.169.254/latest/meta-data/local-ipv4):4317\"; <Application Startup Commands>"
      ],
			"command": [],
			...
}

Step 3: Add Service Name of your Application

To add the service name of your application, you need to set the OTEL_RESOURCE_ATTRIBUTES environment variable of the application container to service.name=<your-service-name>.

In your task definition, you can add the following lines.

...
    ContainerDefinitions:
        - Name: <your-container-name>
          ...
          Environment:
            - Name: OTEL_RESOURCE_ATTRIBUTES
              Value: service.name=<your-service-name>
          ...
...

In case of JSON task definition, you can add the following lines.

...
    "containerDefinitions": [
        {
            "name": "<your-container-name>",
            ...
            "environment": [
                {
                    "name": "OTEL_RESOURCE_ATTRIBUTES",
                    "value": "service.name=<your-service-name>"
                }
            ],
            ...
        }
    ],
...

Step 4: Rebuild and Deploy Application Container

After following previous step, you need to rebuild the application container and deploy it to ECS cluster.

Step 5: Verify Data in SigNoz