ECS Serverless (Fargate) - Configure

The install guide provides a working config: the collector in each task collects task metrics, the logs of your containers, and data from instrumented applications. This page explains what each component in that config does and how to customize it through Foundry's casting.yaml.

Prerequisites

• ECS Serverless Collection Agent installed with Foundry and Terraform (see the install guide)
• SigNoz Cloud account or self-hosted SigNoz instance
• The AWS CLI with access to your ECS cluster
• The casting.yaml, foundryctl, and Terraform you installed with

Apply Changes

For each change:

1. Add the section's snippet to casting.yaml. Foundry already generates the default receivers and processors, so add only what you change.

2. Run this from the directory that contains casting.yaml, then apply the change with your usual Terraform workflow:

   Copy

   foundryctl forge -f casting.yaml

   Terraform updates the collector's config and registers a new revision of your task definition.

3. Run this on the machine where you use the AWS CLI to check that the new task runs the collector:

   Copy

   aws ecs describe-tasks --region <aws-region> --cluster <ecs-cluster> \
     --tasks $(aws ecs list-tasks --region <aws-region> --cluster <ecs-cluster> --family <task-family> --query 'taskArns[0]' --output text) \
     --query 'tasks[0].containers[].[name,lastStatus]' --output text

   Verify these values:

   • <aws-region>: The AWS region of your ECS cluster.
   • <ecs-cluster>: The name or ARN of the cluster your task runs in.
   • <task-family>: The family of your task definition.

   The output shows signoz-collector-sidecar as RUNNING. If it shows STOPPED, see View the Collector's Logs.

4. In SigNoz, confirm data in the Container Metrics - ECS dashboard for task metrics, Logs for container logs, and Traces for application traces.

Send Data from Applications to the Collector

Task metrics and container logs flow as soon as the collector is running.

Where and how to send data from your application

To send traces, metrics, and logs from an application, instrument it with OpenTelemetry (language guides), then add this to its container in container_definitions:

environment = [
  { name = "OTEL_EXPORTER_OTLP_PROTOCOL", value = "http/protobuf" },
  { name = "OTEL_EXPORTER_OTLP_ENDPOINT", value = "http://localhost:4318" },
  { name = "OTEL_SERVICE_NAME", value = "<service-name>" },
  { name = "OTEL_RESOURCE_ATTRIBUTES", value = "service.version=<service-version>" },
]

Verify these values:

• <service-name>: The name your application shows under in SigNoz. It becomes service.name.
• <service-version>: Your application's version. It becomes service.version.

Containers in a Fargate task share one network namespace in the awsvpc network mode, so they reach the collector on localhost. Use port 4317 with OTEL_EXPORTER_OTLP_PROTOCOL=grpc for OTLP over gRPC.

Redeploy the application with your usual Terraform workflow and send it some traffic. Within a minute it shows under Services in SigNoz, and its traces carry aws.ecs.task.family and aws.ecs.task.arn.

Export to SigNoz

The OTLP/HTTP exporter sends the telemetry the collector gathers to SigNoz.

Foundry generates this by default: the otlphttp/signoz exporter sends every pipeline to SIGNOZ_INGESTION_ENDPOINT.

Collect Task Metrics

Use the awsecscontainermetrics receiver to collect task and container metrics from the ECS task metadata endpoint.

Foundry collects task metrics by default, every 20s, and its filter/ecs processor keeps the 8 ecs.task.* metrics. In casting.yaml, add this to also send per-container metrics:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            processors:
              filter/ecs:
                metrics:
                  include:
                    metric_names:
                      - ecs.task.memory.reserved
                      - ecs.task.memory.utilized
                      - ecs.task.cpu.reserved
                      - ecs.task.cpu.utilized
                      - ecs.task.network.rate.rx
                      - ecs.task.network.rate.tx
                      - ecs.task.storage.read_bytes
                      - ecs.task.storage.write_bytes
                      - container.memory.utilized
                      - container.cpu.utilized

Lists replace, so metric_names is restated with Foundry's 8 task metrics. Then apply the change.

Collect Container Logs

Send your containers' stdout and stderr to SigNoz to read them in Logs Explorer. The collector is the task's log router (FireLens), and it receives these logs with the fluentforward receiver.

A container sends its logs to the collector when its definition uses the module's log_configuration output, as in Step 4 of the install guide. To add another container, add this to it in container_definitions, then apply with your usual Terraform workflow:

logConfiguration = module.signoz_collector_sidecar.log_configuration

The container's logs then go to SigNoz instead of CloudWatch Logs, with service.name set to the container's name.

Two rules apply:

• At least one container in the task must keep this setting. The collector is always the task's log router, and ECS rejects a task definition whose log router no container uses, as Route logs to FireLens describes.
• A task can have only one log router. Remove any other FireLens log router container from the task definition before you add the module.

Disable Logs Collection

To stop collecting one container's logs, remove its logConfiguration line, or set its earlier log configuration, then apply with your usual Terraform workflow. Keep the line on at least one other container.

To collect no container logs at all, remove the line from every container, and add this to casting.yaml. It removes the collector's log router setting and the fluentforward receiver, which cannot start without it:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            receivers:
              fluentforward: null
            service:
              pipelines:
                logs:
                  receivers: [otlp/http, otlp/grpc]
  patches:
    - target: collectionagent/collector/sidecar/main.tf.json
      operations:
        - op: remove
          path: /locals/container_collector_sidecar/firelensConfiguration

Lists replace, so the logs pipeline is restated with Foundry's OTLP receivers. Logs that applications send over OTLP keep flowing. Then apply the change.

Resource Detection

The resourcedetection processor adds ECS task and cloud resource attributes, so you can filter and group by task in SigNoz.

Foundry runs the env and ecs detectors by default: ecs adds task and cloud resource attributes such as aws.ecs.cluster.arn, aws.ecs.task.arn, aws.ecs.task.family, and cloud.region, and env adds the collector's OTEL_RESOURCE_ATTRIBUTES. In casting.yaml, set this to add resource attributes to every signal the collector sends:

spec:
  collector:
    spec:
      env:
        OTEL_RESOURCE_ATTRIBUTES: deployment.environment=<DEPLOYMENT_ENVIRONMENT>,service.namespace=<service-namespace>

Verify these values:

• <DEPLOYMENT_ENVIRONMENT>: The deployment environment you set at install, for example production.
• <service-namespace>: A name that groups your services. It becomes service.namespace.

Then apply the change.

Scrape Prometheus Metrics from Containers in the Task

The collector can scrape metrics from containers in the task that expose a Prometheus endpoint. In casting.yaml, add this to scrape a container that serves them on port 9090:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            receivers:
              prometheus/app:
                config:
                  scrape_configs:
                    - job_name: app
                      scrape_interval: 60s
                      static_configs:
                        - targets: [localhost:9090]
            service:
              pipelines:
                metrics/prometheus:
                  receivers: [prometheus/app]
                  processors: [memory_limiter, resourcedetection, batch]
                  exporters: [otlphttp/signoz]

The metrics/prometheus pipeline runs without filter/ecs, so the scraped metrics are kept. Then apply the change.

Tune Batch and Memory Limiter

Adjust the batch processor for throughput and the memory_limiter processor for stability based on your workload. See Collector Configuration Best Practices for detailed guidance.

The batch processor batches spans, metrics, and logs before export. That reduces connections and improves compression. It supports both size-based and time-based batching.

Foundry's defaults are send_batch_size: 1000, send_batch_max_size: 2048, and timeout: 10s, and limit_percentage: 80 and spike_limit_percentage: 20 of the task's memory. In casting.yaml, add this to change the batch sizes and the memory limits:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            processors:
              batch:
                send_batch_size: 2000
                send_batch_max_size: 4000
              memory_limiter:
                limit_percentage: 75
                spike_limit_percentage: 15

Then apply the change.

Resource Requests and Limits

Foundry reserves cpu: 256 and memoryReservation: 256 for the collector container by default. In casting.yaml, add a patch that reserves more:

spec:
  patches:
    - target: collectionagent/collector/sidecar/main.tf.json
      operations:
        - op: replace
          path: /locals/container_collector_sidecar/cpu
          value: 512
        - op: replace
          path: /locals/container_collector_sidecar/memoryReservation
          value: 512

Raise the task's cpu and memory by what you add, to a Fargate task size. Then apply the change.

Add Custom Processors to Pipelines

A processor transforms or enriches telemetry once a pipeline lists it under processors.

Foundry's pipelines are traces, metrics and logs, plus metrics/ecs for task metrics. In casting.yaml, add this to define a processor and add it to the traces pipeline:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            processors:
              attributes/team:
                actions:
                  - key: team
                    value: platform
                    action: insert
            service:
              pipelines:
                traces:
                  processors: [memory_limiter, resourcedetection, attributes/team, batch]

Lists replace, so processors is restated with Foundry's defaults (memory_limiter, resourcedetection, batch), with yours before batch. Then apply the change.

Exposing Ports for Custom Receivers

A receiver that listens on a port needs that port reachable from your applications. The following example adds the statsd receiver on UDP port 8125.

In casting.yaml, add the statsd receiver in its own metrics/statsd pipeline and map the port:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            receivers:
              statsd:
                endpoint: 0.0.0.0:8125
            service:
              pipelines:
                metrics/statsd:
                  receivers: [statsd]
                  processors: [memory_limiter, resourcedetection, batch]
                  exporters: [otlphttp/signoz]
  patches:
    - target: collectionagent/collector/sidecar/main.tf.json
      operations:
        - op: add
          path: /locals/container_collector_sidecar/portMappings/-
          value: {containerPort: 8125, protocol: udp}

Containers in the task send to localhost:8125. Then apply the change.

Disable Metrics Collection

Turn off task metrics when you only need the telemetry your applications send. Foundry collects task metrics by default. In casting.yaml, add this to remove the awsecscontainermetrics receiver, the filter/ecs processor, and the metrics/ecs pipeline:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            receivers:
              awsecscontainermetrics: null
            processors:
              filter/ecs: null
            service:
              pipelines:
                metrics/ecs: null

The metrics pipeline keeps its OTLP receivers, so OTLP metrics from your applications keep flowing. Then apply the change.

Monitor More Task Definitions

The collector sees only the task it runs in, so each task definition you want to monitor needs it.

In the same Terraform root, use the one module in every task definition, as in Step 4 of the install guide, then apply with your usual Terraform workflow. For another Terraform root in the same AWS account and region, give its casting.yaml its own metadata.name, because the module's SSM parameter and IAM policy names come from it.

View the Collector's Logs

Check the collector's status and read its output when it stops or when data stops arriving in SigNoz. The collector container is not essential, so the task keeps running when the collector stops. ECS restarts the collector only when it stops after running for at least 5 minutes, as container restart policies describes. A collector that stops sooner, for example on a config error, stays stopped.

Run this on the machine where you use the AWS CLI to read the collector container's status in the latest task:

aws ecs describe-tasks --region <aws-region> --cluster <ecs-cluster> \
  --tasks $(aws ecs list-tasks --region <aws-region> --cluster <ecs-cluster> --family <task-family> --query 'taskArns[0]' --output text) \
  --query "tasks[0].containers[?name=='signoz-collector-sidecar'].[lastStatus,exitCode]" --output text

Verify these values:

• <aws-region>: The AWS region of your ECS cluster.
• <ecs-cluster>: The name or ARN of the cluster your task runs in.
• <task-family>: The family of your task definition.

The output shows RUNNING, or STOPPED with the collector's exit code.

The collector's own output is not sent anywhere by default. To read it in CloudWatch Logs, add this patch to casting.yaml, as Foundry's ECS Serverless example shows:

spec:
  patches:
    - target: collectionagent/collector/sidecar/main.tf.json
      operations:
        - op: add
          path: /locals/container_collector_sidecar/logConfiguration
          value:
            logDriver: awslogs
            options:
              awslogs-group: <log-group>
              awslogs-region: <aws-region>
              awslogs-stream-prefix: collector

Verify these values:

• <log-group>: An existing CloudWatch Logs log group, for example /ecs/app. The execution role needs permission to write to it, which the AmazonECSTaskExecutionRolePolicy managed policy grants.
• <aws-region>: The AWS region of your ECS cluster.

Apply the change. The collector's output then shows in the log group, under the collector stream prefix, as Send Amazon ECS logs to CloudWatch describes.

Next Steps

• Monitoring ECS Fargate using OpenTelemetry Collection Agents
• Telemetry Data Requirements for Infrastructure Monitoring
• Collector Configuration Best Practices