This guide explains how to install SigNoz on AWS Elastic Container Service (ECS) with the EC2 launch type, on infrastructure that Foundry provisions or on an existing cluster.
Prerequisites
- An AWS account, with credentials that can create the resources listed in the tab you follow.
- Terraform and the AWS CLI installed on the same machine. Terraform reads the same credentials as the AWS CLI.
- Enough capacity for SigNoz. For sizing, see resource planning.
Install SigNoz
Step 1: Install foundryctl
Run this on the machine where you run Terraform:
curl -fsSL https://signoz.io/foundry.sh | bashFor manual install (Windows PowerShell, air-gapped, etc.) or PATH setup, see the foundry getting-started guide.
Step 2: Create casting.yaml
Create a file named casting.yaml. It describes the infrastructure, named foundry:
apiVersion: v1alpha1
kind: Infrastructure
metadata:
name: foundry
annotations:
foundry.signoz.io/ecs-region: <aws-region>
spec:
deployment:
platform: ecs
mode: ec2
flavor: terraform
resource:
spec:
config:
data:
resource.yaml: |
networking:
networkCIDR: 10.0.0.0/16
subnets:
private-a:
type: private
zone: <availability-zone>
cidr: 10.0.0.0/19
public-a:
type: public
zone: <availability-zone>
cidr: 10.0.96.0/22Verify these values:
<aws-region>: The AWS region to create the infrastructure in, for exampleus-east-1.<availability-zone>: An Availability Zone of that region, for exampleus-east-1a.networkCIDRand each subnet'scidr: Change them if they overlap a network you connect to.
The defaults are development sizes; production sizing is under Customize the installation.
metadata.name names everything Foundry creates; the ECS cluster is foundry-cls.
For all options, see Infrastructure and the ECS infrastructure example.
Step 3: Create the infrastructure
Run this from the directory that contains casting.yaml:
foundryctl cast -f casting.yamlThis creates the infrastructure with Terraform in pours/infrastructure/. Run the same command again after editing casting.yaml to apply the change.
Check that the instances have registered with the cluster. Run this with the AWS CLI:
aws ecs describe-clusters --clusters foundry-cls --region <aws-region> \
--query 'clusters[0].registeredContainerInstancesCount'Verify these values:
<aws-region>: The region fromcasting.yaml.
The command prints the registered instance count. Wait until it reaches 4, one per instance in this casting.yaml, before the next step.
Step 4: Create the SigNoz casting
Create a separate casting.yaml for the SigNoz installation, or append it to the end of the one from Step 2 after a --- line. It describes the installation and binds it to the infrastructure:
apiVersion: v1alpha1
kind: Installation
metadata:
name: signoz
annotations:
foundry.signoz.io/ecs-region: <aws-region>
spec:
deployment:
platform: ecs
mode: ec2
flavor: terraform
infrastructure:
name: foundryVerify these values:
<aws-region>: The AWS region from Step 2.
spec.infrastructure.name must match the Infrastructure document's metadata.name. The installation's metadata.name, signoz, prefixes every ECS service and names the DNS namespace signoz-installation.local.
Step 5: Deploy SigNoz
Run this from the directory that contains casting.yaml:
foundryctl cast -f casting.yamlThis installs SigNoz with Terraform in pours/deployment/. Run the same command again after editing casting.yaml to apply the change.
Step 6: Verify the installation
Check that each SigNoz service runs its task. Run this with the AWS CLI:
aws ecs describe-services --cluster foundry-cls --region <aws-region> \
--services signoz-telemetrykeeper-clickhousekeeper-0 signoz-telemetrystore-clickhouse-0-0 \
signoz-metastore-postgres-0 signoz-signoz-0 signoz-ingester \
--query 'services[].[serviceName,desiredCount,runningCount]' --output textVerify these values:
<aws-region>: The region fromcasting.yaml.
Each line shows the service name, the desired task count and the running task count. The output should look similar to the following:
signoz-telemetrykeeper-clickhousekeeper-0 1 1
signoz-telemetrystore-clickhouse-0-0 1 1
signoz-metastore-postgres-0 1 1
signoz-signoz-0 1 1
signoz-ingester 1 1The counts can take a few minutes to match.
ECS runs SigNoz's health check, /api/v1/health, inside the task. Read its result with the AWS CLI:
aws ecs describe-tasks --cluster foundry-cls --region <aws-region> \
--tasks $(aws ecs list-tasks --cluster foundry-cls --region <aws-region> --service-name signoz-signoz-0 --query 'taskArns[0]' --output text) \
--query 'tasks[].healthStatus' --output textThe command prints HEALTHY once the health check passes. The status reads UNKNOWN until the first check passes. Failed checks in the first five minutes after the task starts do not count against it.
SigNoz is reachable at http://signoz-0.signoz-installation.local:8080 from inside the VPC.
Send data to SigNoz
Your applications send traces, metrics, and logs to the SigNoz ingester. Inside the VPC, the ingester listens at:
http://ingester.signoz-installation.local:4318Use this address as the OTLP endpoint in your instrumentation, for example as the value of OTEL_EXPORTER_OTLP_ENDPOINT. For gRPC, use port 4317 instead.
The name resolves inside the VPC. The security group Foundry creates allows traffic between its members, so allow ports 4317 and 4318 from your applications. The signoz in signoz-installation.local is the installation's metadata.name. If you changed it, use yours.
For applications outside the VPC, see the self-hosted ingestion guide.
Customize the installation
Every change follows the same loop: edit casting.yaml, then run foundryctl cast -f casting.yaml again.
-
Component settings (image, shards, replicas): set them in the Installation document's component
spec. For example, to pin SigNoz to a specific release:spec: signoz: spec: image: signoz/signoz:v0.144.0ClickHouse takes shard and replica counts, and ClickHouse Keeper takes a replica count under
spec.telemetrystore.spec.clusterandspec.telemetrykeeper.spec.cluster. Each node runs as its own ECS service. -
Infrastructure (machine types, disks, node counts): override
resource.yamlin the Infrastructure document. For production, the SigNoz capacity guide names the C5 family,c5.4xlargeand up, for ClickHouse, and the T3 family,t3.xlargeand up, for everything else:apiVersion: v1alpha1 kind: Infrastructure metadata: name: foundry annotations: foundry.signoz.io/ecs-region: <aws-region> spec: deployment: platform: ecs mode: ec2 flavor: terraform resource: spec: config: data: resource.yaml: | networking: networkCIDR: 10.0.0.0/16 subnets: private-a: type: private zone: <availability-zone> cidr: 10.0.0.0/19 public-a: type: public zone: <availability-zone> cidr: 10.0.96.0/22 instanceGroups: persistent: machineType: c5.4xlarge minSize: 3 maxSize: 3 dataVolume: size: 500 ephemeral: machineType: t3.xlarge minSize: 3 maxSize: 3Every
resource.yamlfield, with an example per section, is in Infrastructure; what happens to the data on each disk when you change a machine is under Disks. -
Platform settings (CPU, memory): add patches under a document's
spec.patches, against its files inpours/deployment/for the Installation orpours/infrastructure/for the Infrastructure. For example, to give SigNoz 1024 CPU units, in the Installation document:spec: patches: - target: deployment/signoz.tf.json operations: - op: replace path: /locals/containers_signoz_0/0/cpu value: 1024Run
foundryctl forge -f casting.yamlfrom the directory that containscasting.yamland read the generated file to find the path you need. Each run regenerates the files underpours/, so keep every change incasting.yaml.
Troubleshooting
foundryctl fails
Re-run the failing command with --debug for verbose logs.
An instance never registers
List the instances that hold data. Run this with the AWS CLI:
aws ecs list-container-instances --cluster foundry-cls --region <aws-region> \
--filter "attribute:foundry.signoz.io/storage == persistent"Verify these values:
<aws-region>: The region fromcasting.yaml.
Fewer than three entries means an instance has not mounted its data volume yet. See Validate and Changing the infrastructure.
A service never reaches its desired count
Read the service's recent events. Run this with the AWS CLI:
aws ecs describe-services --cluster foundry-cls --region <aws-region> \
--services <service-name> --query 'services[0].events[:5].message'Verify these values:
<aws-region>: The region fromcasting.yaml.<service-name>: A service name from Step 6, for examplesignoz-signoz-0.
The events say why ECS could not place or start the task. How each service is placed on the instances Foundry creates is in the ECS installation example.
For tasks that started and then stopped, list them and read why they stopped:
aws ecs list-tasks --cluster foundry-cls --region <aws-region> \
--service-name <service-name> --desired-status STOPPED
aws ecs describe-tasks --cluster foundry-cls --region <aws-region> \
--tasks <task-arn> --query 'tasks[].stoppedReason'Verify these values:
<task-arn>: A task ARN from thelist-tasksoutput.
Read container logs
The containers write their logs on the container instance with Docker's json-file log driver. Find the instance that runs a service's task. Run this with the AWS CLI:
TASK=$(aws ecs list-tasks --cluster foundry-cls --region <aws-region> --service-name <service-name> --query 'taskArns[0]' --output text)
INSTANCE=$(aws ecs describe-tasks --cluster foundry-cls --region <aws-region> --tasks "$TASK" --query 'tasks[0].containerInstanceArn' --output text)
aws ecs describe-container-instances --cluster foundry-cls --region <aws-region> --container-instances "$INSTANCE" --query 'containerInstances[0].ec2InstanceId' --output textVerify these values:
<aws-region>: The region fromcasting.yaml.<service-name>: A service name from Step 6, for examplesignoz-signoz-0.
Connect to that instance (for example with AWS Systems Manager Session Manager) and run this. The main container of each task carries its service's name:
sudo docker logs $(sudo docker ps -q --filter label=com.amazonaws.ecs.container-name=<service-name>)terraform destroy fails with Instance cannot be destroyed
Destroying SigNoz in pours/deployment/ stops with an error that names aws_ec2_tag.claims, similar to the following:
Error: Instance cannot be destroyed
Resource aws_ec2_tag.claims["vol-0a1b2c3d4e5f67890"] has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed.The claims record which data volume each stateful service owns, and they are protected from deletion. Remove them from the state, then destroy. Run this from the directory that contains casting.yaml:
cd pours/deployment
terraform state rm aws_ec2_tag.claims
terraform destroyThe claim tags stay on the volumes, so a later cast puts each service back on its own data. To remove the infrastructure too, run terraform destroy in pours/infrastructure/ afterwards. That deletes the data volumes and their data.
Lowering a shard or replica count can stop foundryctl cast with the same error. See Changing the installation instead.
Install SigNoz
This path installs SigNoz on an ECS cluster you already run. You need:
- An ECS cluster with registered EC2 container instances.
- A VPC with private subnets that route outbound traffic through a NAT gateway or VPC endpoints.
- A security group for the SigNoz tasks that allows traffic between them (the ports are listed below).
- A durable volume mounted at
/var/lib/foundryon the container instances (see Step 3).
Step 1: Install foundryctl
Run this on the machine where you run Terraform:
curl -fsSL https://signoz.io/foundry.sh | bashFor manual install (Windows PowerShell, air-gapped, etc.) or PATH setup, see the foundry getting-started guide.
Step 2: Create casting.yaml
Create a file named casting.yaml. It describes the installation and names the AWS resources SigNoz runs on:
apiVersion: v1alpha1
kind: Installation
metadata:
name: signoz
annotations:
foundry.signoz.io/ecs-region: <aws-region>
foundry.signoz.io/ecs-cluster-arn: <cluster-arn>
foundry.signoz.io/ecs-vpc-id: <vpc-id>
foundry.signoz.io/ecs-private-subnet-ids: <private-subnet-ids>
foundry.signoz.io/ecs-security-group-ids: <security-group-ids>
spec:
deployment:
platform: ecs
mode: ec2
flavor: terraformVerify these values:
<aws-region>: The AWS region of your ECS cluster, for exampleus-east-1.<cluster-arn>: The ARN of your ECS cluster, for examplearn:aws:ecs:us-east-1:111122223333:cluster/observability.<vpc-id>: The ID of the VPC your container instances run in, for examplevpc-0a1b2c3d4e5f67890.<private-subnet-ids>: Comma-separated IDs of private subnets in the Availability Zones where your container instances run, for examplesubnet-0a1b2c3d4e5f67890,subnet-0f9e8d7c6b5a43210.<security-group-ids>: Comma-separated IDs of the security groups for the SigNoz tasks, for examplesg-0a1b2c3d4e5f67890.
metadata.name prefixes every ECS service name and names the private DNS namespace the components use, signoz-installation.local.
For all options, see the casting file reference and the AWS ECS example.
Step 3: Pin the stateful services to their instances
ClickHouse, ClickHouse Keeper and PostgreSQL keep their data on the instance that runs them. Pin each one so ECS does not move it.
Add this to casting.yaml:
spec:
patches:
- target: deployment/telemetrykeeper.tf.json
operations:
- op: add
path: /resource/aws_ecs_service/telemetrykeeper_clickhousekeeper_0/placement_constraints
value:
- type: memberOf
expression: ec2InstanceId == <keeper-instance-id>
- target: deployment/telemetrystore.tf.json
operations:
- op: add
path: /resource/aws_ecs_service/telemetrystore_clickhouse_0_0/placement_constraints
value:
- type: memberOf
expression: ec2InstanceId == <clickhouse-instance-id>
- target: deployment/metastore.tf.json
operations:
- op: add
path: /resource/aws_ecs_service/metastore_postgres_0/placement_constraints
value:
- type: memberOf
expression: ec2InstanceId == <postgres-instance-id>Verify these values:
<keeper-instance-id>: The ID of the container instance that runs ClickHouse Keeper, for examplei-0123456789abcdef0.<clickhouse-instance-id>: The ID of the container instance that runs ClickHouse, for examplei-0123456789abcdef1.<postgres-instance-id>: The ID of the container instance that runs PostgreSQL, for examplei-0123456789abcdef2.
With more shards or replicas, each node's service gets its own constraint: telemetrykeeper_clickhousekeeper_<replica>, telemetrystore_clickhouse_<shard>_<replica> and metastore_postgres_<replica>.
If casting.yaml already has a spec.patches list, add these entries to it.
Step 4: Deploy
Run this from the directory that contains casting.yaml:
foundryctl cast -f casting.yamlThis installs SigNoz with Terraform in pours/deployment/. Run the same command again after editing casting.yaml to apply the change.
Step 5: Verify the installation
Check that each SigNoz service runs its task. Run this with the AWS CLI:
aws ecs describe-services --cluster <cluster-arn> --region <aws-region> \
--services signoz-telemetrykeeper-clickhousekeeper-0 signoz-telemetrystore-clickhouse-0-0 \
signoz-metastore-postgres-0 signoz-signoz-0 signoz-ingester \
--query 'services[].[serviceName,desiredCount,runningCount]' --output textVerify these values:
<cluster-arn>: The cluster ARN fromcasting.yaml.<aws-region>: The region fromcasting.yaml.
Each line shows the service name, the desired task count and the running task count. The output should look similar to the following:
signoz-telemetrykeeper-clickhousekeeper-0 1 1
signoz-telemetrystore-clickhouse-0-0 1 1
signoz-metastore-postgres-0 1 1
signoz-signoz-0 1 1
signoz-ingester 1 1The counts can take a few minutes to match. If you changed shard or replica counts, aws ecs list-services --cluster <cluster-arn> --region <aws-region> lists every service name.
ECS runs SigNoz's health check, /api/v1/health, inside the task. Read its result with the AWS CLI:
aws ecs describe-tasks --cluster <cluster-arn> --region <aws-region> \
--tasks $(aws ecs list-tasks --cluster <cluster-arn> --region <aws-region> --service-name signoz-signoz-0 --query 'taskArns[0]' --output text) \
--query 'tasks[].healthStatus' --output textVerify these values:
<cluster-arn>: The cluster ARN fromcasting.yaml.<aws-region>: The region fromcasting.yaml.
The command prints HEALTHY once the health check passes. The status reads UNKNOWN until the first check passes. Failed checks in the first five minutes after the task starts do not count against it.
SigNoz is reachable at http://signoz-0.signoz-installation.local:8080 from inside the VPC. The signoz prefix follows metadata.name.
Send data to SigNoz
Your applications send traces, metrics, and logs to the SigNoz ingester. Inside the VPC, the ingester listens at:
http://ingester.signoz-installation.local:4318Use this address as the OTLP endpoint in your instrumentation, for example as the value of OTEL_EXPORTER_OTLP_ENDPOINT. For gRPC, use port 4317 instead.
The name resolves inside the VPC, and the security group must allow ports 4317 and 4318 from your applications. The signoz in signoz-installation.local is metadata.name. If you changed it, use yours.
For applications outside the VPC, see the self-hosted ingestion guide.
Customize the installation
Every change follows the same loop: edit casting.yaml, then run foundryctl cast -f casting.yaml again.
-
Component settings (image, shards, replicas): set them in the component's
spec. For example, to pin SigNoz to a specific release:spec: signoz: spec: image: signoz/signoz:v0.144.0ClickHouse takes shard and replica counts, and ClickHouse Keeper takes a replica count:
spec: telemetrystore: spec: cluster: shards: 2 replicas: 1 telemetrykeeper: spec: cluster: replicas: 3Each node runs as its own ECS service. ClickHouse
replicascounts the copies beside each shard, so this example runs four ClickHouse services,signoz-telemetrystore-clickhouse-0-0tosignoz-telemetrystore-clickhouse-1-1, and three ClickHouse Keeper services. Keep the ClickHouse Keeper count odd. -
Platform settings (CPU, memory): add patches under
spec.patchesagainst the component's file inpours/deployment/. For example, to give SigNoz 1024 CPU units:spec: patches: - target: deployment/signoz.tf.json operations: - op: replace path: /locals/containers_signoz_0/0/cpu value: 1024Run
foundryctl forge -f casting.yamlfrom the directory that containscasting.yamland read the generated file to find the path you need. Each run regenerates the files underpours/, so keep every change incasting.yaml.
Troubleshooting
foundryctl fails
Re-run the failing command with --debug for verbose logs.
Terraform rejects a variable or a subnet
A missing or malformed annotation stops terraform plan with a validation message, for example subnet_ids must be a non-empty list of subnet IDs.
A subnet that assigns public IP addresses on launch stops the plan with a message that names it, for example subnet subnet-0a1b2c3d4e5f67890 assigns public IPs on launch; tasks on the EC2 launch type take no public IP, so state private subnets with a NAT route or VPC endpoints.
Fix the annotation in casting.yaml and run foundryctl cast -f casting.yaml again. What each annotation names is under Annotations.
A service never reaches its desired count
Read the service's recent events. Run this with the AWS CLI:
aws ecs describe-services --cluster <cluster-arn> --region <aws-region> \
--services <service-name> --query 'services[0].events[:5].message'Verify these values:
<cluster-arn>: The cluster ARN fromcasting.yaml.<aws-region>: The region fromcasting.yaml.<service-name>: A service name from Step 5, for examplesignoz-signoz-0.
The events say why ECS could not place or start the task. Each SigNoz task takes its own elastic network interface on the instance, so small instance types fit only a few tasks. See task networking.
For tasks that started and then stopped, list them and read why they stopped:
aws ecs list-tasks --cluster <cluster-arn> --region <aws-region> \
--service-name <service-name> --desired-status STOPPED
aws ecs describe-tasks --cluster <cluster-arn> --region <aws-region> \
--tasks <task-arn> --query 'tasks[].stoppedReason'Verify these values:
<task-arn>: A task ARN from thelist-tasksoutput.
Read container logs
The containers write their logs on the container instance with Docker's json-file log driver. Find the instance that runs a service's task. Run this with the AWS CLI:
TASK=$(aws ecs list-tasks --cluster <cluster-arn> --region <aws-region> --service-name <service-name> --query 'taskArns[0]' --output text)
INSTANCE=$(aws ecs describe-tasks --cluster <cluster-arn> --region <aws-region> --tasks "$TASK" --query 'tasks[0].containerInstanceArn' --output text)
aws ecs describe-container-instances --cluster <cluster-arn> --region <aws-region> --container-instances "$INSTANCE" --query 'containerInstances[0].ec2InstanceId' --output textVerify these values:
<cluster-arn>: The cluster ARN fromcasting.yaml.<aws-region>: The region fromcasting.yaml.<service-name>: A service name from Step 5, for examplesignoz-signoz-0.
Connect to that instance (for example with AWS Systems Manager Session Manager) and run this. The main container of each task carries its service's name:
sudo docker logs $(sudo docker ps -q --filter label=com.amazonaws.ecs.container-name=<service-name>)Removing SigNoz
Run this from the directory that contains casting.yaml:
cd pours/deployment && terraform destroyThis removes the services and everything else the Terraform created.