K8s Serverless (EKS Fargate) - Configure

The install guide provides a working config: the deployment collector collects kubelet metrics from every Fargate node, cluster metrics, Kubernetes events, and data from instrumented applications. The optional sidecar receives an application's telemetry inside its own pod. This page explains what each component in that config does and how to customize it through Foundry's casting.yaml.

Prerequisites

• EKS Fargate Collection Agent installed with Foundry (see the install guide, or the OpenTelemetry Operator guide for the operator)
• SigNoz Cloud account or self-hosted SigNoz instance
• Access to the cluster with kubectl
• The casting.yaml and foundryctl you installed with

Apply Changes

For each change:

1. Add the section's snippet to casting.yaml, in the deployment collector's document (collector.kind: deployment) or the sidecar's (collector.kind: sidecar). Foundry already generates the default receivers and processors, so add only what you change.

2. For the deployment collector, run this from the directory that contains casting.yaml:

   Copy

   foundryctl cast -f casting.yaml

   Foundry updates the collector's config, and the deployment collector's pod restarts with it.

   For the sidecar, follow the way you added it, as Collector Sidecar shows:

   • With a Kustomize patch: forge again, apply your application again, and restart each workload that runs the sidecar.
   • With a pod template: forge again, apply the sidecar's files again, copy the two entries again if they changed, and restart each workload that runs the sidecar.

   For example, for a Deployment, run kubectl -n <your-namespace> rollout restart deployment/<your-workload> with your application's namespace and the workload's name.

3. Check that the collector starts without configuration errors, as View the Collector's Logs shows.

4. In SigNoz, confirm data in Infrastructure Monitoring for node, pod, and container metrics, Logs for Kubernetes events, and Traces for application traces.

With the OpenTelemetry Operator, the operator rolls the collector's workload after foundryctl cast, and the sidecar follows Apply Changes in the OpenTelemetry Operator guide.

Send Data from Applications to the Collector

Kubelet and cluster metrics and Kubernetes events flow as soon as the deployment collector is running.

Where and how to send data from your application

To send traces, metrics, and logs from an application, instrument it with OpenTelemetry (language guides), then set its endpoint on the application's container in its manifest.

In a pod that runs the sidecar, send to the sidecar on localhost:

env:
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
    value: http://localhost:4318
  - name: OTEL_SERVICE_NAME
    value: <service-name>

Verify these values:

• <service-name>: The name your application shows under in SigNoz.

The sidecar adds the pod's Kubernetes resource attributes. Use port 4317 for OTLP over gRPC.

In any other pod, send to the deployment collector's Service:

env:
  - name: K8S_POD_IP
    valueFrom:
      fieldRef:
        fieldPath: status.podIP
  - name: K8S_POD_UID
    valueFrom:
      fieldRef:
        fieldPath: metadata.uid
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
    value: http://signoz-collector-deployment.signoz.svc.cluster.local:4318
  - name: OTEL_SERVICE_NAME
    value: <service-name>
  - name: OTEL_RESOURCE_ATTRIBUTES
    value: k8s.pod.ip=$(K8S_POD_IP),k8s.pod.uid=$(K8S_POD_UID)

With the OpenTelemetry Operator, use http://signoz-collector-deployment-collector.signoz.svc.cluster.local:4318, the Service the operator creates in Step 4: Deploy of the OpenTelemetry Operator guide.

Verify these values:

• <service-name>: The name your application shows under in SigNoz.
• signoz in the address: metadata.name, which is also the namespace unless the foundry.signoz.io/kubernetes-namespace annotation sets another.

With k8s.pod.ip and k8s.pod.uid set, the deployment collector adds the pod's Kubernetes resource attributes. Use port 4317 for OTLP over gRPC.

Redeploy the application and send it some traffic. Within a minute it shows under Services in SigNoz, and its traces carry k8s.namespace.name and k8s.pod.name.

Export to SigNoz

Both collectors send every signal to SIGNOZ_INGESTION_ENDPOINT over OTLP/HTTP.

For SigNoz Cloud, the install guide's documents add the ingestion key as a header.

Collect Kubelet Metrics

The deployment collector collects node, pod, container, and volume metrics from the kubelet of each Fargate node, through the Kubernetes API server.

It scrapes every 30s, with the metrics in the table below enabled. To change the interval and enable one more metric, add this under receivers.receiver_creator.receivers.kubeletstats.config in the deployment collector's document:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            receivers:
              receiver_creator:
                receivers:
                  kubeletstats:
                    config:
                      collection_interval: 60s
                      metrics:
                        k8s.pod.memory.node.utilization:
                          enabled: true

metrics is a map, so the metrics you add join Foundry's defaults.

These container-level metrics are off by default upstream and on in Foundry's config. Keep them enabled to retain full container visibility on the Containers view:

Then apply the change.

Collect Cluster Metrics

The deployment collector collects cluster-level metrics from the Kubernetes API server, every 30s.

To change the interval and report ephemeral storage too, add this under receivers.k8s_cluster in the deployment collector's document:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            receivers:
              k8s_cluster:
                collection_interval: 60s
                allocatable_types_to_report: [cpu, memory, ephemeral-storage]

Lists replace, so allocatable_types_to_report holds only the types you list. Then apply the change.

Collect Kubernetes Events

The deployment collector collects Kubernetes events as logs, for all namespaces.

To watch specific namespaces, add this under receivers.k8s_events in the deployment collector's document:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            receivers:
              k8s_events:
                namespaces: [default, production]

Then apply the change.

Collect Container Logs

Application logs reach SigNoz over OTLP from the application, or through CloudWatch Logs from Fargate's log router. For both routes and the CloudWatch setup, see Collect EKS Fargate Logs with OpenTelemetry and CloudWatch.

Resource Detection and Kubernetes Attributes

Foundry adds these resource attributes by default, in the deployment collector and the sidecar, so you can filter and group by namespace, workload, or pod in SigNoz. They are the namespace, pod name and UID, node, and owning workload of the pod that sent the data, plus k8s.cluster.name from K8S_CLUSTER_NAME, cloud.provider, cloud.platform, and the attributes in the collector's OTEL_RESOURCE_ATTRIBUTES. Workloads that run the sidecar under service accounts of their own need signoz-rolebinding.yaml, as Collector Sidecar shows.

To add a pod label as a resource attribute, add this under processors.k8sattributes in the deployment collector's document:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            processors:
              k8sattributes:
                extract:
                  labels:
                    - tag_name: service.name
                      key: app.kubernetes.io/name
                      from: pod

For the sidecar, add the same block to its document, keyed collector/sidecar/sidecar.yaml. Lists replace, so restating extract.metadata or pod_association drops Foundry's defaults: include every entry you keep, as the warning below describes. Then apply the change.

Scrape Prometheus Metrics from pods

The deployment collector can scrape metrics from pods that expose a Prometheus endpoint. In casting.yaml, add this to the deployment collector's document to scrape pods annotated signoz.io/scrape: "true" every 60s, at the path in signoz.io/path and the port in signoz.io/port:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            receivers:
              prometheus/scraper:
                config:
                  scrape_configs:
                    - job_name: signoz-scraper
                      scrape_interval: 60s
                      kubernetes_sd_configs:
                        - role: pod
                      relabel_configs:
                        - source_labels: [__meta_kubernetes_pod_annotation_signoz_io_scrape]
                          action: keep
                          regex: "true"
                        - source_labels: [__meta_kubernetes_pod_annotation_signoz_io_path]
                          action: replace
                          regex: (.+)
                          target_label: __metrics_path__
                        - source_labels: [__meta_kubernetes_pod_ip, __meta_kubernetes_pod_annotation_signoz_io_port]
                          action: replace
                          separator: ":"
                          target_label: __address__
            service:
              pipelines:
                metrics:
                  receivers: [otlp/http, otlp/grpc, k8s_cluster, receiver_creator, prometheus/scraper]

Lists replace, so the snippet restates the metrics pipeline's default receivers. Then apply the change.

Tune Batch and Memory Limiter

Foundry batches telemetry before export, in the deployment collector and the sidecar, with send_batch_size: 1000, send_batch_max_size: 2048, and timeout: 10s. For sizing guidance, see Collector Configuration Best Practices. To change the batch sizes, add this under processors.batch in the deployment collector's document:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            processors:
              batch:
                send_batch_size: 15000
                send_batch_max_size: 20000

For the sidecar, add the same block to its document, keyed collector/sidecar/sidecar.yaml. Keys you leave out keep Foundry's values. Then apply the change.

Each collector limits its memory use to 80% of its container's memory limit, with a spike allowance of 20%, so it follows the limits in the next section.

Resource Requests and Limits

Fargate sizes each pod from the CPU and memory requests of its containers, rounded up to one of the Fargate configurations. For the configurations, see Fargate pod configuration.

Foundry's defaults set requests and limits to the same values:

• The deployment collector requests 250m CPU and 512Mi memory, so its pod runs in the 0.25 vCPU / 1 GB Fargate configuration.
• The sidecar requests 200m CPU and 256Mi memory in each pod that runs it. A pod whose containers request 0.25 vCPU and 512 MiB runs in the 0.5 vCPU / 1 GB configuration with the sidecar.

In casting.yaml, add a patch to the deployment collector's document to raise its requests and limits:

spec:
  patches:
    - target: collectionagent/collector/deployment/workload.yaml
      operations:
        - op: replace
          path: /spec/template/spec/containers/0/resources
          value:
            requests:
              cpu: 500m
              memory: 1Gi
            limits:
              cpu: 500m
              memory: 1Gi

For the sidecar, add a patch to its document that sets the resources of its container in patch.yaml, and in patch-cronjob.yaml if a CronJob takes the sidecar:

spec:
  patches:
    - target: collectionagent/collector/sidecar/patch.yaml
      operations:
        - op: replace
          path: /spec/template/spec/initContainers/0/resources
          value:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              cpu: 250m
              memory: 512Mi
    - target: collectionagent/collector/sidecar/patch-cronjob.yaml
      operations:
        - op: replace
          path: /spec/jobTemplate/spec/template/spec/initContainers/0/resources
          value:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              cpu: 250m
              memory: 512Mi

With the OpenTelemetry Operator, patch /spec/resources of collectionagent/collector/deployment/opentelemetrycollector.yaml and collectionagent/collector/sidecar/opentelemetrycollector.yaml instead, as Customize the Collection Agent in the OpenTelemetry Operator guide shows.

Then apply the change.

Add Custom Processors to Pipelines

A processor transforms or enriches telemetry once a pipeline lists it under processors.

Foundry's pipelines are traces, metrics and logs, in the deployment collector and the sidecar. In casting.yaml, add this to the deployment collector's document to define a processor and add it to the traces pipeline:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            processors:
              attributes/team:
                actions:
                  - key: team
                    value: platform
                    action: insert
            service:
              pipelines:
                traces:
                  processors: [memory_limiter, resourcedetection, resource/identity, k8sattributes, attributes/team, batch]

For the sidecar, add the same block to its document, keyed collector/sidecar/sidecar.yaml. Lists replace, so restate Foundry's default processors in the order the snippet shows, with yours before batch. Then apply the change.

Exposing Ports for Custom Receivers

A receiver that listens on a port needs that port reachable from your applications. This example adds one on port 8125.

In casting.yaml, add the statsd receiver to the deployment collector's document and list it in the metrics pipeline:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            receivers:
              statsd:
                endpoint: 0.0.0.0:8125
            service:
              pipelines:
                metrics:
                  receivers: [otlp/http, otlp/grpc, k8s_cluster, receiver_creator, statsd]

Lists replace, so the snippet restates the metrics pipeline's default receivers. Then open the port on the collector with a patch in the same document:

spec:
  patches:
    - target: collectionagent/collector/deployment/workload.yaml
      operations:
        - op: add
          path: /spec/template/spec/containers/0/ports/-
          value: {name: statsd, containerPort: 8125, protocol: UDP}
    - target: collectionagent/collector/deployment/service.yaml
      operations:
        - op: add
          path: /spec/ports/-
          value: {name: statsd, port: 8125, targetPort: statsd, protocol: UDP}

Your applications reach the receiver on the deployment collector's Service, at signoz-collector-deployment.signoz.svc.cluster.local:8125, where signoz is the namespace from metadata.name.

With the OpenTelemetry Operator, add the port under /spec/ports of the deployment collector's OpenTelemetryCollector resource, as Customize the Collection Agent in the OpenTelemetry Operator guide describes, and reach it at signoz-collector-deployment-collector.signoz.svc.cluster.local:8125.

In a pod that runs the sidecar, your application reaches the sidecar on localhost, so a receiver needs only its config, in the sidecar's document:

spec:
  collector:
    spec:
      config:
        data:
          collector/sidecar/sidecar.yaml: |
            receivers:
              statsd:
                endpoint: localhost:8125
            service:
              pipelines:
                metrics:
                  receivers: [otlp/http, otlp/grpc, statsd]

Then apply the change.

Disable Metrics Collection

To turn off kubelet and cluster metrics, add this to the deployment collector's document. It removes receivers.receiver_creator and receivers.k8s_cluster:

spec:
  collector:
    spec:
      config:
        data:
          collector/deployment/deployment.yaml: |
            receivers:
              receiver_creator: null
              k8s_cluster: null
            service:
              pipelines:
                metrics:
                  receivers: [otlp/http, otlp/grpc]

Lists replace, so the snippet restates the metrics pipeline with its two OTLP receivers. Kubernetes events keep flowing. Then apply the change.

Add the Sidecar to More Workloads

Each workload takes the sidecar as Collector Sidecar shows. To add it to one more workload:

With a Kustomize patch, add a target for the workload under patches: in your application's kustomization.yaml. A CronJob takes pours/collectionagent/collector/sidecar/patch-cronjob.yaml with kind: CronJob:

 patches:
+  - path: pours/collectionagent/collector/sidecar/patch.yaml
+    target:
+      kind: <your-workload-kind>
+      name: <your-workload>

Verify these values:

• <your-workload-kind>: The workload's kind: Deployment, StatefulSet, DaemonSet, or Job.
• <your-workload>: The workload's name, from its manifest.

If the workload runs under a service account of its own, add the account to the subjects of signoz-rolebinding.yaml. Then apply your application again.

With the pod template, add the initContainers entry and the volumes entry of pours/collectionagent/collector/sidecar/patch.yaml to the workload's pod template. In a CronJob, the pod template sits under spec.jobTemplate. Then apply the workload.

With the OpenTelemetry Operator, add the sidecar.opentelemetry.io/inject annotation to the workload's pod template, as Step 3: Add the sidecar to your workloads in the OpenTelemetry Operator guide shows.

Migrate to Foundry

If you already run the manual sidecar collector, move it to Foundry with these steps. That setup is the k8s-infra chart release my-release without its agent, plus an opentelemetry-collector-contrib sidecar container in your application's Deployments.

1. Install foundryctl and create casting.yaml, as Step 1 and Step 2 of the install guide show.

2. Add your customizations to casting.yaml under spec.collector.spec.config.data, keyed collector/deployment/deployment.yaml for the deployment collector and collector/sidecar/sidecar.yaml for the sidecar, as Customize the Collection Agent shows.

3. Remove the my-release chart release. Run this on the machine where you use kubectl:

   Copy

   helm uninstall my-release --namespace <release-namespace>

   Verify these values:

   • <release-namespace>: The namespace of the my-release release, as helm list -A shows.

4. Deploy the deployment collector with Foundry. Run this from the directory that contains casting.yaml:

   Copy

   foundryctl cast -f casting.yaml

5. Remove the old sidecar from each of your application's Deployments: the opentelemetry-collector-contrib container, the otel-agent-config-vol volume, and serviceAccountName: otel-agent-sidecar.

6. Add Foundry's sidecar as Collector Sidecar shows, or point the application at the deployment collector's Service as Send Data from Applications to the Collector shows. Then apply your application.

7. Remove the old ConfigMap, service account, and RBAC. Run this on the machine where you use kubectl:

   Copy

   kubectl -n <your-namespace> delete configmap otel-agent-sidecar-config
   kubectl -n <your-namespace> delete serviceaccount otel-agent-sidecar
   kubectl delete clusterrolebinding otel-kubeletstats-proxy
   kubectl delete clusterrole otel-kubeletstats-proxy

   Verify these values:

   • <your-namespace>: The namespace of your application pods that ran the old sidecar.

8. Verify the Collection Agent as Step 4 of the install guide shows.

View the Collector's Logs

Run this on the machine where you use kubectl to read the deployment collector's logs:

kubectl logs -n signoz deployment/signoz-collector-deployment --since=10m

To read the sidecar's logs in a workload's pods, name its container:

kubectl logs -n <your-namespace> deployment/<your-workload> -c signoz-collector-sidecar --since=10m

With the OpenTelemetry Operator, read deployment/signoz-collector-deployment-collector and the injected sidecar container otc-container instead, the names Step 5: Verify the Collection Agent and Step 4: Verify the sidecar show.

Verify these values:

• signoz: The namespace from metadata.name, unless the foundry.signoz.io/kubernetes-namespace annotation sets another.
• <your-namespace>: Your application's namespace.
• <your-workload>: The workload's name, from its manifest; for a Job, write job/<your-workload> instead of deployment/<your-workload>.

A collector that starts without configuration errors logs Everything is ready. Begin running and processing data.

Next Steps

• EKS Fargate Collection Agent Overview
• Add the sidecar to your workloads
• Telemetry Data Requirements for Infrastructure Monitoring
• Collector Configuration Best Practices