AKS Metrics & Logging

Setup

This setup is similar to the central collector but with a different function.

helm repo add signoz <https://charts.signoz.io>
helm install -n signoz  --create-namespace kubelet-otel signoz/k8s-infra \\
--set signozApiKey=<ingestionKey> --set otelCollectorEndpoint="ingest.<region>.signoz.cloud:443" --set otelInsecure=false

This should start sending logs and metrics to SigNoz.

Application-Level Tracing

For application-level tracing, you can use the OpenTelemetry SDKs integrated with your application. These SDKs will automatically collect and forward traces to the Central Collector.

Please refer to our SigNoz Tutorials or Blog to find information on how to instrument your application like Spring, FastAPI, NextJS, Node.js, Flask, Django, etc.

# Node.js example
npm install @opentelemetry/api
npm install @opentelemetry/auto-instrumentations-node
npm install @opentelemetry/exporter-trace-otlp-http

Configure the OpenTelemetry SDK

# Set env vars or config file
export OTEL_EXPORTER_OTLP_ENDPOINT="http://otel-collector.kubelet-otel.svc.cluster.local:4318/"

For application-level traces and metrics, configure your application to use the kube-dns name of the Central Collector you set up earlier.

Troubleshooting

If you encounter any issues while setting up logging and metrics for your AKS cluster, follow these troubleshooting steps:

1. Check the logs of the OpenTelemetry Collector:

   kubectl logs -f -n signoz -l app.kubernetes.io/component=otel-agent
   
   

   Review the logs for any error messages or indications of misconfiguration.

2. Verify the rendered configuration:

   kubectl get cm/kubelet-otel-k8s-infra-otel-agent -n signoz -o yaml
   
   

   Ensure that the configuration matches your expected settings, including the SigNoz API key and the OpenTelemetry Collector endpoint.

3. Confirm that the necessary Kubernetes resources are created:

   kubectl get pods,services,configmaps -n signoz
   
   

   Check if the required pods, services, and config maps are running and in a healthy state.

4. Verify network connectivity:

   • Ensure that the AKS cluster has network access to the SigNoz ingestion endpoint (ingest.<region>.signoz.cloud:443).
   • Check if there are any network security groups or firewalls blocking the required ports.

5. Double-check the SigNoz API key:

   • Confirm that the provided signozApiKey is correct and has the necessary permissions to ingest data.