GKE Metrics and Logging

Prerequisites

• GKE cluster
• Install kubectl to access the GKE cluster.
• Install Helm

Quick Start

Check that the nodes in your GKE clusters are in Ready state:

Step 1: Add the SigNoz helm repo using this command

helm repo add SigNoz https://charts.signoz.io

Step 2: Install Otel Agent and Deployment using signoz/k8s-infra Helm chart.

helm install -n signoz  --create-namespace kubelet-otel signoz/k8s-infra \\
--set signozApiKey=<ingestionKey> --set otelCollectorEndpoint="ingest.<region>.signoz.cloud:443" --set otelInsecure=false

After applying the above commands, check whether the signoz namespace pods are running successfully or not:

If running, it should start sending logs and metrics to SigNoz.

Afterward, you could create a pod to check to validate whether the logs and metrics are being sent successfully to SigNoz Cloud.

kubectl run -n signoz test –image=nginx –port=80

Visualize the logs and metrics in SigNoz Cloud

To visualize logs:

Go to your SigNoz UI, and navigate to the SigNoz dashboard. Click on the Logs section to view the logs

For metrics:

1. Log in to your SigNoz cloud account.
2. Navigate to the Dashboards, and add a dashboard
3. Add a Timeseries Panel
4. In Metrics, select any metric
5. In Filter, put name = mention service name
6. Hit “Save Changes”.

After saving changes, it should start showing collected metrics.

Troubleshooting

If you encounter any issues while setting up logging and metrics for your GKE cluster, follow these troubleshooting steps:

1. Check the logs of the OpenTelemetry Collector:

   kubectl logs -f -n signoz -l app.kubernetes.io/component=otel-agent
   

   Review the logs for any error messages or indications of misconfiguration.

2. Verify the rendered configuration:

   kubectl get cm/kubelet-otel-k8s-infra-otel-agent -n signoz -o yaml
   

   Ensure the configuration matches your expected settings, including the SigNoz API key and the OpenTelemetry Collector endpoint.

3. Confirm that the necessary Kubernetes resources are created:

   kubectl get pods,services,configmaps -n signoz
   

   Check if the required pods, services, and config maps are running and in a healthy state.

4. Verify network connectivity:

   • Ensure that the GKE cluster has network access to the SigNoz ingestion endpoint (ingest.<region>.signoz.cloud:443).
   • Check if there are any network security groups or firewalls blocking the required ports.

5. Double-check the SigNoz API key:

   • Confirm that the provided signozApiKey is correct and has the necessary permissions to ingest data.