Configuring OpenTelemetry Kubernetes Metrics Collection for GKE Autopilot

This guide explains how to configure kubelet metrics collection in the k8s-infra OpenTelemetry collection agent for GKE Autopilot clusters, where certain RBAC permissions are restricted.

Prerequisites

Before configuring kubelet metrics collection for GKE Autopilot, ensure you have:

• A SigNoz backend (SigNoz Cloud or self-hosted) to receive OpenTelemetry data K8s-infra OpenTelemetry collection agent installed in your GKE Autopilot cluster
• Basic understanding of Kubernetes RBAC and GKE Autopilot security model

How Kubelet Metrics Collection Works in k8s-infra

The k8s-infra collection agent includes a kubeletMetrics preset that uses the OpenTelemetry Collector's kubeletstats receiver to gather node, pod, container, and volume metrics from the Kubelet API.

Understanding GKE Autopilot Limitations

GKE Autopilot blocks access to the nodes/proxy RBAC resource under its security model. It disables the insecure kubelet read-only port in favor of the more secure, authenticated port 10250 and restricts related proxy access.

Since the kubeletstats receiver needs get permissions for nodes/proxy resources to scrape extra_metadata_labels and {request|limit}_utilization metrics (refer to the kubeletstats receiver RBAC requirements), these metrics cannot be scraped on GKE Autopilot. When enabled, the otel-agent DaemonSet crashes repeatedly due to privilege errors, disrupting metrics collection. Disabling these metrics allows us to match the agent configuration to GKE Autopilot's security policies while still collecting essential kubelet metrics such as node, pod, container, and volume metrics.

Configure Kubelet Metrics Collection for GKE Autopilot

Each setup path below applies these settings for you: on GKE Autopilot, it disables extra_metadata_labels and the {request|limit}_utilization metrics.

global:
  cloud: gcp/autogke

helm upgrade <RELEASE_NAME> signoz/k8s-infra -n <NAMESPACE> -f override-values.yaml

foundryctl cast -f casting.yaml

foundryctl cast -f casting.yaml

Manual Configuration

If you manage your own OpenTelemetry Collector configuration, you must explicitly disable the problematic metrics:

Update your values.yaml or collector configuration:

receivers:
  kubeletstats:
    # Disable extra metadata labels that require nodes/proxy access
    extra_metadata_labels: []
    # Disable metrics that require nodes/proxy permissions
    metrics:
      k8s.pod.cpu_limit_utilization:
        enabled: false
      k8s.pod.cpu_request_utilization:
        enabled: false
      k8s.pod.memory_limit_utilization:
        enabled: false
      k8s.pod.memory_request_utilization:
        enabled: false
      k8s.container.cpu_limit_utilization:
        enabled: false
      k8s.container.cpu_request_utilization:
        enabled: false
      k8s.container.memory_limit_utilization:
        enabled: false
      k8s.container.memory_request_utilization:
        enabled: false

Validate

1. Wait a minute for the updated settings to take effect.
2. Open the SigNoz dashboard and confirm that node and pod metrics are visible (check the infrastructure or metrics page).
3. If metrics do not appear as expected, inspect the k8s-infra collector pod logs for errors or warnings.

Next Steps

1. K8s-Infra Overview
2. Configure K8s-Infra
3. Telemetry Data Requirements for Infrastructure Monitoring
4. **Best Practices for OpenTelemetry Collector Configuration