Monitoring Kubernetes Clusters using Opentelemetry Collection Agents

This guide explains how to monitor Kubernetes clusters using OpenTelemetry collectors through SigNoz's k8s-infra collection agent and gain full observability into your Kubernetes infrastructure.

What is Kubernetes Monitoring with OpenTelemetry?

Kubernetes monitoring involves collecting metrics, logs, and traces from your K8s cluster using OpenTelemetry collectors. The k8s-infra collection agent simplifies OpenTelemetry Collectors deployment by providing pre-configured collectors that automatically discover and monitor all Kubernetes resources.

To learn more about the k8s-infra collection agent architecture, components, and how it works, see the K8s-Infra Overview Guide.

Prerequisites

Before setting up OpenTelemetry monitoring for your Kubernetes cluster, ensure you have:

• A SigNoz backend (SigNoz Cloud or self-hosted) to receive OpenTelemetry data
• kubectl access to your Kubernetes cluster
• Helm v3 installed for deploying the OpenTelemetry collector
• Basic understanding of Kubernetes concepts (pods, nodes, namespaces)

How OpenTelemetry Kubernetes Monitoring Works

The k8s-infra collection agent deploys OpenTelemetry collectors as a DaemonSet and a Deployment across your Kubernetes cluster:

1. Install k8s-infra OpenTelemetry collectors in your K8s cluster
2. OpenTelemetry collectors automatically discover Kubernetes resources (pods, nodes, services)
3. Telemetry data (metrics, logs, traces) is collected using OpenTelemetry standards
4. Data forwarding to SigNoz backend via OTLP (OpenTelemetry Protocol)
5. Visualization and analysis in SigNoz

Step 1: Install OpenTelemetry Collector for Kubernetes

Deploy the k8s-infra Collection agent to your Kubernetes cluster. Follow the complete step-by-step instructions in the K8s-Infra Installation Guide.

Step 2: What Telemetry Does the OpenTelemetry Collector Gather?

Once deployed, the k8s-infra collection agent automatically collects:

Node-level and Pod-level telemtry, including:

• Host Metrics: CPU usage, memory consumption, disk I/O, and network statistics from each node.
• Kubelet Metrics: Data from the Kubelet API on each node, such as pod and container resource utilization.
• Container Logs: Tailing and parsing logs generated by the containers running on the node.
• Collect Traces: Traces from the application will also be collected generated by containers on node.

Cluster metrics and events:

• Cluster-level Metrics: Information about Kubernetes objects such as Deployments, StatefulSets, DaemonSets, and Jobs.
• Kubernetes Events: Significant events within the cluster, useful for troubleshooting and auditing.
• Custom Metrics: It can be configured to collect metrics from other services in the cluster, like Redis or PostgreSQL.
• Prometheus Scraping: Metrics can also be scraped using pod annotations.

Learn more about what the k8s-infra collection agent monitors in the K8s-Infra Overview Guide.

Step 3: Configure Your OpenTelemetry Collector for Kubernetes Monitoring

The k8s-infra collection agent offers extensive configuration options to customize your monitoring setup:

Available Configuration Options

• Log collection presets: Choose between different log parsing and collection strategies
• Metrics collection settings: Enable/disable specific metric types and adjust collection intervals
• Resource filtering: Control which Kubernetes resources to monitor
• Data sampling: Configure trace sampling rates to manage data volume
• Custom processors: Add your own OpenTelemetry processors for data transformation
• Performance tuning: Adjust batch sizes, timeout values, and resource limits

For detailed information on all available configuration options, including log presets, metrics collection settings, and advanced customizations, see the K8s-Infra Configuration Guide.

Step 4: Monitor Multiple Kubernetes Clusters with OpenTelemetry

To monitor multiple Kubernetes clusters with a single SigNoz backend:

1. Deploy k8s-infra in each Kubernetes cluster you want to monitor
2. Set unique cluster names using the cluster identification parameter
3. Point to the same SigNoz endpoint for centralized observability

The installation guide provides specific instructions for multi-cluster setups. See the K8s-Infra Installation Guide for multi-cluster deployment instructions.

Step 5: Visualize Kubernetes Metrics in SigNoz

After installing the OpenTelemetry collector, view your Kubernetes data in SigNoz:

Filter by Kubernetes Cluster

• Navigate to Metrics, Logs, or Traces in SigNoz
• Filter by clusterName attribute to view specific cluster data
• Use the k8s.cluster.name resource attribute for OpenTelemetry standard filtering

Create Kubernetes Dashboards

• Per-cluster dashboards: Monitor individual cluster health and performance
• Multi-cluster comparison: Compare metrics across different Kubernetes environments
• Node-level monitoring: Track resource usage per node
• Pod-level insights: Analyze container performance and failures

Next Steps

1. K8s-Infra Overview
2. Install K8s-Infra
3. Configure K8s-Infra