K8s Infra Metrics and Logs Collection
Overview
To export K8s metrics, you can enable different receivers in OpenTelemetry collector which will send metrics about your Kubernetes (K8s) infrastructure to SigNoz. These OpenTelemetry collectors will act as agents which send metrics about K8s to SigNoz.
OtelCollector agent can also be used to tail and parse logs generated by
container using filelog
receiver and send it to desired receiver.
Based on where you are running SigNoz ( e.g. in an independent VM or K8s cluster), you have to provide the address to send data from the above receivers.
Steps to export K8s metrics to SigNoz
By default, SigNoz Helm chart installs k8s-infra
dependency chart which
handles the task of collecting metrics and logs from the K8s cluster.
K8s-Infra
helm chart mainly handles the following:
- Tails and parses logs generated by containers in K8s cluster and sends to SigNoz
- Collects kubelet metrics and host metrics from each nodes of the K8s cluster
- Collects cluster-level metrics from the Kubernetes API server
- Acts as a gateway to send any incoming OTLP telemetry data to SigNoz OtelCollector
Skip Step 1 if you have single K8s cluster for SigNoz and your applications
as k8s-infra
chart is included in default SigNoz chart installation.
Install K8s-Infra chart
helm install my-release signoz/k8s-infra \
--set otelCollectorEndpoint=<IP-or-Endpoint-of-SigNoz-OtelCollector>:4317infoIf the OtelCollector endpoint is secured, you would have to enable
otelInsecure
configuration and often make other changes such as including either config or path to the TLS certificate and private key.In case of SigNoz Cloud, you would have to set
signozApiKey
configuration.Plot Metrics in SigNoz UI
To plot metrics generated from
k8s-infra
chart, follow the instructions given in the docs here.Check out the List of metrics from Kubernetes receiver.
Import Dashboard with CPU and Memory Metrics
You can import dashboard with CPU and Memory metrics of K8s cluster containers from here.
Import Dashboard with K8s Metrics
You can import dashboard with K8s metrics of K8s cluster from here.
Generate and Import Dashboard with Node Hostmetrics
You can import dashboard with Node Hostmetrics of K8s cluster from here.
In the Dashboard page of SigNoz UI, you can create your own widgets as per you need using metrics from the list below.
List of metrics
Kubernetes Metrics - kubeletstats and k8s_cluster
- container_cpu_time
- container_cpu_utilization
- container_filesystem_available
- container_filesystem_capacity
- container_filesystem_usage
- container_memory_available
- container_memory_major_page_faults
- container_memory_page_faults
- container_memory_rss
- container_memory_usage
- container_memory_working_set
- k8s_container_cpu_limit
- k8s_container_cpu_request
- k8s_container_memory_limit
- k8s_container_memory_request
- k8s_container_ready
- k8s_container_restarts
- k8s_daemonset_current_scheduled_nodes
- k8s_daemonset_desired_scheduled_nodes
- k8s_daemonset_misscheduled_nodes
- k8s_daemonset_ready_nodes
- k8s_deployment_available
- k8s_deployment_desired
- k8s_job_active_pods
- k8s_job_desired_successful_pods
- k8s_job_failed_pods
- k8s_job_max_parallel_pods
- k8s_job_successful_pods
- k8s_namespace_phase
- k8s_node_condition_memory_pressure
- k8s_node_condition_ready
- k8s_node_cpu_time
- k8s_node_cpu_utilization
- k8s_node_filesystem_available
- k8s_node_filesystem_capacity
- k8s_node_filesystem_usage
- k8s_node_memory_available
- k8s_node_memory_major_page_faults
- k8s_node_memory_page_faults
- k8s_node_memory_rss
- k8s_node_memory_usage
- k8s_node_memory_working_set
- k8s_node_network_errors
- k8s_node_network_io
- k8s_pod_cpu_time
- k8s_pod_cpu_utilization
- k8s_pod_filesystem_available
- k8s_pod_filesystem_capacity
- k8s_pod_filesystem_usage
- k8s_pod_memory_available
- k8s_pod_memory_major_page_faults
- k8s_pod_memory_page_faults
- k8s_pod_memory_rss
- k8s_pod_memory_usage
- k8s_pod_memory_working_set
- k8s_pod_network_errors
- k8s_pod_network_io
- k8s_pod_phase
- k8s_replicaset_available
- k8s_replicaset_desired
- k8s_statefulset_current_pods
- k8s_statefulset_desired_pods
- k8s_statefulset_ready_pods
- k8s_statefulset_updated_pods
- k8s_volume_available
- k8s_volume_capacity
- k8s_volume_inodes
- k8s_volume_inodes_free
- k8s_volume_inodes_used
- k8s_node_allocatable_cpu
- k8s_node_allocatable_memory
Node Hostmetrics - hostmetrics
- system_network_connections
- system_disk_weighted_io_time
- system_disk_merged
- system_disk_operation_time
- system_disk_pending_operations
- system_disk_io_time
- system_disk_operations
- system_disk_io
- system_filesystem_inodes_usage
- system_filesystem_usage
- system_cpu_time
- system_memory_usage
- system_network_packets
- system_network_dropped
- system_network_io
- system_network_errors
- system_cpu_load_average_5m
- system_cpu_load_average_15m
- system_cpu_load_average_1m