Cloud Load Balancer Metrics

Prerequisites

• You should have a Google Cloud account with administrative privileges or Cloud Load Balancer Admin privileges both should suffice.
• Cloud Load Balancer
• Access to a project in Google Cloud Platform (GCP).
• The Google Cloud Load Balancer APIs must be enabled. You can follow this guide to learn how to enable an API in Google Cloud.

Setup

Deploy OpenTelemetry to scrape the metrics from Google Cloud Monitoring

Step 1: Install and configure OpenTelemetry for scraping the metrics from Google Cloud Monitoring. Follow OpenTelemetry Binary Usage in Virtual Machine guide for detailed instructions.

Step 2: Create config.yaml.

Under the googlecloudmonitoring receiver, you need to specify the metrics that you want to capture in the metrics_list. You can include all the metrics that you would like to capture for Cloud Load Balancer, see the available metrics for Cloud Load Balancer.

Here is the config.yaml file for capturing Cloud Load Balancer metrics:

receivers:
  googlecloudmonitoring:
    collection_interval: 5m # Can be specified in seconds (s), minutes (m), or hours (h)
    project_id: gcp-project-id
    metrics_list:
      - metric_name: "loadbalancing.googleapis.com/https/backend_latencies"
      - metric_name: "loadbalancing.googleapis.com/https/backend_request_count"
      - metric_name: "loadbalancing.googleapis.com/https/request_bytes_count"
      - metric_name: "loadbalancing.googleapis.com/https/request_count"
      - metric_name: "loadbalancing.googleapis.com/https/response_bytes_count"
      - metric_name: "loadbalancing.googleapis.com/https/total_latencies"
processors:
  resource/env:
    attributes:
    - key: deployment.environment
      value: prod
      action: upsert
  batch: {}
exporters:
  otlp:
    endpoint: "ingest.{region}.signoz.cloud:443"
    tls:
      insecure: false
    headers:
      "signoz-ingestion-key": "<SIGNOZ_INGESTION_KEY>"
service:
  pipelines:
    metrics:
      receivers: [googlecloudmonitoring]
      processors: [resource/env,batch]
      exporters: [otlp]

Depending on the choice of your region for SigNoz cloud, the otlp endpoint will vary according to this table.

After successful configuration, start the OTel Collector using the following command:

./otelcol-contrib --config ./config.yaml &> otelcol-output.log & echo "$!" > otel-pid

Step 3: If the configurations are configured correctly, you can see the output logs from OpenTelemtry as follows:

Send and Visualize the metrics obtained by OpenTelemetry in SigNoz

Step 1: Go to the SigNoz Cloud URL and head over to the dashboard.

Step 2: If not already created, create a new dashboard. You can create the dashboard and multiple panel under it by following the instructions here.

Step 3: While creating the panel, select metric for Cloud Load Balancer.

All metrics starting with loadbalancing_googleapis_com_ have been collected from Cloud Load Balancer.

For example loadbalancing_googleapis_com_https_total_latencies is one of the metrics that was collected.

Here is the sample dashboard for Cloud Load Balancer:

Troubleshooting

If you run into any problems while setting up monitoring for your Cloud Load Balancer metrics with SigNoz, consider these troubleshooting steps:

• Verify Configuration: Double-check your config.yaml file to ensure all settings, including the ingestion key and endpoint, are correct.
• Review Logs: Look at the logs of your Cloud Load Balancer and the OpenTelemetry Collector to identify any error messages or warnings that might provide insights into what’s going wrong.
• Consult Documentation: Review the SigNoz and OpenTelemetry documentation for additional troubleshooting of the common issues.

Prerequisites

• A Google Cloud account with administrative privileges or Cloud Load Balancer Admin privileges.
• Cloud Load Balancer
• Access to a project in Google Cloud Platform (GCP).
• The Google Cloud Load Balancer APIs must be enabled. You can follow this guide to learn how to enable an API in Google Cloud.

Setup

Deploying a Self-Hosted SigNoz in a Docker container

Step 1: The deployment process of the Self-Hosted SigNoz can be found in official documentation.

Step 2: Ensure all the containers are running in a Healthy state.

exporters:
  otlp:
	endpoint: "<endpoint-ip>:4317"
	tls:
  	   insecure: true

receivers:
  googlecloudmonitoring:
    collection_interval: 5m # Can be specified in seconds (s), minutes (m), or hours (h)
    project_id: gcp-project-id
    metrics_list:
      - metric_name: "loadbalancing.googleapis.com/https/backend_latencies"
      - metric_name: "loadbalancing.googleapis.com/https/backend_request_count"
      - metric_name: "loadbalancing.googleapis.com/https/request_bytes_count"
      - metric_name: "loadbalancing.googleapis.com/https/request_count"
      - metric_name: "loadbalancing.googleapis.com/https/response_bytes_count"
      - metric_name: "loadbalancing.googleapis.com/https/total_latencies"
processors:
  resource/env:
    attributes:
    - key: deployment.environment
      value: prod
      action: upsert
  batch: {}
exporters:
  otlp:
	endpoint: "<endpoint-ip>:4317"
	tls:
  	   insecure: true
service:
  pipelines:
    metrics:
      receivers: [googlecloudmonitoring]
      processors: [resource/env,batch]
      exporters: [otlp]

./otelcol-contrib --config ./config.yaml &> otelcol-output.log & echo "$!" > otel-pid

Visualize the metrics in Self-Hosted SigNoz

Step 1: Go to your Self-Hosted SigNoz UI and navigate to the Self-Hosted SigNoz dashboard. Click on the Dashboards section to view the metrics. Create a new dashboard (If not already present )

Step 2: If not already created, create a new dashboard. You can create the dashboard and multiple panel under it by following the instructions here.

Step 3: Select the metric for the Cloud Load Balancer

For example, loadbalancing_googleapis_com_https_total_latencies is one of the metrics that is collected.

Troubleshooting

If you run into any problems while setting up monitoring for your Cloud Load Balancer metrics with SigNoz, consider these troubleshooting steps:

• Verify Configuration: Double-check your config.yaml file to ensure all settings, including endpoint, are correct.
• Verify Docker container logs: Check if all the containers are running properly and do not unexpectedly crash.
• Review Logs: Look at the logs of both your Cloud Load balancer and the OpenTelemetry Collector to identify any error messages or warnings that might provide insights into what’s going wrong.
• Consult Documentation: Review the SigNoz and OpenTelemetry documentation for any additional troubleshooting of the common issues.