Cloud SQL Metrics

Here's a quick summary of what we will be doing in this guide

• Create and configure Cloud SQL
• Create and configure Compute Engine VM instance to deploy OpenTelemetry Collector
• Deploy OpenTelemetry Collector to scrape the metrics from Google Cloud Monitoring
• Send and Visualize the metrics in SigNoz Cloud

Prerequisites

• Google Cloud account with administrative privilege or Cloud SQL Admin privilege.
• SigNoz Cloud Account (we are using SigNoz Cloud for this demonstration, we will also need ingestion details. To get your Ingestion Key and Ingestion URL, sign-in to your SigNoz Cloud Account and go to Settings >> Ingestion Settings)
• Access to a project in GCP

Setup

Get started with Cloud SQL Configuration

Follow the steps mentioned in the Creating Cloud SQL document to create Cloud SQL instance.

Deploy OpenTelemetry Collector to scrape the metrics from Google Cloud Monitoring

Step 1: Install and configure OpenTelemetry for scraping the metrics from Google Cloud Monitoring. Follow OpenTelemetry Binary Usage in Virtual Machine guide for detailed instructions.

Step 2: Create config.yaml.

Under the googlecloudmonitoring receiver, you need to specify the metrics that you want to capture in the metrics_list. You can include all the metrics that you would like to capture for Cloud SQL, see the available metrics for Cloud SQL.

Here is the config.yaml file for capturing Cloud Storage metrics:

receivers:
  googlecloudmonitoring:
    collection_interval: 5m # Can be specified in seconds (s), minutes (m), or hours (h)
    project_id: my-project-id
    metrics_list:
      - metric_name: "cloudsql.googleapis.com/database/cpu/usage_time"
      - metric_name: "cloudsql.googleapis.com/database/cpu/utilization"
      - metric_name: "cloudsql.googleapis.com/database/disk/bytes_used"
      - metric_name: "cloudsql.googleapis.com/database/disk/utilization"
      - metric_name: "cloudsql.googleapis.com/database/mysql/ddl_operations_count"
      - metric_name: "cloudsql.googleapis.com/database/mysql/dml_operations_count"
processors:
  resource/env:
    attributes:
    - key: deployment.environment
      value: prod
      action: upsert
  batch: {}
exporters:
  otlp:
    endpoint: "ingest.{region}.signoz.cloud:443"
    tls:
      insecure: false
    headers:
      "signoz-ingestion-key": "<your-ingestion-key>"
service:
  pipelines:
    metrics:
      receivers: [googlecloudmonitoring]
      processors: [resource/env,batch]
      exporters: [otlp]

Depending on the choice of your region for SigNoz cloud, the otlp endpoint will vary according to this table.

After successful configuration start the OTel Collector using following command:

./otelcol-contrib --config ./config.yaml &> otelcol-output.log & echo "$!" > otel-pid

Step 3: If the configurations are configured correctly, you can see the output logs from OpenTelemtry as follows:

Send and Visualize the metrics obtained by OpenTelemetry in SigNoz

Step 1: Go to the SigNoz Cloud URL and head over to the dashboard.

Step 2: If not already created, create a new dashboard. You can create the dashboard and multiple panel under it by following the instructions here.

Step 3: While creating the panel, select metric for Cloud SQL.

All metrics starting with cloudsql_googleapis_com_ have been collected from Cloud SQL.

For example, cloudsql_googleapis_com_database_cpu_usage_time is one of the metrics which is collected.

Here is the sample dashboard for Cloud SQL:

Troubleshooting

If you run into any problems while setting up monitoring for your Cloud SQL's metrics with SigNoz, consider these troubleshooting steps:

• Verify Configuration: Double-check your config.yaml file to ensure all settings, including the ingestion key and endpoint, are correct.
• Review Logs: Look at the logs of the OpenTelemetry Collector to identify any error messages or warnings that might provide insights into what’s going wrong.
• Update Dependencies: Ensure all relevant packages and dependencies are up-to-date to avoid compatibility issues.
• Consult Documentation: Review the SigNoz and OpenTelemetry documentation for any additional troubleshooting of the common issues.

Here’s a quick summary of what we will be doing in this guide

• Create and configure Cloud SQL
• Create and configure Compute Engine VM instance to deploy OpenTelemetry Collector
• Deploy OpenTelemetry Collector to scrape the metrics from Google Cloud Monitoring
• Visualize the metrics in SigNoz dashboard

Prerequisites

• Google Cloud account with administrative privilege or Cloud SQL Admin privilege.
• Access to a project in GCP

Setup

Get started with Cloud SQL Configuration

Follow the steps mentioned in the Creating Cloud SQL document to create Cloud SQL instance.

Deploy OpenTelemetry Collector to scrape the metrics from Google Cloud Monitoring

Step 1: Install and configure OpenTelemetry for scraping the metrics from Google Cloud Monitoring. Follow OpenTelemetry Binary Usage in Virtual Machine guide for detailed instructions.

Step 2: Create config.yaml.

Under the googlecloudmonitoring receiver, you need to specify the metrics that you want to capture in the metrics_list. You can include all the metrics that you would like to capture for Cloud SQL, see the available metrics for Cloud SQL.

Here is the config.yaml file for capturing Cloud Storage metrics:

receivers:
  googlecloudmonitoring:
    collection_interval: 5m # Can be specified in seconds (s), minutes (m), or hours (h)
    project_id: my-project-id
    metrics_list:
      - metric_name: "cloudsql.googleapis.com/database/cpu/usage_time"
      - metric_name: "cloudsql.googleapis.com/database/cpu/utilization"
      - metric_name: "cloudsql.googleapis.com/database/disk/bytes_used"
      - metric_name: "cloudsql.googleapis.com/database/disk/utilization"
      - metric_name: "cloudsql.googleapis.com/database/mysql/ddl_operations_count"
      - metric_name: "cloudsql.googleapis.com/database/mysql/dml_operations_count"
processors:
  resource/env:
    attributes:
    - key: deployment.environment
      value: prod
      action: upsert
  batch: {}
exporters:
  otlp:
    endpoint: "<IP of machine hosting SigNoz>:4317"
    tls:
      insecure: true
service:
  pipelines:
    metrics:
      receivers: [googlecloudmonitoring]
      processors: [resource/env,batch]
      exporters: [otlp]

After successful configuration start the OTel Collector using following command:

./otelcol-contrib --config ./config.yaml &> otelcol-output.log & echo "$!" > otel-pid

Step 3: If the configurations are configured correctly, you can see the output logs from OpenTelemtry as follows:

Send and Visualize the metrics obtained by OpenTelemetry in SigNoz

Step 1: Go to SigNoz and head over to the dashboard.

Step 2: If not already created, create a new dashboard. You can create the dashboard and multiple panel under it by following the instructions here.

Step 3: While creating the panel, select metric for Cloud SQL.

All metrics starting with cloudsql_googleapis_com_ have been collected from Cloud SQL.

Here is the sample dashboard for CloudSQL:

Troubleshooting

If you run into any problems while setting up monitoring for your Cloud SQL's metrics with SigNoz, consider these troubleshooting steps:

• Verify Configuration: Double-check your config.yaml file to ensure all settings, including the ingestion key and endpoint, are correct.
• Review Logs: Look at the logs of the OpenTelemetry Collector to identify any error messages or warnings that might provide insights into what’s going wrong.
• Update Dependencies: Ensure all relevant packages and dependencies are up-to-date to avoid compatibility issues.
• Consult Documentation: Review the SigNoz and OpenTelemetry documentation for any additional troubleshooting of the common issues.