Here's a quick summary of what we will be doing in this guide

• Create and configure App Engine service to generate the metrics
• Invoke the App Engine service
• Deploy Telegraf to fetch the metrics from Google Cloud monitoring
• Deploy OpenTelemetry to scrape the metrics from Telegraf
• Send and Visualize the metrics obtained by OpenTelemetry in SigNoz Cloud

Prerequisites

• A Google Cloud account with administrative privileges or App Engine Admin privileges
• A SigNoz Cloud account (used for this demonstration). You'll need the ingestion details. To obtain your Ingestion Key and Ingestion URL, log in to your SigNoz Cloud account and navigate to Settings >> Ingestion Settings
• Access to a project in Google Cloud Platform (GCP)
• Cloud Build API is enabled

Setup

Get started with App Engine Configuration

Create the App Engine service using the following steps:

Step 1: Install the Google Cloud CLI.

Step 2: To initialize the gcloud CLI, run the following command:

gcloud init

Step 3: Run the following gcloud command to enable App Engine and create the associated application resources. Note that the location you select cannot be changed later.

gcloud app create

Step 4: In this example, we will be building the Python 3 application. For this,

1. Download the sample application repository using Git:

git clone https://github.com/GoogleCloudPlatform/python-docs-samples

Alternatively, you can download the sample as a zip file and then extract it.

2. Navigate to the directory that contains a copy of the files from the previous step:

cd python-docs-samples/appengine/standard_python3/hello_world

Note that we have app.yaml file located in this folder.

3. When on this folder which contains app.yaml, you can deploy your service using the command:

gcloud app deploy

4. On running the above command, the service will get deployed on the App Engine, and you will get the logs containing the endpoint which can be used to trigger the deployed service (highlighted in red in the screenshot).

Open the URL in the new browser which will invoke the App Engine service.

Deploy Telegraf to fetch the metrics from Google Cloud Monitoring

We will be using Compute Engine instance to install Telegraf. You can follow the instructions in the Creating Copmute Engine document to create the Compute Engine instance. In this tutorial, we will be using the same instance for installing OTel Collector as well. You may choose a different instance as well.

Step 1: Install telegraf which will collect metrics from Google Cloud Monitoring for App Engine, see the available metrics for App Engine. The installation process for the respective operating system can be found in official documentation.

After successful installation, the Telegraf status should be active and running. You can check the Telegraf status by running the command service telegraf status.

The configuration file for telegraf can be found here:

/etc/telegraf/telegraf.conf

Step 2: Configure the Telegraf input and output plugin by adding configurations to the telegraf.conf file.

# Gather timeseries from Google Cloud Platform v3 monitoring API
[[inputs.stackdriver]]
  ## GCP Project
  project = "omni-new"
  ## Include timeseries that start with the given metric type.
  metric_type_prefix_include = [
    "appengine.googleapis.com",
  ]
  ## Most metrics are updated no more than once per minute; it is recommended
  ## to override the agent level interval with a value of 1m or greater.
  interval = "1m"
# Send OpenTelemetry metrics over gRPC
[[outputs.opentelemetry]]
  ## Override the default (localhost:4317) OpenTelemetry gRPC service
  ## address:port
  service_address = "localhost:4317"

The service_address is using localhost as we will be deploying OTelCollector in the same Compute Engine instance. You can replace it with the appropriate IP address in case you choose to deploy OTel Collector on a different instance.

Deploy OpenTelemetry to scrape the metrics from Telegraf

Step 1: Install and configure OpenTelemetry for scraping the metrics from Telegraf. Follow OpenTelemetry Binary Usage in Virtual Machine guide for detailed instructions.

Step 2: After successful configuration start the OTel Collector using following command:

./otelcol-contrib --config ./config.yaml &> otelcol-output.log & echo "$!" > otel-pid

Step 3: Restart the Telegraf service using the command service telegraf restart.

Step 4: If the configurations are configured correctly with Telegraf and we can see the output logs from OpenTelemetry as follows:

Send and Visualize the metrics obtained by OpenTelemetry in SigNoz

Step 1: Go to the SigNoz Cloud URL and head over to the dashboard.

Step 2: If not already created, create a new dashboard and select Time Series.

Step 3: Select metric for App Engine

All metrics starting with appengine_googleapis_com_ have been collected from App Engine.

Troubleshooting

If you run into any problems while setting up monitoring for your App Engine's metrics with SigNoz, consider these troubleshooting steps:

• Verify Configuration: Double-check your config.yaml file to ensure all settings, including the ingestion key and endpoint, are correct.
• Review Logs: Look at the logs of both App Engine and the OpenTelemetry Collector to identify any error messages or warnings that might provide insights into what’s going wrong.
• Update Dependencies: Ensure all relevant packages and dependencies are up-to-date to avoid compatibility issues.
• Consult Documentation: Review the SigNoz, OpenTelemetry, and Telegraf documentation for any additional troubleshooting of the common issues.