Implementing OpenTelemetry in a Gin application

OpenTelemetry can be used to trace Gin applications for performance issues and bugs. OpenTelemetry is an open-source project under the Cloud Native Computing Foundation (CNCF) that aims to standardize the generation and collection of telemetry data like logs, metrics, and traces.

Gin is an HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster.

If you need smashing performance, get yourself some Gin!

Gin framework has a very small footprint and great speed because it's built on HttpRouter, a lightweight, high-performance HTTP request router. HttpRouter and Gin use a radix tree to parse long and complicated route requests quickly.

In this tutorial, we will demonstrate how to use the OpenTelemetry Gin middleware to generate end-to-end tracing. We will also instrument GORM database client using OpenTelemetry libraries.

Before we demonstrate how to implement the OpenTelemetry libraries, let’s have a brief overview of OpenTelemetry.

What is OpenTelemetry?

OpenTelemetry is an open-source vendor-agnostic set of tools, APIs, and SDKs used to instrument applications to create and manage telemetry data (logs, metrics, and traces). It aims to make telemetry data, a built-in feature of cloud-native software applications.

The telemetry data is then sent to an observability tool for storage and visualization.

OpenTelemetry is the bedrock for setting up an observability framework. It also provides you the freedom to choose a backend analysis tool of your choice.

OpenTelemetry and SigNoz

In this tutorial, we will use SigNoz as our backend analysis tool. SigNoz is a full-stack open-source APM tool that can be used for storing and visualizing the telemetry data collected with OpenTelemetry. It is built natively on OpenTelemetry and works on the OTLP data formats.

SigNoz provides query and visualization capabilities for the end-user and comes with out-of-box charts for application metrics and traces.

Now let’s get down to how to implement OpenTelemetry Gin libraries and then visualize the collected data in SigNoz.

Running Gin application with OpenTelemetry

Step 1: Install SigNoz
First, you need to install SigNoz so that OpenTelemetry can send the data to it.

SigNoz can be installed on macOS or Linux computers in just three steps by using a simple installation script.

The install script automatically installs Docker Engine on Linux. However, on macOS, you must manually install Docker Engine before running the install script.

git clone -b main https://github.com/SigNoz/signoz.git
cd signoz/deploy/
./install.sh

You can visit our documentation for instructions on how to install SigNoz using Docker Swarm and Helm Charts.

When you are done installing SigNoz, you can access the UI at http://localhost:3301

Step 2: Get sample Gin app in Golang
Sample Go application

It contains the sample boilerplate code that we will instrument.

If you want to follow the tutorial, then you should follow the without-instrumentation branch.

Step 3: Declare few variables for configuring OpenTelemetry
Declare the following global variables in main.go which we will use to configure OpenTelemetry

var (
	serviceName  = os.Getenv("SERVICE_NAME")
	collectorURL = os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")
	insecure     = os.Getenv("INSECURE_MODE")
)

Step 4: Instrument your Gin application with OpenTelemetry
To configure your application to send data we will need a function to initialize OpenTelemetry. Add the following snippet of code in your main.go file.

import (
  .....

	"github.com/gin-gonic/gin"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"

	"go.opentelemetry.io/otel/sdk/resource"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

func initTracer() func(context.Context) error {

	secureOption := otlptracegrpc.WithTLSCredentials(credentials.NewClientTLSFromCert(nil, ""))
	if len(insecure) > 0 {
		secureOption = otlptracegrpc.WithInsecure()
	}

	exporter, err := otlptrace.New(
		context.Background(),
		otlptracegrpc.NewClient(
			secureOption,
			otlptracegrpc.WithEndpoint(collectorURL),
		),
	)

	if err != nil {
		log.Fatal(err)
	}
	resources, err := resource.New(
		context.Background(),
		resource.WithAttributes(
			attribute.String("service.name", serviceName),
			attribute.String("library.language", "go"),
		),
	)
	if err != nil {
		log.Printf("Could not set resources: ", err)
	}

	otel.SetTracerProvider(
		sdktrace.NewTracerProvider(
			sdktrace.WithSampler(sdktrace.AlwaysSample()),
			sdktrace.WithBatcher(exporter),
			sdktrace.WithResource(resources),
		),
	)
	return exporter.Shutdown
}

Step 5: Initialize the tracer in main.go
Modify the main function to initialise the tracer in main.go. Initiate the tracer at the very beginning of our main function.

func main() {
	cleanup := initTracer()
	defer cleanup(context.Background())

	......
}

Step 6: Add the OpenTelemetry Gin middleware
Configure Gin to use the middleware by adding the following lines in main.go.

import (
	....
  "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

func main() {
	......
	r := gin.Default()
	r.Use(otelgin.Middleware(serviceName))
	......
}

Step 7: Set environment variables and run your Gin application
Now that you have instrumented your Gin application with OpenTelemetry, you need to set some environment variables to send data to the SigNoz backend:

SERVICE_NAME=goApp INSECURE_MODE=true OTEL_EXPORTER_OTLP_ENDPOINT=<IP of SigNoz backend>:4317 go run main.go

IP os SigNoz backend should be without http/https scheme.

SERVICE_NAME: goGinApp (you can name it whatever you want)

OTEL_EXPORTER_OTLP_ENDPOINT: localhost:4317

Since we have installed SigNoz on our local machine, we use the above IP. If you install SigNoz on a different machine, you can update it with the relevant IP.

Hence, the final run command looks like this:

SERVICE_NAME=goGinApp INSECURE_MODE=true OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317 go run main.go

Step 8: Generate some data
In order to monitor your Gin application with SigNoz, you first need to generate some data.

• Create a book

  curl --location --request POST 'localhost:8091/books' \
  --header 'Content-Type: application/json' \
  --data-raw '{
      "title": "book 1",
      "author": "John Doe"
  }'
  

  

• Check the list of books by running the following Curl

  curl --location --request GET 'localhost:8091/books'
  

Step 9: Visualize the collected data in SigNoz
Access the signoz UI on http://localhost:3301/application

Go to Services → goGinApp → you will be able to see the dashboard

You can monitor application metrics like application latency, requests per second, error percentage, etc. with the Services tab of SigNoz.

OpenTelemetry captures tracing data from your Gin application as well. Tracing data can help you visualize how user requests perform across services in a multi-service application.

In the Traces tab of SigNoz, you can analyze the tracing data using filters based on tags, status codes, service names, operations, etc.

You can also visualize your tracing data with the help of flamegraphs and Gantt charts.

SigNoz also provides log management as a feature. With SigNoz you can can have metrics, traces, and logs under a single pane of glass.

You can also monitor Gorm with OpenTelemetry libraries.

Monitoring GORM database client with OpenTelemetry

We have instrumented our Gin which will help us trace HTTP requests but we might want to trace the DB calls as well.

OpenTelemetry provides an otelgorm plugin to monitor GORM database client.

You can follow the below steps to instrument your GORM database client with OpenTelemetry.

Step 1: Initialise GORM to use OpenTelemetry by updating the models/setup.go

func ConnectDatabase() {
	.....
	DB = database
	if err := DB.Use(otelgorm.NewPlugin()); err != nil {
		panic(err)
	}
}  

Step 2: Update all the database calls to use the request context by modifying controllers/books.go

models.DB will be changed to models.DB.WithContext(c.Request.Context())

func FindBooks(c *gin.Context) {
	.....
	models.DB.WithContext(c.Request.Context()).Find(&books)
	.....
}

The above change should be done for all the DB calls.

Now run your application and execute the curl requests again. You will be able to see DB traces as well.

Best Practices and Common Pitfalls

To get the most out of OpenTelemetry in your Gin application:

• Use sampling: For high-traffic applications, implement trace sampling to reduce data volume and costs.
• Secure sensitive data: Be cautious about what information you include in spans and logs to protect user privacy.
• Monitor performance impact: While OpenTelemetry is designed to be lightweight, monitor its impact on your application's performance.
• Implement error handling: Ensure your instrumentation code handles errors gracefully to avoid impacting your application's reliability.

Common pitfalls to avoid:

• Over-instrumenting: Start with basic instrumentation and add more as needed.
• Ignoring context propagation: Ensure you're properly passing context through your application to maintain trace continuity.
• Neglecting to update dependencies: Keep your OpenTelemetry libraries up-to-date to benefit from performance improvements and bug fixes.

Conclusion

Using OpenTelemetry libraries, you can instrument your Gin applications for end-to-end tracing. You can then use an open-source APM tool like SigNoz to ensure the smooth performance of your Gin applications.

OpenTelemetry is the future for setting up observability for cloud-native apps. It is backed by a huge community and covers a wide variety of technology and frameworks. Using OpenTelemetry, engineering teams can instrument polyglot and distributed applications with peace of mind.

SigNoz is an open-source observability tool that comes with a SaaS-like experience. You can try out SigNoz by visiting its GitHub repo.

If you want to get started quickly, try SigNoz cloud.

FAQs

How does OpenTelemetry impact Gin application performance?

OpenTelemetry is designed to have minimal impact on application performance. However, the exact impact depends on factors like the amount of data collected and the sampling rate. Monitor your application's performance before and after implementing OpenTelemetry to assess the impact.

Can I use OpenTelemetry with existing monitoring tools in my Gin app?

Yes, OpenTelemetry is designed to be vendor-neutral. Many existing monitoring tools support OpenTelemetry data, allowing you to integrate it with your current observability stack.

What are the best practices for securing sensitive data in OpenTelemetry traces?

To secure sensitive data:

1. Use sampling to reduce the amount of data collected.
2. Implement data scrubbing to remove sensitive information from spans and logs.
3. Use attribute limits to prevent accidentally logging large amounts of potentially sensitive data.
4. Ensure your chosen backend implements proper access controls and encryption.

How can I correlate traces across multiple Gin-based microservices?

OpenTelemetry supports distributed tracing out of the box. Ensure you're propagating the trace context between services, either manually or using OpenTelemetry's automatic context propagation features. This allows you to view a single trace spanning multiple services in your chosen backend.

What is OpenTelemetry and how does it work with Gin applications?

OpenTelemetry is an open-source observability framework that provides a set of APIs, libraries, and tools to instrument, generate, collect, and export telemetry data (metrics, logs, and traces). When used with Gin applications, OpenTelemetry can help you trace HTTP requests, monitor performance, and identify bottlenecks in your Go web applications.

How do I instrument my Gin application with OpenTelemetry?

To instrument your Gin application with OpenTelemetry:

1. Install the necessary OpenTelemetry packages
2. Initialize a tracer in your main function
3. Add the OpenTelemetry Gin middleware to your application
4. Set environment variables for the OpenTelemetry exporter
5. Run your application and generate some data

Can I monitor database calls with OpenTelemetry in a Gin application?

Yes, you can monitor database calls using OpenTelemetry in a Gin application. For GORM, you can use the otelgorm plugin to trace database operations. This allows you to see detailed information about your database queries within the same tracing context as your HTTP requests.

What backend tool can I use to visualize OpenTelemetry data from my Gin application?

While OpenTelemetry is vendor-agnostic, this article recommends using SigNoz as the backend analysis tool. SigNoz is an open-source APM tool that natively supports OpenTelemetry data formats and provides visualization for metrics, traces, and logs.

How does OpenTelemetry help in monitoring microservices-based applications?

OpenTelemetry is particularly useful for microservices-based applications as it provides distributed tracing capabilities. This allows you to track requests as they move through different services, helping you understand the flow of requests, identify performance bottlenecks, and debug issues across your entire system.

What are the benefits of using OpenTelemetry with Gin compared to other monitoring solutions?

Using OpenTelemetry with Gin offers several advantages:

1. Open-source and vendor-agnostic, giving you flexibility in choosing backend tools
2. Standardized approach to instrumentation across different languages and frameworks
3. Comprehensive observability with metrics, logs, and traces in a single framework
4. Active community support and ongoing development
5. Easy integration with various backend analysis tools

How can I customize the data collected by OpenTelemetry in my Gin application?

You can customize the data collected by OpenTelemetry in your Gin application by:

1. Adding custom attributes to spans
2. Creating custom spans for specific operations
3. Configuring sampling rates
4. Implementing custom exporters
5. Using context propagation to pass additional information between services

Further Reading

Monitor a Golang application using OpenTelemetry and SigNoz

Why is Distributed Tracing needed in microservices-based applications?