This document contains instructions on how to set up OpenTelemetry instrumentation in your Spring Boot applications and view your application traces in SigNoz.
Requirements
Java 8 or higher
Send Traces to SigNoz Cloud
OpenTelemetry provides a handy Java JAR agent that can be attached to any Java 8+ application and dynamically injects bytecode to capture telemetry from a number of popular libraries and frameworks.
Based on your application environment, you can choose the setup below to send traces to SigNoz Cloud.
From VMs, there are two ways to send data to SigNoz Cloud.
Send traces directly to SigNoz Cloud
OpenTelemetry Java agent can send traces directly to SigNoz Cloud.
Step 1. Download otel java binary agent
wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Step 2. Run your application
OTEL_RESOURCE_ATTRIBUTES=service.name=<service_name> \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>" \
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.<region>.signoz.cloud:443 \
java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <my-app>.jar
- Set the
<region>
to match your SigNoz Cloud region - Replace
<your-ingestion-key>
with your SigNoz ingestion key <service_name>
is name of your service
In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.
Send traces via OTel Collector binary
Step 1. Install OTel Collector binary
OTel Collector binary helps to collect logs, hostmetrics, resource and infra attributes.
You can find instructions to install OTel Collector binary here in your VM.
Step 2. Download OTel java binary agent
wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Step 3. Run your application
java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <myapp>.jar
<myapp>
is the name of your application jar file- In case you download
opentelemetry-javaagent.jar
file in different directory than that of the project, replace$PWD
with the path of the otel jar file.
In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.
You can auto-instrument sending traces from Java Springboot application using one of the following methods:
- Using Kubernetes OTel Operator
- Using OTel Collector Agent
For Java Springboot application deployed on Kubernetes, you can auto-instrument the traces using Kubernetes OpenTelemetry Operator.
An OpenTelemetry Operator is a Kubernetes Operator that manages OpenTelemetry Collectors and auto-instrumentation of workloads. It basically simplifies the deployment and management of OpenTelemetry in a Kubernetes environment.
The OpenTelemetry Operator provides two Custom Resource Definitions (CRDs):
OpenTelemetryCollector
Instrumentation
The OpenTelemetryCollector
CRD allows you to deploy and manage OpenTelemetry Collectors in your Kubernetes cluster.
The Instrumentation
CRD allows you to configure and inject OpenTelemetry auto-instrumentation libraries into your workloads.
Here are the steps you need to follow to auto-instrument Springboot application using OTel Operator:
Step 1: Install cert-manager
Run the following commands to apply cert manager.
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml
kubectl wait --for=condition=Available deployments/cert-manager -n cert-manager
Step 2: Install OpenTelemetry Operator
To install the operator in the existing K8s cluster, run the following command:
kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/download/v0.116.0/opentelemetry-operator.yaml
Installing the OpenTelemetry Operator sets up the necessary components and configurations to enable the observability and monitoring of applications running in the cluster.
Step 3: Setup the OpenTelemetry Collector instance
Once the opentelemetry-operator
has been deployed, you can proceed with the creation of the OpenTelemetry Collector (otelcol
) instance. The OpenTelemetry Collector collects, processes, and exports telemetry data.
There are different deployment modes for the OpenTelemetryCollector
, and you can specify them in the spec.mode section of the custom resource. The available deployment modes are:
- Daemonset
- Sidecar
- StatefulSet
- Deployment (default mode)
The default method - the Deployment
mode, will be used here.
To create a simple instance of the OpenTelemetry Collector, create a file otel-collector.yaml
with the following contents:
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
name: otel-collector
spec:
mode: deployment
config: |
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch: {}
resource/env:
attributes:
- key: deployment.environment
value: prod # can be dev, prod, staging etc. based on your environment
action: upsert
exporters:
debug: {}
otlp:
endpoint: "ingest.<region>.signoz.cloud:443" # replace <region> with your region of SigNoz Cloud
tls:
insecure: false
headers:
"signoz-ingestion-key": "<your-ingestion-key>" # Obtain from https://{your-signoz-tenant-url}/settings/ingestion-settings
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch, resource/env]
exporters: [otlp]
- Set the
<region>
to match your SigNoz Cloud region - Replace
<your-ingestion-key>
with your SigNoz ingestion key
Apply the above yaml file using the following command:
kubectl apply -f otel-collector.yaml
Step 4: Setup the Instrumentation instance
Once the OpenTelemetry Collector instance has been deployed, the next step will be to create an instrumentation instance, which will be responsible for sending OTLP data to the OTel Collector.
Create a file instrumentation.yaml
with the following contents:
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
name: traces-instrumentation
spec:
exporter:
endpoint: https://ingest.{region}.signoz.cloud:443 # replace {region} with your region
env:
- name: OTEL_EXPORTER_OTLP_HEADERS
value: signoz-ingestion-key="{signoz-token}" # Obtain from https://{your-signoz-url}/settings/ingestion-settings
- name: OTEL_EXPORTER_OTLP_INSECURE
value: "false"
propagators:
- tracecontext
- baggage
- b3
sampler:
type: parentbased_traceidratio
argument: "1"
java:
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:latest
Apply the above instrumentation using the following command:
kubectl apply -f instrumentation.yaml
Step 5: Auto-instrument your Springboot app with OpenTelemetry
Create deployment.yaml
file for your Springboot application as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-petclinic
spec:
selector:
matchLabels:
app: spring-petclinic
replicas: 1
template:
metadata:
labels:
app: spring-petclinic
annotations:
instrumentation.opentelemetry.io/inject-java: "true"
resource.opentelemetry.io/service.name: "spring-petclinic"
spec:
containers:
- name: app
image: ghcr.io/pavolloffay/spring-petclinic:latest
ports:
- containerPort: 8080
Note that we have created the above deployment file for spring-petclinic Java Springboot application.
It is important to add the following annotation under spec > template > metadata > annotations
:
instrumentation.opentelemetry.io/inject-java: "true"`
This helps in auto-instrumenting the traces from the Java application.
Apply the deployment using the following command:
kubectl apply -f deployment.yaml
With this, the auto-instrumentation of traces for Java Springboot application is ready.
Step 6: Running the Springboot application
In order to run the application on port 8080, run the following commands:
$ export POD_NAME=$(kubectl get pod -l app=<service-name> -o jsonpath="{.items[0].metadata.name}") # service name is `spring-petclinic` in this case.
$ kubectl port-forward ${POD_NAME} 8080:8080
You can now access the application on port 8080.
You can validate if your application is sending traces to SigNoz cloud by following the instructions here.
For Java Springboot application deployed on Kubernetes, you need to install OTel Collector agent in your k8s infra to collect and send traces to SigNoz Cloud. You can find the instructions to install OTel Collector agent here.
Once you have set up OTel Collector agent, you can proceed with OpenTelemetry java instrumentation by following the below steps:
Download otel java binary
wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Run your application
java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <myapp>.jar
<myapp>
is the name of your application jar file- In case you download
opentelemetry-javaagent.jar
file in different directory than that of the project, replace$PWD
with the path of the otel jar file.
Make sure to dockerise your application along with OpenTelemetry instrumentation.
You can validate if your application is sending traces to SigNoz cloud by following the instructions here.
In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.
From Windows, there are two ways to send data to SigNoz Cloud.
- Send traces directly to SigNoz Cloud
- Send traces via OTel Collector binary (recommended) (recommended)
Send traces directly to SigNoz Cloud
OpenTelemetry Java agent can send traces directly to SigNoz Cloud.
Step 1. Download otel java binary agent
wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Step 2. Run your application
setx OTEL_RESOURCE_ATTRIBUTES=service.name=<service_name>
setx OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
setx OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.<region>.signoz.cloud:443
java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <my-app>.jar
- Set the
<region>
to match your SigNoz Cloud region - Replace
<your-ingestion-key>
with your SigNoz ingestion key <service_name>
is name of your service
In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.
Send traces via OTel Collector binary
Step 1. Install OTel Collector binary
OTel Collector binary helps to collect logs, hostmetrics, resource and infra attributes.
You can find instructions to install OTel Collector binary here in your VM.
Step 2. Download OTel java binary agent
wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Step 3. Run your application
java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <myapp>.jar
<myapp>
is the name of your application jar file- In case you download
opentelemetry-javaagent.jar
file in different directory than that of the project, replace$PWD
with the path of the otel jar file.
In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.
Send Traces to Self-Hosted SigNoz
You can use OpenTelemetry Java to send your traces directly to SigNoz. OpenTelemetry provides a handy Java JAR agent that can be attached to any Java 8+ application and dynamically injects bytecode to capture telemetry from a number of popular libraries and frameworks.
Steps to auto-instrument Spring Boot applications for traces
OpenTelemetry Java auto-instrumentation supports collecting telemetry data from a huge number of libraries and frameworks. You can check out the full list here.
Download the latest OpenTelemetry Java JAR agent
Download the latest Java JAR agent. You can also use the terminal to get the file using the following command:wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
Enable the instrumentation agent and run your application
If you run your Spring Boot application as a JAR file, run your application using the following command:OTEL_EXPORTER_OTLP_ENDPOINT="http://<IP of SigNoz Backend>:4317" OTEL_RESOURCE_ATTRIBUTES=service.name=<service_name> java -javaagent:/path/to/opentelemetry-javaagent.jar -jar <myapp>.jar
<service_name>
is name of your servicepath
should be updated to the path of the downloaded Java JAR agent.
In the above command, we are configuring the exporter to send data to SigNoz backend. By default, OpenTelemetry Java agent uses OTLP exporter configured to send data.
Two things to note about the command:
OTEL_EXPORTER_OTLP_ENDPOINT
- This is the endpoint of the machine where SigNoz is installed.path/to
- Update it to the path of your downloaded Java JAR agent.If you have installed SigNoz on your
localhost
and your Java JAR agent is saved at/Users/john/Downloads/
, then the final command looks like:OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" OTEL_RESOURCE_ATTRIBUTES=service.name=javaApp java -javaagent:/Users/john/Downloads/opentelemetry-javaagent.jar -jar target/*.jar
Here’s a handy grid to figure out which address to use to send data to SigNoz.
You can also specify environment variables in the following way:
java -javaagent:/path/opentelemetry-javaagent.jar \ -Dotel.exporter.otlp.endpoint=http://<IP of SigNoz Backend>:4317 \ -Dotel.resource.attributes=service.name=<service_name> \ -jar <myapp>.jar
💡 Remember to allow incoming requests to port 4317 of the machine where SigNoz backend is hosted.
If you want to try this instrumentation with a sample Spring Boot application, visit this GitHub repo.
In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.
Note: By default, Java auto-instrumentation does not create spans for user-defined functions. In such scenarios, we can use annotations to quickly create spans for user-defined functions with minimal code change. You can instrument your Spring Boot application following this guide on annotations.
Validating instrumentation by checking for traces
With your application running, you can verify that you’ve instrumented your application with OpenTelemetry correctly by confirming that tracing data is being reported to SigNoz.
To do this, you need to ensure that your application generates some data. Applications will not produce traces unless they are being interacted with, and OpenTelemetry will often buffer data before sending. So you need to interact with your application and wait for some time to see your tracing data in SigNoz.
Validate your traces in SigNoz:
- Trigger an action in your app that generates a web request. Hit the endpoint a number of times to generate some data. Then, wait for some time.
- In SigNoz, open the
Services
tab. Hit theRefresh
button on the top right corner, and your application should appear in the list ofApplications
. - Go to the
Traces
tab, and apply relevant filters to see your application’s traces.
Configuring the agent
The agent is highly configurable. You can check out all the configuration options available here.
Disabled instrumentations
Some instrumentations can produce too many spans and make traces very noisy. For this reason, the following instrumentations are disabled by default:
jdbc-datasource
which creates spans whenever thejava.sql.DataSource#getConnection
method is called.dropwizard-metrics
, which might create very low-quality metrics data because of the lack of label/attribute support in the Dropwizard metrics API.
To enable them, add the otel.instrumentation.<name>.enabled
system property: -Dotel.instrumentation.jdbc-datasource.enabled=true
Troubleshooting your installation
If spans are not being reported to SigNoz, try running in debug mode by setting OTEL_LOG_LEVEL=debug
:
The debug log level will print out the configuration information. It will also emit every span to the console, which should look something like:
Span {
attributes: {},
links: [],
events: [],
status: { code: 0 },
endTime: [ 1597810686, 885498645 ],
_ended: true,
_duration: [ 0, 43333 ],
name: 'bar',
spanContext: {
traceId: 'eca3cc297720bd705e734f4941bca45a',
spanId: '891016e5f8c134ad',
traceFlags: 1,
traceState: undefined
},
parentSpanId: 'cff3a2c6bfd4bbef',
kind: 0,
startTime: [ 1597810686, 885455312 ],
resource: Resource { labels: [Object] },
instrumentationLibrary: { name: 'example', version: '*' },
_logger: ConsoleLogger {
debug: [Function],
info: [Function],
warn: [Function],
error: [Function]
},
_traceParams: {
numberOfAttributesPerSpan: 32,
numberOfLinksPerSpan: 32,
numberOfEventsPerSpan: 128
},
_spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] }
},
Sample Application
- We have included a sample application at: