Rust Opentelemetry Instrumentation

This doc contains instructions on how to set up OpenTelemetry(OTel) instrumentation in your Rust application. OpenTelemetry, also known as OTel for short, is an open-source observability framework that can help you generate and collect telemetry data - traces, metrics, and logs from your Rust application.

Once the telemetry data is generated, you can configure an exporter to send the data to SigNoz for monitoring and visualization.

There are three major steps to using OpenTelemetry:

  • Instrumenting your Rust application with OpenTelemetry
  • Configuring the exporter to send data to SigNoz
  • Validating that configuration to ensure that data is being sent as expected.

In this tutorial, we will instrument a Rust application for traces and send it to SigNoz.

Requirements

Rust

Send traces to SigNoz Cloud

Based on your application environment, you can choose the setup below to send traces to SigNoz Cloud.

From VMs, there are two ways to send data to SigNoz Cloud.

Send traces directly to SigNoz cloud

Here we will be sending traces to SigNoz cloud in 4 easy steps, if you want to send traces to self hosted SigNoz , you can refer to this.

✅ Info

If you are using the sample Rust application, then you just need to update .env file and you are good to go!

Step 1 : Instrument your application with OpenTelemetry

To configure our Rust application to send data we need to initialize OpenTelemetry, Otel has already created some crates which you need to add into your Cargo.toml file, just below [dependencies] section.

opentelemetry = { version = "0.18.0", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-otlp = { version = "0.11.0", features = ["trace", "metrics"] }
opentelemetry-semantic-conventions = { version = "0.10.0" }
opentelemetry-proto = { version = "0.1.0"}
tokio = { version = "1", features = ["full"] }
tonic = { version = "0.8.2", features = ["tls-roots"] }

after adding these in Cargo.toml , you need to use these in entry point of your Rust application , which is main.rs file in majority of applications.

use opentelemetry::global::shutdown_tracer_provider;
use opentelemetry::sdk::Resource;
use opentelemetry::trace::TraceError;
use opentelemetry::{
    global, sdk::trace as sdktrace,
    trace::{TraceContextExt, Tracer},
    Context, Key, KeyValue,
};
use opentelemetry_otlp::WithExportConfig;
use tonic::metadata::{MetadataMap, MetadataValue};

Step 2: Initialize the tracer and create env file

Add this function in main.rs file, init_tracer is initializing an OpenTelemetry tracer with the OpenTelemetry OTLP exporter which is sending data to SigNoz Cloud.

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    let signoz_access_token = std::env::var("SIGNOZ_ACCESS_TOKEN").expect("SIGNOZ_ACCESS_TOKEN not set");
    let mut metadata = MetadataMap::new();
    metadata.insert(
        "signoz-access-token",
        MetadataValue::from_str(&signoz_access_token).unwrap(),
    );
    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(
            opentelemetry_otlp::new_exporter()
                .tonic()
                .with_metadata(metadata)
                .with_endpoint(std::env::var("SIGNOZ_ENDPOINT").expect("SIGNOZ_ENDPOINT not set")),
        )
        .with_trace_config(
            sdktrace::config().with_resource(Resource::new(vec![
                KeyValue::new(
                    opentelemetry_semantic_conventions::resource::SERVICE_NAME,
                    std::env::var("APP_NAME").expect("APP_NAME not set"),
                ),
            ])),
        )
        .install_batch(opentelemetry::runtime::Tokio)
}

After adding this function, you need to create a .env file in root of project , the structure should look like this.

project_root/
|-- Cargo.toml
|-- src/
|   |-- main.rs
|-- .env

Paste these in .env file

PORT=3000
APP_NAME=rust-sample
SIGNOZ_ENDPOINT=https://ingest.[region].signoz.cloud:443/v1/traces
SIGNOZ_ACCESS_TOKEN=XXXXXXXXXX
VariableDescription
PORT (Optional)If it is a web app pass port or else you can ignore this variable
APP_NAME *Name you want to give to your rust application
SIGNOZ_ENDPOINT *This is ingestion URL which you must have got in mail after registering on SigNoz cloud
SIGNOZ_ACCESS_TOKEN *This is Ingestion Key which you must have got in mail after registering on SigNoz cloud

Step 3: Add the OpenTelemetry instrumentation for your Rust app

Open your Cargo.toml file and paste these below [dependencies]

dotenv = "0.15.0"

Import these at top, so you can use variables from .env file

use dotenv::dotenv;

After importing , just call these functions inside main() function by pasting this at starting of main() function

dotenv().ok();
let _ = init_tracer();

also change

fn main(){
    //rest of the code
}

to

#[tokio::main]
async fn main() {
    //rest of the code
}

Now comes the most interesting part, Sending data to SigNoz to get sense of your traces. After adding the below block you can send data to SigNoz cloud

  let tracer = global::tracer("global_tracer");
    let _cx = Context::new();
  
    tracer.in_span("operation", |cx| {
        let span = cx.span();
        span.set_attribute(Key::new("KEY").string("value"));

        span.add_event(
            format!("Operations"),
            vec![
                Key::new("SigNoz is").string("Awesome"),
            ],
        );
    });
    shutdown_tracer_provider()

Step 4: Set environment variables and run app

Go to your .env file and update the value of required variables i.e APP_NAME, SIGNOZ_ENDPOINT and SIGNOZ_ACCESS_TOKEN

Now run cargo run in terminal to start the application!

Send traces via OTel Collector binary

OTel Collector binary helps to collect logs, hostmetrics, resource and infra attributes. It is recommended to install Otel Collector binary to collect and send traces to SigNoz cloud. You can correlate signals and have rich contextual data through this way.

📝 Note

You can find instructions to install OTel Collector binary here in your VM. Once you are done setting up your OTel Collector binary, you can follow the below steps for instrumenting your Rust application.

Step 1 : Instrument your application with OpenTelemetry

To configure our Rust application to send traces we need to initialize OpenTelemetry, Otel has already created some crates which you need to add into your Cargo.toml file, just below [dependencies] section.

opentelemetry = { version = "0.18.0", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-otlp = { version = "0.11.0", features = ["trace", "metrics"] }
opentelemetry-semantic-conventions = { version = "0.10.0" }
opentelemetry-proto = { version = "0.1.0"}
tokio = { version = "1", features = ["full"] }
tonic = { version = "0.8.2", features = ["tls-roots"] }

after adding these in Cargo.toml , you need to use these in entry point of your Rust application , which is main.rs file in majority of applications.

use opentelemetry::global::shutdown_tracer_provider;
use opentelemetry::sdk::Resource;
use opentelemetry::trace::TraceError;
use opentelemetry::{
    global, sdk::trace as sdktrace,
    trace::{TraceContextExt, Tracer},
    Context, Key, KeyValue,
};
use opentelemetry_otlp::WithExportConfig;
use tonic::metadata::{MetadataMap, MetadataValue};

Step 2: Initialize the tracer and create env file

Add this function in main.rs file, init_tracer is initializing an OpenTelemetry tracer with the OpenTelemetry OTLP exporter which is sending data to SigNoz Cloud.

This tracer initializes the connection with the OTel collector from the system variables passed while starting the app.

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(opentelemetry_otlp::new_exporter().tonic().with_env())
        .with_trace_config(
            sdktrace::config().with_resource(Resource::default()),
        )
        .install_batch(opentelemetry::runtime::Tokio)
}

Step 3: Add the OpenTelemetry instrumentation for your Rust app

You need call init_tracer function inside main() at starting so that as soon as your rust application starts, tracer will be available globally.

let _ = init_tracer();

also change

fn main(){
    //rest of the code
}

to

#[tokio::main]
async fn main() {
    //rest of the code
}

Now comes the most interesting part, Sending data to SigNoz to get sense of your traces. After adding the below block you can send traces to SigNoz cloud

  let tracer = global::tracer("global_tracer");
    let _cx = Context::new();
  
    tracer.in_span("operation", |cx| {
        let span = cx.span();
        span.set_attribute(Key::new("KEY").string("value"));

        span.add_event(
            format!("Operations"),
            vec![
                Key::new("SigNoz is").string("Awesome"),
            ],
        );
    });
    shutdown_tracer_provider()

Step 4: Set environment variables and run app

VariableDescription
OTEL_EXPORTER_OTLP_ENDPOINTThis is the endpoint of your OTel Collector. If you have hosted it somewhere, provide the URL. Otherwise, the default is http://localhost:4317 if you have followed our guide.
OTEL_RESOURCE_ATTRIBUTES=service.nameSpecify as the value. This will be the name of your Rust application on SigNoz services page, allowing you to uniquely identify the application traces.

Now run

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_RESOURCE_ATTRIBUTES=service.name=sample-rust-app cargo run

in terminal to start the application!

Send Traces to Self-Hosted SigNoz

Step 1 : Instrument your application with OpenTelemetry

To configure our Rust application to send traces we need to initialize OpenTelemetry, Otel has already created some crates which you need to add into your Cargo.toml file, just below [dependencies] section.

opentelemetry = { version = "0.18.0", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-otlp = { version = "0.11.0", features = ["trace", "metrics"] }
opentelemetry-semantic-conventions = { version = "0.10.0" }
opentelemetry-proto = { version = "0.1.0"}
tokio = { version = "1", features = ["full"] }
tonic = { version = "0.8.2", features = ["tls-roots"] }

after adding these in Cargo.toml , you need to use these in entry point of your Rust application , which is main.rs file in majority of applications.

use opentelemetry::global::shutdown_tracer_provider;
use opentelemetry::sdk::Resource;
use opentelemetry::trace::TraceError;
use opentelemetry::{
    global, sdk::trace as sdktrace,
    trace::{TraceContextExt, Tracer},
    Context, Key, KeyValue,
};
use opentelemetry_otlp::WithExportConfig;
use tonic::metadata::{MetadataMap, MetadataValue};

Step 2: Initialize the tracer and create env file

Add this function in main.rs file, init_tracer is initializing an OpenTelemetry tracer with the OpenTelemetry OTLP exporter which is sending data to your Self-Hosted SigNoz.

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(opentelemetry_otlp::new_exporter().tonic().with_env())
        .with_trace_config(
            sdktrace::config().with_resource(Resource::default()),
        )
        .install_batch(opentelemetry::runtime::Tokio)
}

Step 3: Add the OpenTelemetry instrumentation for your Rust app

You need call init_tracer function inside main() at starting so that as soon as your rust application starts, tracer will be available globally.

let _ = init_tracer();

also change

fn main(){
    //rest of the code
}

to

#[tokio::main]
async fn main() {
    //rest of the code
}

Now comes the most interesting part, Sending data to SigNoz to get sense of your traces. After adding the below block you can send traces to Self-Hosted SigNoz

  let tracer = global::tracer("global_tracer");
    let _cx = Context::new();
  
    tracer.in_span("operation", |cx| {
        let span = cx.span();
        span.set_attribute(Key::new("KEY").string("value"));

        span.add_event(
            format!("Operations"),
            vec![
                Key::new("SigNoz is").string("Awesome"),
            ],
        );
    });
    shutdown_tracer_provider()

Step 4: Set environment variables and run app

VariableDescription
OTEL_EXPORTER_OTLP_ENDPOINTThis is the url where you have hosted SigNoz
OTEL_RESOURCE_ATTRIBUTES=service.nameSpecify as the value. This will be the name of your Rust application on SigNoz services page, allowing you to uniquely identify the application traces.

Now run

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_RESOURCE_ATTRIBUTES=service.name=sample-rust-app cargo run

in terminal to start the application!

Sample Rust Application

We have included a sample Rust application at Sample Rust App Github Repo,

Tutorial

Here's a tutorial with step by step guide on how to install SigNoz and start monitoring a sample Rust app.

Frequently Asked Questions

  1. How to find what to use in IP of SigNoz if I have installed SigNoz in Kubernetes cluster?

    Based on where you have installed your application and where you have installed SigNoz, you need to find the right value for this. Please use this grid to find the value you should use for IP of SigNoz

  2. I am sending data from my application to SigNoz, but I don't see any events or graphs in the SigNoz dashboard. What should I do?

    This could be because of one of the following reasons:

    1. Your application is generating telemetry data, but not able to connect with SigNoz installation

      Please use this troubleshooting guide to find if your application is able to access SigNoz installation and send data to it.

    2. Your application is not actually generating telemetry data

      Please check if the application is generating telemetry data first. You can use Console Exporter to just print your telemetry data in console first. Join our Slack Community if you need help on how to export your telemetry data in console

    3. Your SigNoz installation is not running or behind a firewall

      Please double check if the pods in SigNoz installation are running fine. docker ps or kubectl get pods -n platform are your friends for this.

What Cloud Endpoint Should I Use?

The primary method for sending data to SigNoz Cloud is through OTLP exporters. You can either send the data directly from your application using the exporters available in SDKs/language agents or send the data to a collector agent, which batches/enriches telemetry and sends it to the Cloud.

My Collector Sends Data to SigNoz Cloud

Using gRPC Exporter

The endpoint should be ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu. Note that the exporter endpoint doesn't require a scheme for the gRPC exporter in the collector.

# Sample config with `us` region
exporters:
    otlp:
        endpoint: "ingest.us.signoz.cloud:443"
        tls:
            insecure: false
        headers:
            "signoz-access-token": "<SIGNOZ_INGESTION_KEY>"

Using HTTP Exporter

The endpoint should be https://ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu. Note that the endpoint includes the scheme https for the HTTP exporter in the collector.

# Sample config with `us` region
exporters:
    otlphttp:
        endpoint: "https://ingest.us.signoz.cloud:443"
        tls:
            insecure: false
        headers:
            "signoz-access-token": "<SIGNOZ_INGESTION_KEY>"

My Application Sends Data to SigNoz Cloud

The endpoint should be configured either with environment variables or in the SDK setup code.

Using Environment Variables

Using gRPC Exporter

Examples with us region

  • OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud:443 OTEL_EXPORTER_OTLP_HEADERS=signoz-access-token=<SIGNOZ_INGESTION_KEY>
Using HTTP Exporter
  • OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud:443 OTEL_EXPORTER_OTLP_HEADERS=signoz-access-token=<SIGNOZ_INGESTION_KEY>

Configuring Endpoint in Code

Please refer to the agent documentation.

Sending Data from a Third-Party Service

The endpoint configuration here depends on the export protocol supported by the third-party service. They may support either gRPC, HTTP, or both. Generally, you will need to adjust the host and port. The host address should be ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu, and port 443 should be used.