Rust Opentelemetry Instrumentation

This doc contains instructions on how to set up OpenTelemetry(OTel) instrumentation in your Rust application. OpenTelemetry, also known as OTel for short, is an open-source observability framework that can help you generate and collect telemetry data - traces, metrics, and logs from your Rust application.

Once the telemetry data is generated, you can configure an exporter to send the data to SigNoz for monitoring and visualization.

There are three major steps to using OpenTelemetry:

• Instrumenting your Rust application with OpenTelemetry
• Configuring the exporter to send data to SigNoz
• Validating that configuration to ensure that data is being sent as expected.

In this tutorial, we will instrument a Rust application for traces and send it to SigNoz.

Requirements

Rust

Send traces to SigNoz Cloud

Based on your application environment, you can choose the setup below to send traces to SigNoz Cloud.

opentelemetry = { version = "0.18.0", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-otlp = { version = "0.11.0", features = ["trace", "metrics"] }
opentelemetry-semantic-conventions = { version = "0.10.0" }
opentelemetry-proto = { version = "0.1.0"}
tokio = { version = "1", features = ["full"] }
tonic = { version = "0.8.2", features = ["tls-roots"] }

use opentelemetry::global::shutdown_tracer_provider;
use opentelemetry::sdk::Resource;
use opentelemetry::trace::TraceError;
use opentelemetry::{
    global, sdk::trace as sdktrace,
    trace::{TraceContextExt, Tracer},
    Context, Key, KeyValue,
};
use opentelemetry_otlp::WithExportConfig;
use tonic::metadata::{MetadataMap, MetadataValue};

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    let signoz_access_token = std::env::var("SIGNOZ_ACCESS_TOKEN").expect("SIGNOZ_ACCESS_TOKEN not set");
    let mut metadata = MetadataMap::new();
    metadata.insert(
        "signoz-access-token",
        MetadataValue::from_str(&signoz_access_token).unwrap(),
    );
    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(
            opentelemetry_otlp::new_exporter()
                .tonic()
                .with_metadata(metadata)
                .with_endpoint(std::env::var("SIGNOZ_ENDPOINT").expect("SIGNOZ_ENDPOINT not set")),
        )
        .with_trace_config(
            sdktrace::config().with_resource(Resource::new(vec![
                KeyValue::new(
                    opentelemetry_semantic_conventions::resource::SERVICE_NAME,
                    std::env::var("APP_NAME").expect("APP_NAME not set"),
                ),
            ])),
        )
        .install_batch(opentelemetry::runtime::Tokio)
}

project_root/
|-- Cargo.toml
|-- src/
|   |-- main.rs
|-- .env

PORT=3000
APP_NAME=rust-sample
SIGNOZ_ENDPOINT=https://ingest.[region].signoz.cloud:443/v1/traces
SIGNOZ_ACCESS_TOKEN=XXXXXXXXXX

dotenv = "0.15.0"

use dotenv::dotenv;

dotenv().ok();
let _ = init_tracer();

fn main(){
    //rest of the code
}

#[tokio::main]
async fn main() {
    //rest of the code
}

  let tracer = global::tracer("global_tracer");
    let _cx = Context::new();
  
    tracer.in_span("operation", |cx| {
        let span = cx.span();
        span.set_attribute(Key::new("KEY").string("value"));

        span.add_event(
            format!("Operations"),
            vec![
                Key::new("SigNoz is").string("Awesome"),
            ],
        );
    });
    shutdown_tracer_provider()

opentelemetry = { version = "0.18.0", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-otlp = { version = "0.11.0", features = ["trace", "metrics"] }
opentelemetry-semantic-conventions = { version = "0.10.0" }
opentelemetry-proto = { version = "0.1.0"}
tokio = { version = "1", features = ["full"] }
tonic = { version = "0.8.2", features = ["tls-roots"] }

use opentelemetry::global::shutdown_tracer_provider;
use opentelemetry::sdk::Resource;
use opentelemetry::trace::TraceError;
use opentelemetry::{
    global, sdk::trace as sdktrace,
    trace::{TraceContextExt, Tracer},
    Context, Key, KeyValue,
};
use opentelemetry_otlp::WithExportConfig;
use tonic::metadata::{MetadataMap, MetadataValue};

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(opentelemetry_otlp::new_exporter().tonic().with_env())
        .with_trace_config(
            sdktrace::config().with_resource(Resource::default()),
        )
        .install_batch(opentelemetry::runtime::Tokio)
}

let _ = init_tracer();

fn main(){
    //rest of the code
}

#[tokio::main]
async fn main() {
    //rest of the code
}

  let tracer = global::tracer("global_tracer");
    let _cx = Context::new();
  
    tracer.in_span("operation", |cx| {
        let span = cx.span();
        span.set_attribute(Key::new("KEY").string("value"));

        span.add_event(
            format!("Operations"),
            vec![
                Key::new("SigNoz is").string("Awesome"),
            ],
        );
    });
    shutdown_tracer_provider()

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_RESOURCE_ATTRIBUTES=service.name=sample-rust-app cargo run

Send Traces to Self-Hosted SigNoz

Step 1 : Instrument your application with OpenTelemetry

To configure our Rust application to send traces we need to initialize OpenTelemetry, Otel has already created some crates which you need to add into your Cargo.toml file, just below [dependencies] section.

opentelemetry = { version = "0.18.0", features = ["rt-tokio", "metrics", "trace"] }
opentelemetry-otlp = { version = "0.11.0", features = ["trace", "metrics"] }
opentelemetry-semantic-conventions = { version = "0.10.0" }
opentelemetry-proto = { version = "0.1.0"}
tokio = { version = "1", features = ["full"] }
tonic = { version = "0.8.2", features = ["tls-roots"] }

after adding these in Cargo.toml , you need to use these in entry point of your Rust application , which is main.rs file in majority of applications.

use opentelemetry::global::shutdown_tracer_provider;
use opentelemetry::sdk::Resource;
use opentelemetry::trace::TraceError;
use opentelemetry::{
    global, sdk::trace as sdktrace,
    trace::{TraceContextExt, Tracer},
    Context, Key, KeyValue,
};
use opentelemetry_otlp::WithExportConfig;
use tonic::metadata::{MetadataMap, MetadataValue};

Step 2: Initialize the tracer and create env file

Add this function in main.rs file, init_tracer is initializing an OpenTelemetry tracer with the OpenTelemetry OTLP exporter which is sending data to your Self-Hosted SigNoz.

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(opentelemetry_otlp::new_exporter().tonic().with_env())
        .with_trace_config(
            sdktrace::config().with_resource(Resource::default()),
        )
        .install_batch(opentelemetry::runtime::Tokio)
}

Step 3: Add the OpenTelemetry instrumentation for your Rust app

You need call init_tracer function inside main() at starting so that as soon as your rust application starts, tracer will be available globally.

let _ = init_tracer();

also change

fn main(){
    //rest of the code
}

to

#[tokio::main]
async fn main() {
    //rest of the code
}

Now comes the most interesting part, Sending data to SigNoz to get sense of your traces. After adding the below block you can send traces to Self-Hosted SigNoz

  let tracer = global::tracer("global_tracer");
    let _cx = Context::new();
  
    tracer.in_span("operation", |cx| {
        let span = cx.span();
        span.set_attribute(Key::new("KEY").string("value"));

        span.add_event(
            format!("Operations"),
            vec![
                Key::new("SigNoz is").string("Awesome"),
            ],
        );
    });
    shutdown_tracer_provider()

Step 4: Set environment variables and run app

Now run

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_RESOURCE_ATTRIBUTES=service.name=sample-rust-app cargo run

in terminal to start the application!

Sample Rust Application

We have included a sample Rust application at Sample Rust App Github Repo,

Tutorial

Here's a tutorial with step by step guide on how to install SigNoz and start monitoring a sample Rust app.

Frequently Asked Questions

1. How to find what to use in IP of SigNoz if I have installed SigNoz in Kubernetes cluster?

   Based on where you have installed your application and where you have installed SigNoz, you need to find the right value for this. Please use this grid to find the value you should use for IP of SigNoz

2. I am sending data from my application to SigNoz, but I don't see any events or graphs in the SigNoz dashboard. What should I do?

   This could be because of one of the following reasons:

   1. Your application is generating telemetry data, but not able to connect with SigNoz installation

      Please use this troubleshooting guide to find if your application is able to access SigNoz installation and send data to it.

   2. Your application is not actually generating telemetry data

      Please check if the application is generating telemetry data first. You can use Console Exporter to just print your telemetry data in console first. Join our Slack Community if you need help on how to export your telemetry data in console

   3. Your SigNoz installation is not running or behind a firewall

      Please double check if the pods in SigNoz installation are running fine. docker ps or kubectl get pods -n platform are your friends for this.

What Cloud Endpoint Should I Use?

The primary method for sending data to SigNoz Cloud is through OTLP exporters. You can either send the data directly from your application using the exporters available in SDKs/language agents or send the data to a collector agent, which batches/enriches telemetry and sends it to the Cloud.

My Collector Sends Data to SigNoz Cloud

Using gRPC Exporter

The endpoint should be ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu. Note that the exporter endpoint doesn't require a scheme for the gRPC exporter in the collector.

# Sample config with `us` region
exporters:
    otlp:
        endpoint: "ingest.us.signoz.cloud:443"
        tls:
            insecure: false
        headers:
            "signoz-access-token": "<SIGNOZ_INGESTION_KEY>"

Using HTTP Exporter

The endpoint should be https://ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu. Note that the endpoint includes the scheme https for the HTTP exporter in the collector.

# Sample config with `us` region
exporters:
    otlphttp:
        endpoint: "https://ingest.us.signoz.cloud:443"
        tls:
            insecure: false
        headers:
            "signoz-access-token": "<SIGNOZ_INGESTION_KEY>"

My Application Sends Data to SigNoz Cloud

The endpoint should be configured either with environment variables or in the SDK setup code.

Using Environment Variables

Using gRPC Exporter

Examples with us region

• OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud:443 OTEL_EXPORTER_OTLP_HEADERS=signoz-access-token=<SIGNOZ_INGESTION_KEY>

Using HTTP Exporter

• OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud:443 OTEL_EXPORTER_OTLP_HEADERS=signoz-access-token=<SIGNOZ_INGESTION_KEY>

Configuring Endpoint in Code

Please refer to the agent documentation.

Sending Data from a Third-Party Service

The endpoint configuration here depends on the export protocol supported by the third-party service. They may support either gRPC, HTTP, or both. Generally, you will need to adjust the host and port. The host address should be ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu, and port 443 should be used.