OpenTelemetry Injector - Zero-Code Setup Guide for Linux

By default the injector instruments every process on the host, and not only the services that you care about. Build tools and helper processes send data too. A dotnet build on the host, for example, makes the Roslyn compiler server appear in SigNoz as a service named VBCSCompiler. Narrow the set before you run the injector on a busy host.

Four settings in /etc/opentelemetry/injector/injector.conf control which processes the injector acts on. Each one takes a comma-separated list of glob patterns:

• include_paths: Instrument only the executables whose path matches.
• exclude_paths: Skip the executables whose path matches.
• include_with_arguments: Instrument only the processes whose command line matches.
• exclude_with_arguments: Skip the processes whose command line matches.

The rules combine as follows. Patterns inside one setting are alternatives, so one match is enough. Different settings all apply, so a process must satisfy each setting that you configure. An exclusion always wins over an inclusion.

A pattern matches the resolved path of the executable, not the path that you typed. On Debian, /usr/bin/java is a symbolic link, so the pattern /usr/bin/java matches nothing. Run readlink -f on the command first and use the path that it prints:

readlink -f /usr/bin/java

This example instruments the programs under /app, skips the ones under /app/system, and skips any Java process that already loads an agent:

include_paths=/app/*
exclude_paths=/app/system/*
exclude_with_arguments=-javaagent:*

To turn off one language, set auto_instrumentation_disabled in the same file. Valid values are dotnet, jvm, nodejs, python, and ruby. The value * turns off all of them:

auto_instrumentation_disabled=dotnet,python

To turn the injector off for a single program, set OTEL_INJECTOR_DISABLED=true in the environment of that program. The injector then changes no environment variable for it.

Set this variable in the systemd unit of the program, or in a shell that the injector has not touched. The injector also runs for your shell, so an interactive shell already exports JAVA_TOOL_OPTIONS, NODE_OPTIONS, and PYTHONPATH. A child process inherits those values. A command such as OTEL_INJECTOR_DISABLED=true java -jar app.jar therefore still starts the agent, because the shell passed the variables down before the injector ran.

The injector reads a set of OTEL_INJECTOR_* variables from the process environment and adds them to OTEL_RESOURCE_ATTRIBUTES. Set them the same way you set OTEL_INJECTOR_SERVICE_NAME, per process:

If OTEL_RESOURCE_ATTRIBUTES already holds a key, the injector keeps the existing value.

Turn on the injector log first. Set OTEL_INJECTOR_LOG_LEVEL=debug in the environment of the application, then restart it. The injector writes its log to the standard error stream of the instrumented process. Read the log of the application to see it. The injector keeps no log file of its own. Valid levels are debug, info, warn, error, and none. The default is error.

Still no data in SigNoz? Work through Debug missing traces, logs, and metrics, which covers agent diagnostics, Collector connectivity, and the common ingestion errors.

No process is instrumented

• Cause: The library is not in /etc/ld.so.preload, or the file is not readable by other users.
• Fix: Run cat /etc/ld.so.preload and make sure that it holds the line /usr/lib/opentelemetry/injector/libotelinject.so. Run ls -l /etc/ld.so.preload and make sure that the mode is 0644. The dynamic linker ignores the file for a non-root process when other users cannot read it.
• Verify: Restart the service with OTEL_INJECTOR_LOG_LEVEL=debug. The injector prints a setting "..." line for the variable of your runtime.

Every service reports under the same name

• Cause: OTEL_SERVICE_NAME or OTEL_INJECTOR_SERVICE_NAME is set in default_env.conf, so all processes share one name.
• Fix: Remove the name from default_env.conf and set OTEL_INJECTOR_SERVICE_NAME in each systemd unit instead.
• Verify: Each service appears under its own name in the SigNoz Services list.

A .NET application sends nothing

• Cause: The application targets .NET 7 or earlier, or its *.deps.json file already references OpenTelemetry* packages.
• Fix: Move the application to .NET 8 or later. If the application already carries the OpenTelemetry SDK, keep the SDK and let the injector stand down. .NET accepts one agent only.
• Verify: The injector debug log records the runtime version that it read from *.runtimeconfig.json.

A Go, Rust, or C++ application sends nothing

• Cause: The injector supports Java, Node.js, .NET, and Python only. A statically linked binary, such as a Go binary built without cgo, never loads the injector at all. The dynamic linker that reads /etc/ld.so.preload does not run for it.
• Fix: Use eBPF instrumentation for these languages, or the matching language guide.
• Verify: Spans from the service appear in SigNoz after you set up the other method.

Spans appear twice

• Cause: The application starts its own agent, and the injector adds a second one.
• Fix: Add exclude_with_arguments=-javaagent:* to /etc/opentelemetry/injector/injector.conf, or set OTEL_INJECTOR_DISABLED=true for that program.
• Verify: A trace in SigNoz holds one span per operation.

A Python application sends nothing and prints a dependency conflict

The agent writes a line like this to the standard error stream of the Python process:

[opentelemetry-python-autoinstrumentation] WARN: cannot auto-instrument Python process: dependency conflicts: {'asgiref': {'version_required': '==3.12.1', 'version_found': '3.6.0'}}

• Cause: The Python agent pins exact versions of its own dependencies. A package that the operating system installed holds a different version, so the agent stops instead of breaking the application. The system Python on Debian 12 hits this with asgiref.
• Fix: Run the application in a virtual environment that does not carry the conflicting package. The agent instruments a virtual environment correctly, because it prepends its own directory to PYTHONPATH. If you cannot use a virtual environment, turn off Python injection with auto_instrumentation_disabled=python in /etc/opentelemetry/injector/injector.conf and follow the Python guide instead.
• Verify: The warning stops, and the service appears in the SigNoz Services list.

Run an OpenTelemetry Collector on the host when you want to process, filter, or route telemetry before it reaches SigNoz. The Collector also gives every service on the host one egress point and one copy of the ingestion key. Follow the Switch to Collector guide for setup instructions.

Then point the agents at the local Collector in /etc/opentelemetry/injector/default_env.conf:

OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

Write the address as 127.0.0.1. Node.js resolves localhost to the IPv6 address ::1 first. A Collector that listens on 127.0.0.1 only rejects that connection. The Node.js agent then fails with connect ECONNREFUSED ::1:4318, while the Java agent on the same host keeps working.