Set Custom Attributes in OpenTelemetry Traces, Logs, Metrics

Overview

A single SigNoz workspace receives telemetry from many applications, teams, and environments. Custom attributes are the key-value pairs that tell that telemetry apart. Add an attribute such as team.name or tenant.id, and you can filter on it in the explorers, group charts by it, and use it as an alert condition.

You can set an attribute in an environment variable, in your application code, or in the OpenTelemetry Collector. This guide covers all three, for traces, logs, and metrics.

Attributes and resource attributes

OpenTelemetry has two kinds of attributes. Pick the kind you need before you write any configuration.

	Resource attribute	Attribute
Describes	The process that produces the telemetry	One span, metric data point, or log record
Value changes	Once per process, at startup	On every request or operation
Applies to	Every span, metric, and log record from that process	Only the item you set it on
Examples	`service.name`, `deployment.environment.name`, `host.name`, `team.name`	`http.request.method`, `db.query.text`, `tenant.id`

Use a resource attribute to answer "where did this come from". Use an attribute to answer "what happened in this operation".

Prefer the names in the OpenTelemetry semantic conventions when one exists. SigNoz builds service maps, APM charts, and infrastructure views from these standard names. Use your own names, such as team.name or tenant.id, only for concepts the conventions do not cover.

Prerequisites

An application that is instrumented with OpenTelemetry. See the instrumentation guides for your language.
A SigNoz account, either SigNoz Cloud or a self-hosted deployment.
An OpenTelemetry Collector, if you plan to set attributes outside your application. See Collector configuration.

Choose where to set an attribute

Method	Use it when	Sets
Environment variable	You want one value for the whole process and you do not want to change code	Resource attributes
SDK code	The value comes from application configuration, or you use an attribute that changes per request	Resource attributes and attributes
Collector	You cannot change the application, or you want one rule for many services	Resource attributes and attributes

Start with environment variables. They need no code change, and every OpenTelemetry SDK reads them.

Set resource attributes with environment variables

Set these variables in the process that runs your application, then restart the application:

export OTEL_SERVICE_NAME="checkout"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment.name=production,service.version=1.4.2,team.name=payments"

OTEL_RESOURCE_ATTRIBUTES takes comma-separated key=value pairs. Percent-encode any comma or equals sign inside a key or a value.

OTEL_SERVICE_NAME sets the service.name resource attribute. If you set service.name in both variables, OTEL_SERVICE_NAME wins.

OpenTelemetry deprecated deployment.environment and replaced it with deployment.environment.name. Use the new name in new configuration. See the deployment attribute registry.

For the full list of variables that SigNoz supports, see OpenTelemetry environment variables.

Set resource attributes in the SDK

Set resource attributes in code when the value comes from application configuration. When the same key is also in OTEL_RESOURCE_ATTRIBUTES, the winner depends on the language. Each tab states the order.

const { NodeSDK } = require('@opentelemetry/sdk-node')
const { defaultResource, resourceFromAttributes } = require('@opentelemetry/resources')
const {
  ATTR_SERVICE_NAME,
  ATTR_SERVICE_VERSION,
} = require('@opentelemetry/semantic-conventions')
 
const sdk = new NodeSDK({
  resource: defaultResource().merge(
    resourceFromAttributes({
      [ATTR_SERVICE_NAME]: 'checkout',
      [ATTR_SERVICE_VERSION]: '1.4.2',
      'deployment.environment.name': 'production',
      'team.name': 'payments',
    })
  ),
})
 
sdk.start()

resourceFromAttributes needs @opentelemetry/resources 2.0.0 or newer. On 1.x, the package exports a Resource class instead. The 1.x class and the SemanticResourceAttributes constants are both deprecated.

NodeSDK replaces the default resource with the one you pass, so merge defaultResource() to keep service.name and the telemetry.sdk.* attributes.

NodeSDK then merges the detected resources over your resource, and the default detectors read OTEL_RESOURCE_ATTRIBUTES. A key in that variable therefore overrides the same key in this code. Remove the key from the variable, or pass resourceDetectors: [] to turn detection off.

from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry import trace
 
resource = Resource.create({
    "service.name": "checkout",
    "service.version": "1.4.2",
    "deployment.environment.name": "production",
    "team.name": "payments",
})
 
trace.set_tracer_provider(TracerProvider(resource=resource))

Pass the same resource object to LoggerProvider and MeterProvider, so that logs and metrics carry the same attributes.

Resource.create reads OTEL_RESOURCE_ATTRIBUTES first and applies the attributes you pass second, so the values here override the environment variable.

The ResourceAttributes class in opentelemetry-semantic-conventions is deprecated since package version 1.25.0. Write the attribute names as plain strings, as shown above.

import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.semconv.ServiceAttributes;
 
Resource resource = Resource.getDefault().toBuilder()
    .put(ServiceAttributes.SERVICE_NAME, "checkout")
    .put(ServiceAttributes.SERVICE_VERSION, "1.4.2")
    .put("deployment.environment.name", "production")
    .put("team.name", "payments")
    .build();

Pass resource to SdkTracerProvider, SdkMeterProvider, and SdkLoggerProvider.

This resource does not read OTEL_RESOURCE_ATTRIBUTES. Only the autoconfigure module and the Java agent read that variable.

If you run the Java agent, the agent builds the resource for you. Use OTEL_RESOURCE_ATTRIBUTES instead of this code.

import (
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/sdk/resource"
	semconv "go.opentelemetry.io/otel/semconv/v1.40.0"
)
 
res, err := resource.Merge(
	resource.Default(),
	resource.NewSchemaless(
		semconv.ServiceName("checkout"),
		semconv.ServiceVersion("1.4.2"),
		semconv.DeploymentEnvironmentNameKey.String("production"),
		attribute.String("team.name", "payments"),
	),
)

Pass res to the tracer, meter, and logger providers with their WithResource option.

The semconv import path must name a package that ships inside your go.opentelemetry.io/otel version. semconv/v1.40.0 is present from v1.42.0 onward. Check the semconv directory at your version tag before you pick a newer path.

resource.NewSchemaless keeps the schema URL of resource.Default(). If you use resource.NewWithAttributes with a schema URL that does not match, resource.Merge returns ErrSchemaURLConflict and drops the schema URL. Handle the error in both cases.

resource.Default() reads OTEL_RESOURCE_ATTRIBUTES, and resource.Merge gives the second resource priority. The values here therefore override the environment variable.

using OpenTelemetry;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
 
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(serviceName: "checkout", serviceVersion: "1.4.2")
        .AddAttributes(new Dictionary<string, object>
        {
            ["deployment.environment.name"] = "production",
            ["team.name"] = "payments",
        }))
    .WithTracing(tracing => tracing.AddOtlpExporter())
    .WithMetrics(metrics => metrics.AddOtlpExporter());

ConfigureResource applies the resource to every signal that you register on the builder. The values here override the same key in OTEL_RESOURCE_ATTRIBUTES.

Add attributes to a single span, metric, or log record

Use an attribute when the value changes per request, for example a tenant ID or a customer plan.

Span attributes

Add the attribute to the span that the instrumentation already started:

const { trace } = require('@opentelemetry/api')
 
trace.getActiveSpan()?.setAttribute('tenant.id', tenantId)

Metric attributes

Pass the attributes when you record a measurement:

orderCounter.add(1, { 'tenant.id': tenantId, 'order.type': orderType })

import (
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/metric"
)
 
orderCounter.Add(ctx, 1, metric.WithAttributes(
	attribute.String("tenant.id", tenantID),
	attribute.String("order.type", orderType),
))

Every unique combination of metric attributes creates a new time series. An attribute such as user.id or request.id creates one series per value, which raises cost and slows queries. Keep high-cardinality values on spans and logs. See Attribute ingestion volume.

Log record attributes

OpenTelemetry log SDKs bridge the logging library that your application already uses, so log record attributes come from the fields that you already log:

Python: pass extra={"tenant.id": tenant_id} to the log call. The OpenTelemetry LoggingHandler copies every non-reserved field on the log record into attributes.
Java: put the value in the MDC, then capture it with OTEL_INSTRUMENTATION_<FRAMEWORK>_APPENDER_EXPERIMENTAL_CAPTURE_MDC_ATTRIBUTES. See Java logs.
Go: pass the key-value pairs to the slog call. The otelslog bridge copies them into log record attributes.

For logs, resource attributes carry most of the filtering value. See Guide to add Resource Attributes for the Collector and Kubernetes options.

Set attributes in the OpenTelemetry Collector

Use the Collector when you cannot change the application, or when one rule applies to many services. Add the snippets below to your existing otel-collector-config.yaml. Do not replace the whole file.

Add resource attributes

The resource processor changes resource attributes:

processors:
  resource/ownership:
    attributes:
      - key: team.name
        value: payments
        action: upsert
      - key: deployment.environment.name
        value: production
        action: insert

Then add resource/ownership to the processors list of each pipeline that needs it. Edit the list your file already has. Do not replace it, because a pipeline often already runs memory_limiter, resourcedetection, or a filter, and removing any of those changes what the Collector sends.

Order inside the list matters. Keep memory_limiter first. Add resource/ownership after it. If the pipeline ends with batch, keep batch last.

The example below shows a pipeline that already had three processors. Your list will differ:

service:
  pipelines:
    traces:
      processors: [memory_limiter, resourcedetection, resource/ownership, batch]

Repeat the edit for the metrics and logs pipelines. Leave the receivers and exporters of each pipeline unchanged.

Change span, metric, and log attributes

The attributes processor changes the attributes on individual spans, data points, and log records:

processors:
  attributes/cleanup:
    actions:
      - key: url.full
        action: delete
      - key: enduser.id
        action: hash

Add attributes/cleanup to your pipelines in the same way as the resource processor.

Actions that both processors support

Action	Result
`insert`	Adds the key only when it does not exist
`update`	Changes the value only when the key exists
`upsert`	Adds the key, or changes it when it exists
`delete`	Removes the key
`hash`	Replaces the value with a hash of the value
`extract`	Splits a value into new keys with a regular expression

Use upsert when you want one value everywhere. Use insert when a value that the application already sends must win.

Detect host and cloud attributes automatically

The resource detection processor reads attributes from the environment, such as the host name, the cloud provider, and the Kubernetes node:

processors:
  # On Collector v0.153.0 and newer, use "resource_detection" to avoid a deprecation warning.
  resourcedetection:
    detectors: [env, system]
    system:
      hostname_sources: ['os']
    timeout: 5s

Add a detector only for the platform that you run on. Each detector costs a lookup at startup, and timeout limits how long the Collector waits for all of them.

Make complex changes with OTTL

The resource and attributes processors handle fixed keys and values. For conditional logic, string functions, or changes that read one field to build another, use the transform processor. See OTTL in SigNoz.

Validate

Send traffic through your application, then check each signal in SigNoz:

Open Traces Explorer. Add a filter on your attribute, for example team.name = payments. The attribute also appears in the attributes table of a single span.
Open Logs Explorer. Add the same filter. Resource attributes appear in the resources section of a log record.
Open Metrics Explorer. Open a metric from your application and group by your attribute.

If the attribute appears in the filter list, SigNoz received it and you can use it in dashboards and alerts.

Troubleshooting

The attribute does not appear in SigNoz

Likely cause: The application did not restart after the configuration change.
Fix: Restart the application. An SDK reads OTEL_RESOURCE_ATTRIBUTES once, at startup.
Verify: Open Traces Explorer and search for the attribute key in the filter list.

The attribute appears on one signal only

Likely cause: The resource is attached to one provider, or the Collector processor is in one pipeline.
Fix: Pass the same resource object to the tracer, meter, and logger providers. In the Collector, add the processor to the traces, metrics, and logs pipelines.
Verify: Filter on the attribute in all three explorers.

The value in code is not the value in SigNoz

Likely cause: Another layer sets the same key later. The Collector runs after the SDK, and upsert in the resource processor overwrites what the application sent.
Fix: Change the action to insert, or remove the key from the Collector configuration.
Verify: Open one span in Traces Explorer and read the value in the attributes table.

The service name is unknown_service

Likely cause: Nothing set service.name.
Fix: Set OTEL_SERVICE_NAME, or set service.name in the SDK resource.
Verify: Open Services and find your service name in the list.

A metric has too many time series

Likely cause: A metric attribute carries a high-cardinality value, such as a user ID or a URL with an ID in the path.
Fix: Remove that attribute from the metric, or delete it with the attributes processor. Keep the value on the span instead.
Verify: Check the metric in Metrics Explorer and read its cardinality.

Next steps

Filter and group by your attributes in the query builder.
Create an alert that uses an attribute as a condition.
Remove resource attributes that you do not need.
Scrub personally identifiable information from attributes before it leaves your network.

Get Help

If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack. If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.