Reduce Metric Sample Volume and Cardinality

This guide covers the primary causes of metric volume growth in SigNoz and how to address each one.

Metrics are measured in Samples: a single (timestamp, value) pair for one time series. Every unique combination of label values creates a distinct time series: a metric with a label carrying 500 unique values generates 500 distinct time series, each emitting a sample on every scrape interval.

Understanding High Cardinality Data: explains why cardinality multiplies time series and the downstream impact on storage and query performance.

Optimization Tiers

Address the problem as close to its source as possible before reaching for Collector processors:

Instrumentation: parameterize dynamic routes. Addresses the problem at source, before it enters the pipeline.
OTel SDK: use the View API to drop attributes or entire metrics before they are serialized to OTLP.
OTel Collector: use processors to aggregate or drop metrics in the pipeline. Covered in the problems below.

Problem 1: Unused Metrics

The simplest reduction is to stop ingesting metrics that serve no purpose in any dashboard or alert. Drop them at the Collector before they reach SigNoz.

Read more: How to Drop OpenTelemetry Metrics

Problem 2: High-Cardinality Labels

OTel Instrumentation libraries commonly emit high-cardinality attributes like k8s.pod.name, client.port, url.full directly on metric data points. If a label like k8s.pod.name has 200 unique values, it multiplies your baseline time series by 200×.

Step 1: Find the Noisy Labels

Go to Metrics. Toggle to the Time Series view.
Select your top ingested metric. Expand the "All Attributes" panel.
Identify labels with hundreds or thousands of unique values.

Step 2: Verify Usage

Before aggregating anything, confirm the label is not in active use. If it appears in any dashboard or alert, do not aggregate it. The aggregation will break those queries.

Open the metric in Metrics Explorer. If it is used in any dashboards or alert rules, the counts appear at the top of the detail view. If neither is shown, the metric is not referenced anywhere and is safe to drop.

Metric detail view showing dashboard and alert rule counts at the top — *Metrics Explorer: dashboard and alert rule counts appear at the top if the metric is in active use*

Step 3: Implement the Fix (Aggregation)

If the label is unused, aggregating it will reduce the number of time series to one per unique combination of the retained labels.

Label	Why it's problematic	Verdict
`k8s.pod.name` / `k8s.pod.uid`	Changes on every pod restart or redeploy. Large multiplier.	Aggregate away
`client.port` / `net.peer.port`	Ephemeral TCP ports (49152–65535).	Aggregate away
`url.full` / `http.target`	Dynamic query params.	Aggregate away
`db.query.text`	Full SQL with literal values.	Aggregate away
`http.route`	Unparameterized raw URLs (`/user/123` instead of `/user/{id}`).	Fix at instrumentation: parameterize route patterns in your SDK or framework router

Using delete_key() via the attributes processor does NOT merge time series. It removes the label name but leaves colliding, undefined data. Use the metricstransform processor's aggregation action to merge series within the dropped dimension instead. metricstransform aggregates only within a single batch passing through the Collector, not across the full set of pods or nodes emitting that dimension, so cardinality reduction is bounded by your batch size and scrape topology.

Read more: How to safely drop metric labels

Problem 3: Cumulative Temporality

By default, OpenTelemetry SDKs use Cumulative Temporality: they emit a sample carrying the running total on every scrape interval, even when the value is unchanged.

Delta Temporality only emits a sample when the value changes. See Metric Types and Aggregation for a detailed explanation of temporality and how it affects aggregation.

When to Switch to Delta

If you have large idle infrastructure (e.g., 1000 pods with only 10 receiving traffic in a given minute), Delta prevents the 990 idle pods from emitting unchanged samples every scrape interval. This applies to synchronous instruments (like a request counter incremented per request); asynchronous instruments whose callback runs on every collection cycle still emit a sample each interval regardless of temporality.

Feature	Cumulative	Delta
Idle series	Always sends	Skips unchanged
Network failure	Next scrape recovers the total (Safe)	Gap is permanently lost
Collector memory	Tracks running totals	Lower (no running totals to maintain)
UpDownCounter / Async UpDownCounter	Cumulative	Cumulative (unaffected; Delta is undefined for bidirectional instruments)

OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta applies Delta to Counter, Async Counter, and Histogram only. UpDownCounter, Async UpDownCounter, Gauge and Async Gauge remain Cumulative regardless.

Migration Warning: Switching a live metric pipeline from Cumulative to Delta mid-flight may cause unexpected values in dashboards during the transition. Panels computing rates or totals from these metrics may show incorrect readings until reporting stabilises. See Metrics temporality FAQ for details on temporality mismatch behaviour.

How to Set Delta:

export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta

Problem 4: Collection Interval

Sample count scales directly with how often each time series is emitted. A metric exported every 5 seconds generates 12x the samples of the same metric exported every 60 seconds, with no change in cardinality.

SDK metrics: the OTel SDK default for OTEL_METRIC_EXPORT_INTERVAL is 60000 (60s). If your service overrides it to export more frequently, for example for a low-latency alert that no longer needs that resolution, raise it back toward the default or higher:

export OTEL_METRIC_EXPORT_INTERVAL=60000  # raise this if your service currently overrides it lower

Collector receivers (for example hostmetrics): the receiver's own default collection_interval is 1m. If your config explicitly sets a shorter interval, raise it:

receivers:
  hostmetrics:
    collection_interval: 60s # raise this if your config currently sets a shorter interval

Increasing the interval reduces the resolution of rate-based queries (for example rate() over a short window) and delays how quickly a spike shows up on dashboards or triggers an alert. Do not raise the interval on metrics feeding low-latency alerts.

Validate Your Changes

The Cost Meter Dashboard uses signoz.meter.* metrics which aggregate hourly. Changes may take up to one hour to appear there. Use the Metrics Explorer for faster feedback.

To confirm the volume reduction:

Navigate to Metrics. In the list view, verify that unused metrics you dropped no longer appear, or that the time series count for your target metric has reduced.
Check the Cost Meter Dashboard after an hour to confirm the datapoint.count trend has dropped.

Next Steps

Configure Cost Meter Alert to be notified when ingestion volume crosses your expected threshold.
Read Understanding Metrics Billing and Reducing Costs for how cardinality maps to your bill and additional cost-driver examples.

Get Help

If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack. If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.