VM Collection Agent - Configure

The install guide provides a working config: one agent on each host collects host metrics and data from instrumented applications, and on Linux the warnings and errors in the systemd journal. This page explains how to customize it.

Prerequisites

The VM Collection Agent installed as the install guide shows
SigNoz Cloud account or self-hosted SigNoz instance
A shell on the host (with sudo on Linux), or PowerShell on Windows (as Administrator for the MSI)
On Linux, the casting.yaml and foundryctl you installed with; on macOS and Windows, the config.yaml you created

Apply Changes

Add the section's snippet to casting.yaml. Foundry already generates the default receivers and processors, so add only what you change. If casting.yaml already has a collector/agent/agent.yaml block, as the install guide's does, add the snippet's lines inside that block.
Run this from the directory that contains casting.yaml to apply the change:
```
sudo foundryctl cast -f casting.yaml
```
Foundry reinstalls the agent's files and restarts it.
View the agent's logs to confirm there are no configuration errors.
In SigNoz, confirm the data in Infrastructure Monitoring for host metrics, Logs Explorer for logs, and Trace Explorer for application traces.

Add the section's snippet to config.yaml, merged into the keys the file already has.
From the directory that contains config.yaml, stop the agent with this, then run Step 4 of the install guide again:
```
kill "$(cat otel-pid)"
```
View the agent's logs to confirm there are no configuration errors.
In SigNoz, confirm the data in Infrastructure Monitoring for host metrics, Logs Explorer for logs, and Trace Explorer for application traces.

Add the section's snippet to config.yaml, merged into the keys the file already has. With the MSI, the file is C:\Program Files\OpenTelemetry Collector\config.yaml; when you run the agent manually, it is the config.yaml next to otelcol-contrib.exe.
Restart the agent. With the MSI, run this in PowerShell as Administrator:
```
Restart-Service otelcol-contrib
```
When you run it manually, stop the agent with this, then run Step 4 of the manual install again:
```
Stop-Process -Id (Get-Content otel-pid)
```
View the agent's logs to confirm there are no configuration errors.
In SigNoz, confirm the data in Infrastructure Monitoring for host metrics, Logs Explorer for logs, and Trace Explorer for application traces.

Send Data from Applications to Collector on VM

Host metrics, and on Linux the warnings and errors in the systemd journal, flow as soon as the agent is running.

Where and how to send data from your application

To send traces, metrics, and logs from an application, instrument it with OpenTelemetry (language guides), then point it at the agent. For an application on the same host, set these in its environment, for example in the shell that starts it:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_SERVICE_NAME="<service-name>"

Verify these values:

<service-name>: The name your application shows under in SigNoz.

From another host, use the agent host's address in place of localhost, with port 4318 open to it. For OTLP over gRPC, set OTEL_EXPORTER_OTLP_PROTOCOL=grpc and port 4317. On Windows, set the same variables in the application's environment.

Restart the application and send it some traffic. Within a minute it shows under Services in SigNoz.

Collect Host Metrics

To collect host metrics every 30s, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              hostmetrics:
                collection_interval: 30s

To stop a scraper, set it to null under scrapers, for example paging: null. Then apply the change.

Exclude filesystem mounts

To stop reporting the filesystem mounted at /mnt/backup, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              hostmetrics:
                scrapers:
                  filesystem:
                    exclude_mount_points:
                      mount_points:
                        - /dev/*
                        - /proc/*
                        - /sys/*
                        - /run/credentials/*
                        - /run/k3s/containerd/*
                        - /var/lib/docker/*
                        - /var/lib/containers/storage/*
                        - /var/lib/kubelet/*
                        - /snap/*
                        - ^/mnt/backup$

Lists replace, so restate the agent's default entries, the ones above ^/mnt/backup$. Then apply the change.

To stop reporting the filesystem mounted at /Volumes/Backup, add this to config.yaml:

receivers:
  hostmetrics:
    scrapers:
      filesystem:
        exclude_mount_points:
          match_type: regexp
          mount_points:
            - /dev/*
            - ^/System/Volumes/(Preboot|VM|Update|xarts|iSCPreboot|Hardware)$
            - ^/Volumes/Backup$

Then apply the change.

To stop reporting the D: drive, add this to config.yaml:

receivers:
  hostmetrics:
    scrapers:
      filesystem:
        exclude_mount_points:
          match_type: regexp
          mount_points:
            - ^D:$

Then apply the change.

Collect Host Logs

On Linux, to also collect journal entries at priority info, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              journald:
                priority: info

priority merges into the agent's journald receiver, so its other settings stay. Then apply the change.

To collect the journal of only some systemd units, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              journald:
                units:
                  - nginx.service
                  - postgresql.service

Then apply the change.

Disable Logs Collection

On Linux, to stop collecting the journal, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              journald: null
            service:
              pipelines:
                logs:
                  receivers: [otlp/http, otlp/grpc]

Lists replace, so the logs pipeline is restated with the agent's two OTLP receivers. Then apply the change.

Collect Application Log Files

To collect the log files that an application writes, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              filelog/app:
                include:
                  - /var/log/myapp/*.log
                include_file_path: true
            service:
              pipelines:
                logs:
                  receivers: [otlp/http, otlp/grpc, journald, filelog/app]

Lists replace, so the logs pipeline restates the agent's receivers with the new one. Then apply the change.

To collect the log files that an application writes, add this to config.yaml:

receivers:
  filelog/app:
    include:
      - /var/log/myapp/*.log
    include_file_path: true
service:
  pipelines:
    logs:
      receivers: [otlp/http, otlp/grpc, filelog/app]

Then apply the change.

To collect the log files that an application writes, add this to config.yaml:

receivers:
  filelog/app:
    include:
      - C:\myapp\logs\*.log
    include_file_path: true
service:
  pipelines:
    logs:
      receivers: [otlp/http, otlp/grpc, filelog/app]

Then apply the change.

For JSON parsing and more filelog options, see Collect Logs from a File.

Other log sources

To receive syslog, or logs forwarded from Fluent Bit, Fluentd, or Logstash, take the receiver from its guide, add it the same way, and list it in the logs pipeline. On Linux, the agent can listen only on ports 1024 and above. Then apply the change.

Resource Detection

To detect only host attributes on a host outside AWS, GCP, and Azure, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            processors:
              resourcedetection:
                detectors: [env, system]

Lists replace, so detectors holds only the detectors you list. Then apply the change.

Custom resource attributes

To add team=backend to all telemetry, add this to casting.yaml:

spec:
  collector:
    spec:
      env:
        OTEL_RESOURCE_ATTRIBUTES: deployment.environment=<your-environment>,team=backend

Verify these values:

<your-environment>: Your deployment environment, for example staging or production.

Then apply the change.

To add team=backend to all telemetry, set this in place of the OTEL_RESOURCE_ATTRIBUTES line of Step 4 of the install guide:

export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=<your-environment>,team=backend"

Verify these values:

<your-environment>: Your deployment environment, for example staging or production.

Then apply the change.

To add team=backend to all telemetry with the MSI, use this line for OTEL_RESOURCE_ATTRIBUTES in Step 3 of the MSI install:

'OTEL_RESOURCE_ATTRIBUTES=deployment.environment=<your-environment>,team=backend'

Verify these values:

<your-environment>: Your deployment environment, for example staging or production.

When you run the agent manually, set this in place of the OTEL_RESOURCE_ATTRIBUTES line of Step 4 of the manual install:

$env:OTEL_RESOURCE_ATTRIBUTES = 'deployment.environment=<your-environment>,team=backend'

Verify these values:

<your-environment>: Your deployment environment, for example staging or production.

Then apply the change.

Scrape Prometheus Metrics

To scrape a Prometheus endpoint every 60s, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              prometheus:
                config:
                  scrape_configs:
                    - job_name: my-app
                      scrape_interval: 60s
                      static_configs:
                        - targets: [localhost:9090]
            service:
              pipelines:
                metrics:
                  receivers: [otlp/http, otlp/grpc, hostmetrics, prometheus]

Lists replace, so the metrics pipeline is restated with the agent's default receivers. Then apply the change.

To scrape a Prometheus endpoint every 60s, add this to config.yaml:

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: my-app
          scrape_interval: 60s
          static_configs:
            - targets: [localhost:9090]
service:
  pipelines:
    metrics:
      receivers: [otlp/http, otlp/grpc, hostmetrics, prometheus]

Then apply the change.

To scrape a Prometheus endpoint every 60s, add this to config.yaml:

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: my-app
          scrape_interval: 60s
          static_configs:
            - targets: [localhost:9090]
service:
  pipelines:
    metrics:
      receivers: [otlp/http, otlp/grpc, hostmetrics, prometheus]

Then apply the change.

Tune Batch and Memory Limiter

For sizing guidance, see Collector Configuration Best Practices.

To send larger batches, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            processors:
              batch:
                send_batch_size: 2000
                send_batch_max_size: 4000

The agent's defaults are send_batch_size: 1000, send_batch_max_size: 2048, and timeout: 10s. Then apply the change.

Memory limit

To cap the agent's memory at 1 GiB, add this patch to casting.yaml:

spec:
  patches:
    - target: collectionagent/collector/agent/signoz-collector-agent.service
      operations:
        - op: add
          path: /Service/MemoryMax
          value: 1G

The agent's memory_limiter processor uses a fixed limit_mib: 4000, so size it to the cap with percentages, in the same document:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            processors:
              memory_limiter:
                limit_mib: null
                spike_limit_mib: null
                limit_percentage: 80
                spike_limit_percentage: 25

Then apply the change.

To size the memory_limiter processor by the Mac's memory, replace its settings in config.yaml with this:

processors:
  memory_limiter:
    check_interval: 5s
    limit_percentage: 80
    spike_limit_percentage: 25

Then apply the change.

To size the memory_limiter processor by the host's memory, replace its settings in config.yaml with this:

processors:
  memory_limiter:
    check_interval: 5s
    limit_percentage: 80
    spike_limit_percentage: 25

Then apply the change.

Add Custom Processors to Pipelines

To add a processor to the traces pipeline, add this to casting.yaml:

spec:
  collector:
    spec:
      config:
        data:
          collector/agent/agent.yaml: |
            processors:
              attributes/team:
                actions:
                  - key: team
                    value: platform
                    action: insert
            service:
              pipelines:
                traces:
                  processors: [memory_limiter, resourcedetection, attributes/team, batch]

Lists replace, so restate the agent's processors memory_limiter, resourcedetection, and batch, with yours before batch. Then apply the change.

To add a processor to the traces pipeline, add this to config.yaml:

processors:
  attributes/team:
    actions:
      - key: team
        value: platform
        action: insert
service:
  pipelines:
    traces:
      processors: [memory_limiter, resourcedetection, attributes/team, batch]

List yours before batch. Then apply the change.

To add a processor to the traces pipeline, add this to config.yaml:

processors:
  attributes/team:
    actions:
      - key: team
        value: platform
        action: insert
service:
  pipelines:
    traces:
      processors: [memory_limiter, resourcedetection, attributes/team, batch]

List yours before batch. Then apply the change.

Common Deployment Scenarios

Run on the same host as self-hosted SigNoz

To run the agent on the host that runs self-hosted SigNoz, add this to casting.yaml:

spec:
  collector:
    spec:
      env:
        SIGNOZ_INGESTION_ENDPOINT: http://localhost:4318
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              otlp/grpc:
                protocols:
                  grpc:
                    endpoint: 0.0.0.0:14317
              otlp/http:
                protocols:
                  http:
                    endpoint: 0.0.0.0:14318
            extensions:
              health_check:
                endpoint: 0.0.0.0:13134
            service:
              telemetry:
                metrics:
                  readers:
                    - pull:
                        exporter:
                          prometheus:
                            host: localhost
                            port: 8889

The agent then receives OTLP on 14317 and 14318, so applications on the host send to localhost:14318. Then apply the change.

To run the agent on the Mac that runs self-hosted SigNoz, add this to config.yaml, and set SIGNOZ_INGESTION_ENDPOINT to http://localhost:4318 in Step 4 of the install guide:

receivers:
  otlp/grpc:
    protocols:
      grpc:
        endpoint: 0.0.0.0:14317
  otlp/http:
    protocols:
      http:
        endpoint: 0.0.0.0:14318
extensions:
  health_check:
    endpoint: 0.0.0.0:13134
service:
  telemetry:
    metrics:
      readers:
        - pull:
            exporter:
              prometheus:
                host: localhost
                port: 8889

The agent then receives OTLP on 14317 and 14318, so applications on the Mac send to localhost:14318. Then apply the change.

To run the agent on the host that runs self-hosted SigNoz, add this to config.yaml, and set SIGNOZ_INGESTION_ENDPOINT to http://localhost:4318 in the agent's environment (Step 3 of the MSI install, or Step 4 of the manual install):

receivers:
  otlp/grpc:
    protocols:
      grpc:
        endpoint: 0.0.0.0:14317
  otlp/http:
    protocols:
      http:
        endpoint: 0.0.0.0:14318
extensions:
  health_check:
    endpoint: 0.0.0.0:13134
service:
  telemetry:
    metrics:
      readers:
        - pull:
            exporter:
              prometheus:
                host: localhost
                port: 8889

The agent then receives OTLP on 14317 and 14318, so applications on the host send to localhost:14318. Then apply the change.

Migrate to Foundry

If you run the agent on Linux as the otelcol-contrib package, or as an otelcol-contrib process you set up yourself, with your own config.yaml, move it to Foundry so that casting.yaml holds its whole setup and every change follows Apply Changes.

To describe the agent in Foundry, install foundryctl, download the OpenTelemetry Collector, and create casting.yaml as the install guide shows. On a host that runs self-hosted SigNoz, also add the change in Run on the same host as self-hosted SigNoz.
To keep your config.yaml customizations, add them to casting.yaml under spec.collector.spec.config.data, keyed collector/agent/agent.yaml, and your environment variables under spec.collector.spec.env. For example, for host metrics every 30s and a team resource attribute:
```
spec:
  collector:
    spec:
      env:
        OTEL_RESOURCE_ATTRIBUTES: deployment.environment=<your-environment>,team=backend
      config:
        data:
          collector/agent/agent.yaml: |
            receivers:
              hostmetrics:
                collection_interval: 30s
```
Verify these values:
- <your-environment>: Your deployment environment, for example staging or production.
Foundry merges the collector/agent/agent.yaml block over its own config. When you move your settings over:
- Restate a whole list to change it, such as a pipeline's receivers or an exclude list. A list you set replaces Foundry's.
- Use Foundry's names: otlp/grpc and otlp/http for your otlp receiver's gRPC and HTTP protocols, and otlphttp/signoz for your otlphttp exporter.
- Keep journald in the logs pipeline's receivers.
- Set your exporter's endpoint and key as SIGNOZ_INGESTION_ENDPOINT and SIGNOZ_INGESTION_KEY under spec.collector.spec.env, not as values in agent.yaml. The install guide's agent.yaml block reads the key from there.
To free the host ports the Foundry agent binds, remove the old collector. For the package on Debian and Ubuntu, run this on the host:
```
sudo apt-get remove otelcol-contrib
```
On RHEL and Fedora (yum on older releases):
```
sudo dnf remove otelcol-contrib
```
For an otelcol-contrib process you set up yourself, stop it.
To start the agent with Foundry, run this from the directory that contains casting.yaml:
```
sudo foundryctl cast -f casting.yaml
```
Then verify the Collection Agent as Step 5 of the install guide shows.

Upgrade the Collection Agent

To replace the collector binary:

Stop the agent. Run this on the host:

sudo systemctl stop signoz-collector-agent

Download the binary as Step 2 of the install guide shows, with the version the install guide names.
Start the agent. Run this from the directory that contains casting.yaml:
```
sudo foundryctl cast -f casting.yaml
```

To replace the collector binary, from the directory that contains config.yaml:

Stop the agent:
```
kill "$(cat otel-pid)"
```
Download and extract the new tarball as Steps 1 and 2 of the install guide show, with the version the install guide names. config.yaml stays as it is.
Start the agent as Step 4 shows.

With the MSI, run these in PowerShell as Administrator:

Save your config, because the MSI replaces config.yaml and removes the service's environment when it upgrades:
```
Copy-Item 'C:\Program Files\OpenTelemetry Collector\config.yaml' "$env:TEMP\otelcol-config.yaml"
```
Install the new MSI as Step 1 of the MSI install shows, with the version the install guide names.

Put your config back:

Copy-Item "$env:TEMP\otelcol-config.yaml" 'C:\Program Files\OpenTelemetry Collector\config.yaml'

Then run Step 3 again.

When you run the agent manually, from the folder that contains config.yaml:

Stop the agent:
```
Stop-Process -Id (Get-Content otel-pid)
```
Download and extract the new tarball as Steps 1 and 2 of the manual install show, with the version the install guide names. config.yaml stays as it is.
Start the agent as Step 4 shows.

View the Collector's Logs

The agent writes its own logs to the systemd journal. To follow them, run this on the host:

sudo journalctl -u signoz-collector-agent -f

With the MSI, the service writes its logs to the Application event log under the otelcol-contrib source, which Event Viewer shows under Windows Logs > Application. To list the latest, run this in PowerShell:

Get-WinEvent -ProviderName otelcol-contrib -MaxEvents 50 | Select-Object TimeCreated, LevelDisplayName, Message

When you run the agent manually, it writes its logs to otelcol-output.log, in the folder you started it from. To follow them, run this there:

Get-Content otelcol-output.log -Wait -Tail 50

Troubleshooting

Metrics or logs stop after adding a receiver

After a change that adds a receiver to the metrics or logs pipeline, host metrics or host logs stop arriving in SigNoz.

Cause: the pipeline's receivers list no longer names hostmetrics or, on Linux, journald. In casting.yaml, a list replaces Foundry's whole list.

Fix: list every receiver the pipeline still needs, as Scrape Prometheus Metrics and Collect Application Log Files show, then apply the change as Apply Changes shows.

After the fix, host metrics and host logs arrive in SigNoz again.

A custom processor has no effect

After a change that adds a processor, its attribute or transformation is missing from the data in SigNoz.

Cause: the processor is defined under processors but not listed in a pipeline's processors.

Fix: list it before batch as Add Custom Processors to Pipelines shows, then apply the change as Apply Changes shows.

After the fix, the attribute or transformation appears on the signal in SigNoz Explorer.

A service's logs do not arrive

On Linux, a service's log lines appear in journalctl -u for the service's unit, but not in the SigNoz Logs Explorer.

Cause: the agent collects journal entries at priority warning and above, and the service writes its lines at a lower priority. Plain standard output and standard error are stored at priority info.

Fix: collect entries at priority info as Collect Host Logs shows, then apply the change as Apply Changes shows.

After the fix, new log lines from the service arrive in SigNoz.

VM Collection Agent - Configure

Prerequisites

Apply Changes

Send Data from Applications to Collector on VM

Where and how to send data from your application

Collect Host Metrics

Exclude filesystem mounts

Collect Host Logs

Disable Logs Collection

Collect Application Log Files

Other log sources

Resource Detection

Custom resource attributes

Scrape Prometheus Metrics

Tune Batch and Memory Limiter

Memory limit

Add Custom Processors to Pipelines

Common Deployment Scenarios

Run on the same host as self-hosted SigNoz

Migrate to Foundry

Upgrade the Collection Agent

View the Collector's Logs

Troubleshooting

Metrics or logs stop after adding a receiver

A custom processor has no effect

A service's logs do not arrive

Next Steps