Infinite Retention of OpenTelemetry Data in AWS S3

To back up all three signals, use named instances with different prefixes. Append these exporters and update each pipeline in your existing config:

exporters:
  awss3/logs:
    s3uploader:
      region: '<region>'
      s3_bucket: '<bucket-name>'
      s3_prefix: 'logs'
      compression: gzip
 
  awss3/metrics:
    s3uploader:
      region: '<region>'
      s3_bucket: '<bucket-name>'
      s3_prefix: 'metrics'
      compression: gzip
 
  awss3/traces:
    s3uploader:
      region: '<region>'
      s3_bucket: '<bucket-name>'
      s3_prefix: 'traces'
      compression: gzip

Then add each exporter to the corresponding pipeline in your existing service.pipelines section:

service:
  pipelines:
    logs:
      exporters: [otlp, awss3/logs]
    metrics:
      exporters: [otlp, awss3/metrics]
    traces:
      exporters: [otlp, awss3/traces]

• Named instances (awss3/logs, awss3/metrics, awss3/traces) allow a different s3_prefix per signal

The exporter supports several optional fields for production use:

Top-level options on the exporter (siblings of s3uploader):

• marshaler: output format. Default: otlp_json (queryable with Amazon Athena). otlp_proto is more compact but harder to query.

Options nested under s3uploader:

• compression: none (default), gzip, or zstd. Use gzip or zstd in production to reduce storage costs.
• s3_partition_format: strftime-based path for time partitioning. Default: year=%Y/month=%m/day=%d/hour=%H/minute=%M
• storage_class: S3 storage class. Default: STANDARD. Valid values: STANDARD, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, GLACIER, DEEP_ARCHIVE. Use STANDARD_IA or INTELLIGENT_TIERING for archival data to reduce costs (see AWS S3 storage classes).
• role_arn: IAM role ARN to assume via STS, instead of static credentials.
• endpoint: custom S3-compatible endpoint URL for MinIO, LocalStack, or other stores.
• s3_force_path_style: set to true when using non-AWS S3-compatible stores.

Example with all optional fields:

exporters:
  awss3/logs:
    marshaler: otlp_json
    s3uploader:
      region: '<region>'
      s3_bucket: '<bucket-name>'
      s3_prefix: 'logs'
      s3_partition_format: 'year=%Y/month=%m/day=%d/hour=%H/minute=%M'
      compression: gzip
      storage_class: STANDARD_IA
      role_arn: '<iam-role-arn>'
      endpoint: '<s3-compatible-endpoint>'
      s3_force_path_style: true

Replace the placeholders:

• <iam-role-arn>: ARN of the IAM role to assume (e.g., arn:aws:iam::123456789012:role/otel-s3-writer)
• <s3-compatible-endpoint>: endpoint URL for non-AWS S3-compatible stores (e.g., http://localhost:9000 for MinIO)

For the full list of configuration options, see the AWS S3 Exporter README.

The awss3 exporter works with any S3-compatible object store, for example MinIO, Cloudflare R2, or Tigris. Set endpoint to the store's URL and enable path-style requests:

exporters:
  awss3/logs:
    s3uploader:
      region: 'auto'
      s3_bucket: '<bucket-name>'
      s3_prefix: 'logs'
      compression: gzip
      endpoint: '<s3-compatible-endpoint>'
      s3_force_path_style: true

Verify these values:

• <bucket-name>: your bucket name
• <s3-compatible-endpoint>: your store's URL (e.g., https://t3.storage.dev for Tigris, http://localhost:9000 for a local MinIO)
• region: defaults to us-east-1. Set the value your store expects: auto for Tigris and Cloudflare R2, us-east-1 for a default MinIO deployment.

Two of the options above behave differently outside AWS:

• Provide credentials as a static key pair via the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. role_arn requires STS, which most S3-compatible stores do not implement.
• storage_class values are AWS-specific. Leave the default unless your store documents support for them.

To validate, point the AWS CLI at the same endpoint:

aws s3 ls s3://<bucket-name>/logs/ --recursive \
  --endpoint-url <s3-compatible-endpoint> | head

No objects appearing in S3

• Verify the exporter is listed in the pipeline's exporters array, not just defined in the exporters: section.
• Check that the Collector is receiving data on the pipeline (e.g., logs are flowing through the logs pipeline).

AccessDenied or NoCredentialProviders in Collector logs

• Verify your AWS credentials are available to the Collector process (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY environment variables, or an attached IAM role).
• Confirm the IAM policy includes s3:PutObject permission on the target bucket.

NoSuchBucket error in Collector logs

• Verify the s3_bucket name and region match your actual S3 bucket.

Collector fails to start with config error

• Confirm you are using the otelcol-contrib distribution, which includes the awss3 exporter. The core otelcol distribution does not include it.
• Check YAML indentation and that exporter names use the type/name format with a forward slash (e.g., awss3/logs, not awss3logs).

SignatureDoesNotMatch or 301 Moved Permanently with an S3-compatible store

• Set region to the value your store expects. Some stores validate the request signing region.
• Confirm s3_force_path_style: true is set. Most non-AWS stores expect path-style requests.