JSON Logs Migration Guide

📌 Overview

SigNoz is moving log bodies from a string column (body) to native JSON storage (body_v2). Every field in the body becomes queryable by path, so log queries are faster and you can filter, group by and aggregate on body fields without a pipeline.

Track this migration in SigNoz/signoz#13376 for more details.

🔍 Why We Are Migrating

String bodies are parsed at query time on every matching row. With ClickHouse's native JSON type, log queries ran about 30% faster and scanned about 99% less data in our benchmarks (deep dive).

📦 Dual Ingestion

During the transition, the collector writes each log body to both columns. Existing dashboards, alerts and saved views keep reading body, so you don't need to do anything yet.

⚙️ Resource Impact

Plan for this extra capacity before upgrading.

To estimate the extra disk, run this on ClickHouse. The logs table will grow by about body_pct:

SELECT
    formatReadableSize(sumIf(column_data_compressed_bytes, column = 'body')) AS body_size,
    formatReadableSize(sum(column_data_compressed_bytes)) AS table_size,
    round(100 * sumIf(column_data_compressed_bytes, column = 'body') / sum(column_data_compressed_bytes), 1) AS body_pct
FROM system.parts_columns
WHERE database = 'signoz_logs' AND table = 'logs_v2' AND active;

The increase builds up gradually and reaches its full size one retention period after the upgrade.