Upgrade to v0.19 from earlier versions

Before upgrading to v0.19, you need to run the migration script to sanitise the alerts and dashboards data.

Steps to run migration script

Command-Line Interface (CLI) Flags

There are is only one flag in the migrate binary:

  • --dataSource : Data Source path. default=signoz.db

For Docker

cd to SigNoz repository and run following commands:

cd deploy/docker/clickhouse-setup

docker run -it -v $PWD/data/signoz/signoz.db:/signoz.db signoz/migrate:0.19

Output should be similar as below:

Data Source path:  signoz.db
2023/05/20 15:28:22 Total Dashboard found: 2
2023/05/20 15:28:22 625fa391-d9d3-47c1-809a-1a147eea229d
2023/05/20 15:28:22 b05af383-23ec-4061-8f57-0765d45ccd51
2023/05/20 15:28:22 Dashboard 625fa391-d9d3-47c1-809a-1a147eea229d updated
2023/05/20 15:28:22 Dashboard b05af383-23ec-4061-8f57-0765d45ccd51 updated
2023/05/20 15:28:22 Dashboards migrated
2023/05/20 15:28:22 Migrating 1 rules
2023/05/20 15:28:22 Migrating rule 1
2023/05/20 15:28:22 Migrated 1 rules

At last, trigger a restart of the query-service container:

docker restart query-service

For Docker Swarm

cd to SigNoz repository and run following commands:

cd deploy/swarm/clickhouse-setup

docker run -it -v $PWD/data/signoz/signoz.db:/signoz.db signoz/migrate:0.19

Output should be similar as below:

Data Source path:  signoz.db
2023/05/20 15:28:22 Total Dashboard found: 2
2023/05/20 15:28:22 625fa391-d9d3-47c1-809a-1a147eea229d
2023/05/20 15:28:22 b05af383-23ec-4061-8f57-0765d45ccd51
2023/05/20 15:28:22 Dashboard 625fa391-d9d3-47c1-809a-1a147eea229d updated
2023/05/20 15:28:22 Dashboard b05af383-23ec-4061-8f57-0765d45ccd51 updated
2023/05/20 15:28:22 Dashboards migrated
2023/05/20 15:28:22 Migrating 1 rules
2023/05/20 15:28:22 Migrating rule 1
2023/05/20 15:28:22 Migrated 1 rules

At last, trigger a restart of the query-service container:

docker restart query-service
✅ Info

In case of multi node swarm cluster, run the above commands in the node where query-service is running. To find out which node: docker service ps query-service.

For Kubernetes

To download migrate binary:

wget https://github.com/signoz/signoz-db-migrations/releases/download/v0.19/migrate-v0.19-linux-amd64 -O migrate

sudo chmod +x migrate

To copy the binary in persistent volume path /var/lib/signoz in query-service:

kubectl cp -n platform ./migrate my-release-signoz-query-service-0:/var/lib/signoz/migrate

To exec into the query-service container:

kubectl -n platform exec -it pod/my-release-signoz-query-service-0 -- sh

Now, change directory to the /var/lib/signoz and run the migration script:

cd /var/lib/signoz

./migrate

You should see output similar to this:

Data Source path:  signoz.db
2023/05/20 15:28:22 Total Dashboard found: 3
2023/05/20 15:28:22 625fa391-d9d3-47c1-809a-1a147eea229d
2023/05/20 15:28:22 b05af383-23ec-4061-8f57-0765d45ccd51
2023/05/20 15:28:22 e730bcd5-5319-4cab-8de7-82edd5f48c72
2023/05/20 15:28:22 Dashboard 625fa391-d9d3-47c1-809a-1a147eea229d updated
2023/05/20 15:28:22 Dashboard b05af383-23ec-4061-8f57-0765d45ccd51 updated
2023/05/20 15:28:22 Dashboard e730bcd5-5319-4cab-8de7-82edd5f48c72 updated
2023/05/20 15:28:22 Dashboards migrated
2023/05/20 15:28:22 Migrating 1 rules
2023/05/20 15:28:22 Migrating rule 1
2023/05/20 15:28:22 Migrated 1 rules

Upgrade to v0.19

Follow the platform specific instructions to upgrade to 0.19 and above.

⚠️ Warning

Prior to upgrading to v0.19, you need to run the migration script.

In case you upgrade and don't run the migration script, you might run into query-service pod crash loop. To solve this, follow the instructions in the section below.

Issue - query-service pod is crashing (Kubernetes)

In case you upgraded to v0.19 prior to running the migration script in Kubernetes, you will see query-service pod is crashing due to invalid alerts data.

To solve this, you will have to use migration init container in query-service pod to run the migration script and then restart the pod.

Follow the steps below:

  1. Make sure you have latest chart information from the Helm repositories:

    helm repo update
    
  2. Include the following in override-values.yaml file:

    queryService:
      initContainers:
        migration:
          enabled: true
          command:
            - sh
            - -c
            - |
              echo "Running migration"
              wget https://github.com/signoz/signoz-db-migrations/releases/download/v0.19/migrate-v0.19-linux-amd64 -O migrate
              chmod +x migrate
              ./migrate --dataSource /var/lib/signoz/signoz.db
              echo "Migration completed"
    
  3. Run the following command to upgrade the chart:

    helm -n platform upgrade my-release signoz/signoz -f override-values.yaml
    
  4. Wait for the migration init container to complete and then restart the query-service pod:

    kubectl -n platform rollout restart sts -l app.kubernetes.io/component=query-service
    
  5. (Optional) Once the query-service pod is running, you can delete the migration init container from the override-values.yaml file which you added in step 2, followed by running the following command to upgrade the chart:

    helm -n platform upgrade my-release signoz/signoz -f override-values.yaml