Configure PagerDuty Channel

Prerequisites

Before setting up PagerDuty as a notification channel in SigNoz, ensure the following:

• Valid Integration Key: You have a valid Integration Key (also known as Routing Key) from PagerDuty.
• SigNoz Version: v0.8.0 or later

There are two main ways to integrate with PagerDuty:

• Global Event Orchestration: Ideal for automating incident creation and management.
• Direct Integration on PagerDuty service: Suitable for specific service-level integrations.

Accessing Alert Channels

To manage your alert channels in SigNoz:

• Navigate to Settings > Alert Channels tab within SigNoz. This tab displays a list of configured alert channels.

  

Obtaining Integration or Routing key

For Global Event Orchestration

1. From the Automation menu, select Event Orchestration
2. Create a new orchestration
3. Click on Global Orchestration Key and copy your integration key and keep it safe for later use.

For PagerDuty Service Integration

1. Navigate to Services > Service Directory and select the service where you’d like to add the integration.
2. Select Integration tab and click Add another integration
3. Select Events API V2 from the list
4. Click Add
5. Locate your integration in the list and click down arrow to view and copy integration key.

For more details on PagerDuty service setup, visit PagerDuty Service and Integrations.

Create a New PagerDuty Channel

To create a new PagerDuty notification channel in SigNoz, follow these steps:

• Navigate to Settings > Alert Channels and click on New Channel.
• Enter a Name for the channel and select PagerDuty as the channel type.
• Routing Key: Enter Routing Key (Integration Key) obtained from PagerDuty.
• More information: Enter more information as necessary. Refer to the Events API V2 Overview for more details.

Test Configuration: Click the Test button to test the connection with your application.

Testing the PagerDuty channel

1. Let's create a simple alert rule that monitors average CPU performance for each host. Go to Alerts page in your SigNoz app and click New Alert button. When the new alert page opens, edit metric query as shown below. Feel free to choose any other metric, the idea is to pick a metric with sufficient data to raise an alert.

   

2. We can now review the graph to identify a threshold that will definitely cause an alert. Here, anything below 0.2 looks like a good condition for threshold.

   

3. Let's set threshold to 0.12 to be sure that alert will be raised in next few minutes.

   

4. Save the alert rule. Feel free to edit severity and labels as necessary.

5. Go to your PagerDuty Alerts Dashboard (PagerDuty Home >> Incident >> Alerts) and wait for a few minutes. If all goes well, you will see an incident. You may have to refresh the page few times to see the alert.

   
   

Turn-off Resolved alerts

The resolve alerts toggle is switched on by default. To turn-off receiveing the Resolved alerts for your Notification Channel, follow these steps:

• Navigate to Settings > Alert Channels
• Click on Edit button for a particular alert.
• Toggle off the Send resolved alerts button and hit Save.

Troubleshooting

If you encounter issues with the PagerDuty integration:

• Check the Integration Key: Ensure that the Routing Key or Integration Key is correctly entered in SigNoz.
• Verify PagerDuty Configuration: Confirm that the PagerDuty service or global event orchestration is set up correctly.
• Test Connectivity: Use the Test button in SigNoz to verify connectivity with PagerDuty. If the test fails, review your network settings and integration key.