Parsing Logs with Regex Parser

Logs are unstructured data, and it is not easy to analyze them. SigNoz provides easy to use Regex Parser to parse unstructured logs and extract information from them using Logs Pipelines.

Prerequisites

• You are sending logs to SigNoz.
• You have access to the Logs Pipelines section in SigNoz UI.

Learn more about SigNoz Logs Pipelines.

About Regex Parser

The Regex processor uses Regular Expressions (based on Go's RE2 syntax) to extract information from unstructured logs. It uses Named Capture Groups to map parts of your log message to attribute names.

The Concept:

You define a pattern containing groups like (?P<attribute_name>pattern). Whatever matches inside the parentheses becomes the value of attribute_name.

Example Log

Throughout this guide, we'll use the following example of a legacy application log entry:

2025-12-27 10:00:00 [INFO] User 123 logged in from 192.168.1.5

Desired Outcome

After parsing, we want to extract specific information into structured attributes.

The final processed log should look like this:

{
  "body": "2025-12-27 10:00:00 [INFO] User 123 logged in from 192.168.1.5",
  "attributes": {
    "level": "INFO",
    "user_id": "123",
    "ip": "192.168.1.5"
  }
}

Creating a Pipeline

Before you can parse logs, you need to create a pipeline that will filter and process them.

Step 1: Navigate to Pipelines Page

In SigNoz, go to Logs → Pipelines

Step 2: Create a New Pipeline

• If you do not have existing pipelines, press the "New Pipeline" button.

• If you already have some pipelines, press the "Enter Edit Mode" button and then click the "Add a New Pipeline" button at the bottom of the list.

Step 3: Configure the Pipeline

Provide details about the pipeline in the Create Pipeline Dialog:

• Name: Provide a descriptive pipeline name.
• Description: Add a detailed description for your pipeline (optional).
• Filter: Use the filter field to select the logs you want to process. For example, service.name = my-app or log.file.name = app.log.
• Filtered Logs Preview: Verify that the logs you want to process are selected.

Press the "Create" button to create the pipeline.

Configuring Regex Parser

Now that we have a pipeline, we will add the Regex Parser processor to extract attributes from our log line.

Step 1: Add a Processor

Expand your pipeline and click the "Add Processor" button.

Step 2: Configure Regex Parser

We will now configure the processor to parse the log message.

Before Parsing

At this stage, your log typically contains the raw message in the body field:

{
  "body": "2025-12-27 10:00:00 [INFO] User 123 logged in from 192.168.1.5"
}

Processor Configuration

Select Regex Parser and configure the fields:

• Name: parse_app_log
• Parse From: body (This is where the raw message resides)
• Parse To: attributes (We want to add extracted values directly to log attributes)
• Pattern:

  ^.*\[(?P<level>\w+)\].*User (?P<user_id>\d+) logged in from (?P<ip>[\d\.]+)
  

Press "Create" to add the processor.

After Parsing

Once the processor is applied, the log structure changes. The captured groups from the regex are added as keys in the attributes object.

{
  "body": "2025-12-27 10:00:00 [INFO] User 123 logged in from 192.168.1.5",
  "attributes": {
    "level": "INFO",
    "user_id": "123",
    "ip": "192.168.1.5"
  }
}

Key Observations

• The original body remains unchanged.
• New fields level, user_id, and ip are now available in attributes, making them queryable in SigNoz.

Simulate and Validate

Before deploying, use the Simulate feature to test your parser:

1. Click the "eye" icon in the actions column for the pipeline to open the Pipeline Preview Dialog.

2. Press the "Simulate Processing" button to see the output.

3. Click the expand button next to processed log line to view the log details.

4. Verify that the extracted attributes (level, user_id, ip) appear correctly.

Deploy the Pipeline

After verifying the simulated logs in Pipeline Preview Dialog, your pipeline is ready.

1. Press the Save Configuration button at the bottom of the pipelines list. This will store the latest state of your pipelines and deploy them for pre-processing.

2. You can track the deployment status using the Change History tab at the top of pipelines.

Final Output

Once deployed, your incoming logs will be transformed automatically.

{
  "body": "2025-12-27 10:00:00 [INFO] User 123 logged in from 192.168.1.5",
  "attributes": {
    "level": "INFO",
    "user_id": "123",
    "ip": "192.168.1.5"
  }
}

Related Guides

• Parsing JSON logs with JSON Parser
• Parsing Unstructured Logs with Grok Parser

Get Help

If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack.

If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.