For the complete documentation index, see llms.txt. Markdown versions are available by appending .md to documentation URLs.

VPC Flow Logs: Setup, Analysis, and Monitoring

Last Updated: November 02, 202627 min read

VPC Flow Logs is a powerful, built-in network monitoring feature provided by cloud platforms (AWS, GCP, Azure), designed to capture detailed metadata about IP traffic flowing through your network interfaces. By enabling seamless logging of accepted and rejected traffic flows, it empowers administrators to troubleshoot connectivity issues, detect anomalies, and comply with regulatory requirements, all without the need for additional agents or hardware.

Each flow log entry (a flow log record) documents a single network flow, characterized by source IP, destination IP, source port, destination port, and protocol. These records are aggregated over a configurable time window (called the aggregation interval) and then delivered to a destination of your choice for storage and analysis. Flow log data is collected outside the path of your network traffic, enabling them has zero impact on your network throughput or latency. VPC Flow Logs don't slow down your instances or degrade application performance.

Why Do You Need VPC Flow Logs?

Before VPC flow logging, teams often relied on host-based tooling or third-party network sensors, which used to increase operational overhead and usually lacks VPC-wide coverage. This approach was complex, added overhead to each instance, and provided a limited view restricted to traffic visible at the instance level. VPC Flow Logs eliminate all of that by providing centralized, agent-free visibility at three levels: the entire VPC, a specific subnet, or an individual network interface (ENI).

Here are the practical reasons to use VPC Flow Logs:

Security Monitoring and Threat Detection

VPC Flow Logs help you detect port scanning, network enumeration attempts, lateral movement after a compromised host, and data exfiltration. If an attacker communicates with a compromised system on a non-standard port (say, using SMTPS for an HTTPS request), flow logs catch it.

Troubleshooting Connectivity Issues

When an application can't reach a database or a service times out, flow logs help you pinpoint whether the traffic is being rejected by a security group, blocked by a network ACL, or it is not reaching the intended destination at all.

Compliance Monitoring

Many compliance standards, like PCI DSS and HIPAA, require organizations to monitor and log network traffic. VPC Flow Logs provide the comprehensive record needed for auditing and compliance reporting. AWS itself recommends enabling VPC Flow Logs for every VPC as a security best practice.

Cost Optimization

By analyzing traffic patterns, you can identify top talkers in your network, discover unnecessary cross-AZ traffic that's inflating your bill, and understand how your network resources are actually being utilized.

How do VPC Flow Logs Work?

The architecture behind VPC Flow Logs involves three stages: Capture, Aggregate, and Deliver.

A high-level overview of the VPC Flow Logs architecture
A high-level overview of the VPC Flow Logs architecture

1. Capture

When you enable VPC Flow Logs, the cloud provider begins capturing metadata about IP traffic to/from the network interface level. On AWS, the flow log service captures connection-level metadata for every IP traffic flow passing through the ENIs (Elastic Network Interface) within your configured scope (VPC, subnet, or individual ENI).In practice, not all traffic types are logged, and in rare cases records can be skipped (log-status = SKIPDATA). The capture happens outside the data path, so there is no performance penalty.

2. Aggregate

Captured flow metadata is aggregated into flow log records (source IP, destination IP, source port, destination port, protocol) and aggregated within a time window called the aggregation interval. By default, this interval is 10 minutes on AWS. You can reduce it to 1 minute for more granular data, but this significantly increases the volume (and cost) of log records.

On Nitro-based instances (most modern EC2 instance types), the aggregation interval is always 1 minute or less, regardless of the configured setting.

3. Deliver

After aggregation, the flow log records are delivered to your configured destination. On AWS, you have three options:

DestinationBest For
Amazon CloudWatch LogsReal-time analysis, quick queries using CloudWatch Insights, metric filters and alerts
Amazon S3Long-term archival, cost-effective storage, big data analysis with Athena
Amazon Data FirehoseStreaming to third-party tools (SigNoz, Datadog, Splunk, etc.), real-time processing pipelines

VPC Flow Logs Across Different Cloud Providers

While this guide primarily focuses on AWS, VPC Flow Logs are available across all major cloud providers. Here's a quick comparison:

FeatureAWS VPC Flow LogsGCP VPC Flow LogsAzure NSG Flow Logs
ScopeVPC, Subnet, ENIVPC Network, Subnet, VLAN Attachment, VPN TunnelNetwork Security Group
Aggregation Interval1 min or 10 min5 sec (default), 30 sec, 1 min, 5 min, 10 min, 15 min1 min
Log DestinationsCloudWatch, S3, FirehoseCloud LoggingAzure Storage (Blob) (primary). Optional downstream analytics/export (e.g., Traffic Analytics / SIEM tooling).
SamplingAll flows capturedConfigurable (50% default, adjustable)All flows captured
FormatSpace-delimited or ParquetJSONJSON
Container-level visibilityYes (ECS, Version 7)Yes (GKE nodes)Yes (AKS Container Network Logs / Advanced Container Networking Services)

GCP's VPC Flow Logs work differently in a few important ways. They use a dynamic primary sampling rate that varies with host load. You also get a configurable secondary sampling rate (default 50%), meaning not all flows are guaranteed to be logged. The tradeoff is lower cost for high-traffic environments.

Azure has evolved from NSG Flow Logs (which log at the security group level) to the newer VNet Flow Logs, which provide virtual network-level logging, closer to what AWS and GCP offer at the VPC level.

VPC Flow Logs Format: Understanding the Record Structure

A flow log record is a space-delimited string, with each field representing a component of the network flow. Understanding the format is critical for writing queries, building dashboards, and debugging network issues.

Default Format (Version 2)

The default format includes 14 fields:

<version> <account-id> <interface-id> <srcaddr> <dstaddr> <srcport> <dstport> <protocol> <packets> <bytes> <start> <end> <action> <log-status>

Here's what a real flow log record looks like:

2 123456789012 eni-abcdef01 10.0.1.1 203.0.113.5 54321 80 6 10 3456 1510868790 1510868850 ACCEPT OK

Let's break down each field:

FieldValue in ExampleWhat it Means
version2Flow log format version
account-id123456789012Your AWS account ID
interface-ideni-abcdef01The ENI where traffic was captured
srcaddr10.0.1.1Source IP address
dstaddr203.0.113.5Destination IP address
srcport54321Source port
dstport80Destination port (HTTP)
protocol6IANA protocol number (6 = TCP, 17 = UDP, 1 = ICMP)
packets10Number of packets in the aggregation interval
bytes3456Number of bytes in the aggregation interval
start1510868790Start time of the capture window (Unix epoch)
end1510868850End time of the capture window (Unix epoch)
actionACCEPTWhether traffic was accepted or rejected
log-statusOKLogging status (OK, NODATA, or SKIPDATA)

Custom Format Fields (Version 3+)

The default format gives you the basics, but AWS has introduced additional fields across multiple versions that provide significantly deeper visibility. You access these by choosing a custom format when creating the flow log.

When using custom format, the ${version} field is automatically set to the highest version number among all the fields you include, you don't manually specify the version. For example, if you include reject-reason (v8), your log records will show version = 8.

Version 3 fields (VPC/Subnet/Instance context):

FieldPurpose
vpc-idIdentifies which VPC the traffic belongs to
subnet-idIdentifies the subnet
instance-idMaps traffic to a specific EC2 instance
tcp-flagsBitmask of TCP flags (SYN, ACK, FIN, RST), it is critical for understanding connection state
typeIPv4 or IPv6
pkt-srcaddrOriginal packet source IP (important when traffic passes through NAT or load balancers)
pkt-dstaddrOriginal packet destination IP

Version 4 fields (Location awareness):

FieldPurpose
regionAWS region of the flow
az-idAvailability Zone ID
sublocation-typeType of sublocation (e.g., wavelength)
sublocation-idID of the sublocation

Version 5 fields (Service and direction awareness):

FieldPurpose
pkt-src-aws-serviceAWS service name for the source (e.g.,S3, DYNAMODB)
pkt-dst-aws-serviceAWS service name for the destination
flow-directionWhether traffic is ingress or egress
traffic-pathThe path traffic takes to exit (e.g., through internet gateway, NAT gateway, VPC peering)

Version 7 fields (ECS container visibility):

Version 7, introduced in 2024, adds 10 new fields specific to Amazon ECS workloads, giving you visibility into container-level traffic for the first time, including task IDs, cluster ARNs, container instance IDs, and service names.

Version 8+ fields (Enhanced troubleshooting)

FieldVersionPurpose
reject-reason8Reason traffic was rejected (currently BPA or EC; ‘-’ otherwise).
resource-id9Resource ID associated with the flow (e.g., NAT gateway ID, transit gateway ID)
encryption-status10Encryption status for the flow

If you're setting this up for the first time, here is a recommended custom format that balances depth and cost. It covers security, troubleshooting, and cost optimization use cases:

${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${vpc-id} ${subnet-id} ${instance-id} ${tcp-flags} ${type} ${pkt-srcaddr} ${pkt-dstaddr} ${flow-direction} ${traffic-path}

Add pkt-src-aws-service and pkt-dst-aws-service if you need to trace traffic to specific AWS services.

Log File Format: Plain Text vs Parquet

When publishing to S3, you can choose between two file formats:

  • Plain Text: The default. Human-readable, space-delimited records. Simple but slower to query at scale.
  • Apache Parquet: A columnar data format. AWS documentation states that queries on Parquet data are 10 to 100 times faster compared to plain text, and the format compresses better, reducing storage costs significantly.

For any production deployment publishing to S3, use Parquet.

VPC Flow Logs Examples: Reading and Interpreting Records

Understanding how to read flow log records is essential for effective troubleshooting. Here are the most common patterns and what they mean:

Example 1: Security Group Rejecting Inbound Traffic

2 123456789012 eni-abcdef01 203.0.113.50 10.0.1.15 49152 3389 6 20 4000 1510868790 1510868850 REJECT OK

This record shows someone from an external IP 203.0.113.50 trying to connect to your instance on port 3389 (RDP). The traffic was rejected, which means your security group or network ACL is working correctly. However, if you see hundreds of these from different source IPs, it likely indicates a brute-force or port scanning attempt.

Example 2: Accepted Traffic to an Unexpected Port

2 123456789012 eni-abcdef01 10.0.1.15 198.51.100.25 43567 8443 6 15 5200 1510868790 1510868850 ACCEPT OK

Your instance at 10.0.1.15 is sending traffic to an external IP on port 8443. If this isn't expected (for example, if your application should only communicate with known API endpoints on port 443), it could indicate a compromised instance or a misconfigured application.

Example 3: NODATA Records

2 123456789012 eni-abcdef01 - - - - - - - 1510868790 1510868850 - NODATA

No traffic was recorded for this ENI (Network Interface) during the aggregation window. This is normal for idle interfaces.

Example 4: Custom Format with Flow Direction and AWS Services

Using version 5 custom format, here's a record showing an EC2 instance calling Amazon S3:

5 123456789012 eni-abcdef01 10.0.1.15 52.217.33.14 43567 443 6 15 5200 1510868790 1510868850 ACCEPT OK vpc-abc123 subnet-def456 i-0123456789 2 IPv4 10.0.1.15 52.217.33.14 AMAZON S3 egress 8

The traffic-path value of 8 indicates the traffic went through an internet gateway. The pkt-dst-aws-service shows it's going to S3. If you're expecting this instance to access S3 via a VPC endpoint (private path), this record reveals it's actually going over the public internet, a common misconfiguration that's both slower and more expensive.

Example 5: TCP Flags for Connection Analysis

When using a custom format with tcp-flags, you can track the full lifecycle of TCP connections:

  • SYN (2): Connection initiation, a client is trying to connect.
  • SYN-ACK (18): Server acknowledging the connection.
  • FIN (1): Graceful connection close.
  • RST (4): Abrupt connection termination, it often indicates a problem.

If you see many SYN packets from a single source, but no SYN-ACK responses, it's likely a SYN flood or port scan.

How to Set Up VPC Flow Logs in AWS

Now that you can read a flow log record, enable logging for your own AWS resources. Choose S3 to query stored files with Athena, or CloudWatch Logs to search records with Logs Insights. Complete only the setup for your chosen destination. For CLI steps, configure AWS credentials and select the region containing your VPC.

Create an S3 bucket where AWS will deliver your flow log files.

Step 1: Create an S3 bucket

You can create the bucket via the AWS Console or CLI. Choose a globally unique bucket name. If the name is taken, try adding your account ID or a random number to the name.

aws s3 mb s3://YOUR_BUCKET_NAME --region YOUR_REGION

Step 2: Attach bucket policy

When you create a flow log via the AWS Console and own bucket, AWS automatically attaches the required bucket policy. You don't need to do anything manually. However, if you're creating the flow log via CLI/Terraform, or using a bucket in a different AWS account, you need to manually add this policy. Save the following as bucket-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AWSLogDeliveryWrite",
      "Effect": "Allow",
      "Principal": {
        "Service": "delivery.logs.amazonaws.com"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control",
          "aws:SourceAccount": "YOUR_ACCOUNT_ID"
        }
      }
    },
    {
      "Sid": "AWSLogDeliveryAclCheck",
      "Effect": "Allow",
      "Principal": {
        "Service": "delivery.logs.amazonaws.com"
      },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
    }
  ]
}

Replace before using:

  • YOUR_BUCKET_NAME - Your actual bucket name (e.g., my-vpc-flow-logs-bucket-0210)
  • YOUR_ACCOUNT_ID - Your AWS account ID (find it by running aws sts get-caller-identity)

Apply the policy:

aws s3api put-bucket-policy --bucket YOUR_BUCKET_NAME --policy file://bucket-policy.json

With the bucket and policy in place, you’re ready to create the flow log itself. You can now skip to How to Enable VPC Flow Logs.

Option 2: Set Up CloudWatch Logs Destination

Create a CloudWatch Logs log group (a container for your flow logs) and an IAM role that gives the VPC Flow Logs service permission to write to it.

Step 1: Create a log group

aws logs create-log-group --log-group-name /vpc/flowlogs/my-vpc

Step 2: Create an IAM Role for flow log delivery

VPC Flow Logs needs permission to write to your CloudWatch log group. When you create the flow log via the Console (covered in the next section), AWS will auto-generate a role with the correct permissions. Your user or role also needs iam:PassRole permission for the delivery role.

For CLI setup, create the role manually:

  1. Create a file called flow-logs-trust-policy.json:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "vpc-flow-logs.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
  2. Create the IAM role:

    aws iam create-role \
      --role-name VPCFlowLogsRole \
      --assume-role-policy-document file://flow-logs-trust-policy.json
  3. Create flow-logs-permissions.json:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "logs:CreateLogGroup",
            "logs:CreateLogStream",
            "logs:PutLogEvents",
            "logs:DescribeLogGroups",
            "logs:DescribeLogStreams"
          ],
          "Resource": "*"
        }
      ]
    }
  4. Attach the permissions to the role:

    aws iam put-role-policy \
      --role-name VPCFlowLogsRole \
      --policy-name VPCFlowLogsPolicy \
      --policy-document file://flow-logs-permissions.json

With the log group and delivery role ready, create the flow log using either the Console or CLI below.

How to Enable VPC Flow Logs in AWS?

Now that your destination is ready, it's time to create the flow log itself. VPC Flow Logs can be enabled at three different levels:

LevelScopeWhen to Use
VPCAll ENIs in the VPCBroad security monitoring, compliance
SubnetAll ENIs in the subnetFocused monitoring on specific tiers (e.g., database subnet)
ENISingle network interfaceDebugging a specific instance or service

We will start with VPC-level flow logs. This covers the network interfaces in that VPC, and you can filter the recorded traffic later in your queries.

The AWS Console provides the most user-friendly way to enable VPC Flow Logs. The steps differ slightly based on whether you're using S3 or CloudWatch Logs as your destination.

For S3 destination:

  1. Open the Amazon VPC Console
  2. In the left sidebar, click Your VPCs
  3. Select the VPC you want to monitor (check the box next to it)
  4. Click Actions → Create flow log
  5. Fill in the form:
    • Name: my-vpc-flow-log (optional but helpful)
    • Filter: Select All (captures both accepted and rejected traffic)
    • Maximum aggregation interval: Select 10 minutes (good balance of detail vs. cost) or 1 minute for more granular data
    • Destination: Select Send to an Amazon S3 bucket
    • S3 bucket ARN: Enter arn:aws:s3:::YOUR_BUCKET_NAME (e.g., arn:aws:s3:::my-vpc-flow-logs-bucket-0210)
    • Log record format: Select AWS default format (14 fields)
    • Log file format: Select Parquet (compressed format, faster queries, lower cost)
    • Partition logs by time: Select Every 24 hours (default)
  6. Click Create flow log

For CloudWatch Logs destination:

  1. Follow steps 1-4 above
  2. Fill in the form:
    • Name: my-vpc-flow-log-cw
    • Filter: Select All
    • Maximum aggregation interval: Select 1 minute (for a shorter aggregation window)
    • Destination: Select Send to CloudWatch Logs
    • Destination log group: Enter /vpc/flowlogs/my-vpc
    • Service access: Select "Create and use a new service role". AWS auto-generates a role name like VPCFlowLogs-Cloudwatch-XXXXXXXXX and creates it with the correct permissions when you click Create. (If you already created a role manually, select "Use an existing service role" and pick VPCFlowLogsRole from the dropdown.)
    • Log record format: Select AWS default format
  3. Click Create flow log

After creation: Typically, logs arrive in ~5 minutes to CloudWatch Logs and ~10 minutes to S3 (delays can be longer). In some cases it can take 10+ minutes for the log group/data to appear.

Method 2: Enable via AWS CLI

For these steps, you need AWS CLI installed. Configure it with credentials that can create flow logs and manage the chosen destination.

  1. Find your VPC ID first

    This lists all your VPCs. Copy the VPC ID you want to monitor (e.g., vpc-07810113181a81357).

    aws ec2 describe-vpcs --query 'Vpcs[*].[VpcId,CidrBlock,IsDefault]' --output table
  2. Create flow log to S3 (with Parquet)

    For an S3 destination, run:

    aws ec2 create-flow-logs \
      --resource-type VPC \
      --resource-ids vpc-YOUR_VPC_ID \
      --traffic-type ALL \
      --log-destination-type s3 \
      --log-destination arn:aws:s3:::YOUR_BUCKET_NAME \
      --max-aggregation-interval 60 \
      --destination-options FileFormat=parquet
  3. Create flow log to CloudWatch Logs

    For a CloudWatch Logs destination, run this command instead:

    aws ec2 create-flow-logs \
      --resource-type VPC \
      --resource-ids vpc-YOUR_VPC_ID \
      --traffic-type ALL \
      --log-destination-type cloud-watch-logs \
      --log-group-name /vpc/flowlogs/my-vpc \
      --deliver-logs-permission-arn arn:aws:iam::YOUR_ACCOUNT_ID:role/VPCFlowLogsRole \
      --max-aggregation-interval 60
  4. Replace these values

    PlaceholderReplace withHow to find it
    vpc-YOUR_VPC_IDYour VPC IDRun aws ec2 describe-vpcs
    YOUR_BUCKET_NAMEYour S3 bucket nameThe bucket you created in prerequisites
    YOUR_ACCOUNT_IDYour 12-digit AWS account IDRun aws sts get-caller-identity
  5. Verify the flow log was created

aws ec2 describe-flow-logs --output table

How to Search VPC Flow Logs with CloudWatch Insights

Once records reach CloudWatch Logs, use Logs Insights to find rejected traffic or identify the hosts transferring the most data. The queries below use the automatically discovered fields for the AWS default flow log format.

How to Open CloudWatch Insights

  1. Open the CloudWatch Console
  2. In the left sidebar, click Logs → Logs Insights
  3. In the Select log group(s) dropdown, choose your flow log group (e.g., /vpc/flowlogs/my-vpc)
  4. Set the time range in the top-right corner (e.g., last 1 hour)
  5. Paste a query from below into the query editor
  6. Click Run query

Useful Queries

1. Verify logs are being ingested (run this first):

fields @timestamp, srcAddr, dstAddr, srcPort, dstPort, action
| sort @timestamp desc
| limit 10

This shows the 10 most recent flow log records. If you see results, your logs are working. If there are no results, check the selected time range and allow for delivery delays before checking the flow log delivery status.

2. Count rejected SSH flow records by source IP:

filter action = "REJECT" and dstPort = 22
| stats count(*) as rejectedFlows by srcAddr
| sort rejectedFlows desc
| limit 25

3. Top talkers — who's sending the most data:

filter action = "ACCEPT"
| stats sum(bytes) as totalBytes by srcAddr, dstAddr
| sort totalBytes desc
| limit 20

4. Find traffic outside an expected port list:

Adjust the port list to match your applications. Traffic outside this list is worth checking, but is not necessarily malicious.

filter action = "ACCEPT" and dstPort not in [80, 443, 22, 3306, 5432, 6379, 53]
| stats count(*) as flowRecords, sum(bytes) as totalBytes by srcAddr, dstAddr, dstPort
| sort flowRecords desc
| limit 25

5. Rejected traffic over time (useful for dashboards):

filter action = "REJECT"
| stats count(*) as rejectedCount by bin(5m)

6. Find all traffic involving a specific IP:

filter srcAddr = "203.0.113.50" or dstAddr = "203.0.113.50"
| sort @timestamp desc
| limit 100

How to Query VPC Flow Logs with Amazon Athena

For flow logs stored in S3, Amazon Athena lets you run standard SQL queries against your log data. Use it when you need SQL analysis of stored flow logs.

Step 1: Open Athena

  1. Open the Athena Console
  2. If this is your first time, Athena will ask you to set a query result location. Click Settings → Manage, then enter an S3 path like s3://YOUR_BUCKET_NAME/athena-results/ and click Save
  3. Make sure you're in the Query editor tab

Step 2: Create the Athena Table

Paste the following SQL into the Athena query editor and click Run. This table reads the default 14 fields from the Parquet files created above. Its partition projection follows the default daily S3 layout, with Hive-compatible prefixes disabled. If you chose a bucket prefix or a different partition layout, adjust the paths to match your files.

CREATE EXTERNAL TABLE IF NOT EXISTS vpc_flow_logs_parquet (
  version int,
  account_id string,
  interface_id string,
  srcaddr string,
  dstaddr string,
  srcport int,
  dstport int,
  protocol bigint,
  packets bigint,
  bytes bigint,
  start bigint,
  `end` bigint,
  action string,
  log_status string
)
PARTITIONED BY (
  `aws-account-id` string,
  `aws-service` string,
  `aws-region` string,
  `year` string,
  `month` string,
  `day` string
)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION 's3://YOUR_BUCKET_NAME/AWSLogs/'
TBLPROPERTIES (
  "projection.enabled" = "true",
  "projection.aws-account-id.type" = "enum",
  "projection.aws-account-id.values" = "YOUR_ACCOUNT_ID",
  "projection.aws-service.type" = "enum",
  "projection.aws-service.values" = "vpcflowlogs",
  "projection.aws-region.type" = "enum",
  "projection.aws-region.values" = "YOUR_REGION",
  "projection.year.type" = "integer",
  "projection.year.range" = "2024,2036",
  "projection.month.type" = "integer",
  "projection.month.range" = "1,12",
  "projection.month.digits" = "2",
  "projection.day.type" = "integer",
  "projection.day.range" = "1,31",
  "projection.day.digits" = "2",
  "storage.location.template" = "s3://YOUR_BUCKET_NAME/AWSLogs/${aws-account-id}/${aws-service}/${aws-region}/${year}/${month}/${day}"
);

Before running, replace these 3 values:

PlaceholderReplace withExample
YOUR_BUCKET_NAMEYour S3 bucket name (appears twice)my-vpc-flow-logs-bucket-0210
YOUR_ACCOUNT_IDYour 12-digit AWS account ID123456789012
YOUR_REGIONYour AWS regionap-south-1

If the query succeeds, you'll see "Query successful" and the table vpc_flow_logs_parquet will appear in the left sidebar under Tables.

Step 3: Useful Athena Queries

1. Verify data is accessible (run this first):

SELECT * FROM vpc_flow_logs_parquet LIMIT 10;

If rows appear, Athena can read your flow log files. If you get zero results, check that your S3 path and account ID are correct.

2. Find all rejected SSH traffic:

SELECT srcaddr, dstaddr, srcport, dstport, action, packets
FROM vpc_flow_logs_parquet
WHERE dstport = 22
  AND action = 'REJECT'
ORDER BY packets DESC
LIMIT 25;

3. Top talkers by bytes transferred:

SELECT srcaddr, dstaddr, SUM(bytes) as total_bytes, SUM(packets) as total_packets
FROM vpc_flow_logs_parquet
WHERE action = 'ACCEPT'
GROUP BY srcaddr, dstaddr
ORDER BY total_bytes DESC
LIMIT 20;

4. Traffic to non-standard ports:

SELECT srcaddr, dstaddr, dstport, protocol, SUM(packets) as total_packets
FROM vpc_flow_logs_parquet
WHERE action = 'ACCEPT'
AND dstport NOT IN (80, 443, 22, 3306, 5432, 6379, 53)
GROUP BY srcaddr, dstaddr, dstport, protocol
ORDER BY total_packets DESC
LIMIT 25;

5. Traffic volume by source IP for a selected month:

Replace the year and month with the period you want to analyze.

SELECT srcaddr,
  SUM(bytes) / 1073741824.0 AS total_gib,
  COUNT(*) as flow_count
FROM vpc_flow_logs_parquet
WHERE year = '2026' AND month = '11'
GROUP BY srcaddr
ORDER BY total_gib DESC
LIMIT 20;

6. Find sources with rejected traffic to more than 20 destination ports:

This can help identify potential port scans. Each count represents flow records, not individual connection attempts.

SELECT srcaddr, COUNT(DISTINCT dstport) as unique_ports, COUNT(*) as rejected_flows
FROM vpc_flow_logs_parquet
WHERE action = 'REJECT'
GROUP BY srcaddr
HAVING COUNT(DISTINCT dstport) > 20
ORDER BY unique_ports DESC;

Monitoring VPC Flow Logs with SigNoz Cloud

The queries above help you investigate individual traffic patterns. For ongoing monitoring, you may also want to compare rejected traffic with application errors or receive alerts when a pattern changes.

You can forward AWS VPC Flow Logs to SigNoz Cloud and view them alongside application logs, metrics, and traces. Follow the AWS VPC log forwarding guide for the S3-to-Lambda pipeline. That guide parses text flow log records, so use plain-text files for that pipeline rather than the Parquet files used in the Athena example. For Google Cloud, follow the GCP VPC logging guide.

Once the flow log fields are parsed into attributes, filter by source IP, destination port, or action. Use a dashboard to compare rejected traffic with application error rates over the same time window, and configure a log-based alert for a sustained rise in rejected flows.

How to Disable or Delete VPC Flow Logs

If you no longer need flow logs for a specific resource (or want to recreate one with a different configuration), here's how to delete them:

Delete via AWS Console

  1. Open the Amazon VPC Console.
  2. In the navigation pane, select Your VPCs (or Subnets or Network Interfaces).
  3. Select the resource.
  4. Go to the Flow logs tab.
  5. Select the flow log you want to delete.
  6. Click Actions → Delete flow logs.
  7. When prompted, type delete to confirm, then click Delete.

Delete via AWS CLI

First, find your flow log ID:

aws ec2 describe-flow-logs --query 'FlowLogs[*].[FlowLogId,FlowLogStatus,LogDestination]' --output table

Then delete it:

aws ec2 delete-flow-logs --flow-log-ids fl-01234567890abcdef

FAQs

Are VPC Flow Logs free?

No. VPC Flow Logs are billed as CloudWatch Vended Logs (data ingestion + archival) regardless of destination, plus destination-specific charges (e.g., S3 storage/requests, CloudWatch Logs storage/queries, Firehose delivery stream costs). Pricing is region and tier dependent.

Do VPC Flow Logs affect network performance?

No. Flow log data is collected outside the path of your network traffic. Enabling them has no impact on network throughput or latency.

Can I enable VPC Flow Logs on a peered VPC?

Yes, but you can only enable flow logs for a peered VPC if the peer VPC is in your own AWS account. You cannot enable flow logs for VPCs peered with VPCs in other accounts.

How long does it take for flow logs to appear?

Typical delivery: ~5 min to CloudWatch Logs, ~10 min to S3, best-effort and may be delayed.

Can I capture only rejected traffic?

Yes. When creating a flow log, set the filter to Reject. This significantly reduces data volume and cost while still providing security-relevant information.

What is the difference between srcaddr and pkt-srcaddr?

srcaddr shows the IP address of the network interface from which traffic was captured. pkt-srcaddr (available in version 3+) shows the original source IP from the packet header. They differ when traffic passes through intermediate services like NAT gateways or load balancers.

How do VPC Flow Logs differ from VPC Traffic Mirroring?

VPC Flow Logs capture connection-level metadata (IP addresses, ports, protocols, and byte counts). VPC Traffic Mirroring captures actual packet contents for deep inspection. Flow logs are for monitoring and analysis; traffic mirroring is for forensic investigation and protocol analysis.

Can I modify an existing VPC Flow Log?

No. Once created, you cannot change a flow log's configuration. You need to delete it and create a new one with the desired settings.

What traffic is not captured by VPC Flow Logs?

DNS queries to Amazon DNS servers, DHCP traffic, instance metadata requests (169.254.169.254), Windows license activation traffic, Amazon Time Sync Service traffic, and traffic to the VPC router reserved IP address. Queries to your own DNS servers are captured.

How do I search VPC Flow Logs?

If your logs are in CloudWatch, use CloudWatch Logs Insights, select the log group containing your flow logs, and use the built-in query language to filter by source IP, port, action, etc. If your logs are in S3, use Amazon Athena to run SQL queries directly against the stored files. The CloudWatch and Athena sections above include example queries for rejected traffic, top talkers, and specific IP addresses.

Is this page helpful

Tags
vpclogging