VPC Flow Logs: Setup, Analysis, and Monitoring
VPC Flow Logs is a powerful, built-in network monitoring feature provided by cloud platforms (AWS, GCP, Azure), designed to capture detailed metadata about IP traffic flowing through your network interfaces. By enabling seamless logging of accepted and rejected traffic flows, it empowers administrators to troubleshoot connectivity issues, detect anomalies, and comply with regulatory requirements, all without the need for additional agents or hardware.
Each flow log entry (a flow log record) documents a single network flow, characterized by source IP, destination IP, source port, destination port, and protocol. These records are aggregated over a configurable time window (called the aggregation interval) and then delivered to a destination of your choice for storage and analysis. Flow log data is collected outside the path of your network traffic, enabling them has zero impact on your network throughput or latency. VPC Flow Logs don't slow down your instances or degrade application performance.
Why Do You Need VPC Flow Logs?
Before VPC flow logging, teams often relied on host-based tooling or third-party network sensors, which used to increase operational overhead and usually lacks VPC-wide coverage. This approach was complex, added overhead to each instance, and provided a limited view restricted to traffic visible at the instance level. VPC Flow Logs eliminate all of that by providing centralized, agent-free visibility at three levels: the entire VPC, a specific subnet, or an individual network interface (ENI).
Here are the practical reasons to use VPC Flow Logs:
Security Monitoring and Threat Detection
VPC Flow Logs help you detect port scanning, network enumeration attempts, lateral movement after a compromised host, and data exfiltration. If an attacker communicates with a compromised system on a non-standard port (say, using SMTPS for an HTTPS request), flow logs catch it.
Troubleshooting Connectivity Issues
When an application can't reach a database or a service times out, flow logs help you pinpoint whether the traffic is being rejected by a security group, blocked by a network ACL, or it is not reaching the intended destination at all.
Compliance Monitoring
Many compliance standards, like PCI DSS and HIPAA, require organizations to monitor and log network traffic. VPC Flow Logs provide the comprehensive record needed for auditing and compliance reporting. AWS itself recommends enabling VPC Flow Logs for every VPC as a security best practice.
Cost Optimization
By analyzing traffic patterns, you can identify top talkers in your network, discover unnecessary cross-AZ traffic that's inflating your bill, and understand how your network resources are actually being utilized.
How do VPC Flow Logs Work?
The architecture behind VPC Flow Logs involves three stages: Capture, Aggregate, and Deliver.

1. Capture
When you enable VPC Flow Logs, the cloud provider begins capturing metadata about IP traffic to/from the network interface level. On AWS, the flow log service captures connection-level metadata for every IP traffic flow passing through the ENIs (Elastic Network Interface) within your configured scope (VPC, subnet, or individual ENI).In practice, not all traffic types are logged, and in rare cases records can be skipped (log-status = SKIPDATA). The capture happens outside the data path, so there is no performance penalty.
2. Aggregate
Captured flow metadata is aggregated into flow log records (source IP, destination IP, source port, destination port, protocol) and aggregated within a time window called the aggregation interval. By default, this interval is 10 minutes on AWS. You can reduce it to 1 minute for more granular data, but this significantly increases the volume (and cost) of log records.
On Nitro-based instances (most modern EC2 instance types), the aggregation interval is always 1 minute or less, regardless of the configured setting.
3. Deliver
After aggregation, the flow log records are delivered to your configured destination. On AWS, you have three options:
| Destination | Best For |
|---|---|
| Amazon CloudWatch Logs | Real-time analysis, quick queries using CloudWatch Insights, metric filters and alerts |
| Amazon S3 | Long-term archival, cost-effective storage, big data analysis with Athena |
| Amazon Data Firehose | Streaming to third-party tools (SigNoz, Datadog, Splunk, etc.), real-time processing pipelines |
VPC Flow Logs Across Different Cloud Providers
While this guide primarily focuses on AWS, VPC Flow Logs are available across all major cloud providers. Here's a quick comparison:
| Feature | AWS VPC Flow Logs | GCP VPC Flow Logs | Azure NSG Flow Logs |
|---|---|---|---|
| Scope | VPC, Subnet, ENI | VPC Network, Subnet, VLAN Attachment, VPN Tunnel | Network Security Group |
| Aggregation Interval | 1 min or 10 min | 5 sec (default), 30 sec, 1 min, 5 min, 10 min, 15 min | 1 min |
| Log Destinations | CloudWatch, S3, Firehose | Cloud Logging | Azure Storage (Blob) (primary). Optional downstream analytics/export (e.g., Traffic Analytics / SIEM tooling). |
| Sampling | All flows captured | Configurable (50% default, adjustable) | All flows captured |
| Format | Space-delimited or Parquet | JSON | JSON |
| Container-level visibility | Yes (ECS, Version 7) | Yes (GKE nodes) | Yes (AKS Container Network Logs / Advanced Container Networking Services) |
GCP's VPC Flow Logs work differently in a few important ways. They use a dynamic primary sampling rate that varies with host load. You also get a configurable secondary sampling rate (default 50%), meaning not all flows are guaranteed to be logged. The tradeoff is lower cost for high-traffic environments.
Azure has evolved from NSG Flow Logs (which log at the security group level) to the newer VNet Flow Logs, which provide virtual network-level logging, closer to what AWS and GCP offer at the VPC level.
VPC Flow Logs Format: Understanding the Record Structure
A flow log record is a space-delimited string, with each field representing a component of the network flow. Understanding the format is critical for writing queries, building dashboards, and debugging network issues.
Default Format (Version 2)
The default format includes 14 fields:
<version> <account-id> <interface-id> <srcaddr> <dstaddr> <srcport> <dstport> <protocol> <packets> <bytes> <start> <end> <action> <log-status>Here's what a real flow log record looks like:
2 123456789012 eni-abcdef01 10.0.1.1 203.0.113.5 54321 80 6 10 3456 1510868790 1510868850 ACCEPT OKLet's break down each field:
| Field | Value in Example | What it Means |
|---|---|---|
| version | 2 | Flow log format version |
| account-id | 123456789012 | Your AWS account ID |
| interface-id | eni-abcdef01 | The ENI where traffic was captured |
| srcaddr | 10.0.1.1 | Source IP address |
| dstaddr | 203.0.113.5 | Destination IP address |
| srcport | 54321 | Source port |
| dstport | 80 | Destination port (HTTP) |
| protocol | 6 | IANA protocol number (6 = TCP, 17 = UDP, 1 = ICMP) |
| packets | 10 | Number of packets in the aggregation interval |
| bytes | 3456 | Number of bytes in the aggregation interval |
| start | 1510868790 | Start time of the capture window (Unix epoch) |
| end | 1510868850 | End time of the capture window (Unix epoch) |
| action | ACCEPT | Whether traffic was accepted or rejected |
| log-status | OK | Logging status (OK, NODATA, or SKIPDATA) |
Custom Format Fields (Version 3+)
The default format gives you the basics, but AWS has introduced additional fields across multiple versions that provide significantly deeper visibility. You access these by choosing a custom format when creating the flow log.
When using custom format, the ${version} field is automatically set to the highest version number among all the fields you include, you don't manually specify the version. For example, if you include reject-reason (v8), your log records will show version = 8.
Version 3 fields (VPC/Subnet/Instance context):
| Field | Purpose |
|---|---|
vpc-id | Identifies which VPC the traffic belongs to |
subnet-id | Identifies the subnet |
instance-id | Maps traffic to a specific EC2 instance |
tcp-flags | Bitmask of TCP flags (SYN, ACK, FIN, RST), it is critical for understanding connection state |
type | IPv4 or IPv6 |
pkt-srcaddr | Original packet source IP (important when traffic passes through NAT or load balancers) |
pkt-dstaddr | Original packet destination IP |
Version 4 fields (Location awareness):
| Field | Purpose |
|---|---|
region | AWS region of the flow |
az-id | Availability Zone ID |
sublocation-type | Type of sublocation (e.g., wavelength) |
sublocation-id | ID of the sublocation |
Version 5 fields (Service and direction awareness):
| Field | Purpose |
|---|---|
pkt-src-aws-service | AWS service name for the source (e.g.,S3, DYNAMODB) |
pkt-dst-aws-service | AWS service name for the destination |
flow-direction | Whether traffic is ingress or egress |
traffic-path | The path traffic takes to exit (e.g., through internet gateway, NAT gateway, VPC peering) |
Version 7 fields (ECS container visibility):
Version 7, introduced in 2024, adds 10 new fields specific to Amazon ECS workloads, giving you visibility into container-level traffic for the first time, including task IDs, cluster ARNs, container instance IDs, and service names.
Version 8+ fields (Enhanced troubleshooting)
| Field | Version | Purpose |
|---|---|---|
reject-reason | 8 | Reason traffic was rejected (currently BPA or EC; ‘-’ otherwise). |
resource-id | 9 | Resource ID associated with the flow (e.g., NAT gateway ID, transit gateway ID) |
encryption-status | 10 | Encryption status for the flow |
Recommended Custom Format for Production
If you're setting this up for the first time, here is a recommended custom format that balances depth and cost. It covers security, troubleshooting, and cost optimization use cases:
${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${vpc-id} ${subnet-id} ${instance-id} ${tcp-flags} ${type} ${pkt-srcaddr} ${pkt-dstaddr} ${flow-direction} ${traffic-path}Add pkt-src-aws-service and pkt-dst-aws-service if you need to trace traffic to specific AWS services.
Log File Format: Plain Text vs Parquet
When publishing to S3, you can choose between two file formats:
- Plain Text: The default. Human-readable, space-delimited records. Simple but slower to query at scale.
- Apache Parquet: A columnar data format. AWS documentation states that queries on Parquet data are 10 to 100 times faster compared to plain text, and the format compresses better, reducing storage costs significantly.
For any production deployment publishing to S3, use Parquet.
VPC Flow Logs Examples: Reading and Interpreting Records
Understanding how to read flow log records is essential for effective troubleshooting. Here are the most common patterns and what they mean:
Example 1: Security Group Rejecting Inbound Traffic
2 123456789012 eni-abcdef01 203.0.113.50 10.0.1.15 49152 3389 6 20 4000 1510868790 1510868850 REJECT OKThis record shows someone from an external IP 203.0.113.50 trying to connect to your instance on port 3389 (RDP). The traffic was rejected, which means your security group or network ACL is working correctly. However, if you see hundreds of these from different source IPs, it likely indicates a brute-force or port scanning attempt.
Example 2: Accepted Traffic to an Unexpected Port
2 123456789012 eni-abcdef01 10.0.1.15 198.51.100.25 43567 8443 6 15 5200 1510868790 1510868850 ACCEPT OKYour instance at 10.0.1.15 is sending traffic to an external IP on port 8443. If this isn't expected (for example, if your application should only communicate with known API endpoints on port 443), it could indicate a compromised instance or a misconfigured application.
Example 3: NODATA Records
2 123456789012 eni-abcdef01 - - - - - - - 1510868790 1510868850 - NODATANo traffic was recorded for this ENI (Network Interface) during the aggregation window. This is normal for idle interfaces.
Example 4: Custom Format with Flow Direction and AWS Services
Using version 5 custom format, here's a record showing an EC2 instance calling Amazon S3:
5 123456789012 eni-abcdef01 10.0.1.15 52.217.33.14 43567 443 6 15 5200 1510868790 1510868850 ACCEPT OK vpc-abc123 subnet-def456 i-0123456789 2 IPv4 10.0.1.15 52.217.33.14 AMAZON S3 egress 8The traffic-path value of 8 indicates the traffic went through an internet gateway. The pkt-dst-aws-service shows it's going to S3. If you're expecting this instance to access S3 via a VPC endpoint (private path), this record reveals it's actually going over the public internet, a common misconfiguration that's both slower and more expensive.
Example 5: TCP Flags for Connection Analysis
When using a custom format with tcp-flags, you can track the full lifecycle of TCP connections:
- SYN (2): Connection initiation, a client is trying to connect.
- SYN-ACK (18): Server acknowledging the connection.
- FIN (1): Graceful connection close.
- RST (4): Abrupt connection termination, it often indicates a problem.
If you see many SYN packets from a single source, but no SYN-ACK responses, it's likely a SYN flood or port scan.
How to Set Up VPC Flow Logs in AWS
Now that you can read a flow log record, enable logging for your own AWS resources. Choose S3 to query stored files with Athena, or CloudWatch Logs to search records with Logs Insights. Complete only the setup for your chosen destination. For CLI steps, configure AWS credentials and select the region containing your VPC.
Option 1: Using Amazon S3 (Recommended for Production)
Create an S3 bucket where AWS will deliver your flow log files.
Step 1: Create an S3 bucket
You can create the bucket via the AWS Console or CLI. Choose a globally unique bucket name. If the name is taken, try adding your account ID or a random number to the name.
aws s3 mb s3://YOUR_BUCKET_NAME --region YOUR_REGIONStep 2: Attach bucket policy
When you create a flow log via the AWS Console and own bucket, AWS automatically attaches the required bucket policy. You don't need to do anything manually. However, if you're creating the flow log via CLI/Terraform, or using a bucket in a different AWS account, you need to manually add this policy. Save the following as bucket-policy.json:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSLogDeliveryWrite",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control",
"aws:SourceAccount": "YOUR_ACCOUNT_ID"
}
}
},
{
"Sid": "AWSLogDeliveryAclCheck",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
}
]
}Replace before using:
YOUR_BUCKET_NAME- Your actual bucket name (e.g.,my-vpc-flow-logs-bucket-0210)YOUR_ACCOUNT_ID- Your AWS account ID (find it by runningaws sts get-caller-identity)
Apply the policy:
aws s3api put-bucket-policy --bucket YOUR_BUCKET_NAME --policy file://bucket-policy.jsonWith the bucket and policy in place, you’re ready to create the flow log itself. You can now skip to How to Enable VPC Flow Logs.
Option 2: Set Up CloudWatch Logs Destination
Create a CloudWatch Logs log group (a container for your flow logs) and an IAM role that gives the VPC Flow Logs service permission to write to it.
Step 1: Create a log group
aws logs create-log-group --log-group-name /vpc/flowlogs/my-vpcStep 2: Create an IAM Role for flow log delivery
VPC Flow Logs needs permission to write to your CloudWatch log group. When you create the flow log via the Console (covered in the next section), AWS will auto-generate a role with the correct permissions. Your user or role also needs iam:PassRole permission for the delivery role.
For CLI setup, create the role manually:
-
Create a file called
flow-logs-trust-policy.json:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vpc-flow-logs.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } -
Create the IAM role:
aws iam create-role \ --role-name VPCFlowLogsRole \ --assume-role-policy-document file://flow-logs-trust-policy.json -
Create
flow-logs-permissions.json:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogGroups", "logs:DescribeLogStreams" ], "Resource": "*" } ] } -
Attach the permissions to the role:
aws iam put-role-policy \ --role-name VPCFlowLogsRole \ --policy-name VPCFlowLogsPolicy \ --policy-document file://flow-logs-permissions.json
With the log group and delivery role ready, create the flow log using either the Console or CLI below.
How to Enable VPC Flow Logs in AWS?
Now that your destination is ready, it's time to create the flow log itself. VPC Flow Logs can be enabled at three different levels:
| Level | Scope | When to Use |
|---|---|---|
| VPC | All ENIs in the VPC | Broad security monitoring, compliance |
| Subnet | All ENIs in the subnet | Focused monitoring on specific tiers (e.g., database subnet) |
| ENI | Single network interface | Debugging a specific instance or service |
We will start with VPC-level flow logs. This covers the network interfaces in that VPC, and you can filter the recorded traffic later in your queries.
Method 1: Enable via AWS Console (Recommended for Beginners)
The AWS Console provides the most user-friendly way to enable VPC Flow Logs. The steps differ slightly based on whether you're using S3 or CloudWatch Logs as your destination.
For S3 destination:
- Open the Amazon VPC Console
- In the left sidebar, click Your VPCs
- Select the VPC you want to monitor (check the box next to it)
- Click Actions → Create flow log
- Fill in the form:
- Name:
my-vpc-flow-log(optional but helpful) - Filter: Select All (captures both accepted and rejected traffic)
- Maximum aggregation interval: Select 10 minutes (good balance of detail vs. cost) or 1 minute for more granular data
- Destination: Select Send to an Amazon S3 bucket
- S3 bucket ARN: Enter
arn:aws:s3:::YOUR_BUCKET_NAME(e.g.,arn:aws:s3:::my-vpc-flow-logs-bucket-0210) - Log record format: Select AWS default format (14 fields)
- Log file format: Select Parquet (compressed format, faster queries, lower cost)
- Partition logs by time: Select Every 24 hours (default)
- Name:
- Click Create flow log
For CloudWatch Logs destination:
- Follow steps 1-4 above
- Fill in the form:
- Name:
my-vpc-flow-log-cw - Filter: Select All
- Maximum aggregation interval: Select 1 minute (for a shorter aggregation window)
- Destination: Select Send to CloudWatch Logs
- Destination log group: Enter
/vpc/flowlogs/my-vpc - Service access: Select "Create and use a new service role". AWS auto-generates a role name like
VPCFlowLogs-Cloudwatch-XXXXXXXXXand creates it with the correct permissions when you click Create. (If you already created a role manually, select "Use an existing service role" and pickVPCFlowLogsRolefrom the dropdown.) - Log record format: Select AWS default format
- Name:
- Click Create flow log
After creation: Typically, logs arrive in ~5 minutes to CloudWatch Logs and ~10 minutes to S3 (delays can be longer). In some cases it can take 10+ minutes for the log group/data to appear.
Method 2: Enable via AWS CLI
For these steps, you need AWS CLI installed. Configure it with credentials that can create flow logs and manage the chosen destination.
-
Find your VPC ID first
This lists all your VPCs. Copy the VPC ID you want to monitor (e.g.,
vpc-07810113181a81357).aws ec2 describe-vpcs --query 'Vpcs[*].[VpcId,CidrBlock,IsDefault]' --output table -
Create flow log to S3 (with Parquet)
For an S3 destination, run:
aws ec2 create-flow-logs \ --resource-type VPC \ --resource-ids vpc-YOUR_VPC_ID \ --traffic-type ALL \ --log-destination-type s3 \ --log-destination arn:aws:s3:::YOUR_BUCKET_NAME \ --max-aggregation-interval 60 \ --destination-options FileFormat=parquet -
Create flow log to CloudWatch Logs
For a CloudWatch Logs destination, run this command instead:
aws ec2 create-flow-logs \ --resource-type VPC \ --resource-ids vpc-YOUR_VPC_ID \ --traffic-type ALL \ --log-destination-type cloud-watch-logs \ --log-group-name /vpc/flowlogs/my-vpc \ --deliver-logs-permission-arn arn:aws:iam::YOUR_ACCOUNT_ID:role/VPCFlowLogsRole \ --max-aggregation-interval 60 -
Replace these values
Placeholder Replace with How to find it vpc-YOUR_VPC_IDYour VPC ID Run aws ec2 describe-vpcsYOUR_BUCKET_NAMEYour S3 bucket name The bucket you created in prerequisites YOUR_ACCOUNT_IDYour 12-digit AWS account ID Run aws sts get-caller-identity -
Verify the flow log was created
aws ec2 describe-flow-logs --output tableHow to Search VPC Flow Logs with CloudWatch Insights
Once records reach CloudWatch Logs, use Logs Insights to find rejected traffic or identify the hosts transferring the most data. The queries below use the automatically discovered fields for the AWS default flow log format.
How to Open CloudWatch Insights
- Open the CloudWatch Console
- In the left sidebar, click Logs → Logs Insights
- In the Select log group(s) dropdown, choose your flow log group (e.g.,
/vpc/flowlogs/my-vpc) - Set the time range in the top-right corner (e.g., last 1 hour)
- Paste a query from below into the query editor
- Click Run query
Useful Queries
1. Verify logs are being ingested (run this first):
fields @timestamp, srcAddr, dstAddr, srcPort, dstPort, action
| sort @timestamp desc
| limit 10This shows the 10 most recent flow log records. If you see results, your logs are working. If there are no results, check the selected time range and allow for delivery delays before checking the flow log delivery status.
2. Count rejected SSH flow records by source IP:
filter action = "REJECT" and dstPort = 22
| stats count(*) as rejectedFlows by srcAddr
| sort rejectedFlows desc
| limit 253. Top talkers — who's sending the most data:
filter action = "ACCEPT"
| stats sum(bytes) as totalBytes by srcAddr, dstAddr
| sort totalBytes desc
| limit 204. Find traffic outside an expected port list:
Adjust the port list to match your applications. Traffic outside this list is worth checking, but is not necessarily malicious.
filter action = "ACCEPT" and dstPort not in [80, 443, 22, 3306, 5432, 6379, 53]
| stats count(*) as flowRecords, sum(bytes) as totalBytes by srcAddr, dstAddr, dstPort
| sort flowRecords desc
| limit 255. Rejected traffic over time (useful for dashboards):
filter action = "REJECT"
| stats count(*) as rejectedCount by bin(5m)6. Find all traffic involving a specific IP:
filter srcAddr = "203.0.113.50" or dstAddr = "203.0.113.50"
| sort @timestamp desc
| limit 100How to Query VPC Flow Logs with Amazon Athena
For flow logs stored in S3, Amazon Athena lets you run standard SQL queries against your log data. Use it when you need SQL analysis of stored flow logs.
Step 1: Open Athena
- Open the Athena Console
- If this is your first time, Athena will ask you to set a query result location. Click Settings → Manage, then enter an S3 path like
s3://YOUR_BUCKET_NAME/athena-results/and click Save - Make sure you're in the Query editor tab
Step 2: Create the Athena Table
Paste the following SQL into the Athena query editor and click Run. This table reads the default 14 fields from the Parquet files created above. Its partition projection follows the default daily S3 layout, with Hive-compatible prefixes disabled. If you chose a bucket prefix or a different partition layout, adjust the paths to match your files.
CREATE EXTERNAL TABLE IF NOT EXISTS vpc_flow_logs_parquet (
version int,
account_id string,
interface_id string,
srcaddr string,
dstaddr string,
srcport int,
dstport int,
protocol bigint,
packets bigint,
bytes bigint,
start bigint,
`end` bigint,
action string,
log_status string
)
PARTITIONED BY (
`aws-account-id` string,
`aws-service` string,
`aws-region` string,
`year` string,
`month` string,
`day` string
)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION 's3://YOUR_BUCKET_NAME/AWSLogs/'
TBLPROPERTIES (
"projection.enabled" = "true",
"projection.aws-account-id.type" = "enum",
"projection.aws-account-id.values" = "YOUR_ACCOUNT_ID",
"projection.aws-service.type" = "enum",
"projection.aws-service.values" = "vpcflowlogs",
"projection.aws-region.type" = "enum",
"projection.aws-region.values" = "YOUR_REGION",
"projection.year.type" = "integer",
"projection.year.range" = "2024,2036",
"projection.month.type" = "integer",
"projection.month.range" = "1,12",
"projection.month.digits" = "2",
"projection.day.type" = "integer",
"projection.day.range" = "1,31",
"projection.day.digits" = "2",
"storage.location.template" = "s3://YOUR_BUCKET_NAME/AWSLogs/${aws-account-id}/${aws-service}/${aws-region}/${year}/${month}/${day}"
);Before running, replace these 3 values:
| Placeholder | Replace with | Example |
|---|---|---|
YOUR_BUCKET_NAME | Your S3 bucket name (appears twice) | my-vpc-flow-logs-bucket-0210 |
YOUR_ACCOUNT_ID | Your 12-digit AWS account ID | 123456789012 |
YOUR_REGION | Your AWS region | ap-south-1 |
If the query succeeds, you'll see "Query successful" and the table vpc_flow_logs_parquet will appear in the left sidebar under Tables.
Step 3: Useful Athena Queries
1. Verify data is accessible (run this first):
SELECT * FROM vpc_flow_logs_parquet LIMIT 10;If rows appear, Athena can read your flow log files. If you get zero results, check that your S3 path and account ID are correct.
2. Find all rejected SSH traffic:
SELECT srcaddr, dstaddr, srcport, dstport, action, packets
FROM vpc_flow_logs_parquet
WHERE dstport = 22
AND action = 'REJECT'
ORDER BY packets DESC
LIMIT 25;3. Top talkers by bytes transferred:
SELECT srcaddr, dstaddr, SUM(bytes) as total_bytes, SUM(packets) as total_packets
FROM vpc_flow_logs_parquet
WHERE action = 'ACCEPT'
GROUP BY srcaddr, dstaddr
ORDER BY total_bytes DESC
LIMIT 20;4. Traffic to non-standard ports:
SELECT srcaddr, dstaddr, dstport, protocol, SUM(packets) as total_packets
FROM vpc_flow_logs_parquet
WHERE action = 'ACCEPT'
AND dstport NOT IN (80, 443, 22, 3306, 5432, 6379, 53)
GROUP BY srcaddr, dstaddr, dstport, protocol
ORDER BY total_packets DESC
LIMIT 25;5. Traffic volume by source IP for a selected month:
Replace the year and month with the period you want to analyze.
SELECT srcaddr,
SUM(bytes) / 1073741824.0 AS total_gib,
COUNT(*) as flow_count
FROM vpc_flow_logs_parquet
WHERE year = '2026' AND month = '11'
GROUP BY srcaddr
ORDER BY total_gib DESC
LIMIT 20;6. Find sources with rejected traffic to more than 20 destination ports:
This can help identify potential port scans. Each count represents flow records, not individual connection attempts.
SELECT srcaddr, COUNT(DISTINCT dstport) as unique_ports, COUNT(*) as rejected_flows
FROM vpc_flow_logs_parquet
WHERE action = 'REJECT'
GROUP BY srcaddr
HAVING COUNT(DISTINCT dstport) > 20
ORDER BY unique_ports DESC;Monitoring VPC Flow Logs with SigNoz Cloud
The queries above help you investigate individual traffic patterns. For ongoing monitoring, you may also want to compare rejected traffic with application errors or receive alerts when a pattern changes.
You can forward AWS VPC Flow Logs to SigNoz Cloud and view them alongside application logs, metrics, and traces. Follow the AWS VPC log forwarding guide for the S3-to-Lambda pipeline. That guide parses text flow log records, so use plain-text files for that pipeline rather than the Parquet files used in the Athena example. For Google Cloud, follow the GCP VPC logging guide.
Once the flow log fields are parsed into attributes, filter by source IP, destination port, or action. Use a dashboard to compare rejected traffic with application error rates over the same time window, and configure a log-based alert for a sustained rise in rejected flows.
How to Disable or Delete VPC Flow Logs
If you no longer need flow logs for a specific resource (or want to recreate one with a different configuration), here's how to delete them:
Delete via AWS Console
- Open the Amazon VPC Console.
- In the navigation pane, select Your VPCs (or Subnets or Network Interfaces).
- Select the resource.
- Go to the Flow logs tab.
- Select the flow log you want to delete.
- Click Actions → Delete flow logs.
- When prompted, type delete to confirm, then click Delete.
Delete via AWS CLI
First, find your flow log ID:
aws ec2 describe-flow-logs --query 'FlowLogs[*].[FlowLogId,FlowLogStatus,LogDestination]' --output tableThen delete it:
aws ec2 delete-flow-logs --flow-log-ids fl-01234567890abcdefFAQs
Are VPC Flow Logs free?
No. VPC Flow Logs are billed as CloudWatch Vended Logs (data ingestion + archival) regardless of destination, plus destination-specific charges (e.g., S3 storage/requests, CloudWatch Logs storage/queries, Firehose delivery stream costs). Pricing is region and tier dependent.
Do VPC Flow Logs affect network performance?
No. Flow log data is collected outside the path of your network traffic. Enabling them has no impact on network throughput or latency.
Can I enable VPC Flow Logs on a peered VPC?
Yes, but you can only enable flow logs for a peered VPC if the peer VPC is in your own AWS account. You cannot enable flow logs for VPCs peered with VPCs in other accounts.
How long does it take for flow logs to appear?
Typical delivery: ~5 min to CloudWatch Logs, ~10 min to S3, best-effort and may be delayed.
Can I capture only rejected traffic?
Yes. When creating a flow log, set the filter to Reject. This significantly reduces data volume and cost while still providing security-relevant information.
What is the difference between srcaddr and pkt-srcaddr?
srcaddr shows the IP address of the network interface from which traffic was captured. pkt-srcaddr (available in version 3+) shows the original source IP from the packet header. They differ when traffic passes through intermediate services like NAT gateways or load balancers.
How do VPC Flow Logs differ from VPC Traffic Mirroring?
VPC Flow Logs capture connection-level metadata (IP addresses, ports, protocols, and byte counts). VPC Traffic Mirroring captures actual packet contents for deep inspection. Flow logs are for monitoring and analysis; traffic mirroring is for forensic investigation and protocol analysis.
Can I modify an existing VPC Flow Log?
No. Once created, you cannot change a flow log's configuration. You need to delete it and create a new one with the desired settings.
What traffic is not captured by VPC Flow Logs?
DNS queries to Amazon DNS servers, DHCP traffic, instance metadata requests (169.254.169.254), Windows license activation traffic, Amazon Time Sync Service traffic, and traffic to the VPC router reserved IP address. Queries to your own DNS servers are captured.
How do I search VPC Flow Logs?
If your logs are in CloudWatch, use CloudWatch Logs Insights, select the log group containing your flow logs, and use the built-in query language to filter by source IP, port, action, etc. If your logs are in S3, use Amazon Athena to run SQL queries directly against the stored files. The CloudWatch and Athena sections above include example queries for rejected traffic, top talkers, and specific IP addresses.