如何利用 CloudWatch Logs Insights 來分析自訂 VPC 流程記錄?

上次更新日期:2022 年 4 月 27 日

我已設定自訂 VPC 流程記錄。如何利用 Amazon CloudWatch Logs Insights 來探索模式及趨勢?

簡短描述

您可利用 CloudWatch Logs Insights 來分析 VPC 流程記錄。CloudWatch Logs Insights 會自動在許多 Amazon 提供的記錄探索欄位以及 JSON 格式的日誌事件,以便輕鬆建構查詢及記錄探索。CloudWatch Logs Insights 會自動探索採用預設格式的 VPC 流程記錄。

然而,VPC 流程記錄是以自訂格式部署。基於這個原因,不會自動探索,您必須修改查詢。本文提供了幾個查詢範例,您可自訂及擴充來符合您的使用案例。

利用以下自訂 VPC 流程記錄格式:

${account-id} ${vpc-id} ${subnet-id} ${interface-id} ${instance-id} ${srcaddr} ${srcport} ${dstaddr} ${dstport} ${protocol} ${packets} ${bytes} ${action} ${log-status} ${start} ${end} ${flow-direction} ${traffic-path} ${tcp-flags} ${pkt-srcaddr} ${pkt-src-aws-service} ${pkt-dstaddr} ${pkt-dst-aws-service} ${region} ${az-id} ${sublocation-type} ${sublocation-id}

解決方式

擷取最新 VPC 流程記錄

由於 CloudWatch Logs Insights 不會自動探索記錄欄位,因此必須利用解析關鍵字來隔離所需欄位。在此查詢中,結果依照流程日誌事件開始時間排序,並限於兩則最新的記錄項目。

查詢

#Retrieve latest custom VPC Flow Logs
parse @message "* * * * * * * * * * * * * * * * * * * * * * * * * * *" as account_id, vpc_id, subnet_id, interface_id,instance_id, srcaddr, srcport, dstaddr, dstport, protocol, packets, bytes, action, log_status, start, end, flow_direction, traffic_path, tcp_flags, pkt_srcaddr, pkt_src_aws_service, pkt_dstaddr, pkt_dst_aws_service, region, az_id, sublocation_type, sublocation_id
| sort start desc
| limit 2

結果


account_id  vpc_id   subnet_id  interface_id instance_id srcaddr srcport
123456789012  vpc-0b69ce8d04278ddd  subnet-002bdfe1767d0ddb0 eni-0435cbb62960f230e 172.31.0.104 55125
123456789012  vpc-0b69ce8d04278ddd1  subnet-002bdfe1767d0ddb0 eni-0435cbb62960f230e 91.240.118.81 49422

依照來源/目標 IP 位址彙總資料傳輸

接下來,依照來源/目的地 IP 位址彙總網路流量。在此範例中,利用總和統計數字彙總位元欄位。這會計算主機之間傳輸的資料的累積總數。為了取得更多內容,因此納入了 flow_direction。然後,把彙總的結果臨時指派到 Data_Transferred 欄位。然後,依照 Data_Testorage 由大到小對結果進行排序,同時返回兩組最大的資料對。

查詢

parse @message "* * * * * * * * * * * * * * * * * * * * * * * * * * *" as account_id, vpc_id, subnet_id, interface_id,instance_id, srcaddr, srcport, dstaddr, dstport, protocol, packets, bytes, action, log_status, start, end, flow_direction, traffic_path, tcp_flags, pkt_srcaddr, pkt_src_aws_service, pkt_dstaddr, pkt_dst_aws_service, region, az_id, sublocation_type, sublocation_id
| stats sum(bytes) as Data_Transferred by srcaddr, dstaddr, flow_direction
| sort by Data_Transferred desc
| limit 2

結果

srcaddr dstaddr flow_direction Data_Transferred
172.31.1.247 3,230,172,154 egress 346952038
172.31.0.46 3,230,172,154 egress 343799447

依照 EC2 執行個體 ID 來分析資料傳輸

您可利用自訂 VPC 流程記錄直接分析Amazon Elastic Compute Cloud (Amazon EC2) 執行個體 ID。透過上述查詢,您現在可利用 instance_id 欄位確定最活躍的 EC2 執行個體。

查詢

parse @message "* * * * * * * * * * * * * * * * * * * * * * * * * * *" as account_id, vpc_id, subnet_id, interface_id,instance_id, srcaddr, srcport, dstaddr, dstport, protocol, packets, bytes, action, log_status, start, end, flow_direction, traffic_path, tcp_flags, pkt_srcaddr, pkt_src_aws_service, pkt_dstaddr, pkt_dst_aws_service, region, az_id, sublocation_type, sublocation_id
| stats sum(bytes) as Data_Transferred by instance_id
| sort by Data_Transferred desc
| limit 5

結果

instance_id Data_Transferred
- 1443477306
i-03205758c9203c979 517558754
i-0ae33894105aa500c 324629414
i-01506ab9e9e90749d 198063232
i-0724007fef3cb06f3 54847643

篩選遭拒的 SSH 流量

為了更瞭解遭到安全群組及網路存取控制清單 (ACL) 拒絕的流量,請篩選遭拒的 VPC 流程記錄。您可進一步縮小此篩選範圍,納入通訊協定和目標連接埠。要識別 SSH 通訊遭拒的主機,請擴大篩選條件納入 TCP 通訊協定 (例如,通訊協定 6) 和目標連接埠為 22 的通訊。

查詢

parse @message "* * * * * * * * * * * * * * * * * * * * * * * * * * *" as account_id, vpc_id, subnet_id, interface_id,instance_id, srcaddr, srcport, dstaddr, dstport, protocol, packets, bytes, action, log_status, start, end, flow_direction, traffic_path, tcp_flags, pkt_srcaddr, pkt_src_aws_service, pkt_dstaddr, pkt_dst_aws_service, region, az_id, sublocation_type, sublocation_id
| filter action = "REJECT" and protocol = 6 and dstport = 22
| stats sum(bytes) as SSH_Traffic_Volume by srcaddr
| sort by SSH_Traffic_Volume desc
| limit 2

結果

srcaddr SSH_Traffic_Volume
23.95.222.129 160
179.43.167.74 80

隔離特定來源/目標對的 HTTP 資料串流

要利用 CloudWatch Logs Insights 進一步調查資料趨勢,請隔離兩個 IP 位址間的雙向流量。在此查詢中,["172.31.1.247","172.31.11.212"] 返回利用 IP 位址作為來源 IP 位址或目標 IP 位址的流程記錄。要隔離 HTTP 流量,篩選條件的敘述會找出符合通訊協定 6 (TCP) 與連接埠 80 的 VPC 流程日誌事件。利用 display 關鍵字返回所有可用欄位的子集。

查詢

#HTTP Data Stream for Specific Source/Destination Pair
parse @message "* * * * * * * * * * * * * * * * * * * * * * * * * * *" as account_id, vpc_id, subnet_id, interface_id,instance_id, srcaddr, srcport, dstaddr, dstport, protocol, packets, bytes, action, log_status, start, end, flow_direction, traffic_path, tcp_flags, pkt_srcaddr, pkt_src_aws_service, pkt_dstaddr, pkt_dst_aws_service, region, az_id, sublocation_type, sublocation_id
| filter srcaddr in ["172.31.1.247","172.31.11.212"] and dstaddr in ["172.31.1.247","172.31.11.212"] and protocol = 6 and (dstport = 80 or srcport=80)
| display interface_id,srcaddr, srcport, dstaddr, dstport, protocol, bytes, action, log_status, start, end, flow_direction, tcp_flags
| sort by start desc
| limit 2

結果

interface_id srcaddr srcport dstaddr dstport 通訊協定 位元組 動作 log_status
eni-0b74120275654905e 172.31.11.212 80 172.31.1.247 29376 6 5160876 接受 確定
eni-0b74120275654905e 172.31.1.247 29376 172.31.11.212 80 6 97380 接受 確定

隔離特定來源/目標對的 HTTP 資料串流

您可利用 CloudWatch Logs Insights 以長條圖或餅狀圖形式來顯示結果。如結果包含 bin() 函數,則返回包含時間戳記的查詢結果。然後,可利用折線圖或堆疊面積圖顯示此時間序列

在上述查詢基礎之上,您可利用 stats sum(bytes) as Data_Trasferred by bin(1m) 計算在一分鐘間隔內傳輸的累積資料。要檢視此視覺化結果,請在 CloudWatch Logs Insights 主控台中的記錄視覺化表格之間進行切換。

查詢

parse @message "* * * * * * * * * * * * * * * * * * * * * * * * * * *" as account_id, vpc_id, subnet_id, interface_id,instance_id, srcaddr, srcport, dstaddr, dstport, protocol, packets, bytes, action, log_status, start, end, flow_direction, traffic_path, tcp_flags, pkt_srcaddr, pkt_src_aws_service, pkt_dstaddr, pkt_dst_aws_service, region, az_id, sublocation_type, sublocation_id
| filter srcaddr in ["172.31.1.247","172.31.11.212"] and dstaddr in ["172.31.1.247","172.31.11.212"] and protocol = 6 and (dstport = 80 or srcport=80)
| stats sum(bytes) as Data_Transferred by bin(1m)

結果

bin(1m) Data_Transferred
2022-04-01 15:23:00.000 17225787
2022-04-01 15:21:00.000 17724499
2022-04-01 15:20:00.000 1125500
2022-04-01 15:19:00.000 101525
2022-04-01 15:18:00.000 81376