针对网络安全组进行流日志记录简介Introduction to flow logging for network security groups

网络安全组 (NSG) 流日志是网络观察程序的一项功能,可用于查看有关通过 NSG 的入口和出口 IP 流量的信息。Network security group (NSG) flow logs are a feature of Network Watcher that allows you to view information about ingress and egress IP traffic through an NSG. 流日志以 JSON 格式编写,并基于每个规则显示出站和入站流、流所适用的网络接口 (NIC)、有关流的 5 元组信息(源/目标 IP 地址、源/目标端口和协议)、是允许还是拒绝流量,版本 2 中还会显示吞吐量信息(字节和数据包)。Flow logs are written in JSON format, and show outbound and inbound flows on a per rule basis, the network interface (NIC) the flow applies to, 5-tuple information about the flow (Source/destination IP, source/destination port, and protocol), if the traffic was allowed or denied, and in Version 2, throughput information (Bytes and Packets).

流日志概述

流日志针对的是 NSG,但其显示方式不同于其他日志。While flow logs target NSGs, they are not displayed the same as the other logs. 流日志仅存储在一个存储帐户中,其采用的日志记录路径如以下示例所示:Flow logs are stored only within a storage account and follow the logging path shown in the following example:

https://{storageAccountName}.blob.core.chinacloudapi.cn/insights-logs-networksecuritygroupflowevent/resourceId=/SUBSCRIPTIONS/{subscriptionID}/RESOURCEGROUPS/{resourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/{nsgName}/y={year}/m={month}/d={day}/h={hour}/m=00/macAddress={macAddress}/PT1H.json

可以使用流量分析来分析流日志并获取网络流量的见解。You can analyze flow logs and gain insights into your network traffic using traffic analytics.

适用于其他日志的保留策略也适用于流日志。The same retention policies seen for other logs apply to flow logs. 可以设置日志保留策略,时间范围为 1 天至 365 天。You can set log retention policy from 1 day to 365 days. 如果未设置保留策略,则会永久保留日志。If a retention policy is not set, the logs are maintained forever.

日志文件Log file

流日志包含以下属性:Flow logs include the following properties:

  • time - 记录事件的时间time - Time when the event was logged
  • systemId - 网络安全组资源 IDsystemId - Network Security Group resource Id.
  • 类别 - 事件的类别。category - The category of the event. 类别始终是 NetworkSecurityGroupFlowEvent The category is always NetworkSecurityGroupFlowEvent
  • resourceid - NSG 的资源 IDresourceid - The resource Id of the NSG
  • operationName - 始终为 NetworkSecurityGroupFlowEventsoperationName - Always NetworkSecurityGroupFlowEvents
  • properties - 流属性的集合properties - A collection of properties of the flow
    • Version - 流日志事件架构的版本号Version - Version number of the Flow Log event schema
    • flows - 流的集合。flows - A collection of flows. 此属性有多个针对不同规则的条目This property has multiple entries for different rules
      • rule - 列出流时所依据的规则rule - Rule for which the flows are listed
        • flows - 流的集合flows - a collection of flows
          • mac - VM 的 NIC 的 MAC 地址,用于收集流mac - The MAC address of the NIC for the VM where the flow was collected
          • flowTuples - 一个字符串,包含逗号分隔格式的流元组的多个属性flowTuples - A string that contains multiple properties for the flow tuple in comma-separated format
            • Time Stamp - 此值为时间戳,表示流的发生时间,采用 UNIX epoch 格式Time Stamp - This value is the time stamp of when the flow occurred in UNIX epoch format
            • Source IP - 源 IPSource IP - The source IP
            • Destination IP - 目标 IPDestination IP - The destination IP
            • Source Port - 源端口Source Port - The source port
            • Destination Port - 目标端口Destination Port - The destination Port
            • Protocol - 流的协议。Protocol - The protocol of the flow. 有效值为 T(表示 TCP)和 U(表示 UDP)Valid values are T for TCP and U for UDP
            • Traffic Flow - 流的方向。Traffic Flow - The direction of the traffic flow. 有效值为 I(表示入站)和 O(表示出站)。Valid values are I for inbound and O for outbound.
            • Traffic Decision - 是允许了还是拒绝了流。Traffic Decision - Whether traffic was allowed or denied. 有效值为 A(表示已允许)和 D(表示已拒绝)。Valid values are A for allowed and D for denied.
            • Flow State - 仅限版本 2 - 捕获流的状态。Flow State - Version 2 Only - Captures the state of the flow. 可能的状态包括 B:创建流时开始。Possible states are B: Begin, when a flow is created. 未提供统计信息。Statistics aren't provided. C:继续执行正在进行的流。C: Continuing for an ongoing flow. 以 5 分钟的时间间隔提供统计信息。Statistics are provided at 5-minute intervals. E:在流终止时结束。E: End, when a flow is terminated. 已提供统计信息。Statistics are provided.
            • Packets - 源到目标 - 仅限版本 2 自上次更新以来,从源发送到目标的 TCP 或 UDP 数据包的总数。Packets - Source to destination - Version 2 Only The total number of TCP or UDP packets sent from source to destination since last update.
            • Bytes sent - 源到目标 - 仅限版本 2 自上次更新以来,从源发送到目标的 TCP 或 UDP 数据包字节的总数。Bytes sent - Source to destination - Version 2 Only The total number of TCP or UDP packet bytes sent from source to destination since last update. 数据包字节包括数据包标头和有效负载。Packet bytes include the packet header and payload.
            • Packets - 目标到源 - 仅限版本 2 自上次更新以来,从目标发送到源的 TCP 或 UDP 数据包的总数。Packets - Destination to source - Version 2 Only The total number of TCP or UDP packets sent from destination to source since last update.
            • Bytes sent - 目标到源 - 仅限版本 2 自上次更新以来,从目标发送到源的 TCP 和 UDP 数据包字节的总数。Bytes sent - Destination to source - Version 2 Only The total number of TCP and UDP packet bytes sent from destination to source since last update. 数据包字节包括数据包标头和有效负载。Packet bytes include packet header and payload.

NSG 流日志版本 2NSG flow logs version 2

版本 2 的日志引入了流状态。Version 2 of the logs introduces flow state. 可以配置要接收的流日志的版本。You can configure which version of flow logs you receive. 要了解如何启用流日志,请参阅启用 NSG 流日志记录To learn how to enable flow logs, see Enabling NSG flow logging.

启动流时记录流状态 B 。Flow state B is recorded when a flow is initiated. 流状态 C 和流状态 E 是分别标记流的延续和终止的状态 。Flow state C and flow state E are states that mark the continuation of a flow and flow termination, respectively. 状态 C 和 E 都包含流量带宽信息 。Both C and E states contain traffic bandwidth information.

示例:介于 185.170.185.105:35370 和 10.2.0.4:23 之间的 TCP 对话中的流元组:Example: Flow tuples from a TCP conversation between 185.170.185.105:35370 and 10.2.0.4:23:

"1493763938,185.170.185.105,10.2.0.4,35370,23,T,I,A,B,,,," "1493695838,185.170.185.105,10.2.0.4,35370,23,T,I,A,C,1021,588096,8005,4610880" "1493696138,185.170.185.105,10.2.0.4,35370,23,T,I,A,E,52,29952,47,27072""1493763938,185.170.185.105,10.2.0.4,35370,23,T,I,A,B,,,," "1493695838,185.170.185.105,10.2.0.4,35370,23,T,I,A,C,1021,588096,8005,4610880" "1493696138,185.170.185.105,10.2.0.4,35370,23,T,I,A,E,52,29952,47,27072"

对于延续 C 和结束 E 流状态,字节和数据包计数是从上一次流元祖记录时集合的计数 。For continuation C and end E flow states, byte and packet counts are aggregate counts from the time of the previous flow tuple record. 引用上一示例会话,传输的数据包的总数是 1021+52+8005+47 = 9125。Referencing the previous example conversation, the total number of packets transferred is 1021+52+8005+47 = 9125. 传输的字节总数是 588096+29952+4610880+27072 = 5256000。The total number of bytes transferred is 588096+29952+4610880+27072 = 5256000.

以下文本是流日志的示例。The text that follows is an example of a flow log. 可以看到,有多个记录遵循前一部分描述的属性列表。As you can see, there are multiple records that follow the property list described in the preceding section.

NSG 流日志记录注意事项NSG flow logging considerations

存储帐户注意事项Storage account considerations:

  • 位置:所用的存储帐户必须与 NSG 位于同一区域中。Location: The storage account used must be in the same region as the NSG.
  • 自行管理密钥轮换:如果你更改/轮换存储帐户的访问密钥,则 NSG 流日志将停止工作。Self-manage key rotation: If you change/rotate the access keys to your storage account, NSG Flow Logs will stop working. 若要解决此问题,必须禁用并重新启用 NSG 流日志。To fix this issue, you must disable and then re-enable NSG Flow Logs.

在附加到资源的所有 NSG 上启用 NSG 流日志记录:Azure 中的流日志记录是在 NSG 资源上配置的。Enable NSG Flow Logging on all NSGs attached to a resource: Flow logging in Azure is configured on the NSG resource. 一个流只与一个 NSG 规则相关联。A flow will only be associated to one NSG Rule. 如果利用多个 NSG,则我们建议在应用资源子网或网络接口的所有 NSG 上启用 NSG 流日志记录,以确保记录所有流量。In scenarios where multiple NSGs are utilized, we recommend that NSG flow logging is enabled on all NSGs applied a resource's subnet or network interface to ensure that all traffic is recorded. 有关详细信息,请参阅网络安全组中的流量评估方式For more information see how traffic is evaluated in Network Security Groups.

流日志记录成本:NSG 流日志记录按生成的日志量计费。Flow Logging Costs: NSG flow logging is billed on the volume of logs produced. 流量较高时,流日志的量和相关成本可能会增大。High traffic volume can result in large flow log volume and the associated costs. NSG 流日志定价不包括基本的存储成本。NSG Flow log pricing does not include the underlying costs of storage. 将保留策略功能与 NSG 流日志记录配合使用意味着在较长时间内会产生单独的存储成本。Using the retention policy feature with NSG Flow Logging means incurring separate storage costs for extended periods of time. 如果不需要使用保留策略功能,我们建议将此值设置为 0。If you do not require the retention policy feature, we recommend that you set this value to 0. 有关详细信息,请参阅网络观察程序定价Azure 存储定价For more information, see Network Watcher Pricing and Azure Storage Pricing for additional details.

入站流被从 Internet IP 记录到了没有公共 IP 的虚拟机:对于没有通过与 NIC 关联的公共 IP 地址分配公共 IP 地址作为实例级公共 IP 的虚拟机,或者是属于基本负载均衡器后端池的一部分的虚拟机,请使用默认SNAT,并使用由 Azure 分配的 IP 地址以便于进行出站连接。Inbound flows logged from internet IPs to VMs without public IPs: VMs that don't have a public IP address assigned via a public IP address associated with the NIC as an instance-level public IP, or that are part of a basic load balancer back-end pool, use default SNAT and have an IP address assigned by Azure to facilitate outbound connectivity. 因此,如果流的目的地是分配给 SNAT 的端口范围内的端口,你可能会看到来自 Internet IP 地址的流的流日志条目。As a result, you might see flow log entries for flows from internet IP addresses, if the flow is destined to a port in the range of ports assigned for SNAT. 虽然 Azure 不允许将这些流传输到 VM,但是按照设计,该尝试会被记录并显示在网络观察程序的 NSG 流日志中。While Azure won't allow these flows to the VM, the attempt is logged and appears in Network Watcher's NSG flow log by design. 我们建议使用 NSG 来显式阻止不需要的入站 Internet 流量。We recommend that unwanted inbound internet traffic be explicitly blocked with NSG.

无状态流的字节和数据包计数不正确网络安全组 (NSG) 是作为有状态防火墙实现的。Incorrect byte and packet counts for Stateless flows: Network Security Groups (NSGs) are implemented as a Stateful firewall. 但是,许多用于控制流量流的默认/内部规则是以无状态方式实现的。However many default/internal rules that control the flow of traffic are implemented in a stateless fashion. 由于平台限制,不会为无状态流(即通过无状态规则的流量)记录字节和数据包计数,只会为有状态流记录它们。Due to platform limitations, the bytes and packets counts are not recorded for stateless flows (that is, traffic flows going through stateless rules), they are recorded only for stateful flows. 因此,NSG 流日志(和流量分析)中报告的字节数和数据包数可能与实际流不同。Consequently the number of bytes and packets reported in NSG Flow Logs (and Traffic Analytics) could be different from actual flows. 计划在 2020 年 6 月之前修复此限制。This limitation is scheduled to be fixed by June 2020.

示例日志记录Sample log records

以下文本是流日志的示例。The text that follows is an example of a flow log. 可以看到,有多个记录遵循前一部分描述的属性列表。As you can see, there are multiple records that follow the property list described in the preceding section.

Note

*flowTuples 属性中的值为逗号分隔列表。Values in the *flowTuples property are a comma-separated list.

版本 1 NSG 流日志格式示例Version 1 NSG flow log format sample

{
    "records": [
        {
            "time": "2017-02-16T22:00:32.8950000Z",
            "systemId": "2c002c16-72f3-4dc5-b391-3444c3527434",
            "category": "NetworkSecurityGroupFlowEvent",
            "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
            "operationName": "NetworkSecurityGroupFlowEvents",
            "properties": {
                "Version": 1,
                "flows": [
                    {
                        "rule": "DefaultRule_DenyAllInBound",
                        "flows": [
                            {
                                "mac": "000D3AF8801A",
                                "flowTuples": [
                                    "1487282421,42.119.146.95,10.1.0.4,51529,5358,T,I,D"
                                ]
                            }
                        ]
                    },
                    {
                        "rule": "UserRule_default-allow-rdp",
                        "flows": [
                            {
                                "mac": "000D3AF8801A",
                                "flowTuples": [
                                    "1487282370,163.28.66.17,10.1.0.4,61771,3389,T,I,A",
                                    "1487282393,5.39.218.34,10.1.0.4,58596,3389,T,I,A",
                                    "1487282393,91.224.160.154,10.1.0.4,61540,3389,T,I,A",
                                    "1487282423,13.76.89.229,10.1.0.4,53163,3389,T,I,A"
                                ]
                            }
                        ]
                    }
                ]
            }
        },
        {
            "time": "2017-02-16T22:01:32.8960000Z",
            "systemId": "2c002c16-72f3-4dc5-b391-3444c3527434",
            "category": "NetworkSecurityGroupFlowEvent",
            "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
            "operationName": "NetworkSecurityGroupFlowEvents",
            "properties": {
                "Version": 1,
                "flows": [
                    {
                        "rule": "DefaultRule_DenyAllInBound",
                        "flows": [
                            {
                                "mac": "000D3AF8801A",
                                "flowTuples": [
                                    "1487282481,195.78.210.194,10.1.0.4,53,1732,U,I,D"
                                ]
                            }
                        ]
                    },
                    {
                        "rule": "UserRule_default-allow-rdp",
                        "flows": [
                            {
                                "mac": "000D3AF8801A",
                                "flowTuples": [
                                    "1487282435,61.129.251.68,10.1.0.4,57776,3389,T,I,A",
                                    "1487282454,84.25.174.170,10.1.0.4,59085,3389,T,I,A",
                                    "1487282477,77.68.9.50,10.1.0.4,65078,3389,T,I,A"
                                ]
                            }
                        ]
                    }
                ]
            }
        },
    "records":
    [

        {
             "time": "2017-02-16T22:00:32.8950000Z",
             "systemId": "2c002c16-72f3-4dc5-b391-3444c3527434",
             "category": "NetworkSecurityGroupFlowEvent",
             "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
             "operationName": "NetworkSecurityGroupFlowEvents",
             "properties": {"Version":1,"flows":[{"rule":"DefaultRule_DenyAllInBound","flows":[{"mac":"000D3AF8801A","flowTuples":["1487282421,42.119.146.95,10.1.0.4,51529,5358,T,I,D"]}]},{"rule":"UserRule_default-allow-rdp","flows":[{"mac":"000D3AF8801A","flowTuples":["1487282370,163.28.66.17,10.1.0.4,61771,3389,T,I,A","1487282393,5.39.218.34,10.1.0.4,58596,3389,T,I,A","1487282393,91.224.160.154,10.1.0.4,61540,3389,T,I,A","1487282423,13.76.89.229,10.1.0.4,53163,3389,T,I,A"]}]}]}
        }
        ,
        {
             "time": "2017-02-16T22:01:32.8960000Z",
             "systemId": "2c002c16-72f3-4dc5-b391-3444c3527434",
             "category": "NetworkSecurityGroupFlowEvent",
             "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
             "operationName": "NetworkSecurityGroupFlowEvents",
             "properties": {"Version":1,"flows":[{"rule":"DefaultRule_DenyAllInBound","flows":[{"mac":"000D3AF8801A","flowTuples":["1487282481,195.78.210.194,10.1.0.4,53,1732,U,I,D"]}]},{"rule":"UserRule_default-allow-rdp","flows":[{"mac":"000D3AF8801A","flowTuples":["1487282435,61.129.251.68,10.1.0.4,57776,3389,T,I,A","1487282454,84.25.174.170,10.1.0.4,59085,3389,T,I,A","1487282477,77.68.9.50,10.1.0.4,65078,3389,T,I,A"]}]}]}
        }
        ,
        {
             "time": "2017-02-16T22:02:32.9040000Z",
             "systemId": "2c002c16-72f3-4dc5-b391-3444c3527434",
             "category": "NetworkSecurityGroupFlowEvent",
             "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
             "operationName": "NetworkSecurityGroupFlowEvents",
             "properties": {"Version":1,"flows":[{"rule":"DefaultRule_DenyAllInBound","flows":[{"mac":"000D3AF8801A","flowTuples":["1487282492,175.182.69.29,10.1.0.4,28918,5358,T,I,D","1487282505,71.6.216.55,10.1.0.4,8080,8080,T,I,D"]}]},{"rule":"UserRule_default-allow-rdp","flows":[{"mac":"000D3AF8801A","flowTuples":["1487282512,91.224.160.154,10.1.0.4,59046,3389,T,I,A"]}]}]}
        }
        ,
        ...

版本 2 NSG 流日志格式示例Version 2 NSG flow log format sample

 {
    "records": [
        {
            "time": "2018-11-13T12:00:35.3899262Z",
            "systemId": "a0fca5ce-022c-47b1-9735-89943b42f2fa",
            "category": "NetworkSecurityGroupFlowEvent",
            "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
            "operationName": "NetworkSecurityGroupFlowEvents",
            "properties": {
                "Version": 2,
                "flows": [
                    {
                        "rule": "DefaultRule_DenyAllInBound",
                        "flows": [
                            {
                                "mac": "000D3AF87856",
                                "flowTuples": [
                                    "1542110402,94.102.49.190,10.5.16.4,28746,443,U,I,D,B,,,,",
                                    "1542110424,176.119.4.10,10.5.16.4,56509,59336,T,I,D,B,,,,",
                                    "1542110432,167.99.86.8,10.5.16.4,48495,8088,T,I,D,B,,,,"
                                ]
                            }
                        ]
                    },
                    {
                        "rule": "DefaultRule_AllowInternetOutBound",
                        "flows": [
                            {
                                "mac": "000D3AF87856",
                                "flowTuples": [
                                    "1542110377,10.5.16.4,13.67.143.118,59831,443,T,O,A,B,,,,",
                                    "1542110379,10.5.16.4,13.67.143.117,59932,443,T,O,A,E,1,66,1,66",
                                    "1542110379,10.5.16.4,13.67.143.115,44931,443,T,O,A,C,30,16978,24,14008",
                                    "1542110406,10.5.16.4,40.71.12.225,59929,443,T,O,A,E,15,8489,12,7054"
                                ]
                            }
                        ]
                    }
                ]
            }
        },
        {
            "time": "2018-11-13T12:01:35.3918317Z",
            "systemId": "a0fca5ce-022c-47b1-9735-89943b42f2fa",
            "category": "NetworkSecurityGroupFlowEvent",
            "resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/FABRIKAMRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/FABRIAKMVM1-NSG",
            "operationName": "NetworkSecurityGroupFlowEvents",
            "properties": {
                "Version": 2,
                "flows": [
                    {
                        "rule": "DefaultRule_DenyAllInBound",
                        "flows": [
                            {
                                "mac": "000D3AF87856",
                                "flowTuples": [
                                    "1542110437,125.64.94.197,10.5.16.4,59752,18264,T,I,D,B,,,,",
                                    "1542110475,80.211.72.221,10.5.16.4,37433,8088,T,I,D,B,,,,",
                                    "1542110487,46.101.199.124,10.5.16.4,60577,8088,T,I,D,B,,,,",
                                    "1542110490,176.119.4.30,10.5.16.4,57067,52801,T,I,D,B,,,,"
                                ]
                            }
                        ]
                    }
                ]
            }
        },
        ...

后续步骤Next steps