Tutorial: Log network traffic to and from a virtual machine using the Azure portal

A network security group (NSG) enables you to filter inbound traffic to, and outbound traffic from, a virtual machine (VM). You can log network traffic that flows through an NSG with Network Watcher's NSG flow log capability.

In this tutorial, you learn how to:

  • Create a VM with a network security group
  • Enable Network Watcher and register the Microsoft.Insights provider
  • Enable a traffic flow log for an NSG, using Network Watcher's NSG flow log capability
  • Download logged data
  • View logged data

If you don't have an Azure subscription, create a trial subscription before you begin.

Prerequisites

  • An Azure account with an active subscription.

Sign in to Azure

Sign in to the Azure portal.

Create a virtual machine

  1. Sign in to the Azure portal.

  2. Select + Create a resource on the upper-left corner of the Azure portal. Enter Windows Server 2022 Datacenter: Azure Edition in the search filter of new page, and then select Windows Server 2022 Datacenter: Azure Edition item in search results.

  3. Select + Create in the Windows Server 2022 Datacenter: Azure Edition page.

  4. Enter or select the following information in Create a virtual machine.

    Setting Value
    Project details
    Subscription Select your subscription.
    Resource group Select Create new.
    Enter myResourceGroup in Name.
    Select OK.
    Instance details
    Virtual machine name Enter myVM.
    Region Select (Asia Pacific) China East.
    Availability options Select No infrastructure redundancy required.
    Image Select Windows Server 2022 Datacenter: Azure Edition - Gen2.
    Size Select a size.
    Administrator account
    Username Enter a username.
    Password Enter a password.
    Confirm password Confirm password.
    Inbound port rules
    Public inbound ports Leave the default of Allow selected ports.
    Select inbound ports Leave the default of RDP (3389).
  5. Select Review + create.

  6. Select Create.

The virtual machine takes a few minutes to create. Don't continue with the remaining steps until the VM has finished creating. While the portal creates the virtual machine, it also creates a network security group with the name myVM-nsg and associates it with the network interface for the VM.

Enable Network Watcher

If you already have a network watcher enabled in the China East region, skip to Register Insights provider.

  1. In the search box at the top of the portal, enter network watcher. Select Network Watcher in the search results.

  2. In the Overview page of Network Watcher, select > icon in Regions column of your subscription, to expand it, and then select ... to the right of China East, as shown in the following picture:

    Enable Network Watcher

  3. Select Enable Network Watcher in popup menu.

Register Insights provider

NSG flow logging requires the Microsoft.Insights provider. To register the provider, complete the following steps:

  1. In the search box at the top of the portal, enter subscriptions. Select Subscriptions in the search results.

  2. Select the subscription you want to enable the provider for in Subscriptions.

  3. Select Resource providers in Settings of your subscription.

  4. Enter Microsoft.Insights in the filter box.

  5. Confirm the status of the provider displayed is Registered. If the status is Unregistered, select the provider then select Register.

    Screenshot of registering microsoft insights provider.

Enable NSG flow log

NSG flow log data is written to an Azure Storage account. Complete the following steps to create a storage account for the log data.

  1. In the search box at the top of the portal, enter storage account. Select Storage accounts in the search results.

  2. In Storage accounts, select + Create.

  3. Enter or select the following information in Create a storage account.

    Setting Value
    Project details
    Subscription Select your subscription.
    Resource group Select myResourceGroup.
    Instance details
    Storage account name Enter a name for your storage account.
    Must be 3-24 characters long, and can contain only lowercase letters and numbers, and must be unique across all Azure Storage.
    Region Select (Asia Pacific) China East.
    Performance Leave the default of Standard.
    Redundancy Leave the default of Geo-redundant storage (GRS).
  4. Select Review.

  5. Select Create.

The storage account may take around a minute to create. Don't continue with the remaining steps until the storage account is created. In all cases, the storage account must be in the same region as the NSG.

  1. In the search box at the top of the portal, enter network watcher. Select Network Watcher in the search results.

  2. Select NSG flow logs in Logs, as shown in the following picture:

    Screenshot shows the Network Watcher NSG flow logs.

  3. From the list of NSGs, select the NSG named myVm-nsg.

  4. Under Flow logs settings, select On.

  5. Select the flow logging version. Version 2 contains flow-session statistics (Bytes and Packets)

    Select flow Logs version

  6. Select the storage account that you created in step 3 of Enable NSG flow log.

    Note

    NSG Flow Logs do not work with storage accounts that have hierarchical namespace enabled.

  7. Set Retention (days) to 5, and then select Save.

Download flow log

  1. From Network Watcher, in the portal, select NSG flow logs under LOGS.

  2. Select You can download flow logs from configured storage accounts, as shown in the following picture:

    Download flow logs

  3. Select the storage account that you configured in step 3 of Enable NSG flow log.

  4. Under Blob service, select Containers, and then select the insights-logs-networksecuritygroupflowevent container.

  5. In the container, navigate the folder hierarchy until you get to a PT1H.json file, as shown in the picture that follows. Log files are written to a folder hierarchy that follows the following naming convention:

    https://{storageAccountName}.blob.core.chinacloudapi.cn/insights-logs-networksecuritygroupflowevent/resourceId=/SUBSCRIPTIONS/{subscriptionID}/RESOURCEGROUPS/{resourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/{nsgName}/y={year}/m={month}/d={day}/h={hour}/m=00/macAddress={macAddress}/PT1H.json

    Flow log

  6. Select ... to the right of the PT1H.json file and select Download.

View flow log

The following example JSON displays data that you'll see in the PT1H.json file for each flow logged:

Version 1 flow log event

{
    "time": "2018-05-01T15:00:02.1713710Z",
    "systemId": "<Id>",
    "category": "NetworkSecurityGroupFlowEvent",
    "resourceId": "/SUBSCRIPTIONS/<subscriptionId>/RESOURCEGROUPS/MYRESOURCEGROUP/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/MYVM-NSG",
    "operationName": "NetworkSecurityGroupFlowEvents",
    "properties": {
        "Version": 1,
        "flows": [
            {
                "rule": "UserRule_default-allow-rdp",
                "flows": [
                    {
                        "mac": "<macAddress>",
                        "flowTuples": [
                            "1525186745,192.168.1.4,10.0.0.4,55960,3389,T,I,A"
                        ]
                    }
                ]
            }
        ]
    }
}

Version 2 flow log event

{
    "time": "2018-11-13T12:00:35.3899262Z",
    "systemId": "<Id>",
    "category": "NetworkSecurityGroupFlowEvent",
    "resourceId": "/SUBSCRIPTIONS/<subscriptionId>/RESOURCEGROUPS/MYRESOURCEGROUP/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/MYVM-NSG",
    "operationName": "NetworkSecurityGroupFlowEvents",
    "properties": {
        "Version": 2,
        "flows": [
            {
                "rule": "DefaultRule_DenyAllInBound",
                "flows": [
                    {
                        "mac": "<macAddress>",
                        "flowTuples": [
                            "1542110402,94.102.49.190,10.5.16.4,28746,443,U,I,D,B,,,,",
                            "1542110424,176.119.4.10,10.5.16.4,56509,59336,T,I,D,B,,,,",
                            "1542110432,167.99.86.8,10.5.16.4,48495,8088,T,I,D,B,,,,"
                        ]
                    }
                ]
            },
            {
                "rule": "DefaultRule_AllowInternetOutBound",
                "flows": [
                    {
                        "mac": "<macAddress>",
                        "flowTuples": [
                            "1542110377,10.5.16.4,13.67.143.118,59831,443,T,O,A,B,,,,",
                            "1542110379,10.5.16.4,13.67.143.117,59932,443,T,O,A,E,1,66,1,66",
                            "1542110379,10.5.16.4,13.67.143.115,44931,443,T,O,A,C,30,16978,24,14008",
                            "1542110406,10.5.16.4,40.71.12.225,59929,443,T,O,A,E,15,8489,12,7054"
                        ]
                    }
                ]
            }
        ]
    }
}

The value for mac in the previous output is the MAC address of the network interface that was created when the VM was created. The comma-separated information for flowTuples is as follows:

Example data What data represents Explanation
1542110377 Time stamp The time stamp of when the flow occurred, in UNIX EPOCH format. In the previous example, the date converts to May 1, 2018 at 2:59:05 PM GMT.
10.0.0.4 Source IP address The source IP address that the flow originated from. 10.0.0.4 is the private IP address of the VM you created in Create a virtual machine.
13.67.143.118 Destination IP address The destination IP address that the flow was destined to.
44931 Source port The source port that the flow originated from.
443 Destination port The destination port that the flow was destined to. Since the traffic was destined to port 443, the rule named UserRule_default-allow-rdp, in the log file processed the flow.
T Protocol Whether the protocol of the flow was TCP (T) or UDP (U).
O Direction Whether the traffic was inbound (I) or outbound (O).
A Action Whether the traffic was allowed (A) or denied (D).
C Flow State Version 2 Only Captures the state of the flow. Possible states are B: Begin, when a flow is created. Statistics aren't provided. C: Continuing for an ongoing flow. Statistics are provided at 5-minute intervals. E: End, when a flow is ended. Statistics are provided.
30 Packets sent - Source to destination Version 2 Only The total number of TCP or UDP packets sent from source to destination since the last update.
16978 Bytes sent - Source to destination Version 2 Only The total number of TCP or UDP packet bytes sent from source to destination since the last update. Packet bytes include the packet header and payload.
24 Packets sent - Destination to source Version 2 Only The total number of TCP or UDP packets sent from destination to source since the last update.
14008 Bytes sent - Destination to source Version 2 Only The total number of TCP and UDP packet bytes sent from destination to source since the last update. Packet bytes include packet header and payload.

Next steps

In this tutorial, you learned how to:

  • Enable NSG flow logging for an NSG
  • Download and view data logged in a file.

The raw data in the JSON file can be difficult to interpret. To visualize Flow Logs data, you can use Azure Traffic Analytics and Microsoft Power BI.

For alternate methods of enabling NSG Flow Logs, see PowerShell, Azure CLI, REST API, and Resource Manager templates.

Advance to the next article to learn how to monitor network communication between two virtual machines: