读取流日志

Artículo
11/11/2024

重要

2027 年 9 月 30 日，网络安全组 (NSG) 流日志将停用。在此停用过程中，从 2025 年 6 月 30 日起，你将无法再创建新的 NSG 流日志。建议迁移到虚拟网络流日志，从而克服 NSG 流日志的限制。自停用日期之后，将不再支持使用 NSG 流日志启用的流量分析，并且订阅中的现有 NSG 流日志资源将被删除。但是，不会删除 NSG 流日志记录，并将继续遵循其各自的保留策略。有关详细信息，请查看官方公告。

本文介绍如何使用 PowerShell 选择性地读取 Azure 网络观察程序流日志的某些部分，而无需分析整个日志。流日志存储于块 blob 中的存储帐户中。每个日志都是一个单独的块 blob，每小时生成一次，每隔几分钟更新一次最新数据。使用本文中提供的脚本，可以从流日志中读取最新数据，而无需下载整个日志。

本文中讨论的概念不局限于 PowerShell，适用于 Azure 存储 API 支持的所有语言。

先决条件

具有活动订阅的 Azure 帐户。创建试用版订阅。
PowerShell 已安装在计算机上。有关详细信息，请参阅在 Windows、Linux 和 macOS 上安装 PowerShell。本文需要 Az PowerShell 模块。有关详细信息，请参阅如何安装 Azure PowerShell。要查找已安装的版本，请运行 Get-Module -ListAvailable Az。
一个或多个区域中的流日志。有关详细信息，请参阅创建网络安全组流日志。
流日志和存储帐户订阅所需的 RBAC 权限。有关详细信息，请参阅网络观察程序 RBAC 权限。

检索阻止列表

网络安全组流日志

下方 PowerShell 脚本设置查询网络安全组流日志 blob 和列出 CloudBlockBlob 块 blob 中的块所需的变量。更新脚本以包含适合你环境的有效值，具体而言就是“yourSubscriptionId”、“FLOWLOGSVALIDATIONWESTCENTRALUS”、“V2VALIDATIONVM-NSG”、“yourStorageAccountName”、“ml-rg”、“000D3AF87856”、“11/11/2018 03:00”。例如，应将 yourSubscriptionId 替换为订阅 ID。

function Get-NSGFlowLogCloudBlockBlob {
    [CmdletBinding()]
    param (
        [string] [Parameter(Mandatory=$true)] $subscriptionId,
        [string] [Parameter(Mandatory=$true)] $NSGResourceGroupName,
        [string] [Parameter(Mandatory=$true)] $NSGName,
        [string] [Parameter(Mandatory=$true)] $storageAccountName,
        [string] [Parameter(Mandatory=$true)] $storageAccountResourceGroup,
        [string] [Parameter(Mandatory=$true)] $macAddress,
        [datetime] [Parameter(Mandatory=$true)] $logTime
    )

    process {
        # Retrieve the primary storage account key to access the network security group logs
        $StorageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroup -Name $storageAccountName).Value[0]

        # Setup a new storage context to be used to query the logs
        $ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey

        # Container name used by network security group flow logs
        $ContainerName = "insights-logs-networksecuritygroupflowevent"

        # Name of the blob that contains the network security group flow log
        $BlobName = "resourceId=/SUBSCRIPTIONS/${subscriptionId}/RESOURCEGROUPS/${NSGResourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/${NSGName}/y=$($logTime.Year)/m=$(($logTime).ToString("MM"))/d=$(($logTime).ToString("dd"))/h=$(($logTime).ToString("HH"))/m=00/macAddress=$($macAddress)/PT1H.json"

        # Gets the storage blog
        $Blob = Get-AzStorageBlob -Context $ctx -Container $ContainerName -Blob $BlobName

        # Gets the block blog of type 'Microsoft.Azure.Storage.Blob.CloudBlob' from the storage blob
        $CloudBlockBlob = [Microsoft.Azure.Storage.Blob.CloudBlockBlob] $Blob.ICloudBlob

        #Return the Cloud Block Blob
        $CloudBlockBlob
    }
}

function Get-NSGFlowLogBlockList  {
    [CmdletBinding()]
    param (
        [Microsoft.Azure.Storage.Blob.CloudBlockBlob] [Parameter(Mandatory=$true)] $CloudBlockBlob
    )
    process {
        # Stores the block list in a variable from the block blob.
        $blockList = $CloudBlockBlob.DownloadBlockListAsync()

        # Return the Block List
        $blockList
    }
}

$CloudBlockBlob = Get-NSGFlowLogCloudBlockBlob -subscriptionId "yourSubscriptionId" -NSGResourceGroupName "FLOWLOGSVALIDATIONWESTCENTRALUS" -NSGName "V2VALIDATIONVM-NSG" -storageAccountName "yourStorageAccountName" -storageAccountResourceGroup "ml-rg" -macAddress "000D3AF87856" -logTime "11/11/2018 03:00" 

$blockList = Get-NSGFlowLogBlockList -CloudBlockBlob $CloudBlockBlob

$blockList 变量返回 blob 中块的列表。每个块 blob 至少包含两个块。第一个块长度为 12 个字节，并包含 JSON 日志的开括号。另一个块是闭括号，其长度为 2 个字节。以下示例日志中有七个单独条目。日志中所有新条目会被添加到末尾、最后一个块之前。

Name                                         Length Committed
----                                         ------ ---------
ZDk5MTk5N2FkNGE0MmY5MTk5ZWViYjA0YmZhODRhYzY=     12      True
NzQxNDA5MTRhNDUzMGI2M2Y1MDMyOWZlN2QwNDZiYzQ=   2685      True
ODdjM2UyMWY3NzFhZTU3MmVlMmU5MDNlOWEwNWE3YWY=   2586      True
ZDU2MjA3OGQ2ZDU3MjczMWQ4MTRmYWNhYjAzOGJkMTg=   2688      True
ZmM3ZWJjMGQ0ZDA1ODJlOWMyODhlOWE3MDI1MGJhMTc=   2775      True
ZGVkYTc4MzQzNjEyMzlmZWE5MmRiNjc1OWE5OTc0OTQ=   2676      True
ZmY2MjUzYTIwYWIyOGU1OTA2ZDY1OWYzNmY2NmU4ZTY=   2777      True
Mzk1YzQwM2U0ZWY1ZDRhOWFlMTNhYjQ3OGVhYmUzNjk=   2675      True
ZjAyZTliYWE3OTI1YWZmYjFmMWI0MjJhNzMxZTI4MDM=      2      True

读取块 blob

在本部分中，将读取 $blocklist 变量以检索数据。在下面的示例中，我们将循环访问阻止列表，从每个块读取字节并将其存储在数组中。使用 DownloadRangeToByteArray 方法来检索数据。

网络安全组流日志

function Get-NSGFlowLogReadBlock  {
    [CmdletBinding()]
    param (
        [System.Array] [Parameter(Mandatory=$true)] $blockList,
        [Microsoft.Azure.Storage.Blob.CloudBlockBlob] [Parameter(Mandatory=$true)] $CloudBlockBlob

    )
    # Set the size of the byte array to the largest block
    $maxvalue = ($blocklist | measure Length -Maximum).Maximum

    # Create an array to store values in
    $valuearray = @()

    # Define the starting index to track the current block being read
    $index = 0

    # Loop through each block in the block list
    for($i=0; $i -lt $blocklist.count; $i++)
    {
        # Create a byte array object to story the bytes from the block
        $downloadArray = New-Object -TypeName byte[] -ArgumentList $maxvalue

        # Download the data into the ByteArray, starting with the current index, for the number of bytes in the current block. Index is increased by 3 when reading to remove preceding comma.
        $CloudBlockBlob.DownloadRangeToByteArray($downloadArray,0,$index, $($blockList[$i].Length)) | Out-Null

        # Increment the index by adding the current block length to the previous index
        $index = $index + $blockList[$i].Length

        # Retrieve the string from the byte array

        $value = [System.Text.Encoding]::ASCII.GetString($downloadArray)

        # Add the log entry to the value array
        $valuearray += $value
    }
    #Return the Array
    $valuearray
}
$valuearray = Get-NSGFlowLogReadBlock -blockList $blockList -CloudBlockBlob $CloudBlockBlob

现在 $valuearray 数组包含每个块的字符串值。若要验证该条目，请通过运行 $valuearray[$valuearray.Length-2] 从数组获取倒数第二个值。不需要最后一个值，因为它是右括号。

此值的结果如下例所示：

网络安全组流日志

{
	"records": [
		{
			"time": "2017-06-16T20:59:43.7340000Z",
			"systemId": "abcdef01-2345-6789-0abc-def012345678",
			"category": "NetworkSecurityGroupFlowEvent",
			"resourceId": "/SUBSCRIPTIONS/00000000-0000-0000-0000-000000000000/RESOURCEGROUPS/MYRESOURCEGROUP/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/MYNSG",
			"operationName": "NetworkSecurityGroupFlowEvents",
			"properties": {
				"Version": 1,
				"flows": [
					{
						"rule": "DefaultRule_AllowInternetOutBound",
						"flows": [
							{
								"mac": "000D3A18077E",
								"flowTuples": [
									"1497646722,10.0.0.4,168.62.32.14,44904,443,T,O,A",
									"1497646722,10.0.0.4,52.240.48.24,45218,443,T,O,A",
									"1497646725,10.0.0.4,168.62.32.14,44910,443,T,O,A",
									"1497646725,10.0.0.4,52.240.48.24,45224,443,T,O,A",
									"1497646728,10.0.0.4,168.62.32.14,44916,443,T,O,A",
									"1497646728,10.0.0.4,52.240.48.24,45230,443,T,O,A",
									"1497646732,10.0.0.4,168.62.32.14,44922,443,T,O,A",
									"1497646732,10.0.0.4,52.240.48.24,45236,443,T,O,A"
								]
							}
						]
					},
					{
						"rule": "DefaultRule_DenyAllInBound",
						"flows": []
					},
					{
						"rule": "UserRule_ssh-rule",
						"flows": []
					},
					{
						"rule": "UserRule_web-rule",
						"flows": [
							{
								"mac": "000D3A18077E",
								"flowTuples": [
									"1497646738,13.82.225.93,10.0.0.4,1180,80,T,I,A",
									"1497646750,13.82.225.93,10.0.0.4,1184,80,T,I,A",
									"1497646768,13.82.225.93,10.0.0.4,1181,80,T,I,A",
									"1497646780,13.82.225.93,10.0.0.4,1336,80,T,I,A"
								]
							}
						]
					}
				]
			}
		}
	]
}

Compartir a través de

读取流日志

先决条件

检索阻止列表

读取块 blob

Recursos adicionales

Compartir a través de

读取流日志

先决条件

检索阻止列表

读取块 blob

相关内容

Recursos adicionales