Azure Structured Firewall Logs

  • Article

Structured logs are a type of log data that are organized in a specific format. They use a predefined schema to structure log data in a way that makes it easy to search, filter, and analyze. Unlike unstructured logs, which consist of free-form text, structured logs have a consistent format that machines can parse and analyze.

Azure Firewall's structured logs provide a more detailed view of firewall events. They include information such as source and destination IP addresses, protocols, port numbers, and action taken by the firewall. They also include more metadata, such as the time of the event and the name of the Azure Firewall instance.

Currently, the following diagnostic log categories are available for Azure Firewall:

  • Application rule log
  • Network rule log
  • DNS proxy log

These log categories use Azure diagnostics mode. In this mode, all data from any diagnostic setting is collected in the AzureDiagnostics table.

With structured logs, you're able to choose to use Resource Specific Tables instead of the existing AzureDiagnostics table. In case both sets of logs are required, at least two diagnostic settings need to be created per firewall.

Resource specific mode

In Resource specific mode, individual tables in the selected workspace are created for each category selected in the diagnostic setting. This method is recommended since it:

  • May reduce overall logging costs by up to 80%.
  • makes it much easier to work with the data in log queries
  • makes it easier to discover schemas and their structure
  • improves performance across both ingestion latency and query times
  • allows you to grant Azure RBAC rights on a specific table

New resource specific tables are now available in Diagnostic setting that allows you to utilize the following categories:

  • Network rule log - Contains all Network Rule log data. Each match between data plane and network rule creates a log entry with the data plane packet and the matched rule's attributes.
  • NAT rule log - Contains all DNAT (Destination Network Address Translation) events log data. Each match between data plane and DNAT rule creates a log entry with the data plane packet and the matched rule's attributes.
  • Application rule log - Contains all Application rule log data. Each match between data plane and Application rule creates a log entry with the data plane packet and the matched rule's attributes.
  • Threat Intelligence log - Contains all Threat Intelligence events.
  • IDPS log - Contains all data plane packets that were matched with one or more IDPS signatures.
  • DNS proxy log - Contains all DNS Proxy events log data.
  • Internal FQDN resolve failure log - Contains all internal Firewall FQDN resolution requests that resulted in failure.
  • Application rule aggregation log - Contains aggregated Application rule log data for Policy Analytics.
  • Network rule aggregation log - Contains aggregated Network rule log data for Policy Analytics.
  • NAT rule aggregation log - Contains aggregated NAT rule log data for Policy Analytics.

Enable structured logs

To enable Azure Firewall structured logs, you must first configure a Log Analytics workspace in your Azure subscription. This workspace is used to store the structured logs generated by Azure Firewall.

Once you configure the Log Analytics workspace, you can enable structured logs in Azure Firewall by navigating to the Firewall's Diagnostic settings page in the Azure portal. From there, you must select the Resource specific destination table and select the type of events you want to log.

Note

There's no requirement to enable this feature with a feature flag or Azure PowerShell commands.

Screenshot of Diagnostics settings page.

Structured log queries

A list of predefined queries is available in the Azure portal. This list has a predefined KQL (Kusto Query Language) log query for each category and joined query showing the entire Azure firewall logging events in single view.

Screenshot showing Azure Firewall queries.

Azure Firewall Workbook

Azure Firewall Workbook provides a flexible canvas for Azure Firewall data analysis. You can use it to create rich visual reports within the Azure portal. You can tap into multiple firewalls deployed across Azure and combine them into unified interactive experiences.

To deploy the new workbook that uses Azure Firewall Structured Logs, see Azure Monitor Workbook for Azure Firewall.

Next steps