Enable VM monitoring in Azure Monitor

This article describes how to enable monitoring for virtual machines, virtual machine scale sets, and Arc-enabled servers at scale by using command line tools. These tools support infrastructure as code (IaC) and automation methods. These methods allow you to consistently deploy monitoring across your VM fleet and integrate monitoring configuration into your DevOps pipelines.

Supported machines

  • Azure virtual machines
  • Azure virtual machine scale sets
  • Arc-enabled servers

For a list of supported operating systems, see Azure Monitor agent supported operating systems.

Prerequisites

  • Azure Monitor workspace if you enable OpenTelemetry metrics (recommended experience for new deployments on Azure VMs and Arc-enabled servers). See Create an Azure Monitor workspace.
  • Log Analytics workspace if you enable logs-based metrics or collect logs. Logs-based metrics are the classic experience and are typically used for compatibility with existing implementations. See Create a Log Analytics workspace.
  • Permissions to create data collection rules (DCRs) and associate them with VMs. See Data collection rule permissions.
  • Azure Connected Machine agent if you're monitoring virtual machines hosted outside of Azure. You must first install the Connected Machine agent so that the machine can be managed through Azure Arc-enabled servers before installing the Azure Monitor agent and enabling monitoring. See Connect a machine to Azure Arc-enabled servers.

Overview

To enable full monitoring by collecting data from the guest operating system and workloads of a virtual machine by Azure Monitor, complete the three steps shown in the following table. When you create a DCR in the Azure portal, the portal automatically completes each of these steps for you.

Step Description
Install the Azure Monitor agent Install the agent on each virtual machine to monitor. You only need to install the agent once because it can use any number of DCRs that each collect different data.
Create data collection rules (DCRs) Each DCR specifies data to collect and where to send it. Create your own DCRs or use existing ones depending on your requirements. You need to understand the different types of DCRs and their purposes to determine which ones to use.
Associate DCRs with VMs When you create an association between a VM and a DCR, the agent downloads that DCR and begins data collection. Create associations with multiple DCRs for the agent to collect different types of data. Remove associations to stop data collection.

Install Azure Monitor agent

The first step is to install the Azure Monitor Agent extension on your virtual machines and Arc-enabled servers. The following example installs the agent on an Azure VM by using the Azure CLI:

# Windows
az vm extension set \
  --name AzureMonitorWindowsAgent \
  --publisher Microsoft.Azure.Monitor \
  --vm-name <vm-name> \
  --resource-group <resource-group>

For the full set of installation options, including Linux, Virtual Machine Scale Sets, Arc-enabled servers, PowerShell, the Azure portal, Azure Resource Manager templates, and automatic upgrade, see Install and manage the Azure Monitor Agent.

Create data collection rules

Data collection rules (DCRs) define what data to collect from the Azure Monitor agent and where to send it. Create different types of DCRs depending on what you want to monitor. Some DCRs enable features in the Azure portal, such as the enhanced monitoring experience for VMs, while others collect specific types of logs or metrics for analysis or alerting.

DCRs are structured in JSON. When you create DCRs using the Azure portal, you don't require any knowledge of the DCR structure. You may need to understand the DCR structure though to create DCRs from scratch or to add advanced functionality to existing DCRs such as adding a transformation.

The following table describes the most common DCR types used for VM monitoring. For a complete list of DCR types and their structures, see Data collection rule structure. For details on creating DCRs, see Data collection rules: Create and edit.

DCR Type Description
OpenTelemetry metrics Collects system-level performance counters by using OpenTelemetry standards. This option is the recommended metrics path for new VM monitoring deployments in the Azure portal. Use the DCR definition in the following section. Modify the counterSpecifiers section to add metrics to collect.
Log based metrics Collects predefined performance counters in a Log Analytics workspace. Enables the classic logs-based experience in the Azure portal. Use the DCR definition in the following section. Don't modify this DCR.
Logs Collect different types of logs from the VM, including Windows events and Syslog. These DCRs don't enable any additional experiences in Azure Monitor. Analyze them by using Log Analytics and use them for alerting. See Collect guest log data from virtual machines with Azure Monitor for a description of the different data sources available. See Data collection rule (DCR) samples in Azure Monitor for sample DCR definitions for log collection.

Use the following DCR definitions to enable enhanced monitoring for a virtual machine. The only modification needed is to update the location and destination workspace in each definition to point to your Azure Monitor workspace for OpenTelemetry metrics or your Log Analytics workspace for logs-based metrics.

Metrics-based experience 
{
  "location": "<location>",
  "properties": {
    "dataSources": {
      "performanceCountersOTel": [
        {
          "streams": [
            "Microsoft-OtelPerfMetrics"
          ],
          "samplingFrequencyInSeconds": 60,
          "counterSpecifiers": [
              "system.filesystem.usage",
              "system.disk.io",
              "system.disk.operation_time",
              "system.disk.operations",
              "system.memory.usage",
              "system.network.io",
              "system.cpu.time",
              "system.network.dropped",
              "system.network.errors",
              "system.uptime"
          ],
          "name": "OtelPerfCounters"
        }
      ]
    },
    "destinations": {
      "monitoringAccounts": [
        {
          "accountResourceId": "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Monitor/accounts/<workspace-name>",
          "name": "MonitoringAccount"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [
            "Microsoft-OtelPerfMetrics"
        ],
        "destinations": [
            "MonitoringAccount"
        ]
      }
    ]
  }
}
Logs-based experience (classic) 
{
    "location": "<location>",
    "properties": {
        "description": "Data collection rule for VM Insights.",
        "dataSources": {
            "performanceCounters": [
                {
                    "name": "VMInsightsPerfCounters",
                    "streams": [
                        "Microsoft-InsightsMetrics"
                    ],
                    "scheduledTransferPeriod": "PT1M",
                    "samplingFrequencyInSeconds": 60,
                    "counterSpecifiers": [
                        "\\VmInsights\\DetailedMetrics"
                    ]
                }
            ]
        },
        "destinations": {
            "logAnalytics": [
                {
                    "workspaceResourceId": "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.OperationalInsights/workspaces/<workspace-name>",
                    "name": "VMInsightsPerf-Logs-Dest"
                }
            ]
        },
        "dataFlows": [
            {
                "streams": [
                    "Microsoft-InsightsMetrics"
                ],
                "destinations": [
                    "VMInsightsPerf-Logs-Dest"
                ]
            }
        ]
    }
}

Save the DCR definition to a JSON file. For the CLI, PowerShell, API, and portal commands to create a DCR from a JSON file, see Create data collection rules (DCRs) using JSON.

Associate DCRs with VMs

The final step is to create associations between your DCRs and your VMs. This step activates the DCRs and tells the Azure Monitor agent to begin collecting data based on the rules defined in the DCR. Create multiple associations for a VM to collect different types of data. Remove associations to stop data collection from specific DCRs without affecting other associations or the agent itself.

Azure VM

az monitor data-collection rule association create \
  --name "dcr-association" \
  --rule-id /subscriptions/<subscription-id>/resourceGroups/<resource-group>/microsoft.insights/datacollectionrules/<dcr-name>
  --resource /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<vm-name>

Azure VM scale set

az monitor data-collection rule association create \
  --name "dcr-association" \
  --rule-id /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/datacollectionrules/<dcr-name>
  --resource /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Compute/virtualMachineScaleSets/<vmss-name>

Arc-enabled server

az monitor data-collection rule association create \
  --name "dcr-association" \
  --rule-id /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/datacollectionrules/<dcr-name> \
  --resource /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.HybridCompute/machines/<arc-server-name>

Deploy a complete onboarding template

The Azure Monitor Community repository hosts ARM and Bicep templates that complete all three onboarding steps in a single deployment. Each template enables a managed identity on the virtual machine, creates or reuses an Azure Monitor workspace and a DCR, associates the DCR, installs the agent, and optionally creates recommended alert rules.

Download the templates and parameter files from VM Insights onboarding templates in the Azure Monitor Community repository. The repository readme documents every parameter and the six deployment scenarios the parameter files cover.

These templates onboard individual Azure virtual machines. For virtual machine scale sets and Arc-enabled servers, follow the command line steps described earlier in this article. The deployment updates the managed identity configuration of an existing virtual machine, so review the identity parameters before deploying to a machine that already uses user-assigned identities.

The following Azure CLI examples use the az deployment sub create and az deployment group create commands.

Bicep

The Bicep template sets targetScope to subscription because its modules deploy into different resource groups. The .bicepparam file resolves the template through its using statement, so the command doesn't need a --template-file argument.

# Set variables
azureRegion="<AzureRegion>"
bicepParameterFile="<PathToBicepParameterFile>"

# Deploy the Bicep template
az deployment sub create \
  --location "$azureRegion" \
  --parameters "$bicepParameterFile"

ARM template

# Set variables
resourceGroupName="<VmResourceGroupName>"
templateFile="<PathToTemplateFile>"
parameterFile="<PathToParameterFile>"

# Deploy the ARM template
az deployment group create \
  --resource-group "$resourceGroupName" \
  --template-file "$templateFile" \
  --parameters "@$parameterFile"

Enable network isolation

VM insights supports two methods for network isolation, as described in the following table.

Method Description
Private link See Enable network isolation for Azure Monitor Agent by using Private Link.

Related content

  • Last updated on