Create and edit data collection rules (DCRs) and associations in Azure Monitor
There are multiple methods for creating a data collection rule (DCR) in Azure Monitor. In some cases, Azure Monitor can create and manage the DCR according to settings that you configure in the Azure portal. In other cases, you need to create your own DCRs to customize particular scenarios.
This article describes the different methods for creating and editing a DCR. For the contents of the DCR itself, see Structure of a data collection rule in Azure Monitor.
Permissions
You require the following permissions to create DCRs and associations:
Built-in role | Scopes | Reason |
---|---|---|
Monitoring Contributor |
|
Create or edit DCRs, assign rules to the machine, deploy associations. |
Virtual Machine Contributor Azure Connected Machine Resource Administrator |
|
Deploy agent extensions on the VM (virtual machine). |
Any role that includes the action Microsoft.Resources/deployments/* |
|
Deploy Azure Resource Manager templates. |
Automated methods to create a DCR
The following table lists methods to create data collection scenarios using the Azure portal where the DCR is created for you. In these cases, you don't need to interact directly with the DCR itself.
Scenario | Resources | Description |
---|---|---|
Container insights | Enable Container Insights | When you enable Container Insights on a Kubernetes cluster, a containerized version of the Azure Monitor agent is installed, and a DCR with association to the cluster is created that collects data according to the configuration you selected. You may need to modify this DCR to add a transformation. |
Workspace transformation | Add a transformation in a workspace data collection rule using the Azure portal | Create a transformation for any supported table in a Log Analytics workspace. This transformation is specified within a DCR, which is linked to the workspace. The transformation is then applied to any data sent to that table from any legacy workloads that don't yet utilize DCR. |
Create a DCR
To create a data collection rule using the Azure CLI, PowerShell, API, or ARM templates, create a JSON file, starting with one of the sample DCRs. Use information in Structure of a data collection rule in Azure Monitor to modify the JSON file for your particular environment and requirements.
Important
Create your data collection rule in the same region as your destination Log Analytics workspace or Azure Monitor workspace. You can associate the data collection rule to machines or containers from any subscription or resource group in the tenant. To send data across tenants, you must first enable Azure Lighthouse.
Create with Azure portal
The Azure portal provides a simplified experience for creating a DCR for virtual machines and virtual machine scale sets. Using this method, you don't need to understand the structure of a DCR unless you want to implement an advanced feature such as a transformation. The process for creating this DCR with various data sources is described in Collect data with Azure Monitor Agent.
Important
Create your data collection rule in the same region as your destination Log Analytics workspace or Azure Monitor workspace. You can associate the data collection rule to machines or containers from any subscription or resource group in the tenant. To send data across tenants, you must first enable Azure Lighthouse.
On the Monitor menu in the Azure portal, select Data Collection Rules > Create to open the DCR creation page.
The Basic page includes basic information about the DCR.
Setting | Description |
---|---|
Rule Name | Name for the DCR. The name should be something descriptive that helps you identify the rule. |
Subscription | Subscription to store the DCR. The subscription doesn't need to be the same subscription as the virtual machines. |
Resource group | Resource group to store the DCR. The resource group doesn't need to be the same resource group as the virtual machines. |
Region | Region to store the DCR. The region must be the same region as any Log Analytics workspace or Azure Monitor workspace used in a destination of the DCR. If you have workspaces in different regions, then create multiple DCRs associated with the same set of machines. |
Platform Type | Specifies the type of data sources that will be available for the DCR, either Windows or Linux. None allows for both. 1 |
Data Collection Endpoint | Specifies the data collection endpoint (DCE) used to collect data. The DCE is only required if you're using Azure Monitor Private Links. This DCE must be in the same region as the DCR. For more information, see How to set up data collection endpoints based on your deployment. |
1 This option sets the kind
attribute in the DCR. There are other values that can be set for this attribute, but they aren't available in the portal.
Add resources
The Resources page allows you to add resources to be associated with the DCR. Select + Add resources to select resources. The Azure Monitor agent will automatically be installed on any resources that don't already have it.
Important
The portal enables system-assigned managed identity on the target resources, along with existing user-assigned identities, if there are any. For existing applications, unless you specify the user-assigned identity in the request, the machine defaults to using system-assigned identity instead.
If the machine you're monitoring isn't in the same region as your destination Log Analytics workspace and you're collecting data types that require a DCE, select Enable Data Collection Endpoints and select an endpoint in the region of each monitored machine. If the monitored machine is in the same region as your destination Log Analytics workspace, or if you don't require a DCE, don't select a data collection endpoint on the Resources tab.
Add data sources
The Collect and deliver page allows you to add and configure data sources for the DCR and a destination for each.
Screen element | Description |
---|---|
Data source | Select a Data source type and define related fields based on the data source type you select. See the articles in Data sources for details on configuring each type of data source. |
Destination | Add one or more destinations for each data source. You can select multiple destinations of the same or different types. For instance, you can select multiple Log Analytics workspaces, which is also known as multihoming. See the details for each data type for the different destinations they support. |
A DCR can contain multiple different data sources up to a limit of 10 data sources in a single DCR. You can combine different data sources in the same DCR, but you will typically want to create different DCRs for different data collection scenarios. See Best practices for data collection rule creation and management in Azure Monitor for recommendations on how to organize your DCRs.
Note
It can take up to 5 minutes for data to be sent to the destinations when you create a data collection rule using the data collection rule wizard.
Edit a DCR
To edit a DCR, you can use any of the methods described in the previous section to create a DCR using a modified version of the JSON.
If you need to retrieve the JSON for an existing DCR, you can copy it from the JSON View for the DCR in the Azure portal. You can also retrieve it using an API call as shown in the following PowerShell example.
$ResourceId = "<ResourceId>" # Resource ID of the DCR to edit
$FilePath = "<FilePath>" # Store DCR content in this file
$DCR = Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2022-06-01") -Method GET
$DCR.Content | ConvertFrom-Json | ConvertTo-Json -Depth 20 | Out-File -FilePath $FilePath
For a tutorial that walks through the process of retrieving and then editing an existing DCR, see Tutorial: Edit a data collection rule (DCR).
Verify data flows and troubleshooting
DCR metrics are collected automatically for all DCRs, and you can analyze them using metrics explorer like the platform metrics for other Azure resources. For more information, see Monitor and troubleshoot DCR data collection in Azure Monitor
Metrics sent to a Log Analytics workspace, are stored in the AzureMetricsV2
table. Use the Log Analytics explorer to view the table and confirm that data is being ingested.
For more information, see Overview of Log Analytics in Azure Monitor.