Ingest syslog and CEF messages to Microsoft Sentinel with the Azure Monitor Agent

Create data collection rule (DCR)

To get started, open either the Syslog via AMA or Common Event Format (CEF) via AMA data connector in Microsoft Sentinel and create a data collection rule (DCR).

1. In the Azure portal, under Configuration, select Data connectors.

2. For syslog, type Syslog in the Search box. From the results, select the Syslog via AMA connector.
   For CEF, type CEF in the Search box. From the results, select the Common Event Format (CEF) via AMA connector.

3. Select Open connector page on the details pane.

4. In the Configuration area, select +Create data collection rule.

   

   

5. In the Basic tab:

   • Type a DCR name.
   • Select your subscription.
   • Select the resource group where you want to locate your DCR.

   

6. Select Next: Resources >.

Define VM resources

In the Resources tab, select the machines on which you want to install the AMA—in this case, your log forwarder machine. If your log forwarder doesn't appear in the list, it might not have the Azure Connected Machine agent installed.

1. Use the available filters or search box to find your log forwarder VM. Expand a subscription in the list to see its resource groups, and a resource group to see its VMs.

2. Select the log forwarder VM that you want to install the AMA on. The check box appears next to the VM name when you hover over it.

   

3. Review your changes and select Next: Collect >.

Select facilities and severities

Be aware that using the same facility for both syslog and CEF messages might result in data ingestion duplication. For more information, see Data ingestion duplication avoidance.

1. In the Collect tab, select the minimum log level for each facility. When you select a log level, Microsoft Sentinel collects logs for the selected level and other levels with higher severity. For example, if you select LOG_ERR, Microsoft Sentinel collects logs for the LOG_ERR, LOG_CRIT, LOG_ALERT, and LOG_EMERG levels.

   

2. Review your selections and select Next: Review + create.

Review and create the rule

After you complete all the tabs, review what you entered and create the data collection rule.

1. In the Review and create tab, select Create.

   

   The connector installs the Azure Monitor Agent on the machines you selected when creating your DCR.

2. Check the notifications in the Azure portal to see when the DCR is created and the agent is installed.

3. Select Refresh on the connector page to see the DCR displayed in the list.

Install the Azure Monitor Agent

Follow the appropriate instructions from the Azure Monitor documentation to install the Azure Monitor Agent on your log forwarder. Remember to use the instructions for Linux, not for Windows.

• Install the AMA using PowerShell
• Install the AMA using the Azure CLI
• Install the AMA using an Azure Resource Manager template

You can create Data Collection Rules (DCRs) using the Azure Monitor Logs Ingestion API. For more information, see Data collection rules in Azure Monitor.

Create the data collection rule

Create a JSON file for the data collection rule, create an API request, and send the request.

1. Prepare a DCR file in JSON format. The contents of this file is the request body in your API request.

   For an example, see Syslog/CEF DCR creation request body. To collect syslog and CEF messages in the same data collection rule, see the example Syslog and CEF streams in the same DCR.

   • Verify that the streams field is set to Microsoft-Syslog for syslog messages, or to Microsoft-CommonSecurityLog for CEF messages.
   • Add the filter and facility log levels in the facilityNames and logLevels parameters. See Examples of facilities and log levels sections.

2. Create an API request in a REST API client of your choosing.

   1. For the request URL and header, copy the following request URL and header.

      PUT https://management.chinacloudapi.cn/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{dataCollectionRuleName}?api-version=2022-06-01
      

      • Substitute the appropriate values for the {subscriptionId} and {resourceGroupName} placeholders.
      • Enter a name of your choice for the DCR in place of the {dataCollectionRuleName} placeholder.

   2. For the request body, copy and paste the contents of the DCR JSON file that you created (in step 1 above) into the request body.

3. Send the request.

   For an example of the response that you should receive, see Syslog/CEF DCR creation response.

Associate the DCR with the log forwarder

Now you need to create a DCR Association (DCRA) that ties the DCR to the VM resource that hosts your log forwarder.

1. Create an API request in a REST API client of your choosing.

2. For the request URL and header, copy the following request URL and the header.

   PUT 
   https://management.chinacloudapi.cn/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{virtualMachineName}/providers/Microsoft.Insights/dataCollectionRuleAssociations/{dataCollectionRuleAssociationName}?api-version=2022-06-01
   

   • Substitute the appropriate values for the {subscriptionId}, {resourceGroupName}, and {virtualMachineName} placeholders.
   • Enter a name of your choice for the DCR in place of the {dataCollectionRuleAssociationName} placeholder.

3. For the request body, copy the following request body.

   {
     "properties": {
       "dataCollectionRuleId": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Insights/dataCollectionRules/{dataCollectionRuleName}"
     }
   }
   

   • Substitute the appropriate values for the {subscriptionId} and {resourceGroupName} placeholders.
   • Enter a name of your choice for the DCR in place of the {dataCollectionRuleName} placeholder.

4. Send the request.

Examples of facilities and log levels sections

Review these examples of the facilities and log levels settings. The name field includes the filter name.

For CEF message ingestion, the value for "streams" should be "Microsoft-CommonSecurityLog" instead of "Microsoft-Syslog".

This example collects events from the cron, daemon, local0, local3 and uucp facilities, with the Warning, Error, Critical, Alert, and Emergency log levels:

    "dataSources": {
      "syslog": [
        {
        "name": "SyslogStream0",
        "streams": [
          "Microsoft-Syslog"
        ],
        "facilityNames": [ 
          "cron",
          "daemon",
          "local0",
          "local3", 
          "uucp"
        ],
        "logLevels": [ 
          "Warning", 
          "Error", 
          "Critical", 
          "Alert", 
          "Emergency"
        ]
      }
    ]
  }

Syslog and CEF streams in the same DCR

This example shows how you can collect syslog and CEF messages in the same DCR.

The DCR collects CEF event messages for:

• The authpriv and mark facilities with the Info, Notice, Warning, Error, Critical, Alert, and Emergency log levels
• The daemon facility with the Warning, Error, Critical, Alert, and Emergency log levels

It collects syslog event messages for:

• The kern, local0, local5, and news facilities with the Critical, Alert, and Emergency log levels
• The mail and uucp facilities with the Emergency log level

    "dataSources": {
      "syslog": [
        {
          "name": "CEFStream1",
          "streams": [ 
            "Microsoft-CommonSecurityLog"
          ],
          "facilityNames": [ 
            "authpriv", 
            "mark"
          ],
          "logLevels": [
            "Info",
            "Notice", 
            "Warning", 
            "Error", 
            "Critical", 
            "Alert", 
            "Emergency"
          ]
        },
        {
          "name": "CEFStream2",
          "streams": [ 
            "Microsoft-CommonSecurityLog"
          ],
          "facilityNames": [ 
            "daemon"
          ],
          "logLevels": [ 
            "Warning", 
            "Error", 
            "Critical", 
            "Alert", 
            "Emergency"
          ]
        },
        {
          "name": "SyslogStream3",
          "streams": [ 
            "Microsoft-Syslog"
          ],
          "facilityNames": [ 
            "kern",
            "local0",
            "local5", 
            "news"
          ],
          "logLevels": [ 
            "Critical", 
            "Alert", 
            "Emergency"
          ]
        },
        {
          "name": "SyslogStream4",
          "streams": [ 
            "Microsoft-Syslog"
          ],
          "facilityNames": [ 
            "mail",
            "uucp"
          ],
          "logLevels": [ 
            "Emergency"
          ]
        }
      ]
    }