Resources for creating Microsoft Sentinel custom connectors

Microsoft Sentinel provides a wide range of out-of-the-box connectors for Azure services and external solutions, and also supports ingesting data from some sources without a dedicated connector.

If you're unable to connect your data source to Microsoft Sentinel using any of the existing solutions available, consider creating your own data source connector.

For a full list of supported connectors, see the Find your Microsoft Sentinel data connector).

Compare custom connector methods

The following table compares essential details about each method for creating custom connectors described in this article. Select the links in the table for more details about each method.

Method description	Capability	Serverless	Complexity
Azure Monitor Agent Best for collecting files from on-premises and IaaS sources	File collection, data transformation	No	Low
Logstash Best for on-premises and IaaS sources, any source for which a plugin is available, and organizations already familiar with Logstash	Supports all capabilities of the Azure Monitor Agent	No; requires a VM or VM cluster to run	Low; supports many scenarios with plugins
Logic Apps High cost; avoid for high-volume data Best for low-volume cloud sources	Codeless programming allows for limited flexibility, without support for implementing algorithms. If no available action already supports your requirements, creating a custom action may add complexity.	Yes	Low; simple, codeless development
PowerShell Best for prototyping and periodic file uploads	Direct support for file collection. PowerShell can be used to collect more sources, but will require coding and configuring the script as a service.	No	Low
Log Analytics API Best for ISVs implementing integration, and for unique collection requirements	Supports all capabilities available with the code.	Depends on the implementation	High
Azure Functions Best for high-volume cloud sources, and for unique collection requirements	Supports all capabilities available with the code.	Yes	High; requires programming knowledge

Connect with the Azure Monitor Agent

If your data source delivers events in text files, we recommend that you use the Azure Monitor Agent to create your custom connector.

• For more information, see Collect logs from a text file with Azure Monitor Agent.

• For an example of this method, see Collect logs from a JSON file with Azure Monitor Agent.

Connect with Logstash

If you're familiar with Logstash, you may want to use Logstash with the Logstash output plug-in for Microsoft Sentinel to create your custom connector.

With the Microsoft Sentinel Logstash Output plugin, you can use any Logstash input and filtering plugins, and configure Microsoft Sentinel as the output for a Logstash pipeline. Logstash has a large library of plugins that enable input from various sources, such as Event Hubs, Apache Kafka, Files, Databases, and Cloud services. Use filtering plug-ins to parse events, filter unnecessary events, obfuscate values, and more.

For examples of using Logstash as a custom connector, see:

• Radware Microsoft Sentinel implementation guide

For examples of useful Logstash plugins, see:

• Cloudwatch input plugin
• Azure Event Hubs plugin

Connect with Logic Apps

Use Azure Logic Apps to create a serverless, custom connector for Microsoft Sentinel.

1. Use one of the following triggers to start your Logic Apps:

   Trigger	Description
   A recurring task	For example, schedule your Logic App to retrieve data regularly from specific files, databases, or external APIs. For more information, see Create, schedule, and run recurring tasks and workflows in Azure Logic Apps.
   On-demand triggering	Run your Logic App on-demand for manual data collection and testing. For more information, see Call, trigger, or nest logic apps using HTTPS endpoints.
   HTTP/S endpoint	Recommended for streaming, and if the source system can start the data transfer. For more information, see Call service endpoints over HTTP or HTTPs.

2. Use any of the Logic App connectors that read information to get your events. For example:

   • Connect to a REST API
   • Connect to a SQL Server
   • Connect to a file system

   Tip

   Custom connectors to REST APIs, SQL Servers, and file systems also support retrieving data from on-premises data sources. For more information, see Install on-premises data gateway documentation.

3. Prepare the information you want to retrieve.

   For example, use the parse JSON action to access properties in JSON content, enabling you to select those properties from the dynamic content list when you specify inputs for your Logic App.

   For more information, see Perform data operations in Azure Logic Apps.

4. Write the data to Log Analytics.

   For more information, see the Azure Log Analytics Data Collector documentation.

For examples of how you can create a custom connector for Microsoft Sentinel using Logic Apps, see:

• Create a data pipeline with the Data Collector API
• Ingesting AlienVault OTX threat indicators into Microsoft Sentinel (blog)

Connect with PowerShell

The Upload-AzMonitorLog PowerShell script enables you to use PowerShell to stream events or context information to Microsoft Sentinel from the command line. This streaming effectively creates a custom connector between your data source and Microsoft Sentinel.

For example, the following script uploads a CSV file to Microsoft Sentinel:

Import-Csv .\testcsv.csv
| .\Upload-AzMonitorLog.ps1
-WorkspaceId '69f7ec3e-cae3-458d-b4ea-6975385-6e426'
-WorkspaceKey $WSKey
-LogTypeName 'MyNewCSV'
-AddComputerName
-AdditionalDataTaggingName "MyAdditionalField"
-AdditionalDataTaggingValue "Foo"

The Upload-AzMonitorLog PowerShell script script uses the following parameters:

Parameter	Description
WorkspaceId	Your Microsoft Sentinel workspace ID, where you'll be storing your data. Find your workspace ID and key.
WorkspaceKey	The primary or secondary key for the Microsoft Sentinel workspace where you'll be storing your data. Find your workspace ID and key.
LogTypeName	The name of the custom log table where you want to store the data. A suffix of _CL will automatically be added to the end of your table name.
AddComputerName	When this parameter exists, the script adds the current computer name to every log record, in a field named Computer.
TaggedAzureResourceId	When this parameter exists, the script associates all uploaded log records with the specified Azure resource. This association enables the uploaded log records for resource-context queries, and adheres to resource-centric, role-based access control.
AdditionalDataTaggingName	When this parameter exists, the script adds another field to every log record, with the configured name, and the value that's configured for the AdditionalDataTaggingValue parameter. In this case, AdditionalDataTaggingValue must not be empty.
AdditionalDataTaggingValue	When this parameter exists, the script adds another field to every log record, with the configured value, and the field name configured for the AdditionalDataTaggingName parameter. If the AdditionalDataTaggingName parameter is empty, but a value is configured, the default field name is DataTagging.

Find your workspace ID and key

Find the details for the WorkspaceID and WorkspaceKey parameters in Microsoft Sentinel:

1. In Microsoft Sentinel, select Settings on the left, and then select the Workspace settings tab.

2. Under Get started with Log Analytics > 1 Connect a data source, select Windows and Linux agents management.

3. Find your workspace ID, primary key, and secondary key on the Windows servers tabs.

Connect with the Log Analytics API

You can stream events to Microsoft Sentinel by using the Log Analytics Data Collector API to call a RESTful endpoint directly.

While calling a RESTful endpoint directly requires more programming, it also provides more flexibility.

For more information, see the Log Analytics Data collector API, especially the following examples:

• C#
• Python

Connect with Azure Functions

Use Azure Functions together with a RESTful API and various coding languages, such as PowerShell, to create a serverless custom connector.

For examples of this method, see:

• Monitoring Zoom with Microsoft Sentinel (blog)
• Deploy a Function App for getting Office 365 Management API data into Microsoft Sentinel (Microsoft Sentinel GitHub community)

Parse your custom connector data

To take advantage of the data collected with your custom connector, develop Advanced Security Information Model (ASIM) parsers to work with your connector. Using ASIM enables Microsoft Sentinel's built-in content to use your custom data and makes it easier for analysts to query the data.

If your connector method allows for it, you can implement part of the parsing as part of the connector to improve query time parsing performance:

• If you've used Logstash, use the Grok filter plugin to parse your data.
• If you've used an Azure function, parse your data with code.

You will still need to implement ASIM parsers, but implementing part of the parsing directly with the connector simplifies the parsing and improves performance.

Next steps

Use the data ingested into Microsoft Sentinel to secure your environment with any of the following processes:

• Visualize and monitor your data
• Investigate incidents
• Detect threats
• Automate threat prevention
• Hunt for threats

Also, learn about one example of creating a custom connector to monitor Zoom: Monitoring Zoom with Microsoft Sentinel.