Integrations overview

The following table summarizes the available connectors and their capabilities:

The following table summarizes the available tools and integrations and their capabilities:

Apache Kafka is a distributed streaming platform for building real-time streaming data pipelines that reliably move data between systems or applications. Kafka Connect is a tool for scalable and reliable streaming of data between Apache Kafka and other data systems. The Kafka Sink serves as the connector from Kafka and doesn't require using code. This is gold certified by Confluent - has gone through comprehensive review and testing for quality, feature completeness, compliance with standards, and for performance.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Logs, Telemetry, Time series
• Underlying SDK: Java
• Repository: Azure - https://github.com/Azure/kafka-sink-azure-kusto/
• Documentation: Ingest data from Apache Kafka
• Community Blog: Kafka ingestion into Azure Data Explorer

Log4J is a popular logging framework for Java applications maintained by the Apache Foundation. Log4j allows developers to control which log statements are output with arbitrary granularity based on the logger's name, logger level, and message pattern. The Apache Log4J 2 sink allows you to stream your log data to your database, where you can analyze and visualize your logs in real time.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Logs
• Underlying SDK: Java
• Repository: Azure - https://github.com/Azure/azure-kusto-log4j
• Documentation: Ingest data with the Apache Log4J 2 connector
• Community Blog: Getting started with Apache Log4J and Azure Data Explorer

Apache Spark is a unified analytics engine for large-scale data processing. The Spark connector is an open source project that can run on any Spark cluster. It implements data source and data sink for moving data to or from Spark clusters. Using the Apache Spark connector, you can build fast and scalable applications targeting data driven scenarios. For example, machine learning (ML), Extract-Transform-Load (ETL), and Log Analytics. With the connector, your database becomes a valid data store for standard Spark source and sink operations, such as read, write, and writeStream.

• Functionality: Ingestion, Export
• Ingestion type supported: Batching, Streaming
• Use cases: Telemetry
• Underlying SDK: Java
• Repository: Azure - https://github.com/Azure/azure-kusto-spark/
• Documentation: Apache Spark connector
• Community Blog: Data preprocessing for Azure Data Explorer for Azure Data Explorer with Apache Spark

The Azure Cosmos DB change feed data connection is an ingestion pipeline that listens to your Cosmos DB change feed and ingests the data into your database.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Change feed
• Documentation: Ingest data from Azure Cosmos DB (Preview)

Azure Data Factory (ADF) is a cloud-based data integration service that allows you to integrate different data stores and perform activities on the data.

• Functionality: Ingestion, Export
• Ingestion type supported: Batching
• Use cases: Data orchestration
• Documentation: Copy data to your database by using Azure Data Factory

Event Grid ingestion is a pipeline that listens to Azure storage, and updates your database to pull information when subscribed events occur. You can configure continuous ingestion from Azure Storage (Blob storage and ADLSv2) with an Azure Event Grid subscription for blob created or blob renamed notifications and streaming the notifications via Azure Event Hubs.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Event processing
• Documentation: Event Grid data connection

Azure Event Hubs is a big data streaming platform and event ingestion service. You can configure continuous ingestion from customer-managed Event Hubs.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Documentation: Azure Event Hubs data connection

Azure Functions allow you to run serverless code in the cloud on a schedule or in response to an event. With input and output bindings for Azure Functions, you can integrate your database into your workflows to ingest data and run queries against your database.

• Functionality: Ingestion, Export
• Ingestion type supported: Batching
• Use cases: Workflow integrations
• Documentation: Integrating Azure Functions using input and output bindings (preview)
• Community Blog: Azure Data Explorer (Kusto) Bindings for Azure Functions

Azure IoT Hub is a managed service, hosted in the cloud, that acts as a central message hub for bi-directional communication between your IoT application and the devices it manages. You can configure continuous ingestion from customer-managed IoT Hubs, using its Event Hubs compatible built in endpoint of device-to-cloud messages.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: IoT data
• Documentation: IoT Hub data connection

Cribl stream is a processing engine that securely collects, processes, and streams machine event data from any source. It allows you to parse and process that data for any destination for analysis.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Machine data processing including logs, metrics, instrumentation data
• Documentation: Ingest data from Cribl Stream into Azure Data Explorer

Fluent Bit is an open-source agent that collects logs, metrics, and traces from various sources. It allows you to filter, modify, and aggregate event data before sending it to storage.

• Functionality: Ingestion
• Ingestion type supported: Batching
• Use cases: Logs, Metrics, Traces
• Repository: fluent-bit Kusto Output Plugin
• Documentation: Ingest data with Fluent Bit into Azure Data Explorer
• Community Blog: Getting started with Fluent bit and Azure Data Explorer

Java Database Connectivity (JDBC) is a Java API used to connect to databases and execute queries. You can use JDBC to connect to Azure Data Explorer.

• Functionality: Query, visualization
• Underlying SDK: Java
• Documentation: Connect to Azure Data Explorer with JDBC

The Microsoft Logic Apps connector allows you to run queries and commands automatically as part of a scheduled or triggered task.

• Functionality: Ingestion, Export
• Ingestion type supported: Batching
• Use cases: Data orchestration
• Documentation: Microsoft Logic Apps and Azure Data Explorer

The Logstash plugin enables you to process events from Logstash into an Azure Data Explorer database for later analysis.

• Functionality: Ingestion
• Ingestion type supported: Batching
• Use cases: Logs
• Underlying SDK: Java
• Repository: Azure - https://github.com/Azure/logstash-output-kusto/
• Documentation: Ingest data from Logstash
• Community Blog: How to migrate from Elasticsearch to Azure Data Explorer

MATLAB is a programming and numeric computing platform used to analyze data, develop algorithms, and create models. You can get an authorization token in MATLAB for querying your data in Azure Data Explorer.

• Functionality: Query
• Documentation: Query data using MATLAB

NLog is a flexible and free logging platform for various .NET platforms, including .NET standard. NLog allows you to write to several targets, such as a database, file, or console. With NLog you can change the logging configuration on-the-fly. The NLog sink is a target for NLog that allows you to send your log messages to your database. The plugin provides an efficient way to sink your logs to your cluster.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Telemetry, Logs, Metrics
• Underlying SDK: .NET
• Repository: Azure - https://github.com/Azure/azure-kusto-nlog-sink
• Documentation: Ingest data with the NLog sink
• Community Blog: Getting started with NLog sink and Azure Data Explorer

Open Database Connectivity (ODBC) is a widely accepted application programming interface (API) for database access. Azure Data Explorer is compatible with a subset of the SQL Server communication protocol (MS-TDS). This compatibility enables the use of the ODBC driver for SQL Server with Azure Data Explorer.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Telemetry, Logs, Metrics
• Documentation: Connect to Azure Data Explorer with ODBC

The OpenTelemetry connector supports ingestion of data from many receivers into your database. It works as a bridge to ingest data generated by Open telemetry to your database by customizing the format of the exported data according to your needs.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Traces, Metrics, Logs
• Underlying SDK: Go
• Repository: Open Telemetry - https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/azuredataexplorerexporter
• Documentation: Ingest data from OpenTelemetry
• Community Blog: Getting started with Open Telemetry and Azure Data Explorer

Serilog is a popular logging framework for .NET applications. Serilog allows developers to control which log statements are output with arbitrary granularity based on the logger's name, logger level, and message pattern. The Serilog sink, also known as an appender, streams your log data to your database, where you can analyze and visualize your logs in real time.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Logs
• Underlying SDK: .NET
• Repository: Azure - https://github.com/Azure/serilog-sinks-azuredataexplorer
• Documentation: Ingest data with the Serilog sink
• Community Blog: Getting started with Serilog sink and Azure Data Explorer

Splunk Enterprise is a software platform that allows you to ingest data from many sources simultaneously.The Azure Data Explorer add-on sends data from Splunk to a table in your cluster.

• Functionality: Ingestion
• Ingestion type supported: Batching
• Use cases: Logs
• Underlying SDK: Python
• Repository: Azure - https://github.com/Azure/azure-kusto-splunk/tree/main/splunk-adx-alert-addon
• Documentation: Ingest data from Splunk
• Splunk Base: Azure Data Explorer Add-On for Splunk
• Community Blog: Getting started with Azure Data Explorer Add-On for Splunk

• Functionality: Ingestion
• Ingestion type supported: Batching
• Use cases: Logs
• Repository: Azure - https://github.com/Azure/azure-kusto-splunk
• Documentation: Ingest data from Splunk Universal Forwarder to Azure Data Explorer
• Community Blog: Ingest data using Splunk Universal forwarder into Azure Data Explorer

Telegraf is an open source, lightweight, minimal memory foot print agent for collecting, processing and writing telemetry data including logs, metrics, and IoT data. Telegraf supports hundreds of input and output plugins. It's widely used and well supported by the open source community. The output plugin serves as the connector from Telegraf and supports ingestion of data from many types of input plugins into your database.

• Functionality: Ingestion
• Ingestion type supported: Batching, Streaming
• Use cases: Telemetry, Logs, Metrics
• Underlying SDK: Go
• Repository: InfluxData - https://github.com/influxdata/telegraf/tree/master/plugins/outputs/azure_data_explorer
• Documentation: Ingest data from Telegraf
• Community Blog: New Azure Data Explorer output plugin for Telegraf enables SQL monitoring at huge scale

Azure CLI lets you manage Kusto resources.

• Functionality: Administration
• Documentation: az kusto

Azure Data Explorer integrates with Azure Blob Storage and Azure Data Lake Storage (Gen1 and Gen2), providing fast, cached, and indexed access to data stored in external storage.

• Functionality: Query
• Documentation: Query data in Azure Data Lake using Azure Data Explorer

The Kusto (KQL) extension for Azure Data Studio enables you to connect and query to Azure Data Explorer clusters. Users can write and run KQL queries and author notebooks with the Kusto kernel complete with IntelliSens

• Functionality: Query
• Documentation: Kusto extension overview, use Kusto, and use Kqlmagic

The Azure Data Explorer supports cross-service queries between Azure Data Explorer, Application Insights (AI), and Log Analytics (LA). You can query your Log Analytics or Application Insights workspace using Azure Data Explorer query tools and in a cross-service query.

• Functionality: Query
• Documentation: Azure Monitor

Kqlmagic is a command that extends the capabilities of the Python kernel in Azure Data Studio notebooks. You can combine Python and Kusto query language (KQL) to query and visualize data using rich Plotly library integrated with render commands. Kqlmagic brings you the benefit of notebooks, data analysis, and rich Python capabilities all in the same location. Supported data sources with Kqlmagic include Azure Data Explorer, Application Insights, and Azure Monitor logs.

• Functionality: Query, Visualization
• Documentation: Azure Notebooks

Delta Kusto is a Command-line interface (CLI) enabling Continuous Integration / Continuous Deployment (CI / CD) automation with Kusto objects (for example: tables, functions, policies, security roles) in Azure Data Explorer databases. It can work on a single database, multiple databases, or an entire cluster. Delta Kusto also supports multitenant scenarios.

• Functionality: Source control
• Documentation: Delta Kusto

Jupyter Notebook is an open-source web application that allows you to create and share documents containing live code, equations, visualizations, and narrative text. It's useful for a wide range of tasks, such as data cleaning and transformation, numerical simulation, statistical modeling, data visualization, and machine learning.

Kqlmagic extends the capabilities of the Python kernel in Jupyter Notebook so you can run Kusto Query Language (KQL) queries natively. You can combine Python and KQL to query and visualize data using the rich Plot.ly library integrated with the render operator. The kqlmagic extension is compatible with Jupyter Lab, Visual Studio Code Jupyter extension, and Azure Data Studio, and supported data sources include Azure Data Explorer, Azure Monitor logs, and Application Insights.

• Functionality: Query, Visualization
• Documentation: Jupyter Notebooks
• Community Blog: Analyze data in Azure Data Explorer using KQL magic for Jupyter Notebook

The open source repo contains C# parser and a semantic analyzer as well as a translator project that generates the same libraries in JavaScript.

• Functionality: language parsing
• Documentation: Kusto Query Language parser

Kusto.Explorer is free software for download and use on your Windows desktop. Kusto.Explorer allows you to query and analyze your data with Kusto Query Language (KQL) in a user-friendly interface.

• Functionality: Query, Visualization
• Documentation: Installation and user interface, using Kusto.Explorer
  • Additional articles include options, troubleshooting, keyboard shortcuts, code features

Kusto.Cli is a command-line utility for sending queries and control commands on a Kusto cluster.

• Functionality: Query
• Documentation: Kusto CLI

Lightingest is a command-line utility for ad-hoc data ingestion into Azure Data Explorer.

• Functionality: Ingestion
• Ingestion type supported: Batching
• Use cases: Historical data load
• Repository: LightIngest
• Documentation: Use LightIngest to ingest data

Microsoft Purview simplifies data governance by offering a unified service to manage your data from various sources.

• Functionality: Governance
• Documentation: Integrate Azure Data Explorer with Microsoft Purview

You can integrate the Monaco Editor with Kusto Query Language support (monaco-kusto) into your app. Integrating monaco-kusto into your app offers you an editing experience such as completion, colorization, refactoring, renaming, and go-to-definition

• Functionality: Query
• Repository: Monaco Editor
• Documentation: Kusto Query Language setup and usage for the Monaco editor - The Kusto Query Language editor that can be embedded in web and electron based applications.
  • Kusto Query Language plugin - Access the Kusto Query Language editor plugin.
  • Embedding the Azure Data Explorer web UI - The Azure Data Explorer web UI can be embedded in an iframe and hosted in third-party websites.

PowerShell scripts can use the Kusto client libraries, as PowerShell inherently integrates with .NET libraries.

• Functionality: Query
• Documentation: Use Kusto .NET client libraries from PowerShell

Sync Kusto is a tool that enables users to synchronize various Kusto schema entities, such as table schemas and stored functions. This synchronization is done between the local file system, an Azure Data Explorer database, and Azure repos.

• Functionality: Source control
• Repository: SyncKusto
• Documentation: Sync Kusto

Azure Data Explorer provides a web experience that enables you to connect to your Azure Data Explorer clusters and write, run, and share Kusto Query Language (KQL) commands and queries

• Functionality: Ingestion, Export, Query, Visualization
• Ingestion type supported: Batching, Streaming
• Documentation: Azure Data Explorer web UI, Get data from file