Connect to Matillion
Matillion ETL is an ETL/ELT tool built specifically for cloud database platforms including Azure Databricks. Matillion ETL has a modern, browser-based UI, with powerful, push-down ETL/ELT functionality.
You can integrate your Databricks SQL warehouses (formerly Databricks SQL endpoints) and Azure Databricks clusters with Matillion.
Connect to Matillion manually
This section describes how to connect an existing SQL warehouse or cluster in your Azure Databricks workspace to Matillion manually.
Requirements
Before you integrate with Matillion manually, you must have the following:
A Matillion ETL instance, which you can launch by using the Azure Marketplace or Azure Resource Manager (ARM).
An Azure Databricks personal access token or Microsoft Entra ID token.
Note
As a security best practice, when you authenticate with automated tools, systems, scripts, and apps, Databricks recommends that you use personal access tokens belonging to service principals instead of workspace users. To create tokens for service principals, see Manage tokens for a service principal.
Steps to connect
To connect to Matillion manually, do the following:
Get the name of the existing compute resource that you want to use (a SQL warehouse or cluster) within your workspace. Later, you will choose that name to complete the connection between your compute resource and your Matillion ETL instance.
- To view SQL warehouses in your workspace, click
SQL Warehouses in the sidebar. To create a new SQL warehouse, see Create a SQL warehouse. - To view the clusters in your workspace, click
Compute in the sidebar. To create a cluster, see Compute configuration reference.
- To view SQL warehouses in your workspace, click
Follow Connect to your Matillion ETL instance and log in to it in the Matillion documentation.
Follow Create a Delta Lake on Databricks project in the Matillion documentation.
For the settings in the Delta Lake Connection section within these instructions, enter the following information:
- For Workspace ID, enter the workspace ID of your Azure Databricks per-workspace URL. Do not include anything after the first "dot" (
.). See Workspace instance names, URLs, and IDs. - For Username, enter the word
token. - For Password, enter the Azure Databricks personal access token or Microsoft Entra ID token.
For the settings in the Delta Lake Defaults section within these instructions, for Cluster, choose the name of the SQL warehouse or cluster.
- For Workspace ID, enter the workspace ID of your Azure Databricks per-workspace URL. Do not include anything after the first "dot" (
Continue with Next steps.
Next steps
Explore one or more of the following resources on the Matillion website: