Ingest data into Lakeflow Designer

This page describes the available options for bringing data into a Visual data prep in Lakeflow Designer. Designer can work with any data accessible through Azure Databricks.

All data ingestion in Designer starts with the Source operator. When you open a Source operator's configuration pane, you have the following options.

Browse existing tables

Click Browse existing to open the asset selector, where you can:

• Browse tables and volumes in Unity Catalog.
• Browse files in the workspace file system.

Use this option when your data is already in Azure Databricks.

Drag and drop a local file

Drag a CSV or Excel file from your local machine directly onto the canvas. Designer uploads the file to the workspace file system and automatically creates a Source operator.

Create a table from a file

Select Create table from file in the Source configuration pane to upload a local CSV or Excel file and create a Unity Catalog table from it. Use this option when you want the data persisted as a managed table rather than stored as a workspace file. Additionally, for large amounts of data, this provides the best performance.

Import from SharePoint

To import a file from SharePoint:

1. In the Source configuration pane, select Import from SharePoint.
2. Select a Unity Catalog connection that has access to your SharePoint site.
3. Provide the direct file link.

This requires a Unity Catalog connection configured for SharePoint access.

Next steps

• How to create a Visual data prep in Lakeflow Designer
• Built-in operators in Lakeflow Designer