Upload files to a Unity Catalog volume

  • Article

The Upload to volume UI allows you to upload files in any format to a Unity Catalog volume, including structured, semi-structured, and unstructured data. See What are Unity Catalog volumes?.

Uploaded files cannot exceed 5 gigabytes.

  • In Databricks Runtime 13.3 LTS and above, Databricks recommends using volumes to store .whl libraries for compute with assigned or shared access modes.
  • In Databricks Runtime 13.3 LTS and above, Databricks recommends using volumes to store JARs and init scripts for compute with assigned or shared access modes.

You can create a Unity Catalog managed table from an uploaded file. See Create a table from data in a volume.

You can also run various machine learning and data science workloads on files uploaded to a volume. Furthermore, you can upload libraries, certificates, and other configuration files of arbitrary formats, such as .whl or .txt, that you want to use to configure cluster libraries, notebook-scoped libraries, or job dependencies.

Where can you access the UI to upload files to a volume?

You can access this UI in the following ways:

  • In the sidebar, click New > Add data > Upload files to volume.
  • In Catalog Explorer, click Add > Upload to volume. You can also upload files directly to a volume or to a directory in a volume while browsing volumes in Catalog Explorer.
  • From within a notebook, by clicking File > Upload files to volume.

Note

Volumes are only supported in Databricks Runtime 13.3 LTS and above. In Databricks Runtime 12.2 LTS and below, operations against /Volumes paths might succeed, but might write data to ephemeral storage disks attached to compute clusters rather than persisting data to Unity Catalog volumes as expected.

Before you begin

Before you upload files to a Unity Catalog volume, you must have the following:

  • A workspace with Unity Catalog enabled. For more information, see Set up and manage Unity Catalog.
  • The WRITE VOLUME privilege on the volume you want to upload files to.
  • The USE SCHEMA privilege on the parent schema
  • The USE CATALOG privilege on the parent catalog.

For more information, see Unity Catalog privileges and securable objects.

Steps to upload files to a volume

To upload files to a Unity Catalog volume, do the following:

  1. Click New > Add Data.
  2. Click Upload files to volume.
  3. Select a volume or a directory inside a volume, or paste a volume path.
    • If no volume exists in the target schema, you can use the dialog to create a new volume.
    • Optionally, you can create a new directory within the target volume by specifying the full path to the target directory.
  4. Click the browse button or drag and drop files directly into the drop zone.

For information about other ways to upload files to Unity Catalog, see Work with files in Unity Catalog volumes.