Manage files in Unity Catalog volumes with the Databricks JDBC Driver

  • Article

This article describes how to upload, download, and delete files in Unity volumes using the Databricks JDBC Driver.

Requirements

  • Databricks JDBC Driver versions 2.6.38 or above.
  • By default, native query mode is enabled. Otherwise, add UseNativeQuery property to the JDBC connection string, setting its value to 1 or 2.

For a complete Java code example showing how to run this article's code snippets in the context of setting up Azure Databricks authentication and running SQL statements withe the Databricks JDBC Driver, see Authentication settings for the Databricks JDBC Driver.

Upload a file

To upload a file to a volume, you must add the StagingAllowedLocalPaths property to the JDBC connection string, setting this property's value to the path of the file to upload. To upload multiple files from separate locations, set this property to a comma-separated list of paths, for example /tmp/,/usr/tmp/.

To override the contents of any existing file in the specified upload location, add OVERWRITE.

The following Java code snippet shows how to upload a file to a volume.

// ...
p.put("StagingAllowedLocalPaths", "/tmp/");

Connection conn = DriverManager.getConnection(url, p);
Statement stmt = conn.createStatement();

stmt.executeQuery("PUT '" +
                  "/tmp/my-data.csv" +
                  "' INTO '" +
                  "/Volumes/main/default/my-volume/my-data.csv" +
                  "' OVERWRITE")
// ...

Download a file

The following Java code snippet shows how to download a file from a volume.

// ...
Connection conn = DriverManager.getConnection(url, p);
Statement stmt = conn.createStatement();

stmt.executeQuery("GET '" +
                  "/Volumes/main/default/my-volume/my-data.csv" +
                  "' TO '" +
                  "/tmp/my-downloaded-data.csv" +
                  "'")
// ...

Delete a file

The following Java code snippet shows how to delete a file from a volume.

// ...
Connection conn = DriverManager.getConnection(url, p);
Statement stmt = conn.createStatement();

stmt.executeQuery("REMOVE '" +
                  "/Volumes/main/default/my-volume/my-data.csv" +
                  "'")
// ...