Use JavaScript SDK in Node.js to manage directories and files in Azure Data Lake Storage

This article shows you how to use Node.js to create and manage directories and files in storage accounts that have a hierarchical namespace.

To learn about how to get, set, and update the access control lists (ACL) of directories and files, see Use JavaScript SDK in Node.js to manage ACLs in Azure Data Lake Storage.

Package (Node Package Manager) | Samples | Give Feedback

Prerequisites

• An Azure subscription. For more information, see Get Azure trial.

• A storage account that has hierarchical namespace enabled. Follow these instructions to create one.

• If you're using this package in a Node.js application, you'll need Node.js 8.0.0 or higher.

Set up your project

Install Data Lake client library for JavaScript by opening a terminal window, and then typing the following command.

npm install @azure/storage-file-datalake

Import the storage-file-datalake package by placing this statement at the top of your code file.

const {
AzureStorageDataLake,
DataLakeServiceClient,
StorageSharedKeyCredential
} = require("@azure/storage-file-datalake");

Connect to the account

To use the snippets in this article, you'll need to create a DataLakeServiceClient instance that represents the storage account.

Connect by using Microsoft Entra ID

You can use the Azure identity client library for JS to authenticate your application with Microsoft Entra ID.

Create a DataLakeServiceClient instance and pass in a new instance of the DefaultAzureCredential class.

function GetDataLakeServiceClientAD(accountName) {

  const dataLakeServiceClient = new DataLakeServiceClient(
      `https://${accountName}.dfs.core.chinacloudapi.cn`,
      new DefaultAzureCredential());

  return dataLakeServiceClient;
}

To learn more about using DefaultAzureCredential to authorize access to data, see Overview: Authenticate JavaScript apps to Azure using the Azure SDK.

Connect by using an account key

You can authorize access to data using your account access keys (Shared Key). This example creates a DataLakeServiceClient instance that is authorized with the account key.

function GetDataLakeServiceClient(accountName, accountKey) {

  const sharedKeyCredential =
     new StorageSharedKeyCredential(accountName, accountKey);

  const dataLakeServiceClient = new DataLakeServiceClient(
      `https://${accountName}.dfs.core.chinacloudapi.cn`, sharedKeyCredential);

  return dataLakeServiceClient;
}

This method of authorization works only for Node.js applications. If you plan to run your code in a browser, you can authorize by using Microsoft Entra ID.

Create a container

A container acts as a file system for your files. You can create one by getting a FileSystemClient instance, and then calling the FileSystemClient.Create method.

This example creates a container named my-file-system.

async function CreateFileSystem(dataLakeServiceClient) {

  const fileSystemName = "my-file-system";

  const fileSystemClient = dataLakeServiceClient.getFileSystemClient(fileSystemName);

  const createResponse = await fileSystemClient.create();

}

Create a directory

Create a directory reference by getting a DirectoryClient instance, and then calling the DirectoryClient.create method.

This example adds a directory named my-directory to a container.

async function CreateDirectory(fileSystemClient) {

  const directoryClient = fileSystemClient.getDirectoryClient("my-directory");

  await directoryClient.create();

}

Rename or move a directory

Rename or move a directory by calling the DirectoryClient.rename method. Pass the path of the desired directory a parameter.

This example renames a subdirectory to the name my-directory-renamed.

async function RenameDirectory(fileSystemClient) {

  const directoryClient = fileSystemClient.getDirectoryClient("my-directory");
  await directoryClient.move("my-directory-renamed");

}

This example moves a directory named my-directory-renamed to a subdirectory of a directory named my-directory-2.

async function MoveDirectory(fileSystemClient) {

  const directoryClient = fileSystemClient.getDirectoryClient("my-directory-renamed");
  await directoryClient.move("my-directory-2/my-directory-renamed");

}

Delete a directory

Delete a directory by calling the DirectoryClient.delete method.

This example deletes a directory named my-directory.

async function DeleteDirectory(fileSystemClient) {

  const directoryClient = fileSystemClient.getDirectoryClient("my-directory");
  await directoryClient.delete();

}

Upload a file to a directory

First, read a file. This example uses the Node.js fs module. Then, create a file reference in the target directory by creating a FileClient instance, and then calling the FileClient.create method. Upload a file by calling the FileClient.append method. Make sure to complete the upload by calling the FileClient.flush method.

This example uploads a text file to a directory named my-directory.`

async function UploadFile(fileSystemClient) {

  const fs = require('fs')

  var content = "";

  fs.readFile('mytestfile.txt', (err, data) => {
      if (err) throw err;

      content = data.toString();

  })

  const fileClient = fileSystemClient.getFileClient("my-directory/uploaded-file.txt");
  await fileClient.create();
  await fileClient.append(content, 0, content.length);
  await fileClient.flush(content.length);

}

Download from a directory

First, create a FileSystemClient instance that represents the file that you want to download. Use the FileSystemClient.read method to read the file. Then, write the file. This example uses the Node.js fs module to do that.

async function DownloadFile(fileSystemClient) {

  const fileClient = fileSystemClient.getFileClient("my-directory/uploaded-file.txt");

  const downloadResponse = await fileClient.read();

  const downloaded = await streamToString(downloadResponse.readableStreamBody);

  async function streamToString(readableStream) {
    return new Promise((resolve, reject) => {
      const chunks = [];
      readableStream.on("data", (data) => {
        chunks.push(data.toString());
      });
      readableStream.on("end", () => {
        resolve(chunks.join(""));
      });
      readableStream.on("error", reject);
    });
  }

  const fs = require('fs');

  fs.writeFile('mytestfiledownloaded.txt', downloaded, (err) => {
    if (err) throw err;
  });
}

List directory contents

This example, prints the names of each directory and file that is located in a directory named my-directory.

async function ListFilesInDirectory(fileSystemClient) {

  let i = 1;

  let iter = await fileSystemClient.listPaths({path: "my-directory", recursive: true});

  for await (const path of iter) {

    console.log(`Path ${i++}: ${path.name}, is directory: ${path.isDirectory}`);
  }

}

See also

• Package (Node Package Manager)
• Samples
• Give Feedback