az ml datastore

Commands for managing and using datastores with the Azure ML Workspace.

Commands

az ml datastore attach-adls

Attach an ADLS datastore.

az ml datastore attach-adls-gen2

Attach a ADLS Gen2 datastore.

az ml datastore attach-blob

Attach a blob storage datastore.

az ml datastore attach-dbfs

Attach a Databricks File System datastore.

az ml datastore attach-file

Attach a file share datastore.

az ml datastore attach-mysqldb

Attach an Azure MySQL datastore.

az ml datastore attach-psqldb

Attach an Azure PostgreSQL datastore.

az ml datastore attach-sqldb

Attach an Azure SQL datastore.

az ml datastore detach

Detach a datastore by name.

az ml datastore download

Download files from a Datastore.

az ml datastore list

List datastores in the workspace.

az ml datastore set-default

Set the workspace default datastore by name.

az ml datastore show

Show a single datastore by name.

az ml datastore show-default

Show the workspace default datastore.

az ml datastore upload

Upload files to a Datastore.

az ml datastore attach-adls

Attach an ADLS datastore.

az ml datastore attach-adls --client-id
                            --client-secret
                            --name
                            --store-name
                            --tenant-id
                            [--adls-resource-group]
                            [--adls-subscription-id]
                            [--authority-url]
                            [--include-secret]
                            [--output-metadata-file]
                            [--path]
                            [--resource-group]
                            [--resource-url]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--client-id

The service principal's client/application ID.

--client-secret

The service principal's secret.

--name -n

The datastore name.

--store-name -c

The ADLS store name.

--tenant-id

The service principal Tenant ID.

Optional Parameters

--adls-resource-group

The resource group the ADLS store belongs to.

--adls-subscription-id

The ID of the subscription the ADLS store belongs to.

--authority-url

Authority url used to authenticate the user.

default value: <a href="https://login.microsoftonline.com">https://login.microsoftonline.com</a>
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--resource-url

Determines what operations will be performed on the data lake store. Defaults to https://datalake.azure.net/ allowing for filesystem operations.

default value: <a href="https://datalake.azure.net/">https://datalake.azure.net/</a>
--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-adls-gen2

Attach a ADLS Gen2 datastore.

az ml datastore attach-adls-gen2 --account-name
                                 --client-id
                                 --client-secret
                                 --file-system
                                 --name
                                 --tenant-id
                                 [--authority-url]
                                 [--endpoint]
                                 [--include-secret]
                                 [--output-metadata-file]
                                 [--path]
                                 [--protocol]
                                 [--resource-group]
                                 [--resource-url]
                                 [--subscription-id]
                                 [--workspace-name]

Required Parameters

--account-name -a

The name of the storage account.

--client-id

The service principal's client/application ID.

--client-secret

The service principal's secret.

--file-system -c

The file system name of the ADLS Gen2.

--name -n

The datastore name.

--tenant-id

The service principal Tenant ID.

Optional Parameters

--authority-url

Authority url used to authenticate the user.

default value: <a href="https://login.microsoftonline.com">https://login.microsoftonline.com</a>
--endpoint

The endpoint of the storage account. Defaults to core.windows.net.

default value: core.windows.net
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--protocol

Protocol to use to connect to the blob container. If not specified, defaults to https.

default value: https
--resource-group -g

Resource group name.

--resource-url

Determines what operations will be performed on the data lake store. Defaults to https://storage.azure.com/ allowing for filesystem operations.

default value: <a href="https://storage.azure.com/">https://storage.azure.com/</a>
--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-blob

Attach a blob storage datastore.

az ml datastore attach-blob --account-name
                            --container-name
                            --name
                            [--account-key]
                            [--endpoint]
                            [--grant-workspace-msi-access]
                            [--include-secret]
                            [--output-metadata-file]
                            [--path]
                            [--protocol]
                            [--resource-group]
                            [--sas-token]
                            [--storage-account-resource-group]
                            [--storage-account-subscription-id]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--account-name -a

The name of the storage account.

--container-name -c

The blob container name.

--name -n

The datastore name.

Optional Parameters

--account-key -k

The storage account key.

--endpoint

The endpoint of the storage account. Defaults to core.windows.net.

default value: core.windows.net
--grant-workspace-msi-access

Grants Workspace Managed Service Identity (MSI) access to the storage account. Defaults to False. This should be set if the storage account is in a VNet. If set to True, we will use the current user's identity to grant the Workspace MSI access to the storage account. It may take up to 5 minutes for the granted access to take effect so the first time you run this, it can fail. If such an event occurs, please rerun this command after 5 minutes. If this error persists, please contact Azure Support.

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--protocol

Protocol to use to connect to the blob container. If not specified, defaults to https.

default value: https
--resource-group -g

Resource group name.

--sas-token

A SAS token for the blob container.

--storage-account-resource-group

The resource group of the storage account.

--storage-account-subscription-id

The subscription ID of the storage account.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-dbfs

Attach a Databricks File System datastore.

az ml datastore attach-dbfs --name
                            [--output-metadata-file]
                            [--path]
                            [--resource-group]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-file

Attach a file share datastore.

az ml datastore attach-file --account-name
                            --name
                            --share-name
                            [--account-key]
                            [--endpoint]
                            [--include-secret]
                            [--output-metadata-file]
                            [--path]
                            [--protocol]
                            [--resource-group]
                            [--sas-token]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--account-name -a

The name of the storage account.

--name -n

The datastore name.

--share-name -c

The file share name.

Optional Parameters

--account-key -k

The storage account key.

--endpoint

The endpoint of the storage account. Defaults to core.windows.net.

default value: core.windows.net
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--protocol

Protocol to use to connect to the blob container. If not specified, defaults to https.

default value: https
--resource-group -g

Resource group name.

--sas-token

A SAS token for the blob container.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-mysqldb

Attach an Azure MySQL datastore.

az ml datastore attach-mysqldb --database-name
                               --name
                               --password
                               --server-name
                               --user-id
                               [--endpoint]
                               [--include-secret]
                               [--output-metadata-file]
                               [--path]
                               [--port]
                               [--resource-group]
                               [--subscription-id]
                               [--workspace-name]

Required Parameters

--database-name -d

The database name.

--name -n

The datastore name.

--password -p

The password.

--server-name

The SQL/PostgreSQL/MySQL server name.

--user-id -u

The user ID.

Optional Parameters

--endpoint

The endpoint of the server. Defaults to mysql.database.azure.com.

default value: mysql.database.azure.com
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--port

The port number.

default value: 5432
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-psqldb

Attach an Azure PostgreSQL datastore.

az ml datastore attach-psqldb --database-name
                              --name
                              --password
                              --server-name
                              --user-id
                              [--endpoint]
                              [--enforce-ssl]
                              [--include-secret]
                              [--output-metadata-file]
                              [--path]
                              [--port]
                              [--resource-group]
                              [--subscription-id]
                              [--workspace-name]

Required Parameters

--database-name -d

The database name.

--name -n

The datastore name.

--password -p

The password.

--server-name

The SQL/PostgreSQL/MySQL server name.

--user-id -u

The user ID.

Optional Parameters

--endpoint

The endpoint of the server. Defaults to postgres.database.azure.com.

default value: postgres.database.azure.com
--enforce-ssl

This sets the ssl value of the server. Defaults to true if not set.

default value: 1
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--port

The port number.

default value: 5432
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-sqldb

Attach an Azure SQL datastore.

az ml datastore attach-sqldb --database-name
                             --name
                             --server-name
                             [--authority-url]
                             [--client-id]
                             [--client-secret]
                             [--endpoint]
                             [--include-secret]
                             [--output-metadata-file]
                             [--password]
                             [--path]
                             [--resource-group]
                             [--resource-url]
                             [--subscription-id]
                             [--tenant-id]
                             [--username]
                             [--workspace-name]

Required Parameters

--database-name -d

The database name.

--name -n

The datastore name.

--server-name

The SQL/PostgreSQL/MySQL server name.

Optional Parameters

--authority-url

Authority url used to authenticate the user.

default value: <a href="https://login.microsoftonline.com">https://login.microsoftonline.com</a>
--client-id

The service principal/application ID.

--client-secret

The service principal's secret.

--endpoint

The endpoint of the sql server. Defaults to database.windows.net.

default value: database.windows.net
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--password

The password of the database user to access the database.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--resource-url

Determines what operations will be performed on the database.

default value: <a href="https://database.windows.net/">https://database.windows.net/</a>
--subscription-id

Specifies the subscription Id.

--tenant-id

The service principal Tenant ID.

--username

The username of the database user to access the database.

--workspace-name -w

Workspace name.

az ml datastore detach

Detach a datastore by name.

az ml datastore detach --name
                       [--output-metadata-file]
                       [--path]
                       [--resource-group]
                       [--subscription-id]
                       [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore download

Download files from a Datastore.

az ml datastore download --name
                         --target-path
                         [--hide-progress]
                         [--output-metadata-file]
                         [--overwrite]
                         [--path]
                         [--prefix]
                         [--resource-group]
                         [--subscription-id]
                         [--workspace-name]

Required Parameters

--name -n

The datastore name.

--target-path -d

Target path for the downloaded files.

Optional Parameters

--hide-progress

Whether to hide progress of operation.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--overwrite

Overwrite target files if they exist.

--path

Path to a root directory for run configuration files.

default value: .
--prefix -p

Path filter for files to download. If none is provided, downloads everything.

--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore list

List datastores in the workspace.

az ml datastore list [--include-secret]
                     [--output-metadata-file]
                     [--path]
                     [--resource-group]
                     [--subscription-id]
                     [--workspace-name]

Optional Parameters

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore set-default

Set the workspace default datastore by name.

az ml datastore set-default --name
                            [--output-metadata-file]
                            [--path]
                            [--resource-group]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore show

Show a single datastore by name.

az ml datastore show --name
                     [--include-secret]
                     [--output-metadata-file]
                     [--path]
                     [--resource-group]
                     [--subscription-id]
                     [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore show-default

Show the workspace default datastore.

az ml datastore show-default [--include-secret]
                             [--output-metadata-file]
                             [--path]
                             [--resource-group]
                             [--subscription-id]
                             [--workspace-name]

Optional Parameters

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore upload

Upload files to a Datastore.

az ml datastore upload --name
                       --src-path
                       [--hide-progress]
                       [--output-metadata-file]
                       [--overwrite]
                       [--path]
                       [--resource-group]
                       [--subscription-id]
                       [--target-path]
                       [--workspace-name]

Required Parameters

--name -n

The datastore name.

--src-path -p

Path from which to upload data.

Optional Parameters

--hide-progress

Whether to hide progress of operation.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--overwrite

Overwrite target files if they exist.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--target-path -u

Path to upload data in the container. Uploads to the root by default.

--workspace-name -w

Workspace name.