bundle command group

The bundle command group within the Databricks CLI enables you to programmatically validate, deploy, and run Azure Databricks workflows such as Azure Databricks jobs, Delta Live Tables pipelines, and MLOps Stacks. See What are Databricks Asset Bundles?.

You run bundle commands by appending them to databricks bundle. To display help for the bundle command, run databricks bundle -h.

Create a bundle from a project template

To create a Databricks Asset Bundle by using the default Databricks Asset Bundle template for Python, run the bundle init command as follows, and then answer the on-screen prompts:

databricks bundle init

To create a Databricks Asset Bundle by using a non-default Databricks Asset Bundle template, run the bundle init command as follows:

databricks bundle init <project-template-local-path-or-url> \
--project-dir="</local/path/to/project/template/output>"

See also:

• Databricks Asset Bundle project templates
• Develop a job on Azure Databricks by using Databricks Asset Bundles
• Develop Delta Live Tables pipelines with Databricks Asset Bundles
• Databricks Asset Bundles for MLOps Stacks

Display the bundle configuration schema

To display the Databricks Asset Bundle configuration schema, run the bundle schema command, as follows:

databricks bundle schema

To output the Databricks Asset Bundle configuration schema as a JSON file, run the bundle schema command and redirect the output to a JSON file. For example, you can generate a file named bundle_config_schema.json within the current directory, as follows:

databricks bundle schema > bundle_config_schema.json

Validate a bundle

To validate that your bundle configuration files are syntactically correct, run the bundle validate command from the bundle project root, as follows:

databricks bundle validate

By default this command returns a summary of the bundle identity:

Name: MyBundle
Target: dev
Workspace:
  Host: https://my-host.cloud.databricks.com
  User: someone@example.com
  Path: /Users/someone@example.com/.bundle/MyBundle/dev

Validation OK!

Sync a bundle's tree to a workspace

Use the bundle sync command to do one-way synchronization of a bundle's file changes within a local filesystem directory, to a directory within a remote Azure Databricks workspace.

databricks bundle sync commands work in the same way as databricks sync commands and are provided as a productivity convenience. For command usage information, see sync command group.

Generate a bundle configuration file

You can use the bundle generate command to generate resource configuration for a job or pipeline that already exists in your Databricks workspace. This command generates a *.yml file for the job or pipeline in the resources folder of the bundle project and also downloads any notebooks referenced in the job or pipeline configuration. Currently, only jobs with notebook tasks are supported by this command.

Run the bundle generate command as follows:

databricks bundle generate [job|pipeline] --existing-[job|pipeline]-id [job-id|pipeline-id]

For example, the following command generates a new hello_job.yml file in the resources bundle project folder containing the YAML below, and downloads the simple_notebook.py to the src project folder.

databricks bundle generate job --existing-job-id 6565621249

# This is the contents of the resulting hello_job.yml file.
resources:
  jobs:
    6565621249:
      name: Hello Job
      format: MULTI_TASK
      tasks:
        - task_key: run_notebook
          existing_cluster_id: 0704-xxxxxx-yyyyyyy
          notebook_task:
            notebook_path: ./src/simple_notebook.py
            source: WORKSPACE
          run_if: ALL_SUCCESS
      max_concurrent_runs: 1

Bind bundle resources

The bundle deployment bind command allows you to link bundle-defined jobs and pipelines to existing jobs and pipelines in the Azure Databricks workspace so that they become managed by Databricks Asset Bundles. If you bind a resource, existing Azure Databricks resources in the workspace are updated based on the configuration defined in the bundle it is bound to after the next bundle deploy.

databricks bundle deployment bind [resource-key] [resource-id]

For example, the following command binds the resource hello_job to its remote counterpart in the workspace. The command outputs a diff and allows you to deny the resource binding, but if confirmed, any updates to the job definition in the bundle are applied to the corresponding remote job when the bundle is next deployed.

databricks bundle deployment bind hello_job 6565621249

Use bundle deployment unbind if you want to remove the link between the job or pipeline in a bundle and its remote counterpart in a workspace.

databricks bundle deployment unbind [resource-key]

Deploy a bundle

To deploy a bundle to the remote workspace, run the bundle deploy command from the bundle project root. If no command options are specified, the default target as declared within the bundle configuration files is used.

databricks bundle deploy

To deploy the bundle to a specific target, set the -t (or --target) option along with the target's name as declared within the bundle configuration files. For example, for a target declared with the name dev:

databricks bundle deploy -t dev

A bundle can be deployed to multiple workspaces, such as development, staging, and production workspaces. Fundamentally, the root_path property is what determines a bundle's unique identity, which defaults to ~/.bundle/${bundle.name}/${bundle.target}. Therefore by default, a bundle's identity is comprised of the identity of the deployer, the bundle's name, and the bundle's target name. If these are identical across different bundles, deployment of these bundles will interfere with one another.

Furthermore, a bundle deployment tracks the resources it creates in the target workspace by their IDs as a state that is stored in the workspace file system. Resource names are not used to correlate between a bundle deployment and a resource instance, so:

• If a resource in the bundle configuration does not exist in the target workspace, it is created.
• If a resource in the bundle configuration exists in the target workspace, it is updated in the workspace.
• If a resource is removed from the bundle configuration, it is removed from the target workspace if it was previously deployed.
• A resource's association with a bundle can only be forgotten if you change the bundle name, the bundle target, or the workspace. You can run bundle validate to output a summary containing these values.

Run a bundle

To run a specific job or pipeline, use the bundle run command. You must specify the resource key of the job or pipeline declared within the bundle configuration files. By default, the environment declared within the bundle configuration files is used. For example, to run a job hello_job in the default environment, run the following command:

databricks bundle run hello_job

To run a job with a key hello_job within the context of a target declared with the name dev:

databricks bundle run -t dev hello_job

If you want to do a pipeline validation run, use the --validate-only option, as shown in the following example:

databricks bundle run --validate-only my_pipeline

To pass job parameters, use the --params option, followed by comma-separated key-value pairs, where the key is the parameter name. For example, the following command sets the parameter with the name message to HelloWorld for the job hello_job:

databricks bundle run --params message=HelloWorld hello_job

To cancel and restart an existing job run or pipeline update, use the --restart option:

databricks bundle run --restart hello_job

Destroy a bundle

To delete jobs, pipelines, and artifacts that were previously deployed, run the bundle destroy command. The following command deletes all previously-deployed jobs, pipelines, and artifacts that are defined in the bundle configuration files:

databricks bundle destroy

By default, you are prompted to confirm permanent deletion of the previously-deployed jobs, pipelines, and artifacts. To skip these prompts and perform automatic permanent deletion, add the --auto-approve option to the bundle destroy command.