Train AI and ML models

Azure Databricks offers compute options for machine learning needs from managed clusters.

Compute Description
Databricks Runtime for Machine Learning Classic compute environment with pre-built libraries for classic machine learning and deep learning workloads.

Databricks Runtime for Machine Learning

Databricks Runtime for Machine Learning is a specialized runtime that automates the creation of compute resources with pre-built infrastructure. It is designed for users who want a comprehensive, ready-to-use environment for both classic machine learning and deep learning.

Key features include:

  • Pre-installed libraries: Includes popular libraries like PyTorch, TensorFlow, and XGBoost, which receive frequent updates and optimized support.

  • Compute versatility: Supports both CPU and GPU-based instance types, including AWS Graviton for improved price-to-performance.

  • Optimization: Offers integration with Photon to accelerate Spark SQL, DataFrames, and feature engineering tasks.

  • Access control: Requires dedicated access mode for secure data access through Unity Catalog.