August 2026

Azure Databricks released these features and improvements in August 2026.

Pipeline events system table is in Beta

August 31, 2026

The pipeline_events system table is in Beta. This table records Lakeflow pipelines event log entries for pipelines in your account, capturing lifecycle transitions, flow progress, data quality metrics, errors, and other operational data across all pipelines and workspaces in a region. Use it to query historical pipeline activity, build alerts on pipeline failures, and correlate pipeline behavior with other Lakeflow system tables.

Restrict service principal OAuth secrets to specific API scopes

August 29, 2026

You can now restrict a service principal's OAuth secret to specific API scopes, such as sql or jobs, instead of all APIs. A token minted from a scoped secret can't exceed those scopes, which limits the impact of a leaked secret. See Authorize service principal access to Azure Databricks with OAuth.

Zhipu AI GLM 5.3 now available as a Databricks-hosted model

August 29, 2026

On Azure Databricks, GLM 5.3 is available as a Databricks-hosted model in supported US regions through ADI Services, provided by Databricks. This text-only mixture of experts (MoE) model supports a context window of up to 1,048,576 tokens, up to 65,536 output tokens, function calling, structured output, coding, agentic tool use, and always-on reasoning. For setup and access, see ADI Services.

Unity Catalog Skills (Beta)

August 28, 2026

You can now publish, govern, and share agent skills with Unity Catalog. A skill is a first-class Unity Catalog securable (catalog.schema.skill) that follows the SKILL.md specification, so you register it once and control who can discover, use, and update it with the same permissions, tags, and audit that protect your tables. Coding agents download published skills or load them live over MCP.

Monitor and cap external provider costs

August 28, 2026

Unity Gateway can now track and cap spend on external model providers, such as Amazon Bedrock and Azure AI Foundry, not just Azure Databricks-provided models. If your organization already has capacity commitments directly with a provider, Unity Gateway becomes the single point to track that usage and enforce spend limits across teams and use cases. To turn it on, enable External Model Spend in Budgets (Beta) on the account console Previews page, then select Include external model usage when you create a budget scoped to the Unity Gateway resource type.

Lakeflow Designer August Release

August 27, 2026

The Lakeflow Designer August release adds the Unique operator, expands the Aggregate, Select, and Python operators, and improves the canvas, preview, and version-history experience. The following updates are now available:

• Unique operator: Use the new Unique operator to remove duplicate rows, optionally keying on a subset of columns and ordering the rows to control which duplicate is kept. See Unique.
• Count distinct: The Aggregate operator now supports a COUNT DISTINCT function to count the unique values in a column. See Aggregate.
• Dynamic column selection: The Select operator can now choose columns dynamically by data type, name pattern, or formula, so the output adapts as the input schema changes. See Select.
• Python preview mode: The Python operator can run different code during previews than during full runs by checking config["is_preview"], so you can skip side effects such as writes and notifications while previewing. See Python preview mode.
• Streaming Python output: Python operator output now streams into the output pane as the operator runs, so you can see results before the run finishes. See Python.
• Enable and disable operators and groups: Disable individual operators or entire groups to exclude them from runs, then enable them again when you need them. See Group.
• Operator lineage highlighting: Select an operator to highlight its lineage from upstream sources to downstream sinks on the canvas. See Operators.
• Run all: Run every operator in the visual data prep at once with the new Run all button. See Run the full visual data prep.
• Exact row count for each operator: You can now easily see the exact row count for each operator. Run an operator in Max mode to show its exact output row count beneath the operator on the canvas. See Preview results.
• Apply results-table changes to the canvas as operators: Filter and transform your data directly in the output table. After renaming, filtering, or inserting a column in the output table, click Apply to canvas to create operators that reproduce exactly what you did in the results table. See Transform data directly in the results table.
• Load more filter values: When you use the Filter operator, you can now load more of a column's values on demand instead of only a sample. See Filter.
• @ mention menu for columns and functions: Type @ in an expression editor to search the operator's input columns and the built-in SQL functions, then insert the one you want. See Custom columns.
• Visual canvas diff in version history: Version history now shows a visual diff on the canvas, highlighting the operators that were added, removed, or changed between versions. See version history.

Genie Code for AI Runtime (Public Preview)

August 27, 2026

Use Genie Code to develop and troubleshoot deep learning workloads in notebooks connected to AI Runtime. Now in Public Preview, it generates distributed training code, resolves environment and dependency issues, and debugs GPU and distributed workload failures.

Share foreign Delta tables with OpenSharing (Public Preview)

August 27, 2026

You can now use OpenSharing to share foreign Delta tables, which are Delta-format tables registered in a foreign catalog such as OneLake, a Hive metastore, or AWS Glue. Providers add foreign Delta tables to a share, and recipients access the shared data in read-only format. See Add foreign Delta tables to a share and Read data in a shared foreign Delta table.

Query text is now masked by default in system tables and APIs

August 26, 2026

Because SQL statements can contain sensitive information, Azure Databricks now masks query text in the following places:

• statement_text in system.query.history returns <REDACTED>.
• query_text in the Query History API and the List Queries API returns <REDACTED>.
• Keys in request_params in system.access.audit that contain SQL definitions are omitted.

Account admins and members of the databricks_pii_access account group can read unmasked query text. To grant access, an account admin creates the group and adds members. See Create and manage the databricks_pii_access group. For details about the masked fields, see Access masked statement text and Access masked request parameters.

Zhipu AI GLM 5.3 Flash is now available on Unity Gateway

August 26, 2026

On Azure Databricks, Zhipu AI GLM 5.3 Flash is available through ADI Services with unified governance through Unity Gateway, for multimodal reasoning, coding, and agentic workloads. Access it through the same governed API as other models, with centralized permissions, guardrails, spend controls, and observability, and switch models without changing application code.

ABAC GRANT policies are generally available for models and services

August 31, 2026

ABAC GRANT policies for models, model services, model provider services, Model Context Protocol (MCP) services, and agent services are now generally available. These policies dynamically grant Unity Catalog privileges to securable objects whose governed tags match a condition, instead of granting privileges on each object individually.

Unity Gateway now supports Provisioned Throughput

August 24, 2026

Govern Provisioned Throughput (PT) traffic through Unity Gateway, bringing PT models under the same access controls, usage tracking, rate limits, payload logging, and guardrails as other model traffic.

Connect to the Databricks Genie app in Microsoft Teams with your workspace URL (Public Preview)

August 24, 2026

When you sign in to the Databricks Genie app in Microsoft Teams, you can now enter your workspace URL instead of a server hostname to connect to your Azure Databricks account.

Default Python package repositories are GA for Lakeflow pipelines and classic compute

August 21, 2026

Configuring default Python package repositories is now generally available for Lakeflow pipelines and for classic compute created via the UI and API. Workspace admins can set private or authenticated registries as the workspace default for these workloads. See Configure default package repositories.

Context attributes in ABAC column mask and row filter policies (Beta)

August 21, 2026

Context attributes for attribute-based access control (ABAC) policies are in Beta. You can now write Unity Catalog ABAC policy conditions that target the context of a request, such as which OAuth application is calling (request.client_id) and whether the request runs on behalf of a user (request.is_on_behalf_of). The has_context_attribute and has_context_attribute_value functions are supported in the WHEN clause of row filter and column mask policies.

A common use is to restrict the data an agent reads on a user's behalf when it connects through a registered OAuth application, while the user can still read the full data when they query it directly in the workspace.

See Context attribute functions (Beta).

Performance mode now applies to materialized view and streaming table refreshes from dbt tasks

August 20, 2026

The job-level performance mode setting now propagates to materialized view and streaming table refreshes triggered by dbt tasks. You can set these jobs to Standard mode to optimize for cost, or keep the default Performance-optimized mode for faster startup. Previously, dbt tasks always ran refreshes in Performance-optimized mode.

Standard mode can increase refresh startup latency from under a minute to four to six minutes in exchange for lower compute costs. This change affects only refreshes that materialize as an materialized view or streaming table, so most dbt projects, which create regular tables, are unaffected.

See Select a performance mode for scheduled refreshes.

Write data back to Azure Databricks from Google Sheets

August 20, 2026

The Databricks Connector for Google Sheets now lets you write data from Google Sheets back to a Unity Catalog table. You can create a new table or overwrite an existing table without leaving Google Sheets.

Role-based access control (RBAC) is generally available

August 19, 2026

Role-based access control (RBAC) is now generally available. RBAC lets users assume a role in Azure Databricks. When a user assumes a role, only that role's permissions apply, rather than the user's accumulated permissions. This enables exclusive access. Users must actively assume a role to access sensitive data, preventing them from accessing it under their own identity and from mixing data across use cases, clinical trials, projects, or clients.

To enable RBAC, an account admin must turn on the feature at the account level. Workspace admins then enable it in each workspace.

See Role-based access control (RBAC).

ai_extract precision mode is now generally available

August 19, 2026

You can now use ai_extract precision mode for advanced extraction workloads, including multi-page documents, schemas with 50 or more fields, many extracted items per document, and complex conditional field logic.

Manage project environments with %uv commands

August 17, 2026

You can now use %uv init, %uv add, and %uv sync in serverless notebooks running environment version 5 or above to create a Python project, add dependencies, and sync its environment. See Create reproducible project environments with %uv.

Session restore for serverless jobs is now in Beta

August 17, 2026

Restore the Python variables and Spark session from a serverless compute job run into a new interactive notebook, so you can debug a failure or explore results without rerunning the job. Azure Databricks automatically captures a state snapshot when an eligible run fails, runs 30 minutes or longer, or is cancelled, and keeps it for 7 days after the run completes. This feature is in Beta. See Session restore for serverless jobs.

Base environments are now supported on classic compute (Beta)

August 17, 2026

You can now use base environments to manage Python dependencies on classic compute. You can use Databricks-provided environments or custom workspace-level environments.

Sharing managed Iceberg tables is now generally available

August 14, 2026

Sharing managed Iceberg tables in Databricks-to-Databricks sharing and open sharing is now generally available. Providers can now also share managed Iceberg tables to external Iceberg clients. See Add managed Iceberg tables to a share and Read shared managed Iceberg tables.

SQL alert task in Lakeflow Jobs is now available by default for workspaces with the compliance security profile enabled

August 13, 2026

The SQL alert task in Lakeflow Jobs now supports all compliance security profile standards that Azure Databricks supports and is available by default for workspaces with the compliance security profile enabled.

Use the task to evaluate a Databricks SQL alert as part of a Lakeflow Job. The task returns its evaluation state as a task output value, so downstream tasks can branch on the result.

See SQL alert task for jobs.

Smart Routing: Cut AI costs by routing each task to the right model

August 13, 2026

Smart Routing (Beta) automatically picks the right model and agent harness for each task, based on complexity, permissions, and budget. It shifts routine work to lower-cost models and reserves frontier models for hard problems, and every routing decision is visible and auditable. Smart Routing works with Claude Code and Codex when you run them through the Unity Gateway CLI or Omnigent, and routes across both proprietary and open models. Only Omnigent can route across coding harnesses, not just models. In internal benchmarks, it matched frontier-level quality while cutting overall task cost by more than 30%.

Block or redact sensitive data in requests and responses

August 13, 2026

The Sensitive Data Detection guardrail (Beta) detects structured sensitive data (such as Social Security numbers, credit card numbers, IBANs, and phone numbers) and either blocks the interaction or redacts the matched values in place. Detection is deterministic (regular expressions plus checksums and context keywords, with no evaluator model), so it adds under 50 ms for a 100-turn conversation and returns the same result every time. To use it, enable the Enhanced Unity Gateway preview on the account console Previews page.

This release also includes the following improvements:

• Blocked interactions now return a standard HTTP 200 response with the block reason, instead of a 4xx error, so coding agents that resend history don't re-trigger the same block every turn.
• Built-in LLM-as-a-judge guardrails can now evaluate a window of recent conversation turns, or the entire conversation history, instead of only the latest message.
• A new execution flow view shows the order in which guardrails run, end to end.

Share data behind a firewall with SecureConnect is generally available

August 13, 2026

OpenSharing SecureConnect is now generally available. Providers can share data from cloud storage behind a firewall or private endpoint without adding each recipient's network to an allowlist, and recipients access the shares using their existing OpenSharing setup. See Share data behind a firewall with SecureConnect and OpenSharing recipient firewall configuration for SecureConnect .

SecureConnect is being made available by default for provider accounts on a rolling basis. See OpenSharing SecureConnect will be available by default for all accounts.

Azure Databricks is expanding SecureConnect networking billing to more charge types and access scenarios, including charges for recipients. See Expanded networking billing for OpenSharing SecureConnect.

Google Gemini 3.7 Flash is now available on Unity Gateway

August 13, 2026

Google Gemini 3.7 Flash is now available on Unity Gateway for fast, cost-efficient everyday tasks. Access it through the same governed API as other models, with centralized permissions, guardrails, spend controls, and observability, and switch models without changing application code.

Sharing tables backed by default storage is now generally available

August 13, 2026

OpenSharing support for tables backed by default storage is now generally available and is being enabled by default for accounts on a rolling basis. Providers can share these tables with any recipient, whether open or Azure Databricks, including recipients that use classic compute. If the feature is not yet on by default for your account, an account admin can enable it in the account console.

For more details about the feature, see Limitations and Create shares for OpenSharing. For more details about enablement, see Sharing tables backed by default storage will be enabled by default for all accounts.

READ METADATA privilege is GA

August 10, 2026

The READ METADATA privilege is now generally available in Unity Catalog.

READ METADATA provides read-only visibility into the same metadata available to the object's owner or a user with MANAGE, including security-sensitive information that BROWSE doesn't expose, such as privilege grants, row filters and column masks, and ABAC policies. It's intended for users who inspect and debug access controls, such as security auditors, data governance teams, and SREs.

You can grant READ METADATA on a range of object types. For the full list of applicable object types, see READ METADATA.

See READ METADATA.

Orchestrate continuous pipelines with continuous jobs

August 10, 2026

You can now run a continuous Lakeflow pipeline with a continuous job. When a continuous job orchestrates a pipeline, the job determines the execution mode, and the pipeline can use job-level features such as serverless Standard performance mode and additional notification destinations. Databricks recommends this pattern over the pipeline's built-in continuous setting, which is not being removed.

See Run a continuous pipeline with a continuous job and Run a pipeline continuously with a continuous job.

Target an Azure capacity reservation group for classic compute

August 10, 2026

You can now target an Azure capacity reservation group when you create or edit a classic cluster or instance pool, using the new capacity_reservation_group field in azure_attributes. Capacity reservations guarantee compute capacity for constrained VM types, including GPUs, that might otherwise fail to launch because of Azure stockouts. The field is available through the Clusters and Instance Pools APIs and the Azure Databricks SDKs on workspaces that use VNet injection.

Alert system tables are in Public Preview

August 10, 2026

The system.alert schema is in Public Preview. This schema contains the alerts table, which records the configuration of every alert, and the alert_evaluation_history table, which records one row per alert evaluation. Use these tables to audit alert definitions, analyze evaluation trends, and monitor alert workloads at scale.

Zerobus Ingest is now enabled by default for workspaces with the compliance security profile

August 6, 2026

The Zerobus Ingest connector in Lakeflow Connect is now enabled by default for workspaces with the compliance security profile enabled.

Unity Gateway API: Manage resources with SDKs, the CLI, and Terraform

August 6, 2026

You can manage Unity Gateway resources programmatically instead of clicking through the UI, with full create, read, update, list, and delete support for endpoints, model providers, and MCP servers. The API is available across standard Azure Databricks developer tools:

• Python SDK: databricks-sdk 0.125.0 and later
• Go SDK: databricks-sdk-go v0.170.0 and later
• Java SDK: databricks-sdk-java 0.143.0 and later
• Databricks CLI: v1.11.0 and later
• Terraform provider: 1.124.0 and later

See the Unity Gateway API reference.

JAR tasks on serverless compute are now generally available

August 6, 2026

Package Scala or Java code as a JAR and run it as a task in a Lakeflow Job on serverless compute, without provisioning or managing clusters. See Create and run JARs on serverless compute.

Search and replace text across files

August 5, 2026

You can now search and replace text across files in an Azure Databricks Git folder or Declarative Automation Bundles project from the Search pane. See Search and replace across files.

Genie One is now available in Google Sheets

August 5, 2026

You can now use Genie One directly in the Databricks Connector for Google Sheets. Query your governed Azure Databricks data in natural language and import the results into Google Sheets as native rows and columns. See Use Genie One in Google Sheets .

Genie One is now available in Microsoft Excel

August 5, 2026

You can now use Genie One directly in the Azure Databricks Excel Add-in. Query your governed Azure Databricks data in natural language and import the results into Microsoft Excel as native rows and columns.

Unity Gateway is now generally available

August 4, 2026

Unity Gateway is now generally available. Unity Gateway is the Azure Databricks governance solution for enterprise AI, part of Unity Catalog. With Unity Gateway you can:

• Control which AI services teams can use.
• Route and manage AI traffic across providers.
• Govern MCP servers to control access and costs.
• Monitor usage, cost, access, and lineage from one place.

Some capabilities, including service policies and agent services, remain in Beta.

Sharing Delta tables with Iceberg reads enabled is now generally available

August 4, 2026

Sharing Delta tables with Iceberg reads enabled is now generally available. See Delta Lake feature support matrix.

Sharing Delta tables with deletion vectors and column mapping is now generally available

August 4, 2026

Sharing Delta tables with deletion vectors or column mapping enabled is now generally available. See Add tables with deletion vectors or column mapping to a share.

SharePoint connector (GA)

August 4, 2026

Both SharePoint connectors in Lakeflow Connect are now generally available: the managed connector for fully managed ingestion pipelines, and the standard connector for custom pipelines built with Spark and SQL APIs.

Format AI/BI dashboard widget titles and descriptions with rich text

August 4, 2026

Visualization widget titles and descriptions support rich text formatting. Click a widget's title or description to open a formatting toolbar, where you can apply text styling, links, and tables, or type markdown directly.

See Format widget titles and descriptions.

NetSuite connector (GA)

August 4, 2026

The managed NetSuite connector in Lakeflow Connect is now generally available. The connector allows you to ingest ERP and financial data from Oracle NetSuite into Azure Databricks.

Full page Genie Code is now generally available

August 4, 2026

Full page Genie Code is now generally available. Full page Genie Code is a command center experience where the active chat is shown prominently, notebooks and files open as tabs alongside it, and you can run multiple chats in parallel. Start work directly from Genie Code, and personalize how it works with skills, instructions, and MCP servers.

Databricks Runtime maintenance updates (08/04)

August 4, 2026

New maintenance updates are available for supported Databricks Runtime versions. These updates include bug fixes, security patches, and performance improvements. For details, see:

• Databricks Runtime 18 LTS
• Databricks Runtime 18.2
• Databricks Runtime 18.1
• Databricks Runtime 17.3 LTS
• Databricks Runtime 16.4 LTS
• Databricks Runtime 15.4 LTS
• Databricks Runtime 14.3 LTS
• Databricks Runtime 13.3 LTS

Secrets in Unity Catalog is GA

August 3, 2026

Secrets in Unity Catalog is now generally available. You can store, govern, and access secrets as securable objects in Unity Catalog. A Unity Catalog secret uses the three-level namespace (catalog.schema.secret) and is governed by Unity Catalog privileges, so you can apply the same access controls and auditing that you use for other data assets.

Organize your work with spaces

August 3, 2026

You can now use spaces to organize your work in the Azure Databricks editor. Each space lets you focus on a specific folder or project while preserving your open tabs, so you can easily pick up where you left off.

Serverless compute access control is now generally available

August 3, 2026

Serverless compute access control is now generally available. Workspace admins can govern who runs serverless workloads through two built-in serverless compute objects: Default Interactive Compute (governs access to notebooks and Databricks Connect) and Default Automated Compute (governs access to jobs and Spark Declarative Pipelines on Lakeflow). All workspace users have Can Use on both objects by default, so existing workloads continue to run without changes.

Automatic cost attribution for materialized views and streaming tables in Databricks SQL

August 3, 2026

Materialized view and streaming table refreshes in Databricks SQL now inherit custom tags from the enclosing SQL warehouse environment. These tags propagate to billing surfaces like system.billing.usage, enabling proper cost attribution of materialized view and streaming table updates back to the source SQL warehouse. See Attribute costs to the SQL warehouse with custom tags.

MANAGE no longer requires USE CATALOG or USE SCHEMA on the same object

August 3, 2026

The MANAGE privilege no longer requires a usage privilege (USE CATALOG or USE SCHEMA) on the object where MANAGE is granted. This change makes MANAGE behave more consistently with object ownership.

Previously, MANAGE on a catalog or schema took effect only if you also held the appropriate usage privileges on that object. You still need usage privileges on any parent containers:

• MANAGE on a catalog requires no usage privileges.
• MANAGE on a schema requires USE CATALOG on the parent catalog, but no longer requires USE SCHEMA on the schema.
• MANAGE on a table, view, volume, or function is unchanged. It still requires USE CATALOG on the parent catalog and USE SCHEMA on the parent schema.

All other privileges are unaffected and still require usage privileges as prerequisites, even for users with MANAGE. For example, querying a table still requires SELECT on the table, plus USE CATALOG and USE SCHEMA on the parent catalog and schema.

If you granted MANAGE broadly without also granting the corresponding usage privileges, those grants are now active, and holders can manage the object. Azure Databricks recommends auditing existing MANAGE grants to confirm they reflect your intended access.

See Usage privilege requirements for MANAGE.

Genie One and Genie Agents free usage extended through January 31, 2027

August 3, 2026

Free usage of Genie One and Genie Agents is extended through January 31, 2027. This promotion was previously set to end on July 31, 2026. Budget controls do not apply to these products during the promotional period.

The promotion covers usage by users only. Service principals are excluded, and their Genie One and Genie Agents usage continues to be billed.

Genie Code is not included in the free usage promotion. Each Genie Code user continues to receive 150 free Databricks units (DBUs) per month, and usage above that allowance receives a 25% promotional discount through January 31, 2027. The promotional discount is combined with applicable discounts on the Serverless Real-Time Inference SKU. For more information about pricing, see Azure Databricks pricing.

Filter updates for the Azure Databricks Excel Add-in

August 3, 2026

The Azure Databricks Excel Add-in now supports cascading filters and case-insensitive string matching. Additionally, the LIKE, NOT LIKE, IN, NOT IN, and ILIKE filters are no longer available in the filters list.

Refresh materialized views and streaming tables without exempting the pipeline owner from ABAC policies

August 3, 2026

You can now refresh a pipeline-managed materialized view or streaming table even when the pipeline owner or run-as identity is subject to an attribute-based access control (ABAC) policy.

Previously, refreshing a materialized view or streaming table managed by a pipeline required the pipeline owner or run-as identity to be exempt from any applicable ABAC policy. That limitation has been removed.

When a pipeline refreshes a materialized view or streaming table, policies are evaluated using the pipeline owner's or run-as identity. If you don't want the persisted data to be masked or filtered, add the pipeline owner or run-as identity to the EXCEPT clause of ABAC policies associated with all the tables being read during the pipeline refresh. Use the TO clause to specify which users and groups receive masked or filtered data.

See ABAC policies on materialized views and streaming tables.