Confidential VM option for Azure Databricks Preview

Azure Databricks & Azure Confidential Computing are excited to announce the Public Preview of Azure Databricks support for Azure Confidential Computing options for data plane.

Feature Overview  

Azure Databricks now supports using Confidential computing VM options while creating clusters. This new feature emphasizes our commitment to cloud security. With this new feature, now you can run your Azure Databricks workload securely & confidentially on Azure. Azure confidential computing generally attracts customers in highly regulated industries & regions but is equally applicable to any business that is looking to run their analytics, machine learning and other workloads in a privacy preserving way. 

Security Benefits

Azure already encrypts data at rest and in transit, and confidential computing helps protect data in use, including the cryptographic keys.  Azure confidential computing helps customers prevent unauthorized access to data in use, including from the cloud operator, by processing data in a hardware-based and attested Trusted Execution Environment (TEE). 

ACC Technology & Offerings

Please start here to understand the technology behind Azure Confidential Computing in depth and also learn about the GA and in-preview offerings.

Azure confidential computing | Microsoft Learn

Supported VM Types for Confidential Databricks Data Plane

Please read this article to find out the DC* & EC* series VM types supported in the new feature release.

Azure Confidential virtual machine options on AMD processors | Microsoft Learn

Current Limitations

The feature release allows adding Confidential VM options while creating clusters in the customer Data plane only. More confidential features will be coming soon.

Configuring Confidential VMs on Azure Databricks

When creating a data plane cluster, select one of the confidential VM options as discussed here. By selecting a confidential VM as the compute option, you would be able to run your workloads in a privacy preserving way & confidential manner. Confidential VMs are available currently in East US, West US, North EU, West EU, with additional regions to be added later. 

How to create a Confidential Data plane cluster - https://aka.ms/ADB-ACC-demo

Note: To migrate existing clusters to Confidential compute, simply edit the cluster and select the VM type as discussed above.