Avoiding Issues: Monitoring Query Pushdowns in Databricks Federated Queries
A foreign catalog in Databricks is a specialized type of catalog that enables users to access and query data stored in external databases as i...
3 days ago
Seequality
Terraforming Databricks #3: Lakehouse Federation
In today’s post, the third in the Terraforming Databricks series, we’ll break down the process of setting a connection to an Azure...
15 days ago
Microsoft Fabric: Using Workspace Identity for Authentication
One of the newest features available in Microsoft Fabric is the ability to use Workspace Identity to authenticate with external Data Lake Stor...
1 month ago
Terraforming Databricks #2: Catalogs & Schemas
In the first post of this series, we discussed the process of creating a metastore, which is essential for enabling workspaces for the Unity C...
1 month ago
Terraforming Databricks #1: Unity Catalog Metastore
Over the past two years, we have participated in numerous projects where Databricks was implemented on the Azure cloud from the ground up. Eac...
1 month ago
Executing SQL queries from Azure DevOps using Service Connection credentials
Executing SQL queries from Azure DevOps to an Azure SQL Database can be a complex and challenging task, particularly when it comes to establis...
2 months ago
Utilizing YAML Anchors in Databricks Asset Bundles
We all know what YAML is – it’s like JSON, just with indentation instead of brackets. Easier to write and read. That’s it, i...
2 months ago
Setup Git credentials for Service Principal in Azure Databricks
Jobs can be executed using notebooks or Python code from a remote Git repository or a Databricks Git folder. This approach simplifies the crea...
2 months ago
Databricks: MERGE WITH SCHEMA EVOLUTION
Anyone who has ever designed an ETL process involving more than a few tables of data has likely encountered the need to build a metadata-drive...
2 months ago
Terraforming ADF: Shared Self-Hosted Integration Runtime
In one of our previous posts, we explained what self-hosted integration runtimes are and how to fully configure them using Terraform. Today, w...
2 months ago