Re: Modern Power BI architecture choices for reporting on Azure Databricks: A performance benchmark for Power BI storage modes
We need to separate shortcuts into two scenarios: shortcuts to internal tables and shortcuts to external tables.
For internal tables, the scenario was effectively similar to Direct Lake on SQL Analytics Endpoint at the time of the PoC for a customer prior to this white paer, so we did not test it separately. Instead, for this white paper, we are testing Direct Lake on OneLake, which was not available when the original PoC was conducted.
For shortcuts to external ADLS Gen2 tables written by Databricks, we did test this in the PoC. However, there were three main concerns:
- Databricks does not understand or benefit from V-Order.
- Databricks writes can overwrite the V-Order optimization.
- Having both Databricks and Fabric modify the same Delta tables introduces additional data-management and supportability complexity.
In addition, even with Date partitioning, the External Shortcut scenario did not show a clear performance benefit in the PoC.
For these reasons, we decided to exclude Direct Lake on External Shortcut tables from the white paper comparison. Hope this helps.
Published on:
Learn moreWe can help you with Re: Modern Power BI architecture choices for reporting on Azure Databricks: A performance benchmark for Power BI storage modes
If you want help implementing, troubleshooting, or improving this product, contact us and we’ll point you in the right direction.