Microsoft Fabric - Processing Bronze to Silver using Fabric Notebooks
Notebooks in Fabric are a lot like notebook experiences in other tools you're probably already familiar with. They allow us to write code in a variety of languages and create a commentary alongside our code using interactive cells. Fabric notebooks also have built in integration with Lakehouses, and provide a built-in filesystem that can be used to store arbitrary files somehow associated with the notebook.
Notebooks are our go-to tool for wrangling Bronze data into Silver, which tends to follow standard processing patterns. In this video we'll do a whirlwind tour of notebooks in Microsoft Fabric, and take a look at how our "Process to Silver" notebook takes shape. The full transcript is available below.
The talk contains the following chapters:
- 00:00 Intro
- 00:10 Creating a blank notebook
- 01:04 Mounting a Lakehouse in a Fabric Notebook
- 02:05 Querying files in a Lakehouse from a Fabric Notebook
- 03:28 Query PySpark DataFrames directly with SQL
- 04:12 Built-in notebook filesystem
- 05:14 Walkthrough of "Process to Silver" notebook - reading data and transforming
- 08:18 Walkthrough of "Process to Silver" notebook - merging and writing data
- 10:03 Walkthrough of "Process to Silver" notebook - reading back written data
- 10:15 Roundup & Outro
Useful links:
- 📖 Microsoft Fabric Notebooks
- 📖 Connect Lakehouses and Fabric Notebooks
- 📖 Built-in Notebook filesystem
- 📖 Delta Lake Merge
Microsoft Fabric End to End Demo Series:
- Part 1 - Lakehouse & Medallion Architecture
- Part 2 - Plan and Architect a Data Project
- Part 3 - Ingest Data
- Part 4 - Creating a shortcut to ADLS Gen2 in Fabric
- Part 5 - Local OneLake Tools
- Part 6 - Role of the Silver Layer in the Medallion Architecture
- Part 7 - Processing Bronze to Silver using Fabric Notebooks
Microsoft Fabric First Impressions:
Decision Maker's Guide to Microsoft Fabric
and find all the rest of our content here.
Published on:
Learn more