Medallion Architecture in Practice

In this episode, Carmel Eve delves deeper into the connection between the medallion architecture and the semantic layer, exploring when data becomes truly production ready.

Carmel recaps the three tiers of the medallion architecture: raw bronze, cleaned silver, and opinionated gold layers. The focus is placed on the semantic layer, which adds context and meaning to the gold layer data, making it useful for humans, machines, and AI. This semantic layer includes human or machine-friendly names, advanced data types, metadata for governance, and relationships between data objects.

Carmel also discusses the environments for software development—development, testing, and production—and how this multi-environment system applies to the medallion architecture. Finally, we highlight the advantages of using this pattern, such as supporting multiple use cases, enhancing data lineage, and ensuring data's flexibility and security.

Chapters

• 00:00 Introduction and Recap
• 00:06 Understanding the Semantic Layer
• 01:21 Defining the Semantic Layer
• 03:02 Data Production Readiness
• 03:13 Development, Testing, and Production Environments
• 04:43 Applying the Medallion Architecture
• 09:30 Conclusion and Key Takeaways