Accelerate metadata heavy workloads with Metadata Caching preview for Azure Premium Files SMB & REST

Azure Files previously announced the limited preview of Metadata caching highlighting improvements on the metadata latency (up to 55%) for workloads running on Azure Premium Files using SMB & REST. Now, we are excited to announce the unlimited public preview lighting up this capability on both new and existing shares in a broader set of regions. You can now automatically onboard your subscriptions to leverage this functionality using feature registration (AFEC) in supported regions. 

Feature Overview

Metadata Caching is an enhancement aimed at reducing metadata latency up to 55% for file workloads running on Windows/Linux environments.  In addition to lower metadata latency, workloads will observe a 2-3x improvement in latency consistency making metadata intensive workloads more predictable and deterministic.  Workloads that perform a high volume of metadata operations (e.g. AI/ML) will see the bigger benefit compared to workloads with high data IO (e.g. databases). Reduced metadata latency will also translate up to 3x increase in metadata scale, and up to 60% increase in data IOPS (reads/writes) and throughput.  

Example of metadata heavy workloads include:

• Web\App Services: Frequently accessed files for CMS\LMS services such as Moodle\WordPress.
• Indexing\Batch Jobs: Large scale processing using Azure Kubernetes or Azure Batch.
• Virtual Desktop Infrastructure: Azure Virtual Desktop\Citrix users with home directories or VDI applications management needs.
• Business Application: Custom line of business or legacy application with “Lift and shift” needs.
• CI\CD DevOps Pipeline: Building, testing, and deployment workloads such as Jenkins open-source automation

Building DevOps solutions using Metadata Caching

• Moodle deployment + Azure Premium Files with Metadata Caching

  • Moodle consists of server hosting (cloud platforms), a database (MySQL, PostgreSQL), file storage (Azure Premium Files), and a PHP-based web server.  It is used for course management (uploading materials, assignments, quizzes), user interaction (students accessing resources, submitting work, and discussions), and performance monitoring (tracking progress, reporting).

    Metadata Cache Benefit: Provides a faster and more consistent user experience.

• GitHub Actions + Azure Premium Files with Metadata Caching

  • GitHub Actions is an automation tool integrated with GitHub that allows developers to build, test, and deploy code directly from their repositories. It uses workflows, defined in YAML files, to automate tasks such as running tests, building software, or deploying applications. These workflows can be triggered by events like code pushes, pull requests, or scheduled times. 
    Metadata Cache Benefit: Shorter build and deployment times when using Azure Premium Files with Metadata cache as the build artifact.

How to get started

To get started, register your subscription with the Metadata Cache feature using Azure portal or PowerShell.

For Regional Availability please visit the following link

Note: As we extend region support for the Metadata Cache feature, Premium File Storage Accounts in those regions will be automatically onboarded for all subscriptions registered with the Metadata Caching feature.

Who should participate?

Whether it is a new workload looking to leverage file shares or existing ones looking for improvements. Any workloads/usage patterns that contains metadata should be encouraged to onboard, specifically metadata heavy workloads that consist primarily of Create/Open/Close or Delete requests.

To determine if your workload contains metadata, can use Azure Monitor to split the transactions by API dimension as described in the following article

Thanks

Azure Files Team

For questions, please email: [email protected]