Combine FinOps best practices and Microsoft tools to streamline and optimize your workloads
Cloud adoption is not a one-and-done endeavor, but an iterative process. Constant innovation means there are new and better cloud-based solutions available every day. That impacts your cloud workloads and the way you manage them. And as your use of cloud grows, so does the scale of your cloud deployments. If you’re a large enterprise organization, you may have hundreds or even thousands of cloud subscriptions. To keep costs in check, you need to continuously examine your workloads or subscriptions to understand how to get the most from cloud. You need a way to cut through the noise so you can prioritize your attention and resources and focus on your desired business outcomes.
Workload management is the process of defining how your resources are running in a given Azure environment. It means running resources only when necessary and at the level or capacity needed for the active workload. An effective workload management and automation plan can significantly reduce costs by adjusting configuration to match supply to demand dynamically, ensuring the most effective utilization. Are your workloads running all the time or are they running based on your business or organizational needs? Learn to optimize your cloud and build your skills to make the most of Azure. Once you understand how your workloads operate day-to-day you can automate certain processes to keep everything running at optimized levels.
FinOps best practices tie in closely here. FinOps encourages collaboration within your organization, increasing visibility into your cloud investment to all levels of the organization while helping you gain more financial control and predictability to optimize cloud spend.
This blog was written to help you better understand workload management and automation within the FinOps Framework and how to implement it in Azure using specific tools within Microsoft Cost Management and Advisor. Let’s take a closer look at each of these tools and how you can use them in conjunction with FinOps capabilities to understand and forecast your bill, optimize workload costs, and control your spending.
Microsoft Cost Management
Once you’ve begun migrating to Azure, you can use Microsoft Cost Management to continually track and monitor your cloud spending and improve the cost-effectiveness of your current investments. Microsoft Cost Management is available to anyone with access to a billing account, subscription, resource group or management group. You can access Cost Management within the billing and resource management experiences or separately as a standalone tool optimized for FinOps teams who manage cost across multiple scope. For ad-hoc exploration of cost data you can use cost analysis in Azure portal. If you need to integrate with your custom reporting, you can automate cost data exports.
Microsoft Cost Management enables you to access your cost datasets through Exports and the exported data can be analyzed using tools like Microsoft Fabric. Let’s take a closer look at these two tools.
A new and improved Exports experience
Data is important if you want to build and analyze custom reports, but FinOps datasets are often large and challenging to manage. We recently improved the Exports experience to help you streamline your FinOps practice. The new Exports experience, currently in preview, includes automatic exports of more cost-impacting datasets, including price sheets, reservation recommendations, reservation details, and reservation transactions, optimized to handle large datasets while enhancing the user experience.
The new Exports functionality also improves file manageability and reduces download latency. File partitioning, which breaks files into manageable smaller chunks, and file overwrite, which replaces the previous day’s file with an updated file each day in daily export, helps you save on storage and network costs. We’ve also added support for the FinOps Open Cost and Usage Specification (FOCUS) dataset, which combines amortized and actual costs and reduces data processing times. If you want to understand the details of an invoice and the resources that went into those charges, use Exports to download the data into a storage account. Depending on the file size, you can then use a custom reporting system solution such as Fabric, Excel or PowerBI to analyze the data.
Use Fabric to simplify your analytics needs
Once you have the data you want, how do you visualize it, analyze it, and build custom reports? That’s where Microsoft Fabric comes in. Microsoft Fabric is an all-in-one analytics solution. With Fabric, you don’t need to piece together different services from multiple vendors because Fabric offers a comprehensive suite of services, including data lake, data engineering, and data integration, all in one place, giving you a highly integrated, end-to-end, and easy-to-use product designed to simplify your analytics needs. By creating a single platform with tools for every data professional in a unified experience and architecture, Fabric reduces the cost and effort of integrating analytics services.
Fabric also includes Copilot functionalities. With Copilot in Microsoft Fabric, you can use natural language to create dataflows and pipelines, write SQL statements, build reports, or even develop machine learning models. Copilot in Fabric is currently in public preview with Power BI, Data Factory, Data Engineering, and Data Science experiences. For example, in Power BI, you can create reports and summarize your insights into narrative summaries in seconds. In Data Factory, you can simply describe how you want to ingest and transform the data using natural language and Copilot does the rest. When working in a notebook in Data Engineering or Data Science, you can more quickly enrich, model, analyze, and explore your data.
Azure Advisor
Azure Advisor is designed to help you focus your efforts and save time on cloud optimization. It orchestrates and guides your journey to create and maintain well-architected workloads on Azure. Several tools within Advisor can help you optimize your workloads. Let’s take a closer look at them:
Get tailored recommendations with Advisor Reviews
How do you determine how to get the most bang for your buck when it comes to optimizing your Azure workloads? Advisor Reviews, which is only available to those who have field assistance, allows you to apply a set of filters and get automated recommendations to improve each of your resources and subscriptions. And the Azure recommendations under this scenario are personalized even more. A Microsoft representative learns about your mission-critical workloads, analyzes your applications, and manually inputs tailored recommendations to help you improve resiliency. The recommendations are prioritized according to our best estimate of significance to your environment and available through Advisor so you have a single point of entry to view them.
View recommendation details – such as recommended actions and impacted resources – and solution options, including postponing or dismissing a recommendation. If you accept recommendations, you can then check your Advisor Score to see if the changes you invested in improved your score. Periodically return to Advisor to review and implement recommendations to continuously improve your workloads.
Identify gaps with Advisor Assessments
We recently introduced a new Assessment tab within Advisor that you can use to improve your workloads in the areas that matter most to your business. Advisor Assessments is a new experience in Azure Advisor that curates a set of questions about your infrastructure to help us identify gaps and opportunities in your workloads across five pillars: reliability, security, cost optimization, operational excellence, and performance efficiency. This is especially useful for early cloud adopters or early Azure customers. Based on the type of assessment you choose, you will be guided through a series of curated questions about your Azure infrastructure, workload etc. Based on your responses, we will generate tailored and prioritized recommendations to improve your Azure architecture. To use these new assessments, you’ll need Contributor Access on a subscription to take the assessment and Reader Access on a subscription to view the generated recommendation.
Previously, the results of these assessments were available for export where you could work on them offline. Now, everything is available in one place, so you can take assessments on, and receive recommendations directly within, the Advisor platform. There is no limit on the number of assessments that can be taken for a subscription, although you’ll be notified if someone else has already created an assessment for the same subscription/workload. To see all Microsoft assessment choices, go to the Learn platform > Assessments.
Understand where you can reduce spend with the Cost Optimization workbook
The Cost Optimization Workbook brings together all vital information about potential savings and opportunities for rate optimization and highlights tactical recommendations to help you understand how and where you can reduce spend. For example, the workbook will highlight any discounts available to you that you might not yet be taking advantage of. Within the workbook, you can use the Tabs function to focus your cost-reduction efforts in the areas of Compute (including Savings Plan), Azure Hybrid Benefit, Storage, and Networking. Use Filters to focus on a specific workload, apply a recommended optimization directly from the workbook, or share insights with your team.
We’re currently working to bake automation into the workbook for use cases within compute, storage, and networking so you can take an iterative approach to reducing costs. For example, if you find a workload that has virtual machines (VMs) that are not allocated but still incurring costs, the workbook will search for them and send an alert so you can choose to stop it. Not only will the Cost Optimization Workbook provide you an opportunity to find potential savings, but the workload management automation piece impacts other capabilities like your budget and real-time decision-making.
Estimate the cost of creating new or expanding existing deployments with the Pricing Calculator
The Azure Pricing Calculator is a web-based tool that makes it easier to plan and estimate the cost of creating new or expanding existing deployments by providing cost estimates based on the services used. With this tool, you can estimate the cost of running your workloads and explore various configurations of many different Azure services as you identify which SKUs and how much usage keeps you within your desired price range. The calculator also provides a cost estimate for your Azure consumption with your negotiated or discounted prices and strategic pricing offers such as Azure Hybrid Benefit, Reserved Instances and Azure savings plan for compute. The estimate is split into upfront and monthly costs. Your input from the Pricing Calculator can also help estimate the cost of any changes to your deployed workloads. That data can help inform more specific recommendations based on your current usage.
We’ve recently added six example scenarios to the Pricing Calculator, such as “add an enterprise data warehouse” or “add advanced analytics on big data.” Depending on your chosen scenario, the Pricing Calculator will walk you through the pieces you’ll need to build that scenario and the estimated cost. For example, if you want to add Azure Web Apps to deliver value faster to your customers with a continuous integration and continuous deployment (CI/CD) pipeline that pushes each of your changes automatically to Web Apps, you will need a VM, storage, bandwidth, and several services or functions. Once you select those pieces, you can choose how and where to deploy them and apply any savings plans. Not only does the Pricing Calculator help you understand the ingredients you’ll need to achieve your new scenario, it also provides a cost estimate based on your choices so you can better plan your next move.
Reducing costs by ensuring effective utilization
Each of the tools discussed here can be used to enable FinOps capabilities and help you develop an effective workload management and automation plan to reduce costs by adjusting configuration to match supply to demand, ensuring the most effective utilization. In addition, we recently released a FinOps Review Assessment based on guidance from the FinOps Foundation framework that can help you work to align your business with FinOps practices and principles.
You can learn even more about using FinOps best practices for workload management and automation at aka.ms/finops/solutions.