Upgrade to Azure Synapse runtimes for Apache Spark 3.4 & previous runtimes deprecation
It is important to stay ahead of the curve and keep services up to date. That's why we encourage all Azure Synapse customers with Apache Spark workloads to migrate to the newest GA version, Azure Synapse Runtime for Apache Spark 3.4. The update brings Apache Spark to version 3.4 and Delta Lake to version 2.4, introduces Mariner as the new operating system, and updates Java from version 8 to 11.
Within a few days/weeks, we are disabling Apache Spark 2.4, 3.1, 3.2 job execution. If you are affected you have already been notified. Using the runtime after EOS date is at one's own risk, and with the agreement and acceptance of the risks that jobs will eventually stop executing. All support tickets will be auto-resolved. Learn more about the Microsoft Lifecycle Policy.
Migrate to the latest GA version of Azure Synapse runtimes for Apache Spark 3.4 before the deprecation and disablement of previous versions.
Please refer to the following article for more information on the lifecycle and supportability of our runtimes: Azure Synapse runtimes.
Go to Synapse runtime for Apache Spark lifecycle and supportability - Azure Synapse Analytics | Microsoft Learn and Migration Guide: Spark Core - Spark 3.4.1 Documentation (apache.org) for more details on how to migrate and how to change Apache Spark-based runtime in Azure Synapse Analytics.
Published on:
Learn moreRelated posts
Primer: Output Data Generated with an Azure Automation Runbook to a SharePoint List
The second part of the Azure Automation runbook primer brings us to output, specifically how to create items generated by a runbook in a Share...
Databricks vs Azure Synapse Analytics: A Comprehensive Comparison for Modern Data Solutions
Table of Contents Introduction Data is at the core of modern business decision-making. As companies increasingly rely on data-driven insights,...
Primer: How to Use Azure Automation to Run Microsoft Graph PowerShell SDK Scripts
A reader asked why it seems so difficult to use Azure Automation runbooks to process Microsoft 365 data. In fact, it's not so hard, and here's...
Extending Regular Expressions (Regex) Support on Azure SQL Managed Instance (MI)
We are happy to announce the Private Preview of Regular Expressions (Regex) support on Azure SQL Managed Instance (MI). This new feature bring...
Shield your Copilot with Azure AI
Azure Confidential Clean Rooms demonstration
Final Days for the MSOnline and AzureAD PowerShell Modules
After many twists and turns since August 2021, the MSOnline module retirement will happen in April 2025. The AzureAD module will then retire i...
Join the Conversation: Call for Proposals for Azure Cosmos DB Conf 2025!
Are you passionate about Azure Cosmos DB? Do you have insights, experiences, or innovations that the developer community would love to hear? N...