Accelerated Networking with H-series VMs on Azure for older OS distributions
The accelerated networking update for the HPC SKUs on Azure has caused problems for older OS distributions or any MPI versions that do not use the latest UCX. This is due to inconsistent naming for the IB devices. My recent patch to rdma-core can be used to provide consistent naming with udev rules. This following script can be used when building an image:
yum install -y cmake libnl3-devel git clone https://github.com/linux-rdma/rdma-core.git cd rdma-core bash build.sh cp build/bin/rdma_rename /usr/lib/udev/ cat <<EOF >/etc/udev/rules.d/60-ib.rules # Accelnet board ACTION=="add", ATTR{board_id}=="MSF0010110035", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_an0" # HBv2 board ACTION=="add", ATTR{board_id}=="MT_0000000223", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_ib0" # HC board ACTION=="add", ATTR{board_id}=="MT_0000000010", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_ib0" EOF
This will name the accelerated networking mlx5_an0 and the infiniband to mlx5_ib0. Now, you can use the older MPI/UCX versions by setting:
export UCX_NET_DEVICES=mlx5_ib0:1
The script includes rules that will work for HB, HC, HBv2 and NDv2.
Published on:
Learn moreRelated posts
🚀 Introducing the New VS Code Extension for Azure Cosmos DB
We’re excited to share that the Azure Databases extension for Visual Studio Code is now officially rebranded as the Azure Cosmos DB extension!...
AI-based T-SQL Refactoring: an automatic intelligent code optimization with Azure OpenAI
This article presents an AI-powered approach to automating SQL Server code analysis and refactoring. The system intelligently identifies ineff...
Azure Boards integration with GitHub Copilot (Private Preview)
Several months ago, GitHub introduced the public preview of its Copilot coding agent, a powerful new capability that allows you to assign GitH...
What is Azure Key Vault and How It Secures Microsoft Dynamics 365 CRM Systems?
Azure Key Vault is a service by Microsoft Azure that helps securely store and manage sensitive information such as API keys, connection string...
Azure AI Foundry Model In Copilot Studio Custom Prompts
Any custom model created in Azure AI Foundry can be used in Copilot Studio. This ... The post Azure AI Foundry Model In Copilot Studio Custom ...
Running Teams PowerShell Cmdlets in Azure Automation
This article describes the prerequisites and how to run cmdlets from the Teams PowerShell module in Azure Automation runbooks. We also conside...
Azure Storage APIs gain Entra ID and RBAC support
To align with security best practices, Microsoft Entra ID and RBAC support is now generally available for several Azure Storage data plane API...
Introducing the Azure Cosmos DB Account Overview Hub
A Simpler Way to Navigate, Learn, and Optimize your Azure Cosmos DB Account within the Azure Portal. Whether you are just getting started with...