Accelerated Networking with H-series VMs on Azure for older OS distributions
The accelerated networking update for the HPC SKUs on Azure has caused problems for older OS distributions or any MPI versions that do not use the latest UCX. This is due to inconsistent naming for the IB devices. My recent patch to rdma-core can be used to provide consistent naming with udev rules. This following script can be used when building an image:
yum install -y cmake libnl3-devel git clone https://github.com/linux-rdma/rdma-core.git cd rdma-core bash build.sh cp build/bin/rdma_rename /usr/lib/udev/ cat <<EOF >/etc/udev/rules.d/60-ib.rules # Accelnet board ACTION=="add", ATTR{board_id}=="MSF0010110035", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_an0" # HBv2 board ACTION=="add", ATTR{board_id}=="MT_0000000223", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_ib0" # HC board ACTION=="add", ATTR{board_id}=="MT_0000000010", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_ib0" EOF
This will name the accelerated networking mlx5_an0 and the infiniband to mlx5_ib0. Now, you can use the older MPI/UCX versions by setting:
export UCX_NET_DEVICES=mlx5_ib0:1
The script includes rules that will work for HB, HC, HBv2 and NDv2.
Published on:
Learn moreRelated posts
Microsoft Purview: Data Lifecycle Management- Azure PST Import
Azure PST Import enables Microsoft 365 admins to import PST files from Azure Blob Storage into Exchange Online mailboxes using PowerShell. The...
April Patches for Azure DevOps Server
We are releasing patches for our self‑hosted product, Azure DevOps Server. We strongly recommend that all customers remain on the latest, most...
Integration Testing Azure Functions with Reqnroll and C#, Part 5 - Using Corvus.Testing.ReqnRoll in a build pipeline
If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. In the final post in this serie...
Integration Testing Azure Functions with Reqnroll and C#, Part 4 - Controlling your functions with additional configuration
If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. In the fourth of this series of...
Integration Testing Azure Functions with Reqnroll and C#, Part 3 - Using hooks to start Functions
If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. In the third of a series of pos...
Integration Testing Azure Functions with Reqnroll and C#, Part 2 - Using step bindings to start Functions
If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. In the second of a series of po...
Integration Testing Azure Functions with Reqnroll and C#, Part 1 - Introduction
If you use Azure Functions on a regular basis, you'll likely have grappled with the challenge of testing them. In the first of a series of pos...