Accelerated Networking with H-series VMs on Azure for older OS distributions
The accelerated networking update for the HPC SKUs on Azure has caused problems for older OS distributions or any MPI versions that do not use the latest UCX. This is due to inconsistent naming for the IB devices. My recent patch to rdma-core can be used to provide consistent naming with udev rules. This following script can be used when building an image:
yum install -y cmake libnl3-devel git clone https://github.com/linux-rdma/rdma-core.git cd rdma-core bash build.sh cp build/bin/rdma_rename /usr/lib/udev/ cat <<EOF >/etc/udev/rules.d/60-ib.rules # Accelnet board ACTION=="add", ATTR{board_id}=="MSF0010110035", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_an0" # HBv2 board ACTION=="add", ATTR{board_id}=="MT_0000000223", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_ib0" # HC board ACTION=="add", ATTR{board_id}=="MT_0000000010", SUBSYSTEM=="infiniband", PROGRAM="rdma_rename %k NAME_FIXED mlx5_ib0" EOF
This will name the accelerated networking mlx5_an0 and the infiniband to mlx5_ib0. Now, you can use the older MPI/UCX versions by setting:
export UCX_NET_DEVICES=mlx5_ib0:1
The script includes rules that will work for HB, HC, HBv2 and NDv2.
Published on:
Learn moreRelated posts
Fabric Mirroring for Azure Cosmos DB: Public Preview Refresh Now Live with New Features
We’re thrilled to announce the latest refresh of Fabric Mirroring for Azure Cosmos DB, now available with several powerful new features that e...
Power Platform – Use Azure Key Vault secrets with environment variables
We are announcing the ability to use Azure Key Vault secrets with environment variables in Power Platform. This feature will reach general ava...
Validating Azure Key Vault Access Securely in Fabric Notebooks
Working with sensitive data in Microsoft Fabric requires careful handling of secrets, especially when collaborating externally. In a recent cu...
Azure Developer CLI (azd) – May 2025
This post announces the May release of the Azure Developer CLI (`azd`). The post Azure Developer CLI (azd) – May 2025 appeared first on ...
Azure Cosmos DB with DiskANN Part 4: Stable Vector Search Recall with Streaming Data
Vector Search with Azure Cosmos DB In Part 1 and Part 2 of this series, we explored vector search with Azure Cosmos DB and best practices for...
General Availability for Data API in vCore-based Azure Cosmos DB for MongoDB
Title: General Availability for Data API in vCore-based Azure Cosmos DB for MongoDB We’re excited to announce the general availability of the ...
Efficiently and Elegantly Modeling Embeddings in Azure SQL and SQL Server
Storing and querying text embeddings in a database it might seem challenging, but with the right schema design, it’s not only possible, ...