Authors:

Jason Adrian – General Manager, Azure Platform Architecture

Laurentiu Olariu – Power Architect, Azure Platform Architecture

Banha Sok – Power Engineer, System Design & Development

Hyperscale datacenters are continually evolving and undergoing significant changes, with the rise of AI representing one of the most substantial shifts to date. The introduction of AI systems has brought forth novel challenges and disruptions to the infrastructure that supports hyperscale datacenters. While compute and storage systems for the cloud usually have rack power densities below 20kW, AI systems are increasing rack power to hundreds of kW. To adapt to this fast-changing segment, we began to look at every layer of our infrastructure to optimize for these changes. Our solution is to separate the single rack into an server rack and a power rack, each optimized for its primary function.

Figure 1 – AI System Disaggregation

This modular methodology allows us to adjust the power in the disaggregated power rack according to the changing demands of different inferencing and training SKUs. Additionally, it facilitates the reuse of this validated design across a variety of silicon solutions.  

The Evolution of Power Delivery – Mt Diablo

Traditional rack solutions integrate the power and server infrastructure in a single rack, but with Mt. Diablo we are moving all the power conversion into a separate disaggregated power rack. There are several key reasons for adopting disaggregated power in the datacenter:

• Space Optimization: Disaggregated power enables the entire server rack to be used for AI accelerators and scale up network switches to enable larger pods. This optimization is crucial for performance and efficiency, enabling up to 35% more AI accelerators in each server rack.
• Scalability and Future Proofing: The need for scalability and future-proofing is driven by high-power server racks, which will exceed a few hundred kilowatts and are moving towards a megawatt. With this approach, we can right-size the power shelf count to meet each configuration’s unique needs.
• Power Conversion Efficiency:  Today’s power solutions convert AC inputs into 48Vdc outputs for distribution to the server trays. To improve efficiency, we can convert to 400Vdc (High Voltage Direct Current or HVDC), monopolar or bipolar, to enable better efficiency relative to the needs of high-power server racks. With 400V we expect improvements and incremental evolution in improved efficiency, like what we have seen in the 48Vdc conversion space.
• Modular Design: The modular design allows for multiple developments in parallel. This includes HVDC power shelves with specific power supply units (PSUs) that provide HVDC output to a dedicated busbar, cross-rack power distribution to the server rack, in rack energy storage, and AC voltage distribution within the power rack.

All of the benefits of disaggregated power highlighted above make this approach a forward-thinking strategy for datacenter infrastructure.

400Vdc & Industry Alignment

While the first disaggregated power racks will use the current 48Vdc ecosystem, the real enhancements come with the 400Vdc power distribution. The high-level proposal for a 400Vdc disaggregated power rack enables an improved solution compared to prior 12Vdc and 48Vdc solutions and aims to encourage industry alignment and commonality in several areas:

• Connectivity Solutions:  The 400Vdc connection solutions will differ significantly from the previous 12Vdc and 48Vdc solutions, highlighting the need for industry-wide standardization.
• Power Rack Form Factor/Dimensions: Establishing common dimensions for power racks to ensure compatibility and ease of integration.
• AC to DC PSU Topology: Addressing the differences between single-phase and three-phase input to create a unified approach.
• DC to DC Modules in Server Rack: Standardizing the modules used within server racks to ensure consistency and reliability.
• Redundancy: Defining redundancy configurations, like single feed or dual feed, or N+x power module redundancy to enhance system reliability.
• Safety Standards: Developing safety standards for 400Vdc distribution and liquid cooled bus solutions to ensure safe operation.
• Data/Power Management Backplane: Creating a standardized backplane for data and power management, including communication protocols, firmware updates, power control, and failure management.

This alignment aims to streamline the development and deployment of disaggregated power solutions, making it easier for industry to adopt and implement these new technologies, and enables partnerships like Microsoft and Meta that are supporting this initiative.

Conclusion

The disaggregated power rack enables scalability and flexibility in a time where innovation and time to market is of paramount importance.  In an effort to move fast and shift the industry to HVDC power distribution, it’s critical to foster a healthy ecosystem and partnerships to drive commonality. This is why we are excited to announce our upcoming contribution of this architectural specification to the OCP community in collaboration with Meta.