Loading...

What runs GPT-4o and Microsoft Copilot? | Largest AI supercomputer in the cloud | Mark Russinovich

What runs GPT-4o and Microsoft Copilot? | Largest AI supercomputer in the cloud | Mark Russinovich

Microsoft has built the world’s largest cloud-based AI supercomputer that is already exponentially bigger than it was just 6 months ago, paving the way for a future with agentic systems.

For example, its AI infrastructure is capable of training and inferencing the most sophisticated large language models at massive scale on Azure. In parallel, Microsoft is also developing some of the most compact small language models with Phi-3, capable of running offline on your mobile phone. 

Watch Azure CTO and Microsoft Technical Fellow Mark Russinovich demonstrate this hands-on and go into the mechanics of how Microsoft is able to optimize and deliver performance with its AI infrastructure to run AI workloads of any size efficiently on a global scale. 

This includes a look at: how it designs its AI systems to take a modular and scalable approach to running a diverse set of hardware including the latest GPUs from industry leaders as well as Microsoft’s own silicon innovations; the work to develop a common interoperability layer for GPUs and AI accelerators, and its work to develop its own state-of-the-art AI-optimized hardware and software architecture to run its own commercial services like Microsoft Copilot and more.

► QUICK LINKS:
00:00 - AI Supercomputer
01:51 - Azure optimized for inference
02:41 - Small Language Models (SLMs)
03:31 - Phi-3 family of SLMs
05:03 - How to choose between SLM & LLM
06:04 - Large Language Models (LLMs)
07:47 - Our work with Maia
08:52 - Liquid cooled system for AI workloads
09:48 - Sustainability commitments 
10:15 - Move between GPUs without rewriting code or building custom kernels. 11:22 - Run the same underlying models and code on Maia silicon
12:30 - Swap LLMs or specialized models with others
13:38 - Fine-tune an LLM
14:15 - Wrap up

 

► Unfamiliar with Microsoft Mechanics? 

As Microsoft's official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.

• Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries

• Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog

• Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast

 

► Keep getting this insider knowledge, join us on social:

• Follow us on Twitter: https://twitter.com/MSFTMechanics 

• Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/

• Enjoy us on Instagram: https://www.instagram.com/msftmechanics/

• Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics

 

Published on:

Learn more
Microsoft Mechanics Podcast
Microsoft Mechanics Podcast

Made for tech enthusiasts and IT professionals. Expanded coverage of your favorite technologies across Microsoft; including Office, Azure, Windows and Data Platforms. We'll even bring you broader topics such as device innovation with Surface, machine learning, and predictive analytics.

Share post:

Related posts

Microsoft Copilot: New Microsoft Teams notifications for Microsoft Copilot Academy users

New Microsoft Teams notifications for Microsoft Copilot Academy users will inform them about new courses starting late June 2025. Requires Tea...

13 hours ago

Microsoft Copilot (Microsoft 365): Add topic(s) through Copilot to your existing presentation

Microsoft PowerPoint for Windows desktop will introduce a Copilot feature in July 2025, allowing users to generate and insert new topic-specif...

13 hours ago

Microsoft Copilot (Microsoft 365): Add topic(s) through Copilot to your existing presentation – Windows

Microsoft PowerPoint for Windows desktop will introduce a Copilot feature in July 2025, allowing users to generate and insert new topics and s...

13 hours ago

Microsoft Copilot (Microsoft 365): Add topic(s) through Copilot to your existing presentation

Microsoft PowerPoint for Mac will introduce a Copilot feature in July 2025, allowing users to generate new topic ideas and slides that match t...

13 hours ago

Microsoft Copilot Studio – Enhance governance with an admin control to block agents

We are announcing a new control for admins, allowing them to block agents created in Microsoft Copilot Studio. This feature provides a PowerSh...

3 days ago

Microsoft Copilot (Microsoft 365): Copilot uses enterprise assets hosted on SharePoint OAL or Templafy when creating a slide

Once your organization’s assets are integrated into a SharePoint Organization Asset Library (OAL) or connected through Templafy to Microsoft 3...

3 days ago

Microsoft Copilot (Microsoft 365): Expanded Input Box on Copilot Chat

Users will be able to expand the Input Box to a full screen view when they type long prompts with lengthy context or instructions. When users ...

4 days ago

Microsoft Edge: Summarization, Translation, and Citation will soon be supported by Microsoft Copilot and Microsoft 365 Copilot Chat in our menus

Menus in Edge where the user has highlighted content in the browser will soon support a summarization, translation, and citation with Microsof...

4 days ago

Microsoft Copilot (Microsoft 365): Increased efficiency with auto-generated Copilot Pages

Copilot Chat can automatically create and open a Page side-by-side when it detects a relevant scenario, enabling instant editing, steering, an...

5 days ago

Microsoft Copilot: Access SharePoint agents in the Microsoft 365 Copilot app

Access recently used SharePoint agents directly within the Microsoft 365 Copilot app, starting mid-July 2025. This feature aims to reduce cont...

6 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy