Loading...

What runs GPT-4o and Microsoft Copilot? | Largest AI supercomputer in the cloud | Mark Russinovich

What runs GPT-4o and Microsoft Copilot? | Largest AI supercomputer in the cloud | Mark Russinovich

Microsoft has built the world’s largest cloud-based AI supercomputer that is already exponentially bigger than it was just 6 months ago, paving the way for a future with agentic systems.

For example, its AI infrastructure is capable of training and inferencing the most sophisticated large language models at massive scale on Azure. In parallel, Microsoft is also developing some of the most compact small language models with Phi-3, capable of running offline on your mobile phone. 

Watch Azure CTO and Microsoft Technical Fellow Mark Russinovich demonstrate this hands-on and go into the mechanics of how Microsoft is able to optimize and deliver performance with its AI infrastructure to run AI workloads of any size efficiently on a global scale. 

This includes a look at: how it designs its AI systems to take a modular and scalable approach to running a diverse set of hardware including the latest GPUs from industry leaders as well as Microsoft’s own silicon innovations; the work to develop a common interoperability layer for GPUs and AI accelerators, and its work to develop its own state-of-the-art AI-optimized hardware and software architecture to run its own commercial services like Microsoft Copilot and more.

► QUICK LINKS:
00:00 - AI Supercomputer
01:51 - Azure optimized for inference
02:41 - Small Language Models (SLMs)
03:31 - Phi-3 family of SLMs
05:03 - How to choose between SLM & LLM
06:04 - Large Language Models (LLMs)
07:47 - Our work with Maia
08:52 - Liquid cooled system for AI workloads
09:48 - Sustainability commitments 
10:15 - Move between GPUs without rewriting code or building custom kernels. 11:22 - Run the same underlying models and code on Maia silicon
12:30 - Swap LLMs or specialized models with others
13:38 - Fine-tune an LLM
14:15 - Wrap up

 

► Unfamiliar with Microsoft Mechanics? 

As Microsoft's official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.

• Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries

• Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog

• Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast

 

► Keep getting this insider knowledge, join us on social:

• Follow us on Twitter: https://twitter.com/MSFTMechanics 

• Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/

• Enjoy us on Instagram: https://www.instagram.com/msftmechanics/

• Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics

 

Published on:

Learn more
Need help with this product?

We can help you with What runs GPT-4o and Microsoft Copilot? | Largest AI supercomputer in the cloud | Mark Russinovich

If you want help implementing, troubleshooting, or improving this product, contact us and we’ll point you in the right direction.

Microsoft Mechanics Podcast
Microsoft Mechanics Podcast

Made for tech enthusiasts and IT professionals. Expanded coverage of your favorite technologies across Microsoft; including Office, Azure, Windows and Data Platforms. We'll even bring you broader topics such as device innovation with Surface, machine learning, and predictive analytics.

Share post:

Related posts

Build apps by describing what you want | Copilot Code

Turn your app idea into a working, interactive web app with Copilot Code. Describe what you want in plain language, ground it in your work dat...

9 hours ago

Manage AI Cost + ROI with Copilot and Agent 365

Take control of AI spend across Microsoft Copilot, Cowork, and agents inside and outside of Microsoft. Track your own Copilot Credit usage tas...

20 hours ago

Azure SQL Database Hyperscale - Now for any size workload

Start small and scale up to your most demanding workloads on Azure SQL Database Hyperscale. Run serverless compute that scales with demand and...

1 day ago

New Microsoft Copilot Harness | More power, token efficiency & control for agents in Foundry

Build agents you can trust with your company's data, apps, and resources using Microsoft Foundry. Start with a fully managed prompt agent and ...

9 days ago

New Conditional Access policy plans in Microsoft Entra

Group related Conditional Access policies into policy plans and define users, apps and conditions once for every rule in the plan. Add a rule ...

22 days ago

Fight AI with AI | Multi-Agent Security Defense

Match AI-powered attacks at machine speed with Project Perception. Point red, blue, and green AI agent teams at a known threat actor, confirm ...

1 month ago

Build Automated Agents, Workflows, and Apps - New in Copilot Studio

Take a business process from prompt to production by building agents, workflows, and apps in one place with Copilot Studio. Describe the agent...

1 month ago

Create agents fast with Agent Builder in Copilot

Build a specialist AI agent without writing a line of code. Describe what you want in one prompt, and Copilot Agent Builder generates the name...

1 month ago

New Agent 365 controls for Microsoft admins

Govern every AI agent running across your organization with Agent 365. Track agents from Microsoft, Amazon, Google, and Salesforce in one regi...

2 months ago

Windows 365: Five Years In, See What's New

An AI agent works inside your Windows 365 Cloud PC from a Teams chat on your phone — finding downloaded files, editing documents, and drafting...

3 months ago

Newsletter

Get the latest Dynamics 365 and Power Platform content in your inbox

A curated digest of community blogs, product news, videos, and podcasts — delivered without the noise.

Weekly updates Unsubscribe anytime Fresh community picks
We use your email only for the newsletter and you can unsubscribe at any time.
By subscribing, you agree to the privacy policy.