What runs GPT-4o and Microsoft Copilot? | Largest AI supercomputer in the cloud | Mark Russinovich
Microsoft has built the world’s largest cloud-based AI supercomputer that is already exponentially bigger than it was just 6 months ago, paving the way for a future with agentic systems.
For example, its AI infrastructure is capable of training and inferencing the most sophisticated large language models at massive scale on Azure. In parallel, Microsoft is also developing some of the most compact small language models with Phi-3, capable of running offline on your mobile phone.
Watch Azure CTO and Microsoft Technical Fellow Mark Russinovich demonstrate this hands-on and go into the mechanics of how Microsoft is able to optimize and deliver performance with its AI infrastructure to run AI workloads of any size efficiently on a global scale.
This includes a look at: how it designs its AI systems to take a modular and scalable approach to running a diverse set of hardware including the latest GPUs from industry leaders as well as Microsoft’s own silicon innovations; the work to develop a common interoperability layer for GPUs and AI accelerators, and its work to develop its own state-of-the-art AI-optimized hardware and software architecture to run its own commercial services like Microsoft Copilot and more.
► QUICK LINKS:
00:00 - AI Supercomputer
01:51 - Azure optimized for inference
02:41 - Small Language Models (SLMs)
03:31 - Phi-3 family of SLMs
05:03 - How to choose between SLM & LLM
06:04 - Large Language Models (LLMs)
07:47 - Our work with Maia
08:52 - Liquid cooled system for AI workloads
09:48 - Sustainability commitments
10:15 - Move between GPUs without rewriting code or building custom kernels. 11:22 - Run the same underlying models and code on Maia silicon
12:30 - Swap LLMs or specialized models with others
13:38 - Fine-tune an LLM
14:15 - Wrap up
► Unfamiliar with Microsoft Mechanics?
As Microsoft's official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.
• Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries
• Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog
• Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast
► Keep getting this insider knowledge, join us on social:
• Follow us on Twitter: https://twitter.com/MSFTMechanics
• Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/
• Enjoy us on Instagram: https://www.instagram.com/msftmechanics/
• Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics
Published on:
Learn moreMade for tech enthusiasts and IT professionals. Expanded coverage of your favorite technologies across Microsoft; including Office, Azure, Windows and Data Platforms. We'll even bring you broader topics such as device innovation with Surface, machine learning, and predictive analytics.
Related posts
{How to } View billing and usage in the admin center on Microsoft Copilot Studio
Hello Everyone,Today I am going to share my thoughts on the viewing billing and usage in the admin center microsoft copilot studio.Let's get's...
Microsoft Copilot for Microsoft 365: Copilot chat active users in Teams counted as Teams Copilot active users
Microsoft Copilot for Microsoft 365 is undergoing some changes, with the active users of Copilot chat in Teams set to count towards Teams Copi...
Microsoft Copilot (Microsoft 365): Use Copilot to search for answers from the web
This post discusses using Microsoft Copilot, a feature within Microsoft 365, to search for answers from the web and add them to an Excel workb...
Microsoft Viva: Viva Pulse – Generate Pulse summary with Microsoft Copilot in Viva Pulse
Microsoft Viva has introduced a new feature called Viva Pulse, which allows users to receive an AI-generated summary of Viva Pulse report resu...
Microsoft Copilot in Teams will use the meeting chat as a data source
Microsoft Teams is introducing a new feature called Copilot, which will use meeting chat as a data source to provide meeting invitees with ans...
{Do you know} Employ user authentication and parameters in plugins in Microsoft Copilot Studio
Hello Everyone,Today I am going to share my thoughts on employ authentication and parameters in plugins in Microsoft Copilot Studio.Let's get'...
Microsoft Copilot for Microsoft 365: New Scheduled prompts feature
Microsoft 365 has added a new feature called Scheduled Prompts for Copilot that enables automated prompts at specific times in Teams, Office.c...
Outlook: Microsoft Copilot available in Outlook mobile
The module switcher in Microsoft Outlook for iOS and Android will now include the Microsoft Copilot experience, previously available in new Ou...
Microsoft Copilot (Microsoft 365): Automatic summary of documents on file-open in Word
Microsoft Copilot, an add-on for Microsoft 365, now generates automatic summaries of documents when they are opened in Word. The summary appea...
Microsoft Copilot (Microsoft 365): Add brand images in M365 Copilot from SharePoint organization asset libraries
Incorporating brand-approved images in your presentations and documents just got easier with Microsoft Copilot. Microsoft SharePoint organizat...