Abstracts: December 6, 2023
Members of the research community at Microsoft work continuously to advance their respective fields. Abstracts brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.
In this episode, Xing Xie, a Senior Principal Research Manager of Microsoft Research Asia, joins host Dr. Gretchen Huizinga to discuss “Evaluating General-Purpose AI with Psychometrics.” As AI capabilities move from task specific to more general purpose, the paper explores psychometrics, a subfield of psychology, as an alternative to traditional methods for evaluating model performance and for supporting consistent and reliable systems.
Read the paper: Evaluating General-Purpose AI with Psychometrics
Published on:
Learn more
Related posts
Can we AI our way to a more sustainable world?
Doug Burger, sustainability expert Amy Luers, and optimization researcher Ishai Menache examine the global emissions implications of datacente...
Ideas: Steering AI toward the work future we want
Microsoft Chief Scientist Jaime Teevan and researchers Jenna Butler, Jake Hofman, and Rebecca Janssen unpack the New Future of Work Report 202...
Will machines ever be intelligent?
Are machines truly intelligent? AI researchers Subutai Ahmad and Nicolò Fusi join Doug Burger to compare transformer-based AI with the human b...
Trailer: The Shape of Things to Come
Microsoft research lead Doug Burger introduces his new podcast series, The Shape of Things to Come, an exploration into the fundamental truths...
Ideas: Community building, machine learning, and the future of AI
As the Women in Machine Learning Workshop (WiML) marks its 20th annual gathering, cofounders, friends, and collaborators Jenn Wortman Vaughan ...
Ideas: More AI-resilient biosecurity with the Paraphrase Project
Microsoft’s Eric Horvitz and bioscience experts Tessa Alexanian, James Diggans, and Bruce Wittmann discuss the Paraphrase Project, a red-teami...
Coauthor roundtable: Reflecting on healthcare economics, biomedical research, and medical education
For the series finale, Peter Lee, Carey Goldberg, and Dr. Zak Kohane compare their predictions to insights from the series’ most recent guests...
Reimagining healthcare delivery and public health with AI
Former Washington State Secretary of Health Dr. Umair Shah and Mayo Clinic CEO Dr. Gianrico Farrugia explore how healthcare leaders are approa...
Navigating medical education in the era of generative AI
Next-generation physicians Morgan Cheatham and Daniel Chen discuss how generative AI is transforming medical education, exploring how students...
AI Testing and Evaluation: Reflections
In the series finale, Amanda Craig Deckard returns to examine what Microsoft has learned about testing as a governance tool. She also explores...