{Do you know} Analyze quality of responses that use generative AI in Copilot Studio

Today I am going to share my thoughts on Copilot Studio's new feature, the quality of responses that use generative AI.

Let's get started.

Microsoft's Copilot Studio introduced a new feature on June 17, 2025, enabling administrators, makers, marketers, and analysts to automatically analyse the quality of responses generated by AI copilots. This feature aims to provide actionable insights to enhance agent performance.

The evaluation framework for generative AI (GenAI) systems has been evolving to address the limitations of traditional lab-based assessments.

A comprehensive evaluation framework was proposed to assess GenAI systems in real-world scenarios, emphasising the need for dynamic and ongoing assessments that consider user intent, social dynamics, and emergent behaviours.

Additionally, Google Cloud introduced the GenAI evaluation service within Vertex AI, allowing users to evaluate generative models against predefined or custom criteria. This service supports various metrics, including accuracy, relevance, and user satisfaction, facilitating a more nuanced understanding of AI performance.

These developments reflect a broader trend towards more sophisticated and context-aware evaluations of generative AI systems, moving beyond traditional benchmarks to emcompass  real-world performance and user experience.

That's it for today.

I hope this helps.

Malla Reddy Gurram(@aka UK365GUY)