{Do you know}Evaluate test sets with multiple graders in Copilot Studio
.png "{Do you know}Evaluate test sets with multiple graders in Copilot Studio")
.png)
- You can attach several graders to the same test set, such as general quality, text similarity, and exact match.
- Each grader can have its own pass criteria.
- When you run the evaluation, Copilot Studio applies all selected graders to every test case in that run.
- Results show up as separate columns per grader, plus an evaluation summary with aggregated results.
- You can assess different aspects of agent quality in one execution instead of rerunning the same test set multiple times.
- Microsoft's guidance also recommends combining multiple evaluation approaches rather than relying on a single grading method.
- Test sets can contain up to 100 test cases.
- You can create test sets by generating them in Copilot Studio, importing a .csv or .txt file, writing cases manually, or using production data themes.
- Go to your agent's Evaluation page.
- Create or open a test set.
- Add multiple graders for the test.
- Define pass thresholds for each grader.
- Run the evaluation and compare the grader-specific result columns and summary.
Published on:
Learn moreWe can help you with {Do you know}Evaluate test sets with multiple graders in Copilot Studio
If you want help implementing, troubleshooting, or improving this product, contact us and we’ll point you in the right direction.
Related posts
{How to} use code interpreter on customer uploaded files in agent conversations in copilot studio
Hello Everyone,Today I am going to share my thoughts on the use of the code interpreter on customer-uploaded files in agent conversations in c...
{How to } Make Power Page Portal WCAG 2.1 Level A and AA Standards
Hello Everyone,Today I am going to share my thoughts on how to make power pages portal according to the WCAG 2.1 Level A and AA Standards.Let'...
{How to }Make your copilot multilingual in Microsoft Copilot Studio
Hello Everyone,Today I am going to show how to enable Microsoft Copilot Studio Multilingual.Let's get started.To make your Microsoft Copilot (...
{How to} Use IVR with Omnichannel for customer service in Copilot Studio
Hello Everyone,Today i am going to share my thoughts on using IVR with Omnichannel for Customer Service in Copilot Studio.Let's get started.To...
{How to} Enhance IVR with generative answers optimized for voice in Copilot Studio
Hello Everyone,Today I am going to share my thoughts on the Enhancing IVR with generative answers optimised for voice in Copilot Studio.To enh...
{How to} Use answering machine detection and proactive engagement tools in Copilot Studio
Hello Everyone,Today I am going to share my thoughts on how answering machine detection and proactive engagement tools work.Let's get started....
{How to } Publish Copilot Agents to WhatsApp
Hello Everyone,Today i am going to share my thoughts on how to publish the copilot agent to whatsapp channel.Let's get started.To publish a Mi...
{How to} Request Information from humans in the loop in Copilot Studio Agent flows
Hello Everyone,Today I am going to share my thoughts on how to request information from humans in the loop in Copilot Studio Agent flows.Let's...
{How to} Reassign an copilot agent's owner with Power Platform API
Hello Everyone,Today I am going to share my thoughts on how to reassign an copilot agent's owner with the Power Platform API.Let's get started...
{Do you know} Business impact of Copilot Studio Agents in Viva Insights
Hello Everyone,Today I am going to share my thoughts on analysing the business impact of Copilot Studio Agents in Viva Insights.Let's get star...