woah look at this

[rsamf]

/evaluate

[rsamf]

/evaluate

[github-actions]

Starting evaluation! Check the Actions tab for progress, or wait for a comment with the results.

[pamelafox]

ooo, I imagine you broke the app with that retrieval mode change! That's likely why the action failed. Thanks for coming to the session!

For your reference, here are my slides:
https://speakerdeck.com/pamelafox/github-universe-evaluating-rag-apps-in-github-actions

And the main repo this was based on:
https://github.com/pamelafox/rag-postgres-openai-python/
with evaluation guide here:
https://github.com/Azure-Samples/rag-postgres-openai-python/blob/main/docs/evaluation.md

To evaluate on Azure,
Azure AI Eval SDK docs are here:
https://learn.microsoft.com/en-us/azure/ai-studio/how-to/develop/evaluate-sdk
And if you're interested in the Azure AI CI/CD private preview, sign up here:
https://aka.ms/genAI-CI-CD-private-preview