How to make this evaluation higher quality?

[findmyway]

The key to the quality of evaluation is the extraction step. For models like gpt4, the instruction ability is quite strong. We can easily parse code snippets from it. However, for many existing open source LLMs we have to handle many corner cases.

As always, contributions are welcomed!