[DocQA] add: new feature using DocQA to evaluate given file

[JLin-dev]

This pull request introduces a new evaluate.py file along with updates to the existing DocQA module. The primary goal of this update is to enhance the evaluation capabilities of the DocQA system by providing two distinct evaluation methods: Retriever Evaluation and End-to-End Evaluation.

Key Features

1. Evaluator Integration in Executor DocQA:

   • The updated evaluate.py can be uploaded to the Executor DocQA, enabling it to perform two different types of evaluations:
     • Retriever Evaluation: This involves passing chunks and questions to the LLM to assess the relevance of the retrieved chunks, ensuring that the retriever has accurately identified related content.
     • End-to-End Evaluation: This checks whether the LLM-generated answer correctly addresses the given question, thus evaluating the overall performance of the LLM.

2. Basic Evaluation Functions:

   • The updated files provide essential functions that facilitate both types of evaluations. Additionally, the in-app results feature allows users to assess the yield results directly within the chat interface.

3. Future Improvement Paths:

   • Path 1: Mistral LLM Integration:
     • Future work can involve integrating the open-source Mistral LLM, along with a tuned critic bot, to evaluate the performance of the retriever and LLM more effectively for end-to-end scenarios.
   • Path 2: RAGAS Framework Integration:
     • The evaluation process can be further enhanced by using formats such as the DRCD structure (e.g., DRCD) to pass data into the RAGAS framework. The evaluate.py file also includes functionality to handle DRCD-formatted data for this purpose.

These updates are a significant step forward in improving the evaluation and performance measurement of the DocQA system. I look forward to any feedback and suggestions for further enhancements.