issue #9, Developing a Search Function Test Utilizing LLM

[JolanThomassin]

Content Analysis and Scoring System for QA Using OpenAI and Azure

Issue Description:

We aim to create a comprehensive system that extracts URLs from a web page chunk, leverages the OpenAI API through Azure to generate relevant questions and answers, and subsequently employs our search function to score the accuracy of the first ten search results in relation to the generated questions and answers.

The generated questions and answers provide a benchmark for the quality of the content analysis. When we use our search function to retrieve information related to these questions, we can assess the accuracy of the search results. This quality assurance step is crucial for ensuring the reliability of the information we collect.

https://gist.github.com/JolanThomassin/1cba745ddb3aae7f433dfb4d85298fd9

Steps to Implement:

1. URL Extraction: Extract chunk using query on the database.

2. Question-Answer Generation: Leverage the Azure OpenAI API to generate questions and corresponding answers based on the content. Store this question-answer pairs along with their associated URLs for future reference.

• Chunk Extraction: Implement a method to extract chunks of text from the collected data.
• Generated Q&A: Create a dataset of Q&A pairs from the conversational agent's responses.

[rngadam]

Does it look as it should? It's my first time doing unit tests, so I would appreciate feedback.

There's many aspects to make good unit tests which impact both your test code and your code itself.

• Right now, the test only covers the "happy path"; tests should cover both success and failures cases.
• if necessary, you may have to have to change your code to retrofit testability (which is why we prefer to TDD - test-driven development, to ensure we construct testable code in the first place)
  • minimize side-effects of your functions; in this case, you are saving to a file; move this out as a separate step and validate the return of the function
    • Reading material: https://jerrynsh.com/how-to-write-testable-code-in-python/
    • you can test save to file separately
  • don't sys.exit(1) in functions; raise custom exceptions (which you should trigger in a unit test)
• a test is made up of three parts: setup, execute, assert.
  • You want to use assert to check the results, not output to console (which is I/0)

[ibrahim-kabir]

This PR needs to be merged to start working on the QnA Documents Diverge from Azure Index Document Source issue.

[rngadam]

This PR needs to be merged to start working on the QnA Documents Diverge from Azure Index Document Source issue.

that's not correct, can branch off this PR and work on a PR/branch to this PR/branch to help @JolanThomassin advance his work while letting him review changes before integrating them.

We won't just merge nilly-willy without tests or checks passing.

[JolanThomassin]

Does it look as it should? It's my first time doing unit tests, so I would appreciate feedback.

There's many aspects to make good unit tests which impact both your test code and your code itself.

• Right now, the test only covers the "happy path"; tests should cover both success and failures cases.

• if necessary, you may have to have to change your code to retrofit testability (which is why we prefer to TDD - test-driven development, to ensure we construct testable code in the first place)

  • minimize side-effects of your functions; in this case, you are saving to a file; move this out as a separate step and validate the return of the function

    • Reading material: https://jerrynsh.com/how-to-write-testable-code-in-python/
    • you can test save to file separately

  • don't sys.exit(1) in functions; raise custom exceptions (which you should trigger in a unit test)

• a test is made up of three parts: setup, execute, assert.

  • You want to use assert to check the results, not output to console (which is I/0)

For now I'm testing one case if it work, and a failure one.

• test_generate_question
• test_generate_question_db_connection_fail

I had to change the code a little so the test fit properly (b86fbe9).

I removed sys.exit(1) (b7e5b56)

For the three parts, setup, execute, assert.

• Setup: In the setUp method, I'm setting up the testing environment.
• Execute: In my test methods test_generate_question and test_generate_question_db_connection_fail, I'm calling the method generate_question which is the part of the code we are actually testing.
• Assert: The assertIsNone and assertIsNotNone methods are the assertions.

For the saving part, I'll change the code so the saving is not done by the same function.

[rngadam]

still not addressed @JolanThomassin

[rngadam]

Let's merge this because it's been pending for way too long AFTER we've linked to a new issue outlining the work to be done.

[JolanThomassin]

@rngadam , before I merge this PR, do you agree with this issue to continue the work? If yes, I would add the assignee, the project, etc...

Issue: #73

[rngadam]

@rngadam , before I merge this PR, do you agree with this issue to continue the work? If yes, I would add the assignee, the project, etc...

that issue is ok, but really very specific to one aspect of pending work.

we want issues to track the following conversations that have not been addressed yet:

[JolanThomassin]

@rngadam , before I merge this PR, do you agree with this issue to continue the work? If yes, I would add the assignee, the project, etc...

that issue is ok, but really very specific to one aspect of pending work.

we want issues to track the following conversations that have not been addressed yet:

On the 5 unresolved conversations only the first one deserve an issue, the 3 ones with Guy is about testing the function, and everything is already here and working. And the 2nd one from you was to change the query so we don't go trough all the documents and the modification is done and working too.

I'll add the issue about question quality here