Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System Parameters #1

Open
Reham-Osama opened this issue Apr 1, 2024 · 3 comments
Open

System Parameters #1

Reham-Osama opened this issue Apr 1, 2024 · 3 comments

Comments

@Reham-Osama
Copy link

Hello,
I have a couple of questions related to the parameters used to run the system

  1. How to specify the number of questions to generate in the benchmark?
  2. I managed to run the system with DBpedia, But I got 10 output files, is this the final output or do you have another step to merge them in one file or choose from the generated questions?

Thanks

@aorogat
Copy link
Owner

aorogat commented Apr 3, 2024

Hi Reham,
I wanted to inform you that I made some adjustments to the system regarding the generation of multiple benchmarks, but I forgot to revert it back to its original state. Rectifying this is quite simple on your end. You can accomplish it by uncommenting lines 72 and 73 in the code found here: link to the relevant GitHub file.

As indicated in this line of code, the variable Settings.benchmarkSizeBeforePrune determines the number of questions generated before the pruning process, where we select the properties of the questions for the final benchmark.

Upon reviewing the code, I noticed that this particular process hasn't been updated. I'll be adding the necessary updates shortly by uploading the latest version of the code.

@Reham-Osama
Copy link
Author

Hi Abdelgany,

I uncommented the two lines that you mentioned, but still I got numbers of questions more than what is specified in Settings.benchmarkSizeBeforePrune. (I got 1272 questions while I asked for 100)
I think it is because generatedQuestions is reinitialized at every loop so in the condition Line 72, the condition is never satisfied so no break happens. I reached this conclusion because while the system is running, I keep getting this message Generated Questions tell now: 0

@aorogat
Copy link
Owner

aorogat commented Apr 5, 2024

You're correct, Reham. There are some additional changes because I configured it to generate multiple benchmarks, and I need to reset the code. I was attempting to update it and refresh the GitHub repository, but encountered a conflict, so I need to review it carefully. I'll try to address it in the near future. Please note that the component package requires code refactoring. I had begun refactoring the entire project but didn't finish it before starting on another project. I can send you my local version by email until I update the GitHub repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants