Lucene and TREC benchmark

[nigel-v-thomas]

Hi @isoboroff

First of all thank you for creating and sharing this. This really isn't an issue, just a clarification of a couple of questions related to the trec-demo:

• The lucene benchmarks package (http://goo.gl/FXais) comes with a set of parsing utilities for TREC 4+5 (-CR) collections, qrles and topics, and search evaluation tools, it is fairly straight forward to set this up for repeated batch runs. Have you considered extending this? I am using the lucene benchmark package at present and would be interested in understanding if there was a reason why you chose not to use it, incase I missed something!
• Sharing the index is great 👍, although I wish I had found this before purchasing the TREC 4 and 5 disks. However does it break the license agreement for the original data? Given that it may be possible to construct the original documents from the index. Or is the data cleaned some way?

Thanks again!