README.md

This directory has notebooks covering all essential functionality of FlexNeuART. The notebooks should be explored in the following order:

1. Installation

2. Basic data preparation and processing (including generation of dense embeddings)

3. Basic indexing (inverted and forward indices)

4. Training a neural model: vanilla BERT ranker

5. Training IBM Model 1 (non-neural lexical translation model)

6. Basic experimentaion:

   i. Tuning BM25 and IBM Model 1

   ii. Testing models that do not require training a fusion model (no learning to rank)

   iii. Learning to rank (LETOR): training & testing fusion models

7. Python API

8. Using k-NN search on dense and dense-sparse embeddings

9. Data processing: Special cases (e.g., answer-based QRELs and bitext from QRELs)