Related or similar questions

[arvindpdmn]

Sites such as StackOverflow and Quora have this feature. Given a question, the site will list related or similar questions. In addition, this list will be sorted by relevance and popularity.

We define two types of relations, both of which should be handled by the NLP model:

• Related questions: For example, What are the benefits of using CSS? and Which are the key features of CSS? are related since they both are about CSS. However, they're asking different things.
• Similar questions: For example, How is TensorFlow different from other ML/DL platforms? and Could you compare PyTorch with TensorFlow? are "more or less" asking the same thing.

Similar questions should have higher ranking (stronger association) compared to related questions.

Every article in Devopedia has a Discussion section. Content in this section is presented as Q&A. We may say that questions within an article are related. They're unlikely to be similar.

Every article in Devopedia is linked to other related articles in the See Also section. Questions in all related articles can be analyzed to check if any two questions are similar.

In any case, articles in Devopedia can be useful as training data. We could also see if StackOverflow, Quora, or other public datasets can be used for training.

[arvindpdmn]

Resources:

• https://cloud.google.com/architecture/building-real-time-embeddings-similarity-matching-system
• https://github.com/UKPLab/sentence-transformers
• https://github.com/oborchers/Fast_Sentence_Embeddings