CentraleSupélec - Reinforcement Learning Project - 2024
Explore the docs »
Table of Contents
Pedantle is a game part of the cemantle games, a collection of games on words.
The first game to appear was Semantle, created by David Turner. The game is a words game, where the goal is to find the random word of the day. The player can submit any amount of words, and the game will tell him if how close his submitted word is to the random word of the day. The game is based on word2vec, which maps any words of the english dictionnary (and even more, such as conjugated verbs or some family names). The word2vec used by Semantle is the Google one, available here.
Enigmatix, inspired by Semantle, created Cémantix, the same game as Semantle but with French words. The Google word2vec being applicable only for english words, Cémantix is based on another word2vec, the one of Jean-Philippe Fauconnier. Cémantix was the first game of the Cémantix collection. Enigmatix launched another game, Pédantix, a game where the goal is to find the title of a random French wikipedia page each day. The player submits words and the game reveals how close this word is to the ones from the article. As Cémantix, the game is based on Fauconnier's word2vec.
Eventually, Julie, based on Pédantix, created Pedantle, the same game as Pédantix but with English wikipedia articles. The game is thus based on the Google word2vec, as Semantle. Julie also created Cemantle, the exact same game as Semantle but with Pedantle/Cémantix/Pédantix interface.
The goal of this project is to create an agent that can play Pedantle, using Reinforcement Learning. We made the pedantle environment from scratch, using the gym library. We use Google's word2vec to compute similarity between words.
To get a local copy of the project up and running, follow these few steps.
- Clone the repo
git clone https://github.com/GabrielBeFr/Pedantle-RL.git
- Install the above mentioned required packages
pip install -r requirements.txt
- Install the Pedantle environment
pip install -e gym-examples
- Download the pre-trained word2vec model here and add it to the data/ directory. You can find this file from this Google Code Archive, in the Pre-trained word and phrase vectors part.
- (Optional - heavy in storage) You can also download the true wikipedia dataset from the Hugging Face datasets library, here, by running the dataset_downloader.py file:
python dataset_downloader.py
- (Read carefully) You have to create a faiss index to search words from the word2vec vocabulary quicker. By running the following command, you will initiate a (very) long and greedy process of faiss index construction. First, a small index named "word2vec_test.faiss" will be created. This should be fast. But the second index building should take a long time to finish. It thus advised that you end the process manually after having created the test index. However, for the agent to perform with the whole word2vec vocabulary, the large index is mandatory.
python faiss_index_maker.py
- You can now run the main.py file to see the agent playing Pedantle:
In the main.py file, you can change the parameters of the agent, such as the number of episodes, the number of steps per episode, the learning rate, etc. but also activate or deactivate the human display mode, the test mode, and change the wikipedia dataset path. Default is:
python main.py
If you want to run the agent with the whole vocabulary and the true wikipedia dataset, you can change to:env = gym.make( "gym_examples/Pedantle-v0", render_mode="human", # else None test_model=True, logging = logging, )
env = gym.make( "gym_examples/Pedantle-v0", render_mode=None, # else "human" test_model=True, wiki_file="data/wikipedia_dataset.csv", logging = logging, )
- Eventually, you can visualize the results of the agent by running the following command:
Don't forget to modify the results.py file with the correct path to the results file (You should find it in the results/ directory).
python results.py
Project Link: https://github.com/GabrielBeFr/Pedantle-RL