Skip to content

Latest commit

 

History

History
29 lines (20 loc) · 949 Bytes

README.md

File metadata and controls

29 lines (20 loc) · 949 Bytes

Masked Word Prediction using BERT

Overview

This project involves creating an AI to predict a masked word in a text sequence using the BERT language model. The project is divided into two main parts:

  1. Implementing a program to predict masked words using BERT.
  2. Analyzing the attention diagrams generated by the model to understand what BERT's attention heads focus on.

Requirements

  • Python Version: The latest version of Python you should use is Python 3.12.
  • Dependencies: Install the required dependencies using pip3 install -r requirements.txt.

How to Run

To run the program, use the following command:

$ python mask.py

Usage

Text: We turned down a narrow lane and passed through a small [MASK].

We turned down a narrow lane and passed through a small field.
We turned down a narrow lane and passed through a small clearing.
We turned down a narrow lane and passed through a small park.