Bengali Abstractive Summarization using deep learning
Text summarization using deep learning has become quite a well studied research problem for high resource languages such as English and other European languages. However, very few works have been done for poorly available resource languages such as Indian subcontinent languages, African languages across the Internet. The resource scarce languages have limited scope in natural language processing (NLP) so far with lack of proper parallel corpus, parser, chunkers, tokenizer, POS taggers, etc. In this paper, we propose an abstractive text summarization sequence to sequence deep learning model for Bengali. We performed end-to-end word-level cross entropy training of the sequence to sequence model on Bengali news articles scrapped from a major daily news paper in Bangladesh. We also trained Word2vec on the dataset to generate Bengali word embeddings. The model used is based on encoder-decoder architecture. The encoder has multilayered Bidirectional Long Short-Term Memory (LSTM) layers to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. The decoder consists of a uni-directional LSTM-RNN with the same hidden-state size, and an attention mechanism over the source-hidden states and a soft-max layer over target vocabulary to generate abstractive summary sentences. We have evaluted the summary generation using ROUGE-1, ROUGE-2, ROUGE-L scores to evaluate the deep learning model. We also presented and analyzed both the good quality and bad quality summary generated by our model. We also used beam size of 10 for beam search decoder to generate summaries.