dp-predict-stock

Final project of NTUST CS5144701 Practices of Deep Learning course.

The goal of this project is to improve the performance of this paper by using RNN-LSTM model instead of EB-CNN for prediction.

Training Guide

Script	Description
`process_data.ipynb`	Fetch data from stock price API and news dataset and generate data frame pickle file `train.pkl`.
`reverb.ipynb`	Preprocess data for reverb OpenIE phrase extraction `reverb_pre.txt` and extract the result `reverb_result.txt` for training model `train_reduce.pkl`.
`word2Vec.ipynb`	Produce word embedding model `word2vec.model`.
`main.ipynb`	Model training and evaluation.

Download jar from the Reverb Website Prepare newsdata.txt and run:

java -Xmx512m -jar ./test-reverb/reverb.jar reverb_pre.txt reverb_result.txt

News dataset: financial-news-dataset includes 450,341 news from Bloomberg and 109,110 news from Reuters within the date range from 2000 to 2013.
Stock price dataset: alphavantage API.

Decide using which dataset, including stock data and news data (2018-05-20).
To know how to use reverb to split one news title sentence (2018-05-25).
Process the news data: got the title and time (2018-05-27).
Process the stock data: got the price (2018-05-28).
Reconstruct the stock price data, only save the result that compared with last day price.If today's price is smaller than last days, it will be saved on 0 and other situation will be saved on 1 (2018-05-29).
Using the Reverb technology to split a news title sentence of all news's data to the three part format (2018-05-29).
Generate the word_embedding model, using reverb-result (2018-05-30).
Generate embedding_matrix, using Keras's tool (2018-05-30).
Construct the neural network model(Event-LSTM) (2018-05-30).
Train and predict (2018-05-30).
If the predict's result dont bigger than 65%, to find and modify the errors (2018-06-01).
Generate the final project documents and slides (2018-06-18).