aupolitics

Australian politics research, based on data from Twitter and other open sources

The workflows at the moment, listed below in the order required to run

Tweets text analysis

1. data-collection.R - scrapping Australian parlament website for data of Members of Parlaments (MPs) Reference to blog post with more detailed explanation of the approach - https://levashov.biz/scrapping-data-about-australian-politicians-with-rselenium/
2. tweets-collection.R - collecting tweets from MPs
3. analysis.R - analysis of data

Data collection to data file stored at AWS S3 on regular basis and futher visualization

1. dbcollection-rtweet.R initial collection of data (larger number), saving to results to the database (DB was used in past, not anymore)
2. tweet-update.R script is running daily on schedule, collects the data and saves to the data file on S3
3. visualization.R some visualization
4. AUPolVizual/app.R Shiny R Application with visualization, available publically at https://rserv.levashov.biz/shiny/rstudio/

Older versions of file working with DB

Retainer with DB suffix