Skip to content

PowerBee is a reddit downloader, allowing you to save submissions, comments, and content

Notifications You must be signed in to change notification settings

aperrien/PowerBee

Folders and files

NameName
Last commit message
Last commit date

Latest commit

3fa2b23 · Jul 27, 2019

History

35 Commits
Jun 27, 2019
Jul 27, 2019
Jun 26, 2019
Jun 12, 2019
Jul 27, 2019
Jun 21, 2019
Jun 17, 2019
Jun 14, 2019
Jun 27, 2019
Jul 16, 2019
Jun 21, 2019
Jun 27, 2019
Jun 12, 2019

Repository files navigation

PowerBee

PowerBee is a reddit downloader, allowing you to save submissions, comments, and content. This is still alpha code! but it is useful.

Usage: After unpacking and installing dependencies, you will need to create an clientID and client secret (OAuth2 tokens) for reddit. You can do that by following the directions given here:

https://github.com/reddit-archive/reddit/wiki/OAuth2-Quick-Start-Example#first-steps

After you have those, enter those values into your PowerBeeConfig.yaml file, replacing the placeholder values. Initialize the main SQLite database by running InitalizeDatabase.py

Now you can begin dowloading from reddit. To start, run "LoadSubredditsToDB.py". This will load the database with all your subscribed subreddits. This process may take a while, depending on reddit's server load.

Next, you can run the "RedditCommentExtractor003.py" script. This script iterates through all your saved subreddits, searching for saved submissions. It will then download up to 1750 of the "best" comments per submission. Note that this can can take a very long time, if you follow a lot of subreddits, and they are very active. For example, I have downloaded 40,099 submissions, from 733 subreddits, totaling some 8,975,214 comments. This took about six months to do, however since PowerBee doesn't repeat itself on downloads, subsequent scheduled runs are now relatively fast.

Dependencies:

Python 3

praw

pywebcopy

pyyaml (Should be in most distributions)

About

PowerBee is a reddit downloader, allowing you to save submissions, comments, and content

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published