autoautelion

Automated reporting of revenue and royalty fee for Autelion-registered authors.

Features

The following tool automates reporting by:

• periodically web scraping Autelion (legacy CGI with invalid HTML)
• caching status in Redis for quick responses
• sending email notifications on status changes
• providing web based view at http://autoautelion.herokuapp.com/
• providing JSON API endpoint at http://autoautelion.herokuapp.com/api
• securing both endpoints with HTTP Basic Authentication

Development

Clone the repository from GitHub:

$ git clone [email protected]:bzaczynski/autoautelion.git

Install requirements in a virtual environment:

$ cd autoautelion
$ pipenv install --dev

Install local redis server:

$ sudo apt install redis-server

Export environment variables:

$ export AUTELION_USERNAME=<your-username>
$ export AUTELION_PASSWORD=<your-password>
$ export REDIS_URL=redis://localhost
$ export EMAIL_ADDRESS=<your-email>
$ export SENDGRID_API_KEY=<your-api-key>

Run updater job locally:

$ python updaterjob.py

Run web server locally:

$ FLASK_APP=webapp.py flask run

...alternatively:

$ gunicorn webapp:app

Cloud Provisioning

Install Heroku command-line client:

$ sudo snap install heroku --classic

Log in to your Heroku account, either via web browser:

$ heroku login

...or from the terminal:

$ heroku login -i

This will create and store a new session in ~/.netrc file.

Change directory to the cloned GitHub repository:

$ cd autoautelion

If you had already created a Heroku app through their website, then add a Git remote with the corresponding app name, e.g.

$ heroku git:remote -a autoautelion

Otherwise, simply create a new app. If you omit the name a random one will be chosen automatically.

$ heroku create autoautelion

Install add-ons. Note this requires account verification by providing your credit card details to Heroku (for abuse prevention).

$ heroku addons:create heroku-redis:hobby-dev -a autoautelion
$ heroku addons:create scheduler:standard
$ heroku addons:create sendgrid:starter
$ heroku addons:create papertrail

This will take a while, so you may want to check the creation status:

$ heroku addons

Schedule a background job to run daily:

$ heroku addons:open scheduler

...then enter python updaterjob.py as the command.

Grab SendGrid auto-generated username and password:

$ heroku config | grep SENDGRID

Navigate your browser to https://app.sendgrid.com/settings/api_keys and log in using the obtained credentials. Click the Create API Key button, choose full access key and then provide key name e.g. autoautelion. Copy the key and define a Heroku environment variable. Note that SendGrid will not show the key again!

$ heroku config:set SENDGRID_API_KEY=<your-api-key>

Scale the web process:

$ heroku ps:scale web=1

Configuration

To set or update remote configuration on Heroku:

$ heroku config:set AUTELION_USERNAME=<your-username> AUTELION_PASSWORD=<your-password>
$ heroku config:set SENDGRID_API_KEY=<your-api-key> EMAIL_ADDRESS=<your-email>

Note that REDIS_URL variable is configured automatically.

Confirm remote configuration validity. This will also display other environment variables, e.g. for add-ons:

$ heroku config

Changing configuration creates and deploys a new release. To see all releases:

$ heroku releases

Releasing

If Heroku app has been connected to a GitHub account then pushing directly to origin master or accepting a pull request will trigger the build and deployment of a new release.

Otherwise use Heroku Git remote:

$ git push heroku master

To rollback a release to the immediately previous version:

$ heroku rollback

...or to a specific one:

$ heroku rollback v123

Testing and Debugging

To run locally:

$ heroku local

To open deployed app on Heroku:

$ heroku open

If something goes wrong, take a look at the logs:

$ heroku logs --tail

Search through the logs persisted in Papertrail add-on:

$ heroku addons:open papertrail

List dynos for the app:

$ heroku ps

Run a one-off dyno (temporary container) with /bin/bash to debug:

$ heroku run bash

Log in to Redis:

$ heroku redis:cli --confirm autoautelion