Introduction

This is the code repository for Meteor (specifically the Meteor API server), which contains an interlinked knowledge graph of polticial parties, datasets, news sources, organizations, tools and much more (for details see the OPTED main page).

Meteor is an API that allows other applications to interact with the database:

• browse & query the Meteor database
• Add new entries to the database
• edit and update existing entries

Technical Details

• developed in the Python framework Flask (version 2.3.x)
• uses DGraph as backend / data storage

Getting started

1. Clone the repository
2. create a new virtual environment with python3 -m venv .venv and activate it with source .venv/bin/activate
3. Install the requirements via pip (python3 -m pip install -r requirements.txt)
4. Install DGraph on your local machine (see official documentation)
5. Launch your local DGraph instances (alpha and zero)
6. Launch the inventory via python3 run.py
   • alternatively you can specify a configuration via python3 run.py --config config.json
7. Set the data schema to your DGraph instance via python3 tools/setschema.py
8. Add sample data to your DGraph instance using python3 tools/sample_data.py
9. Open your browser at your localhost with port 5000 (http://127.0.0.1:5000)

Run Tests

Make sure you set the schema and loaded the sample data to dgraph (see above)

• Test the API with python3 tests/test_api.py
• External services with python3 tests/test_external.py (make sure you have all API keys)
• DOI resolution / ORCID tests with python3 tests/test_doi.py
• additionally, can test: schema, notification, api_sanitizer, and queries.

Other tests might be removed later

Quick Codebase Walkthrough

├── data                  # contains sample data in various formats and the dgraph schem
├── meteor                # root folder for flask
├── README.md             # this file
├── requirements.txt      # python requirements
├── run.py                # Flask launcher
└── tools                 # collection of helper scripts and files for server deployment

Contributing & Bug Fixes

Please contact Paul Balluff for contribution. If you found a problem, just raise an issue here in this repository.

Other Notes

User Permissions

Level	Name	Permissions
0	Anonymous	View entries
1	Contributor	Add entries
2	Reviewer	+ Review & Edit Entries, Invite Users
10	Admin	+ Change User Permissions

Backups

Currently backups are uploaded to ucloud (for data safety and convenience). If the amount of database dumps gets too large, a different solution would be preferred.

Backups are run by a user with root privileges, because database dumps are created by the dgraph user and thus, cannot be manipulated directly by other users. Of course it would be possible to run some chown magic though.

Make sure that the requests module is installed for the root user.

Create Backup Settings

create a settings file (backup.json) inside the tools folder:

{
    "ucloud_url": "https://ucloud.univie.ac.at/public.php/webdav/",
    "ucloud_token": "secret",
    "backup_path": "/mnt/public/backup/"
}

Make sure the directory backup_path exists and that the dgraph user has rights to write to it.

Run backup

Simply execute the script tools/backup.py.

Add it as cronjob with crontab -e

0 1 * * * /usr/bin/python3 /home/user/wp3inventory/tools/backup.py

Restore Backup

• Get backup files and run the following command: dgraph live --files g01.rdf.gz --schema g01.schema --zero localhost:5080

Acknowledgements

This project has received funding from the European Union’s Horizon 2020 research & innovation programme under grant agreement No 951832. The document reflects only the authors’ views. The European Union is not liable for any use that may be made of the information contained herein.