pyontutils

python utilities for working with ontologies

Installation

pyontutils is slowly approaching stability. You can obtain it and other related packages from pypi and install them as you see fit (e.g. pip install --user pyontutils). If you need a bleeding edge version I reccomend installing it into your environment (virtual or otherwise) using pip install --user --editable .[dev,test] run from from your local copy of this repo.

Configuration

pyontutils makes use of 3 configuration files:

1. ~/.config/pyontutils/devconfig.yaml which is not installed by default. To created it you should run ontutils devconfig --write and then edit it to match your system.
2. secrets.yaml that you can put wherever you want by editing the secrets_file: entry in devconfig.yaml. The file mode needs to be set to 0600 so that only you can read and write it. It is also advisable to place it inside a folder with a mode set to 0700 since some editors do not preserve file modes. pyontutils will fail loudly if this happens.
3. ./nifstd/scigraph/curie_map.yaml or ~/.config/pyontutils/curie_map.yaml if a pyontutils git repository is not found. pyontutils will retrieve the latest version of this file from github on first run if it cannot find a local copy.

If you are going to use the SciCrunch SciGraph production instance follow the instructions in the sparc curation setup guide to obtain an API key and put it in the right place. Alternately you can set the key using the SCICRUNCH_API_KEY environment variable (e.g., by running export SCICRUNCH_API_KEY=$(cat path/to/my/apikey)) or by whatever means you prefer for managing your keys.

Development Installation

From the directory that contains this readme run the following. Refer to .travis.yml for full details.

for f in {librdflib,htmlfn,ttlser,.,neurondm,nifstd}; do pushd $f; pip install --user --pre --editable . ; popd; done

If you need even more information there is fairly exhaustive doccumentation located in the sparc curation setup doc.

Requirements

This repo requires PyPy3 or >=Python3.6. See and setup.py and Pipfile for additional requirements. ontload requires Java8 and >=maven3.3 in order to build SciGraph. parcellation requires FSL to be installed or you need to obtain the atlases in some other way. In order to build the packages required by this repo you will need gcc (and toolchain) installed and will need to have the development packages for libxml installed. To build the development dependencies you will also need the development packages for postgresql, and protobuf installed on your system. Building the documentation for the ontology requires pandoc and emacs along with orgstrap. See .travis.yml for an example of how to bootstrap a working dev environment.

Utility Scripts

pyontutils provides a set of scripts that are useful for maintaining and managing ontologies using git, and making them available via SciGraph. Note that if you choose the development installation option you will need to ln -sT the scripts to your preferred bin folder. For the full list please see the documentation.

1. ttlfmt Reserialize ontology files using deterministic turtle (spec).
2. ontutils Various useful and frequently needed commands for ontology processes as well as less frequent refactorings.
3. ontload Load an ontology managed by git into SciGraph for easy deployment of services.
4. qnamefix Set qnames based on the curies defined for a given ontology.
5. necromancy Find dead ids in an ontology and raise them to be owl:Classes again.
6. scigraph-codegen Generate a rest client against a SciGraph services endpoint.
7. scig Run queries against a SciGraph endpoint from the command line.
8. graphml_to_ttl Convert yEd graphml files to ttl.
9. ontree Run a webserver to query and view hierarchies from the ontology.

NIF-Ontology

Many of these scripts are written for working on the NIF standard ontology found here.

SciGraph

scigraph.py is code geneator for creating a python client library against a SciGraph REST endpoint. scigraph_client.py is the client library generated against the nif development scigraph instance. ontload can be used to load your ontology into SciGraph for local use.

Building releases

Test sdist packaging

for f in {htmlfn,ttlser,.,neurondm,nifstd}; do pushd $f; python setup.py sdist; popd; done

Build wheels from the sdist NOT straight from the repo because wheels ignore the manifest file. Make sure to clean any previous builds first.

for f in {htmlfn,ttlser,neurondm,nifstd}; do
pushd $f/dist;
tar xvzf *.tar.gz;
pushd $f*/;
python setup.py bdist_wheel;
mv dist/*.whl ../;
popd;
rm ./$f*/ -r;
popd;
done