From f9d8f9ba23edc9e35335a369e275d11fd2b663bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= <43052541+kysrpex@users.noreply.github.com> Date: Fri, 25 Mar 2022 13:32:27 +0100 Subject: [PATCH 1/3] Interpret `rdfs:Class` as classes on SimPhoNy v3, include dcmitype and dcterms namespaces (#201) Documentation for [PR #760](https://github.com/simphony/osp-core/pull/760) with the same title from the OSP-core repository. --- docs/source/general_architecture.md | 4 +- docs/source/index.md | 5 +- docs/source/jupyter/cuds_api.ipynb | 4 +- docs/source/jupyter/ontology_interface.ipynb | 4 +- docs/source/ontologies_included.md | 203 ++++++++++++++++-- docs/source/ontology_intro.md | 22 +- docs/source/owl.md | 105 --------- docs/source/utils.md | 3 +- .../{yaml.md => working_with_ontologies.md} | 152 +++++++++++-- requirements.txt | 2 + 10 files changed, 354 insertions(+), 150 deletions(-) delete mode 100644 docs/source/owl.md rename docs/source/{yaml.md => working_with_ontologies.md} (66%) diff --git a/docs/source/general_architecture.md b/docs/source/general_architecture.md index 4050b6e..d3163fe 100644 --- a/docs/source/general_architecture.md +++ b/docs/source/general_architecture.md @@ -133,8 +133,8 @@ It is independent of any backend and provides the basic ontology based data stru ### Ontology file OSP-core requires an ontology file to create the appropriate CUDS classes. -Said ontology must be either in a YAML format as defined by [our specification](yaml.md) -or [one of the supported owl ontologies](owl.md). +Said ontology must be either in a YAML format as defined by [our specification](working_with_ontologies.md#osp-core-yaml-ontology-format) +or [one of the supported owl ontologies](working_with_ontologies.md#owl-ontologies-and-rdfs-vocabularies).
YAML Ontology sample diff --git a/docs/source/index.md b/docs/source/index.md index 8ec2b7d..88b613e 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -77,9 +77,9 @@ installation :caption: Usage Guide :maxdepth: 2 +utils jupyter/cuds_api jupyter/sessions_and_vars -utils jupyter/multiple_wrappers jupyter/import_export jupyter/simlammps @@ -92,9 +92,8 @@ jupyter/quantum_espresso :maxdepth: 2 ontology_intro +working_with_ontologies ontologies_included -yaml -owl jupyter/ontology_interface ``` diff --git a/docs/source/jupyter/cuds_api.ipynb b/docs/source/jupyter/cuds_api.ipynb index 5fb9905..7133b13 100644 --- a/docs/source/jupyter/cuds_api.ipynb +++ b/docs/source/jupyter/cuds_api.ipynb @@ -63,7 +63,7 @@ "\n", "- `attributes`: The values of the ontology attributes of an individual (also known as [data properties](https://www.w3.org/TR/owl2-syntax/#Data_Properties) in [OWL](https://en.wikipedia.org/wiki/Web_Ontology_Language)) may also be accessed and modified as python properties of the CUDS objects. For example: `cuds_object.name = \"Bob\"`.\n", "\n", - "There are some advanced functionalities NOT covered in this tutorial. Among them, we highlight the `update` method, which is covered in the [wrapper tutorial](multiple-wrappers.ipynb), where it can be seen in action. For a complete list of available methods and properties, check the API reference. That whole set of methods and attributes constitutes the CUDS [API](https://en.wikipedia.org/wiki/API).\n", + "There are some advanced functionalities NOT covered in this tutorial. Among them, we highlight the `update` method, which is covered in the [wrapper tutorial](multiple_wrappers.ipynb), where it can be seen in action. For a complete list of available methods and properties, check the API reference. That whole set of methods and attributes constitutes the CUDS [API](https://en.wikipedia.org/wiki/API).\n", "\n", "[//]: # (TODO: Link to API reference.)" ] @@ -88,7 +88,7 @@ } }, "source": [ - "The first step is to install the city ontology. Use the tool pico for this. If you want to know more about ontology installation, check the documentation on the [pico ontology installation tool](https://simphony.readthedocs.io/en/latest/utils.html#pico-installs-cuds-ontologies), [YAML ontology installation files](https://simphony.readthedocs.io/en/latest/yaml.html), and [installing OWL ontologies](https://simphony.readthedocs.io/en/latest/owl.html)." + "The first step is to install the city ontology. Use the tool pico for this. If you want to know more about ontology installation, check the documentation on the [pico ontology installation tool](https://simphony.readthedocs.io/en/latest/utils.html#pico-installs-cuds-ontologies), [YAML ontology installation files](https://simphony.readthedocs.io/en/latest/working_with_ontologies.html#osp-core-yaml-ontology-format), and [installing OWL ontologies](https://simphony.readthedocs.io/en/latest/working_with_ontologies.html#owl-ontologies-and-rdfs-vocabularies)." ] }, { diff --git a/docs/source/jupyter/ontology_interface.ipynb b/docs/source/jupyter/ontology_interface.ipynb index 2e259aa..f691158 100644 --- a/docs/source/jupyter/ontology_interface.ipynb +++ b/docs/source/jupyter/ontology_interface.ipynb @@ -28,7 +28,7 @@ "source": [ "In an ontological framework, ontology entities are used as a knowledge representation form. Those can be further categorized in two groups: ontology individuals ([assertional knowledge](https://en.wikipedia.org/wiki/Abox)), and ontology classes, relationships and attributes ([terminological knowledge](https://en.wikipedia.org/wiki/Tbox)).\n", "\n", - "In a [previous tutorial](./cuds_api.html), we have discussed how to work with CUDS objects, which represent ontology individuals. In this tutorial, we present the API of all the other entities instead: ontology classes, relationships and attributes. These are defined in an ontology installation file in [YAML](../yaml.md) or [OWL](../owl.md) format. The presented API enables you to access the entities and navigate within an ontology." + "In a [previous tutorial](./cuds_api.html), we have discussed how to work with CUDS objects, which represent ontology individuals. In this tutorial, we present the API of all the other entities instead: ontology classes, relationships and attributes. These are defined in an ontology installation file in [YAML](../working_with_ontologies.md#osp-core-yaml-ontology-format) or [OWL](../working_with_ontologies.md#owl-ontologies-and-rdfs-vocabularies) format. The presented API enables you to access the entities and navigate within an ontology." ], "metadata": {} }, @@ -195,7 +195,7 @@ "source": [ "The **most convenient way** to access an ontology entity is using the **dot notation** in python. For example, `city.Citizen`. This method is a shorthand for fetching by suffix or label: \n", "\n", - "- When the keyword `reference_by_label` is set to `True` (enabled) in the [ontology YAML installation file](../owl.md), the dot notation is a shorthand for fetching by label. This keyword is **enabled** in the `math` namespace.\n", + "- When the keyword `reference_by_label` is set to `True` (enabled) in the [ontology YAML installation file](../working_with_ontologies.md), the dot notation is a shorthand for fetching by label. This keyword is **enabled** in the `math` namespace.\n", "\n", "- When the keyword `reference_by_label` is set to `False` (disabled) or not set, the dot notation is a shorthand for fetching by suffix instead. This keyword is **disabled** in the `city` namespace." ], diff --git a/docs/source/ontologies_included.md b/docs/source/ontologies_included.md index c14b2e9..4961328 100644 --- a/docs/source/ontologies_included.md +++ b/docs/source/ontologies_included.md @@ -1,38 +1,66 @@ # Included ontologies -## The City ontology +As described on the [working with ontologies](../working_with_ontologies.md) +section, to use an ontology you first have to install it, and to do so +usually you have either to define a `yml` +[configuration file](working_with_ontologies.md#owl-ontologies-and-rdfs-vocabularies) +(for OWL ontologies and RDFS vocabularies) or provide the ontology in the +[OSP-core YAML ontology format](working_with_ontologies.md#osp-core-yaml-ontology-format). -OSP-core currently ships with two ontologies. -The first one, called `city` is a simple example ontology. -You can use it to play around and get familiar with OSP-core. -We will also use it a lot in this documentation as an example. +However, in order to make using ontologies easier, we bundle a few of these +files with OSP-core to enable rapid installation of common, +well-known ontologies. Do not hesitate to [contact us](contact.md) if you want +your ontology to be shipped with SimPhoNy. -The city ontology provides the concepts to describe people and -buildings in a city. In this graph we show the different entities in the -ontology. We used [Ontology2Dot](utils.md#ontology2dot) for that: +The included ontologies, together with their domains of application, are +listed below. -![ontology2dot sample image](_static/img/ontology2dot.png) +- [Elementary Multiperspective Material Ontology (EMMO)](ontologies_included.html#elementary-multiperspective-material-ontology-emmo) + \- Applied sciences +- [Dublin Core Metadata Initiative (DCMI)](ontologies_included.html#dublin-core-metadata-initiative-dcmi) + \- Metadata description +- [Data Catalog Vocabulary - Version 2 (DCAT2)](ontologies_included.html#data-catalog-vocabulary-version-2-dcat2) + \- Data catalogue information +- [Friend of a Friend (FOAF)](ontologies_included.html#friend-of-a-friend-foaf) + \- People and information on the web +- [The PROV Ontology (PROV-O)](ontologies_included.html#the-prov-ontology-prov-o) + \- Provenance information +- [The City Ontology](ontologies_included.html#the-city-ontology) + \- Example ontology aimed at demonstrating the usage of SimPhoNy OSP-core -To use the city ontology you have to install it using the tool [Pico](utils.md#pico-installs-cuds-ontologies): +The ontologies can be installed providing the right +_[package identifier](working_with_ontologies.md#keywords)_ to +[pico](utils.md#pico-installs-cuds-ontologies). You can find such +package identifier and additional information on each ontology by clicking on +the links from the list above. -```sh -pico install city -``` +## Elementary Multiperspective Material Ontology (EMMO) -Take a look at our [examples](jupyter/cuds_api.md) to see how you can build your own city! +```{eval-rst} +.. epigraph:: -## Working with EMMO + EMMO is a multidisciplinary effort to develop a standard representational + framework (the ontology) for applied sciences. It is based on physics, + analytical philosophy and information and communication technologies. It + has been instigated by materials science to provide a framework for + knowledge capture that is consistent with scientific principles and + methodologies. It is released under a Creative Commons + `CC BY 4.0 `_ + license. -The second ontology that is ready to be used out of the box is the Elementary Multiperspective Material Ontology, -or EMMO in short. For a short introduction, see the [fundamentals](fundamentals.md#emmo). + -- `About EMMO section `_, + from the + `official EMMO GitHub repository `_ +``` -You can install EMMO using [Pico](utils.md#pico-installs-cuds-ontologies). +For a short introduction on this ontology, see the [fundamentals](fundamentals.md#emmo) +section. To install the [EMMO ontology](https://emmo-repo.github.io/), use ```sh pico install emmo ``` -Start creating cuds objects: +and then just start creating cuds objects ```py >>> from osp.core.namespaces import math @@ -45,7 +73,138 @@ Start creating cuds objects: 12 ``` -## More ontologies +## Dublin Core Metadata Initiative (DCMI) + +```{eval-rst} +.. epigraph:: + + The Dublin Core™ Metadata Initiative, or "DCMI", is an organization + supporting innovation in metadata design and best practices across the + metadata ecology. DCMI works openly, and it supported by a + `paid-membership model `_. + + -- `About DCMI `_ +``` + +The Dublin Core™ Metadata Initiative has published, among others, the +[DCMI Metadata Terms](https://www.dublincore.org/specifications/dublin-core/dcmi-terms/) +specification, which establishes a set of core metadata terms enabling +cross-domain description of resources on the web. + +```{eval-rst} +.. epigraph:: + + Included are the fifteen terms of the Dublin Core™ Metadata Element Set + (also known as "the Dublin Core") plus several dozen properties, classes, + datatypes, and vocabulary encoding schemes. [...] These terms are intended + to be used in combination with metadata terms from other, compatible + vocabularies in the context of application profiles. + + -- `DCMI Metadata Terms `_ +``` + +To install the `dcmitype` and `dcterms` RDFS vocabularies from the [Dublin +Core Metadata Initiative (DCMI)](https://www.dublincore.org/specifications/dublin-core/dcmi-terms/), use + +```sh +pico install dcmitype dcterms +``` + +Note that due to the fact that +[RDFS properties are not supported by OSP-core](working_with_ontologies.html#rdfs-vocabularies), +the properties in these two vocabularies will be ignored. Only the classes will +be detected. + +## Data Catalog Vocabulary - Version 2 (DCAT2) + +```{eval-rst} +.. epigraph:: + + DCAT is an RDF vocabulary designed to facilitate interoperability + between data + catalogs published on the Web. [...] + + DCAT enables a publisher to describe datasets and data services in a catalog + using a standard model and vocabulary that facilitates the consumption and + aggregation of metadata from multiple catalogs. This can increase the + discoverability of datasets and data services. It also makes it possible to + have a decentralized approach to publishing data catalogs and makes + federated search for datasets across catalogs in multiple sites possible + using the same query mechanism and structure. Aggregated DCAT metadata + can serve as a manifest file as part of the digital preservation process. + + -- `Data Catalog Vocabulary (DCAT) - Version 2 `_ +``` + +To install the [DCAT2 ontology](https://www.w3.org/TR/vocab-dcat-2/), use + +```sh +pico install dcat2 +``` + +## Friend of a Friend (FOAF) + +```{eval-rst} +.. epigraph:: + + FOAF is a project devoted to linking people and information using the Web. + Regardless of whether information is in people's heads, in physical or + digital documents, or in the form of factual data, it can be linked. + FOAF integrates three kinds of network: social networks of human + collaboration, friendship and association; representational networks that + describe a simplified view of a cartoon universe in factual terms, and + information networks that use Web-based linking to share independently + published descriptions of this inter-connected world. FOAF does not + compete with socially-oriented Web sites; rather it provides an approach + in which different sites can tell different parts of the larger story, + and by which users can retain some control over their information in a + non-proprietary format. + + -- `FOAF Vocabulary Specification `_ +``` + +To install the [FOAF ontology](http://xmlns.com/foaf/spec/), use + +```sh +pico install foaf +``` + +## The PROV Ontology (PROV-O) + +```{eval-rst} +.. epigraph:: + + The PROV Ontology (PROV-O) expresses the PROV Data Model + [`PROV-DM `_] + using the OWL2 Web Ontology Language (OWL2) + [`OWL2-OVERVIEW `_]. + It provides a set of classes, properties, and restrictions that can be used + to represent and interchange provenance information generated in different + systems and under different contexts. It can also be specialized to create + new classes and properties to model provenance information for different + applications and domains. + + -- `PROV-O: The PROV Ontology `_ +``` + +To install the [PROV-O ontology](https://www.w3.org/TR/prov-o/), use + +```sh +pico install prov +``` + +## The City ontology + +The City ontology is a +[simple, example ontology](ontology_intro.html#an-example-the-city-ontology) +included with OSP-core. It provides a collection of concepts to describe +people and buildings in a city, and is aimed at demonstrating the usage of +SimPhoNy OSP-core. + +To install the +[city ontology](ontology_intro.html#an-example-the-city-ontology), use + +```sh +pico install city +``` -To create you own or use an existing ontology, see the upcoming sections. -Contact us, if you want your ontology to be shipped with SimPhoNy. diff --git a/docs/source/ontology_intro.md b/docs/source/ontology_intro.md index fe6a55d..d7f0765 100644 --- a/docs/source/ontology_intro.md +++ b/docs/source/ontology_intro.md @@ -1,6 +1,6 @@ # Introduction on ontologies -What is an ontology? +**What is an ontology?** An ontology defines a set of representational primitives with which to model a domain of knowledge or discourse. The representational @@ -11,3 +11,23 @@ and constraints on their logically consistent application. (Source: ) [//]: # (TODO Extend) + +## An example: the City ontology + +OSP-core ships with one simple example ontology, called `city`. +You can use it to play around and get familiar with OSP-core. +We will also use it a lot in this documentation as an example. + +The city ontology provides the concepts to describe people and +buildings in a city. In this graph we show the different entities in the +ontology. We used [Ontology2Dot](utils.md#ontology2dot) for that: + +![ontology2dot sample image](_static/img/ontology2dot.png) + +To use the city ontology you have to install it using the tool [Pico](utils.md#pico-installs-cuds-ontologies): + +```sh +pico install city +``` + +Take a look at our [examples](jupyter/cuds_api.md) to see how you can build your own city! diff --git a/docs/source/owl.md b/docs/source/owl.md deleted file mode 100644 index 2311ae2..0000000 --- a/docs/source/owl.md +++ /dev/null @@ -1,105 +0,0 @@ -# How to work with owl ontologies - -To install owl ontologies in OSP-core, you have to create a configuration yaml file similar -to the following one: - -```yaml -identifier: emmo -ontology_file: https://raw.githubusercontent.com/emmo-repo/EMMO/master/emmo-inferred.owl -format: turtle -reference_by_label: True -namespaces: - mereotopology: http://emmo.info/emmo/top/mereotopology - physical: http://emmo.info/emmo/top/physical - top: http://emmo.info/emmo/top - semiotics: http://emmo.info/emmo/top/semiotics - perceptual: http://emmo.info/emmo/middle/perceptual - reductionistic: http://emmo.info/emmo/middle/reductionistic - holistic: http://emmo.info/emmo/middle/holistic - physicalistic: http://emmo.info/emmo/middle/physicalistic - math: http://emmo.info/emmo/middle/math - properties: http://emmo.info/emmo/middle/properties - materials: http://emmo.info/emmo/middle/materials - metrology: http://emmo.info/emmo/middle/metrology - models: http://emmo.info/emmo/middle/models - manufacturing: http://emmo.info/emmo/middle/manufacturing - isq: http://emmo.info/emmo/middle/isq - siunits: http://emmo.info/emmo/middle/siunits -active_relationships: - - http://emmo.info/emmo/top/mereotopology#EMMO_8c898653_1118_4682_9bbf_6cc334d16a99 - - http://emmo.info/emmo/top/semiotics#EMMO_60577dea_9019_4537_ac41_80b0fb563d41 -default_relationship: http://emmo.info/emmo/top/mereotopology#EMMO_17e27c22_37e1_468c_9dd7_95e137f73e7f -``` - -## Keywords - -**identifier**: Can be any alphanumerical string. It is the name of the package -that contains multiple namespaces. Will be used for uninstallation: `pico uninstall emmo`. -(In YAML ontologies this package name or identifier is the same as the namespace name). - -**ontology_file**: Path to the inferred owl ontology. That means you should -have executed a reasoner on your ontology, e.g. by using the `Export inferred axioms` -functionality of [Protégé](https://protege.stanford.edu/). - -**format** (optional): File format of the ontology file to be parsed. We -support all the -formats that -[RDFLib](https://rdflib.readthedocs.io/en/stable/plugin_parsers.html) supports: -XML (`xml`, `application/rdf+xml`, default), Turtle (`turtle`, `ttl`, -`text/turtle`), N3 (`n3`,`text/n3`), NTriples (`nt`, `nt11`, -`application/n-triples`), N-Quads (`nquads`, `application/n-quads`), -TriX (`trix`, `application/trix`) and TriG (`trig`, `application/trig`). -When not provided, it will be guessed from the file extension. However, such -guess may not always be correct. - -**reference_by_label** (default False): Whether the label should be used or the IRI suffix to reference -entity from within OSP-core. In case of EMMO it is true, because IRI suffixes are not -human friendly. In this case all labels should be unique and not contain whitespaces. -If False, use dot notation to get by IRI square brackets (`__getitem__`) to get by label. -The latter will return a list of all entities with the same label. - -**namespaces**: mapping from namespace name (used to import the namespace) to iri prefix. -If IRI doesn't end with "/" or "#", "#" will be added. - -**active relationships**: -List of iris of active relationships. - -**default relationship**: -The default relationship. - -## Installation - -Name the yaml file as you would any yaml file `.yml`, where `` should be replaced by a user defined name. - -Then you can use pico to install the tool [Pico](utils.md#pico-installs-cuds-ontologies) -to install the ontology: - -```sh -pico install -``` - -## Limitations - -At the moment, only the most basic predicates of owl ontologies are used: - -- `RDF.type` to determine the type of the entities. -- `RDFS.label` to get the entities by label. -- `RDFS.isDefinedBy` to get a descriptions for the entities. -- `RDFS.subClassOf` / `RDFS.subPropertyOf` for subclasses. -- `OWL.inverseOf` for inverse relationships. -- `RDFS.range` to determine the datatype of `DataProperties`. These are the supported - datatypes: - - `XSD.boolean` - - `XSD.integer` - - `XSD.float` - - `XSD.string` -- To get the attributes of an owl class, we use - - The `RDFS.domain` of the `DatatypeProperties`, if it is a simple class. - - Restrictions on the ontology classes. - - Furthermore, all DataProperties are considered functional, see [this issue](https://github.com/simphony/osp-core/issues/416). - -No reasoner is included. We plan to include a reasoner in the -future. - -We try to extend this list over time and support more of the -OWL DL standard. diff --git a/docs/source/utils.md b/docs/source/utils.md index d8a1b40..c614d39 100644 --- a/docs/source/utils.md +++ b/docs/source/utils.md @@ -1,6 +1,7 @@ # Utilities In this section we will compile a list of useful utility functions, tools and examples of their usage. -This functions are part of OSP-core and are used as an extension of the [main API](./detailed_design#cuds-api). +These functions are part of OSP-core and are used as an extension of the +[main API](./detailed_design.md#cuds-api). ## Pico Installs Cuds Ontologies Our tool for installing ontologies is called `pico`. diff --git a/docs/source/yaml.md b/docs/source/working_with_ontologies.md similarity index 66% rename from docs/source/yaml.md rename to docs/source/working_with_ontologies.md index 77e7961..c6a98d9 100644 --- a/docs/source/yaml.md +++ b/docs/source/working_with_ontologies.md @@ -1,6 +1,134 @@ -# Description of YAML format specification +# How to work with ontologies -This file describes how you can create ontologies using YAML. +OSP-core supports ontologies in the following formats: +- [OWL ontologies](working_with_ontologies.md#owl-ontologies-and-rdfs-vocabularies) +- [RDFS vocabularies](working_with_ontologies.md#owl-ontologies-and-rdfs-vocabularies) (_[limited support](#rdfs-vocabularies)_) +- [OSP-core YAML ontology format](working_with_ontologies.md#osp-core-yaml-ontology-format) + +## OWL ontologies and RDFS vocabularies + +To install OWL ontologies or RDFS vocabularies in OSP-core, you have to +create a configuration yaml file similar to the following one: + +```yaml +identifier: emmo +ontology_file: https://raw.githubusercontent.com/emmo-repo/EMMO/master/emmo-inferred.owl +format: turtle +reference_by_label: True +namespaces: + mereotopology: http://emmo.info/emmo/top/mereotopology + physical: http://emmo.info/emmo/top/physical + top: http://emmo.info/emmo/top + semiotics: http://emmo.info/emmo/top/semiotics + perceptual: http://emmo.info/emmo/middle/perceptual + reductionistic: http://emmo.info/emmo/middle/reductionistic + holistic: http://emmo.info/emmo/middle/holistic + physicalistic: http://emmo.info/emmo/middle/physicalistic + math: http://emmo.info/emmo/middle/math + properties: http://emmo.info/emmo/middle/properties + materials: http://emmo.info/emmo/middle/materials + metrology: http://emmo.info/emmo/middle/metrology + models: http://emmo.info/emmo/middle/models + manufacturing: http://emmo.info/emmo/middle/manufacturing + isq: http://emmo.info/emmo/middle/isq + siunits: http://emmo.info/emmo/middle/siunits +active_relationships: + - http://emmo.info/emmo/top/mereotopology#EMMO_8c898653_1118_4682_9bbf_6cc334d16a99 + - http://emmo.info/emmo/top/semiotics#EMMO_60577dea_9019_4537_ac41_80b0fb563d41 +default_relationship: http://emmo.info/emmo/top/mereotopology#EMMO_17e27c22_37e1_468c_9dd7_95e137f73e7f +``` + +### Keywords + +**identifier**: Can be any alphanumerical string. It is the name of the package +that contains multiple namespaces. Will be used for uninstallation: `pico uninstall emmo`. +(In YAML ontologies this package name or identifier is the same as the namespace name). + +**ontology_file**: Path to the inferred owl ontology. That means you should +have executed a reasoner on your ontology, e.g. by using the `Export inferred axioms` +functionality of [Protégé](https://protege.stanford.edu/). + +**format** (optional): File format of the ontology file to be parsed. We +support all the +formats that +[RDFLib](https://rdflib.readthedocs.io/en/stable/plugin_parsers.html) supports: +XML (`xml`, `application/rdf+xml`, default), Turtle (`turtle`, `ttl`, +`text/turtle`), N3 (`n3`,`text/n3`), NTriples (`nt`, `nt11`, +`application/n-triples`), N-Quads (`nquads`, `application/n-quads`), +TriX (`trix`, `application/trix`) and TriG (`trig`, `application/trig`). +When not provided, it will be guessed from the file extension. However, such +guess may not always be correct. + +**reference_by_label** (default False): Whether the label should be used or the IRI suffix to reference +entity from within OSP-core. In case of EMMO it is true, because IRI suffixes are not +human friendly. In this case all labels should be unique and not contain whitespaces. +If False, use dot notation to get by IRI square brackets (`__getitem__`) to get by label. +The latter will return a list of all entities with the same label. + +**namespaces**: mapping from namespace name (used to import the namespace) to iri prefix. +If IRI doesn't end with "/" or "#", "#" will be added. + +**active relationships**: +List of iris of active relationships. + +**default relationship**: +The default relationship. + +### Installation + +Name the yaml file as you would any yaml file `.yml`, where `` should be replaced by a user defined name. + +Then you can use pico to install the tool [Pico](utils.md#pico-installs-cuds-ontologies) +to install the ontology: + +```sh +pico install +``` + +### Limitations + +At the moment, there are a few limitations on the supported features of OWL +ontologies and RDFS vocabularies. + +#### OWL ontologies + +Not all predicates of OWL ontologies are taken into +consideration. Among the used ones are: + +- `RDF.type` to determine the type of the entities. +- `RDFS.label` to get the entities by label. +- `RDFS.isDefinedBy` to get a descriptions for the entities. +- `RDFS.subClassOf` / `RDFS.subPropertyOf` for subclasses. +- `OWL.inverseOf` for inverse relationships. +- `RDFS.range` to determine the datatype of `DataProperties`. These are the supported + datatypes: + - `XSD.boolean` + - `XSD.integer` + - `XSD.float` + - `XSD.string` +- To get the attributes of an owl class, we use + - The `RDFS.domain` of the `DatatypeProperties`, if it is a simple class. + - Restrictions on the ontology classes. + - Furthermore, all DataProperties are considered functional, see [this issue](https://github.com/simphony/osp-core/issues/416). +- Restrictions and compositions are also supported. They can be consulted + using the [`axioms` attribute of ontology classes](jupyter/ontology_interface.html#Operations-specific-to-ontology-axioms). + +No reasoner is included. We plan to include a reasoner in the +future. + +We try to extend this list over time and support more of the +OWL DL standard. + +#### RDFS vocabularies + +With respect to RDFS vocabularies, the `RDFS.Class` predicate is supported, +but the `RDFS.Property` predicate is not. This means that the main +limitation when using RDFS vocabularies is that only their classes are +detected, but their properties are ignored. + +## OSP-core YAML ontology format + +This section describes how you can create ontologies using YAML. ```{tip} If you have an ontology where all entity names are in ALL_UPPERCASE, @@ -8,14 +136,14 @@ This file describes how you can create ontologies using YAML. OSP-core to transform it to an ontology with CamelCase entity names. ``` -## Introduction +### Introduction In this file we will give a description of how an Ontology can be represented in a yaml file format and how to interpret such files. For simplicity reasons in the following we will give examples from the **\* example ontology \*** file which can be found in osp/core/ontology/yml/ontology.city.yml. -## Naming of the files and installation +### Naming of the files and installation Name any ontology `.ontology.yml`, where `` should be replaced by a user defined name. @@ -26,7 +154,7 @@ to install the ontology: pico install ``` -## Syntax of the .yml ontology +### Syntax of the .yml ontology `version`: string @@ -63,7 +191,7 @@ pico install > The value of the mapping is a mapping whose format is detailed in the > [Ontology entities format](#ontology-entities-format). -## Ontology entities format +### Ontology entities format Every declaration of an ontology entity must have the following keys: @@ -91,7 +219,7 @@ For relationship entities, these keys are described in For attributes, these keys are described in [Attribute format](#attribute-format) section. -## The CUBA namespace +### The CUBA namespace The CUBA namespace contains a set of Common Universal Basic entities, that have special meaning in OSP-core. The CUBA namespace is always installed in OSP-core. @@ -117,7 +245,7 @@ The CUBA namespace is always installed in OSP-core. `cuba.passiveRelationship` > The inverse of `cuba.activeRelationship`. -## Attribute format +### Attribute format Every attribute is a subclass of `cuba.attribute`. The declaration of an attribute is a special case of the declaration of an entity. @@ -161,7 +289,7 @@ It can additionally have the following keys: > EMMO has established an appropriate wait of representing them > ``` -## Class expressions +### Class expressions A class expression describes a subset of individuals. They are similar to classes, but do not have a name in the ontology. @@ -224,7 +352,7 @@ or: This describes the set of all individuals that are a city or have a neighbourhood. -## CUDS classes format +### CUDS classes format The declaration of a cuds entity is a special case of the declaration of an entity. It must have the keys described in [Ontology entities format](#ontology-entities-format). @@ -283,7 +411,7 @@ It can contain further information: > > Here every geographical place that has citizens as inhabitants is automatically a populated place. -## Relationship format +### Relationship format Every relationship is a subclass of `cuba.relationship`. The declaration of a relationship is a special case of the declaration of an entity. @@ -319,7 +447,7 @@ Furthermore, it can contain the following information: A subclass of a relationship is called a sub-relationship. -## Limitations +### Limitations `Class expressions`, `domain`, `range`, `characteristics`, `equivalent_to`, `disjoint_with` are currently not parsed by OSP-core. diff --git a/requirements.txt b/requirements.txt index a8f844b..780f246 100644 --- a/requirements.txt +++ b/requirements.txt @@ -10,3 +10,5 @@ jupyter==1.0.0 sphinx-autobuild==2021.3.14 sphinx-panels==0.5.2 jinja2==2.11.3 +markupsafe==2.0.1 + From c828a07fb457fd5116fc51e55898aa1f3ebb83d6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= <43052541+kysrpex@users.noreply.github.com> Date: Fri, 25 Mar 2022 16:07:24 +0100 Subject: [PATCH 2/3] Document usage of `pico` as a module. (#200) --- docs/source/api_ref.md | 8 +++- docs/source/utils.md | 104 ++++++++++++++++++++++++++++++----------- requirements.txt | 1 - 3 files changed, 83 insertions(+), 30 deletions(-) diff --git a/docs/source/api_ref.md b/docs/source/api_ref.md index 4c0f6b0..03caa91 100644 --- a/docs/source/api_ref.md +++ b/docs/source/api_ref.md @@ -84,4 +84,10 @@ This document is for developers and/or advanced users of OSP-core, it contains a .. automodule:: osp.core.utils :imported-members: :members: -``` \ No newline at end of file +``` + +### pico +```{eval-rst} +.. automodule:: osp.core.pico + :members: install, uninstall, packages, namespaces +``` diff --git a/docs/source/utils.md b/docs/source/utils.md index c614d39..f12ff62 100644 --- a/docs/source/utils.md +++ b/docs/source/utils.md @@ -1,27 +1,34 @@ # Utilities In this section we will compile a list of useful utility functions, tools and examples of their usage. -These functions are part of OSP-core and are used as an extension of the -[main API](./detailed_design.md#cuds-api). +These functions are part of OSP-core and are used as an extension of the [main API](./detailed_design.md#cuds-api). -## Pico Installs Cuds Ontologies +## pico (Pico Installs Cuds Ontologies) Our tool for installing ontologies is called `pico`. It is a recursive acronym that stands for _Pico Installs Cuds Ontologies_. -There are 3 main things that can be done with pico: +There are 3 main operations that can be done with pico: - Install ontologies. - List the installed ontologies. - Remove installed ontologies. -There are different possible levels of log available, and they can be set via +`pico` can be used both from the [command-line](utils.md#using-pico-from-the-command-line) +and [as a Python module within the Python shell](utils.md#using-pico-as-a-python-module). + +### Using pico from the command line + +There are different possible logging levels available, and they can be set via `--log-level `. The default value is `INFO`. -### Pico installs -_Usage:_ `pico install |city|cuba` +#### pico installs +_Usage:_ +- `pico install ` +- `pico install ...` +- `pico install city foaf emmo dcat2` (the installation of these specific + well-known ontologies is available via this shortcut) _Behaviour:_ -- The ontology file is parsed, and the entities mapped to python objects. -- The python classes can be imported via their namespace - `from osp.core.namespaces import namespace` +- The ontology file is parsed, and the entities mapped to Python objects. +- The Python objects can be imported via their namespace `from osp.core.namespaces import namespace`. _Example:_ ```console @@ -33,12 +40,13 @@ INFO [osp.core.ontology.parser]: Loaded 367 ontology triples in total INFO [osp.core.ontology.installation]: Installation successful ``` -### Pico lists +#### pico lists _Usage:_ `pico list` _Behaviour:_ - The installed namespaces and packages are printed out. A package can be -uninstalled and can contain many namespaces. A namespace can be imported in code. +uninstalled and can contain many namespaces. A namespace can be imported + within the Python shell. _Example:_ ```console @@ -56,16 +64,18 @@ Namespaces: - city ``` -### Pico uninstalls -_Usage:_ `pico uninstall |all`. -Note that to select all the packages, `all` must be quoted. +#### pico uninstalls +_Usage:_ +- `pico uninstall ` +- `pico uninstall all` _Behaviour:_ -- All installed packages / namespaces are uninstalled. -- All namespaces except the uninstalled ones are re-installed. +- The specified packages are uninstalled. +- All packages except the uninstalled ones are re-installed. _Example:_ ```console +(venv) user@PC:~$ pico uninstall city INFO [osp.core.ontology.installation]: Will install the following namespaces: ['qe'] INFO [osp.core.ontology.yml.yml_parser]: Parsing YAML ontology file /home//.osp_ontologies/qe.yml INFO [osp.core.ontology.yml.yml_parser]: You can now use `from osp.core.namespaces import qe`. @@ -73,17 +83,7 @@ INFO [osp.core.ontology.parser]: Loaded 205 ontology triples in total INFO [osp.core.ontology.installation]: Uninstallation successful ``` -### Ontology installation folder - -The installed ontologies are stored in the directory `~/.osp-ontologies` by -default. On Windows, `~` usually refers to the path -`C:\Users\`. - -The installation directory can be changed by setting the -environment variable `OSP_ONTOLOGIES_DIR`. Such action would move it to -`$OSP_ONTOLOGIES_DIR/.osp-ontologies`. - -### Conflicts with other "pico" installations +#### Conflicts with other "pico" installations Some Operating Systems might have a pre-existing tool called _pico_. In most cases, the previous commands should work, but if any problem arises, you can use the following alternative: @@ -97,6 +97,54 @@ For example: python -m osp.core.pico install city ``` +### Using pico as a Python module + +`pico` can also be used within the Python shell. In particular, four +functions are available to be imported from the `osp.core.pico` module, + +```python +from osp.core.pico import install, namespaces, packages, uninstall +``` + +that cover the three main operations that pico is meant to perform: installing +ontologies (`install`), uninstalling ontologies (`uninstall`), and listing the +installed ontologies (`packages`, `namespaces`). + +Each function is used in a similar way to its command-line counterpart. + +- `install`: accepts _one or more_ positional arguments of string +type, which can be either paths to `yml` ontology installation files or +names of ontologies that can be installed via this shortcut. It is meant to +clone the +[behavior of its command-line counterpart](https://simphony.readthedocs.io/en/latest/utils.html#pico-installs). + +- `uninstall`: accepts _one or more_ positional arguments of string type, + which must be names of already installed ontology packages. It also + clones the [behavior of its command-line counterpart](https://simphony.readthedocs.io/en/latest/utils.html#pico-uninstalls). + +- `packages`: accepts no arguments and returns an [iterator](https://wiki.python.org/moin/Iterator) + over the names of the installed packages. + +- `namespaces`: accepts no arguments and returns an iterator yielding one + [`OntologyNamespace` object](https://simphony.readthedocs.io/en/latest/api_ref.html#osp.core.ontology.namespace.OntologyNamespace) for each installed namespace. + +Usage examples: + +- `install('city', 'path/to/ontology_yml_file.yml')`, `install('foaf', 'dcat2')` +- `uninstall('city', 'foaf')` +- `print(list(packages()))` +- `print(list(namespaces()))` + +### Ontology installation folder + +The installed ontologies are stored in the directory `~/.osp-ontologies` by +default. On Windows, `~` usually refers to the path +`C:\Users\`. + +The installation directory can be changed by setting the +environment variable `OSP_ONTOLOGIES_DIR`. Such action would move it to +`$OSP_ONTOLOGIES_DIR/.osp-ontologies`. + ## Tips and tricks The following are some utility functions and shortcuts for working with cuds. For those that are present in the util package, the import is `from osp.core import utils`. diff --git a/requirements.txt b/requirements.txt index 780f246..ac74d52 100644 --- a/requirements.txt +++ b/requirements.txt @@ -11,4 +11,3 @@ sphinx-autobuild==2021.3.14 sphinx-panels==0.5.2 jinja2==2.11.3 markupsafe==2.0.1 - From db01dacbf7c706508f18da2dfb133c9a4b52423d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= <43052541+kysrpex@users.noreply.github.com> Date: Thu, 7 Apr 2022 09:47:28 +0200 Subject: [PATCH 3/3] Code formatting with black and isort: documentation (#202) --- docs/source/contribute.md | 118 +++++++++++++++++++++++++++----------- 1 file changed, 84 insertions(+), 34 deletions(-) diff --git a/docs/source/contribute.md b/docs/source/contribute.md index 129b197..af31c46 100644 --- a/docs/source/contribute.md +++ b/docs/source/contribute.md @@ -6,25 +6,31 @@ in order to help those that want to contribute to SimPhoNy. ### Tools The following are some of the technologies and concepts we use regularly. It might be useful to become familiar with them: - - Version control: Git, GitHub and GitLab - - Python virtual environments/conda - - Docker - - Unittesting - - Benchmarks + - Version control: [Git](https://git-scm.com/), + [GitHub](https://github.com/about) and + [GitLab](https://about.gitlab.com/) + - [Unittest](https://docs.python.org/3/library/unittest.html) - Continuous integration + - Python virtual environments/[conda](https://docs.conda.io) + - [Docker](https://www.docker.com/resources/what-container/) + - Benchmarks ### Code Organisation There are 3 main categories of repos: - - [_OSP-core_](https://github.com/simphony/osp-core) contains the nucleus of SimPhoNy, the base on which the wrappers build. + - [_OSP-core_](https://github.com/simphony/osp-core) contains the nucleus of + SimPhoNy, the base on which the wrappers build. - Each _wrapper_ will be in its own repository on GitHub or GitLab, - mimicking [wrapper_development](https://github.com/simphony/wrapper-development). - - [_docs_](https://github.com/simphony/docs) holds the source for this documentation. + mimicking + [wrapper_development](https://github.com/simphony/wrapper-development). + - [_docs_](https://github.com/simphony/docs) + holds the source for this documentation. There are also 4 types of branches: - `master/main` contains all the releases, and should always be stable. - `dev` holds the code for the newest release that is being developed. - `issue branch` is where an specific issue is being solved. - - `hotfix branch` is where a critical software bug detected on the stable release (more on this later) is being solved. + - `hotfix branch` is where a critical software bug detected on the stable + release (more on this later) is being solved. All wrappers and OSP-core are part of a common directory structure: - _`osp/`_: contains all the SimPhoNy source code. @@ -38,10 +44,12 @@ All wrappers and OSP-core are part of a common directory structure: - Every new feature or bug is defined in an issue and labelled accordingly. If there is something that is missing or needs improving, make an issue in the appropriate project. -- Generally, the issues are fixed by creating a new `issue branch` from the `dev` branch, committing to that branch and making a new Pull/Merge Request when done. - An owner of the project should be tagged for review. - They will review and merge the PR if the fix is correct, deleting the `issue branch` afterwards. - The changes should be clearly explained in the issue/Pull Request. +- Generally, the issues are fixed by creating a new `issue branch` from the + `dev` branch, committing to that branch and making a new Pull/Merge + Request when done. An owner of the project should be tagged for review. + They will review and merge the PR if the fix is correct, deleting the + `issue branch` afterwards. The changes should be clearly explained in the + issue/Pull Request. ```{warning} If the issue is a critical software bug detected in the stable release, a `hotfix branch` should be created from the `master/main` branch @@ -54,15 +62,17 @@ All wrappers and OSP-core are part of a common directory structure: `hotfix branch`. ``` - Once the features for a release have been reached, `dev` will be merged to - `master/main`. Every new commit in the `master/main` branch generally corresponds - to a new release, which is labelled with a + `master/main`. Every new commit in the `master/main` branch generally + corresponds to a new release, which is labelled with a [git tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) matching its - version number. An exception to this rule may apply, for example when several - critical hotfixes are applied in a row, as it would then be better to just to - publish a single release afterwards. In regard to version numbering, we adhere to the + version number. An exception to this rule may apply, for example when + several critical hotfixes are applied in a row, as it would then be + better to just to publish a single release afterwards. In regard to + version numbering, we adhere to the [_Semantic versioning_](https://semver.org/) rules. -In the next image it can be seen how the branches usually look during this workflow, and the different commits used to synchronise them: +In the next image it can be seen how the branches usually look during this +workflow, and the different commits used to synchronise them:
@@ -73,32 +83,71 @@ In the next image it can be seen how the branches usually look during this workf ## Coding ### Documenting - All code must be properly documented with meaningful comments. -- For readability, we now follow the [Google docstring format](https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings). +- For readability, we now follow the + [Google docstring format](https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings). - If some behaviour is very complex, in-line comments can be used. However, proper naming and clear operations are always preferred. +### Code style +- Code should follow + [PEP8 code style conventions](https://peps.python.org/pep-0008/). +- All Python code should be validated by the + [Flake8](https://github.com/pycqa/flake8) tool. The validation is also + enforced on the repository by the + [continuous integration](contribute.md#continuous-integration). Click + [here](https://github.com/simphony/osp-core/blob/master/.github/workflows/ci.yml#L12) + to see the specific options with which Flake8 is launched. +- All Python code should be reformatted with + [black](https://github.com/psf/black) and + [isort](https://github.com/PyCQA/isort). The use of said tools is + enforced by the + [continuous integration](contribute.md#continuous-integration). Therefore, + we strongly recommend that you use the + [configuration file](https://github.com/simphony/osp-core/blob/master/.pre-commit-config.yaml) + bundled with the repository to + [install](https://pre-commit.com/#installation) the + [pre-commit framework](https://pre-commit.com/), that automates the task + using git pre-commit hooks. +- A few + [other style conventions](https://github.com/simphony/osp-core/blob/master/.pre-commit-config.yaml) + are also enforced by the continuous integration through + [pre-commit](https://pre-commit.com/) (such as empty lines at the end of + text files). If you decide not to use it, the CI will let you know what + to correct. + ### Testing -- All Python code should pass Flake8 tests. - All complex functionality must be tested. -- If some implementation can not be checked through unittest, it should be at least manually run in different systems to assure an expected behaviour. +- If some implementation can not be checked through unittest, it should be + at least manually run in different systems to assure an expected behaviour. ### Continuous Integration - We currently run the CI through Github Actions/GitLab CI. -- Flake8 and unittests are automatically run for all PR. -- For the OSP-core code, benchmarks are run after every merge to `dev`. Benchmark results are - available [here](https://simphony.github.io/osp-core/dev/bench/index.html). - The CI will report a failure when a benchmark is 50% slower than in the previous run, in addition to automatically commenting on the commit. +- Code style conventions are enforced through the use of Flake8, black, isort, + and various + [pre-commit](https://pre-commit.com/) + [hooks](https://github.com/simphony/osp-core/blob/master/.pre-commit-config.yaml). +- Tests are automatically run for all pull requests. +- For the OSP-core code, benchmarks are run after every merge to `dev`. + Benchmark results are available + [here](https://simphony.github.io/osp-core/dev/bench/index.html). The CI + will report a failure when a benchmark is 50% slower than in the previous + run, in addition to automatically commenting on the commit. ### Naming conventions -- Use `cuds_object` as the argument name of your methods (not `entity`, `cuds`, `cuds_instance`...). -- The official spelling is `OSP-core` (as opposed to _osp core_, _OSP-Core_ or similar). +- Use `cuds_object` as the argument name of your methods (not `entity`, + `cuds`, `cuds_instance`...). +- The official spelling is `OSP-core` (as opposed to _osp core_, _OSP-Core_ + or similar). ## Contribute to OSP-core -If you are not a member of the [SimPhoNy organisation](https://github.com/simphony), rather than creating a branch -from `dev`, you will have to fork the repository and create the Pull Request. +If you are not a member of the +[SimPhoNy organisation](https://github.com/simphony), rather than creating +a branch from `dev`, you will have to fork the repository and create the +Pull Request. ## Contribute to wrapper development -For a sample wrapper, visit the [wrapper_development](https://github.com/simphony/wrapper-development) repo. +For a sample wrapper, visit the +[wrapper_development](https://github.com/simphony/wrapper-development) repo. README files should include: - Information regarding the purpose of the wrapper and the backend used. @@ -108,6 +157,7 @@ README files should include: - Any other necessary information for users and other developers. ## Contribute to the docs -If you have any suggestion for this documentation, whether it is something that needs more explanation, is inaccurate or simply a note on anything that could be improved, you can open an issue [here](https://github.com/simphony/docs/issues), and we will look into it!. - - +If you have any suggestion for this documentation, whether it is something +that needs more explanation, is inaccurate or simply a note on anything +that could be improved, you can open an issue +[here](https://github.com/simphony/docs/issues), and we will look into it!.