diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 000000000..e69de29bb diff --git a/404.html b/404.html new file mode 100644 index 000000000..3d7d7d958 --- /dev/null +++ b/404.html @@ -0,0 +1,758 @@ + + + + + + + + + + + + + + + + + + CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ +

404 - Not found

+ +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/Adding_classes_from_another_ontology/index.html b/Adding_classes_from_another_ontology/index.html new file mode 100644 index 000000000..c0208635d --- /dev/null +++ b/Adding_classes_from_another_ontology/index.html @@ -0,0 +1,858 @@ + + + + + + + + + + + + + + + + + + + + + + Importing classes from another ontology - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

How to add (import) classes to the Cell Ontology (CL) from another ontology

+

1. Follow steps 1 - 5 under the heading Protege-based declaration.

+

NB: Even though the instructions state that this workflow is to be avoided, the other solutions in the current documentation are out of date.

+

2. Refresh the imports

+

To refresh the imports, open Docker so it is running in the background. Then open Terminal, navigate to src/ontology directory in the cell-ontology repository and run:

+

sh run.sh make imports/merged_import.owl

+

Running the above command requires > 8GB RAM and sufficient computational power. If the refresh fails to complete due to hardware limitations, create a new issue in GitHub detailing which class(es) need to be imported and another editor can add it on your behalf.

+

Once the imports are refreshed, return to Protégé, add the logical axioms that include the newly imported class(es) and create a pull request per standard procedure.

+

Note that the import refresh process seems to be quite laborious/computationally expensive as-is, and a centralised database approach may be an improved longterm solution.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/CL-logo.jpeg b/CL-logo.jpeg new file mode 100644 index 000000000..854c2fdab Binary files /dev/null and b/CL-logo.jpeg differ diff --git a/Create_upper_level_slim/index.html b/Create_upper_level_slim/index.html new file mode 100644 index 000000000..806b35390 --- /dev/null +++ b/Create_upper_level_slim/index.html @@ -0,0 +1,982 @@ + + + + + + + + + + + + + + + + + + SOP for adding a new slim: - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

SOP for adding a new slim:

+

Intro

+

An upper slim is a set of terms for summarising annotations. CL has both a general slim and domain-specific slims allowing for the generation of general summaries or for domain specific ones. All slims have a context that covers all cells in the domain of interest. In order to fullfil the summarising use-case, a slim should have good % coverage of the domain and, if possible, avoid excluding major cell types. The following are potentially a problem for the grouping use-case:

+
    +
  • Classes in the slim with very small numbers of subclasses: Having 0-1 subclass => no capacity to summarise.
    • +
  • Classes with disproportionately large numbers of subclasses.
    • +
  • Overlapping classes - these clash with some types of summary - e.g. pie charts.
    • +
+

Potential clashing concerns: It seems reasonable to want to make sure that very important cell types are covered and are not obscured in generating summaries, but this desire can clash with the considerations above. Some judgement is needed to balance these concerns.

+

There is no perfect solution, but some solutions are better (at fulfilling the use case) than others.

+

Name and definition

+

It is important that the name of the slim accurately reflects the content it covers. Specifically, if the intention is to cover all cell types that are specific for an organ, the slim's name should be in the format of "organ_upper_slim". This ensures clarity and helps users understand the scope of the slim.

+

Furthermore, the description of the slim should follow a consistent pattern. It is recommended to use the following structure: "a subset of general classes related to specific cell types in the [organ or specific context]". This format provides a concise and informative description of the slim, helping users identify its purpose and content.

+

What files to create and edit

+

1. Preparing the subset:

+
    +
  • Create XXX_upper_slim in Protege (change "XXX" to the subset label). See Adding a new Subset.
    • +
  • Create a CSV table with the following characteristics (Find examples in src/templates):
    • +
  • 3 columns
    • +
+ + + + + + + + + + + + + + + + + + + + + + + + + +
IDsubsetlabel
IDAI oboInOwl:inSubset
CL:#######http://purl.obolibrary.org/obo/cl#XXX_upper_slimCL term
...http://purl.obolibrary.org/obo/cl#XXX_upper_slim...
+
    +
  • +

    Save the CSV file with the name 'XXX_upper_slim.csv' in the src/templates directory.

    +
    • +
  • +

    Modify src/ontology/cl-odk.yaml introducing new lines for the new slim (change "XXX" to the subset label):

    +

    - id: XXX_upper_slim

    +

    image

    +
    • +
+

- filename: XXX_upper_slim.owl + use_template: True + templates: + - XXX_upper_slim.csv

+
![image](https://github.com/obophenotype/cell-ontology/assets/94959119/254ad25f-7bf2-4ac2-afe2-9ad067d9c1ea)
+
+

2. Generating the Slim OWL file:

+
    +
  • Navigate to the src/ontology file in the terminal. Make sure Docker is running.
    • +
  • Run the command:
    • +
+

sh run.sh make update_repo

+

3. Modifying the Catalog:

+
    +
  • Open the src/ontology/catalog-v001.xml file.
    • +
  • Add the following line (change "XXX" to the subset label):
    • +
+

<uri name="http://purl.obolibrary.org/obo/cl/components/XXX_upper_slim.owl" uri="components/XXX_upper_slim.owl"/>` + - image

+

4. Preparing the Upper Level Slim import to CL:

+
    +
  • Open src/ontology/cl-edit.owl.
    • +
  • +

    Add the following import statement (change "XXX" to the subset label):

    +

    Import(<http://purl.obolibrary.org/obo/cl/components/XXX_upper_slim.owl>) +image

    +
    • +
+

5. Updating the slim owl file:

+
    +
  • +

    Run the command:

    +

    sh run.sh make all_subsets -B + - Alternatively, run the following command to run it anyway even if it says it is up to date (change "XXX" to the subset label):

    +

    sh run.sh make components/XXX_upper_slim.owl -B

    +
    • +
+

6. Testing Slim Coverage:

+
    +
  • Open src/ontology/cl.Makefile.
    • +
  • +

    Add the subset label to SLIM_TEMPLATES (without _upper_slim!!!).

    +

    image

    +
    • +
  • +

    Add the term that will be used to test coverage

    +

    image

    +
    • +
  • +

    Add:

    +
    • +
+

$(REPORTDIR)/XXX_upper_slim.csv: $(TEMPLATEDIR)/XXX_upper_slim.csv + $(eval TERM_ID := $(YYY)) + $(COVERAGECMD) + (substitute "XXX" to the subset label and YYY for the tested label)

+
 ![image](https://github.com/obophenotype/cell-ontology/assets/94959119/7eb18255-0ef7-4fbc-9f7f-e582372165bf)
+
+
    +
  • Using the terminal, navigate to src/ontology.
    • +
  • +

    Run the command:

    +

    sh run.sh make slim_coverage

    +
    • +
+

Understanding reports

+

The reports can be accessed at src/ontology/reports/XXX_upper_slim.csv. First, the coverage percentage is displayed, indicating the proportion of cells covered. Secondly, the number of cells covered by each term of the subset is provided. Finally, a list is presented, indicating all the terms that were expected to be covered but are not currently included.

+

Ideally, terms would have more than 1 cell covered. Furthermore, a term covering hundreds of cells might indicate that it is too general, and a more specific term (or multiple) should be evaluated, specially if it overlaps with other terms of the subset. Example: For the eye_upper_slim, 'retinal cell' is a (too) general term that overlaps other grouping terms such as 'retinal bipolar neuron', 'retina horizontal cell' or 'amacrine cell'.

+

In the case that there is overlapping of terms (term A in the subset covers term B of the subset), a coverage file will be created and it can be accessed at src/ontology/reports/overlapping_terms_XXX_upper_slim.csv.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/Fixing_xsdstring_diffs/index.html b/Fixing_xsdstring_diffs/index.html new file mode 100644 index 000000000..a2b7e2950 --- /dev/null +++ b/Fixing_xsdstring_diffs/index.html @@ -0,0 +1,855 @@ + + + + + + + + + + + + + + + + + + + + + + Fixing unintended ^^xsd:string diffs - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Fixing ^^xsd:string Diffs

+

When you make edits, sometimes there will be large amounts of unintended differences that show up that involves the removal of ^^xsd:string. If so, you can resolve them by following normalising your cl-edit.owl file.

+

SOP

+
    +
  1. +

    Update your file from Master (see 'How to resolve merge conflicts' for instructions on how to do this including how to resolve clashes while doing this).

    +
    2. +
  2. +

    in the terminal, set directory to the ontology folder in CL: cd .../GitHub/cell-ontology/src/ontology

    +
    3. +
  3. +

    Run the normaliser in terminal:

    +
    4. +
+

If you have docker installed: sh run.sh make normalise_xsd_string

+

If you do not have docker installed: make normalise_xsd_string

+

If make is not installed, on MAC:

+
sed -i '' -E "s/Annotation[(](oboInOwl[:]hasDbXref [\"][^\"]*[\"])[)]/Annotation(\1^^xsd:string)/g" cl-edit.owl
+
+

This should resolve your ^^xsd:string issue, after which, you can handle your pull request as per usual.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/Keeping_ontology_terms_up_to_date/index.html b/Keeping_ontology_terms_up_to_date/index.html new file mode 100644 index 000000000..1aaff566a --- /dev/null +++ b/Keeping_ontology_terms_up_to_date/index.html @@ -0,0 +1,914 @@ + + + + + + + + + + + + + + + + + + + + + + Keep terms up-to-date - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Keep terms up-to-date

+ +

Keeping cell ontology annotation up to date

+

Cell ontology identifiers (IRIs) are never lost, but they are occasionally deprecated. On the rare occasions that this happens, all logical links to other ontology terms (e.g. recording classification or partonomy) are removed and term is tagged with the annotation owl:deprecated True. To aid migration of annotations to the latest standard, these terms are also annotated with either a term_replaced_by or a consider tag. A term_replaced_by annotation is used to record the ID of a term it is safe to auto-migrate annotations to. More rarely, consider is used to record multiple potential replacement terms requiring human consideration to map. In these cases, a comment will be present to provide guidance for mapping.

+

The Ontology Lookup Service API provides a convenient way to check for deprecated terms & find replacements.

+

The term http://purl.obolibrary.org/obo/CL_0000375 has been deprecated and has that tag term_replaced_by

+

Querying the OLS API for this:

+

https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375. (Note the query IRI must be double encoded)

+

Returns:

+
{
+  "iri" : "http://purl.obolibrary.org/obo/CL_0000375",
+  "label" : "obsolete osteoprogenitor cell",
+  "description" : null,
+  "annotation" : {
+    "database_cross_reference" : [ "BTO:0002051" ],
+    "has_obo_namespace" : [ "cell" ],
+    "term replaced by" : [ "CL:0007010" ]
+  },
+  "synonyms" : null,
+  "ontology_name" : "cl",
+  "ontology_prefix" : "CL",
+  "ontology_iri" : "http://purl.obolibrary.org/obo/cl.owl",
+  "is_obsolete" : true,
+  "term_replaced_by" : "CL:0007010",
+  "is_defining_ontology" : true,
+  "has_children" : false,
+  "is_root" : true,
+  "short_form" : "CL_0000375",
+  "obo_id" : "CL:0000375",
+  "in_subset" : null,
+  "obo_definition_citation" : null,
+  "obo_xref" : [{"database":"BTO","id":"0002051","description":null,"url":"http://purl.obolibrary.org/obo/BTO_0002051"}],
+  "obo_synonym" : null,
+  "is_preferred_root" : false,
+  "_links" : {
+    "self" : {
+      "href" : "https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375"
+    },
+    "graph" : {
+      "href" : "https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375/graph"
+    }
+  }
+}
+
+

The term_replaced_by key points to the ID of a safe replacement term: CL:0007010. This is a CURIE for http://purl.obolibrary.org/obo/CL_0007010 *

+

consider

+

The term http://purl.obolibrary.org/obo/CL_0000144 has been deprecated and has a consider tag pointing to multiple possible replacement terms, along with a comment for guidance.

+

Querying the OLS API for this: https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144

+

Returns

+
{
+  "iri" : "http://purl.obolibrary.org/obo/CL_0000144",
+  "label" : "obsolete cell by function",
+  "description" : [ "OBSOLETE: A classification of cells by their primary end goal or behavior." ],
+  "annotation" : {
+    "comment" : [ "This term was made obsolete because there is no difference in meaning between it and 'cell', as any cell can be classified by its function or behavior. If you have used this term in annotation, please replace it with cell (CL:0000000), native cell (CL:0000003), or cell in vitro (CL:0001034) as appropriate." ],
+    "consider" : [ "CL:0001034", "CL:0000000", "CL:0000003" ],
+    "has_obo_namespace" : [ "cell" ]
+  },
+  "synonyms" : null,
+  "ontology_name" : "cl",
+  "ontology_prefix" : "CL",
+  "ontology_iri" : "http://purl.obolibrary.org/obo/cl.owl",
+  "is_obsolete" : true,
+  "term_replaced_by" : null,
+  "is_defining_ontology" : true,
+  "has_children" : false,
+  "is_root" : true,
+  "short_form" : "CL_0000144",
+  "obo_id" : "CL:0000144",
+  "in_subset" : null,
+  "obo_definition_citation" : [{"definition":"OBSOLETE: A classification of cells by their primary end goal or behavior.","oboXrefs":[{"database":"FB","id":"ma","description":null,"url":"http://flybase.org/reports/ma.html"}]}],
+  "obo_xref" : null,
+  "obo_synonym" : null,
+  "is_preferred_root" : false,
+  "_links" : {
+    "self" : {
+      "href" : "https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144"
+    },
+    "graph" : {
+      "href" : "https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144/graph"
+    }
+  }
+}
+
+

* Warning - due to legacy issues, the values of these tags are either a curie (CL:0000123) or short_form ID (CL_0000123) rather than an iri. Handling code needs to deal with both of these formats.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/annotation_properties/index.html b/annotation_properties/index.html new file mode 100644 index 000000000..d029bf542 --- /dev/null +++ b/annotation_properties/index.html @@ -0,0 +1,1086 @@ + + + + + + + + + + + + + + + + + + + + + + CL annotation properties - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + Skip to content + + +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Annotation Properties

+

Note- this page is currently under development.

+

The Cell Ontology has the following annotation properties:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Annotation propertyDescriptionExample termExample annotationMust have?Only one use per term is allowed?
considerTo be used on obsoleted classes, to point to a term that should be considered by curators for use in place of the obsoleted term. Multiple consider terms are allowed. It can be useful to combine this with a comment to indicate when replacement would be appropriate.CL:0000610 obsolete plant cellPO:0009002NoNo
created_byAdded automatically on term creation with standard Protege settings. Ideally, this should use the "supplied user name" in the Protege User Details preference pane. This has been inconsistently applied in the past.CL:0002518tmeehanShouldYes
creation_dateAdded automatically on term creation with standard Protege settings.CL:00025182011-02-08T10:46:34ZShouldYes
database_cross_referenceCitable references that have helped generate the term and term's definition. Includes PubMed IDs (in the format PMID:XXXXXXXX).CL:0011005 GABAergic interneuronPMID:29724907ShouldNo
dc:contributorUse this to annotate a whole ontology file with the identifier of a contributor. ORCID preferred.N/Ahttps://orcid.org/0000-0001-9990-8331Nice to have, if applicableNo
dc:creatorComing SoonCL:0001201 B cell, CD19-positivehttps://orcid.org/0000-0001-9990-8331NoYes
dc:dateComing soonCL:0001065 innate lymphoid cell2017-01-30T20:20:48ZNoYes
dc:descriptionUse this to annotate a whole ontology file with a brief description of the ontology.N/AAn ontology of cell types.NoNo
dc:titleUse this to annotate an ontology, giving it a human readable title.N/ACell OntologyNoNo
dcterms:licenseUse to attach a license to the whole ontology file.N/Ahttp://creativecommons.org/licenses/by/4.0/NoNo
definitionThe textual definition for the ontology class.CL:0000946 antibody secreting cellA lymphocyte of B lineage that is devoted to secreting large amounts of immunoglobulin.MustYes
'expand expression to'Coming soonComing soonComing soonNoNo
foaf:depicted_byUse this to add a link to an image that depicts an example of an entity referred to by the termComing soonComing soonNoNo
has_alternative_idIn CL this is a legacy property. Do not use.CL:0000059 ameloblastCL:0000053NoNo
has_broad_synonymUsed for synonyms where the primary definition accurately describes the synonym, but the definition of the synonym may encompass other structures as well. In some cases where a broad synonym is given, it will be a broad synonym for more than one ontology term. You are encouraged to add a reference that uses the term in this way.CL:0000365 animal zygotezygoteNoNo
has_exact_synonymUsed for synonyms where the definition of the synonym is exactly the same as primary term definition. This is used when the same class can have more than one name. You are encouraged to add a reference that uses the term in this way.CL:0000622 acinar cellacinic cellNice to have, if applicableNo
has_narrow_synonymUsed for synonyms where the definition of the synonym is the same as the primary definition, but has additional qualifiers. You are encouraged to add a reference that uses the term in this way.CL:0000362 epidermal cellepithelial cell of skinNoNo
has_obo_namespaceThis is a legacy annotation property. Do not add this manually.CL:0001061 abnormal cellcellNoNo
has_related_synonymThis scope is applied when a word of phrase has been used synonymously with the primary term name in the literature, but the usage is not strictly correct. That is, the synonym in fact has a slightly different meaning than the primary term name. Since users may not be aware that the synonym was being used incorrectly when searching for a term, related synonyms are included.CL:0000902 induced T-regulatory celladaptive TregNoNo
has_synonym_typeThe target of this relation must be an annotation property of type 'synonym_type_property'.N/AN/ANoNo
IAO_0000116Coming soonComing soonComing soonNoNo
idAutomatically added by some pathways. Do not add manually. If duplicating a term (with the duplicate getting a new ID), it should be deleted.CL:2000074 splenocyteCL:2000074YesYes
in_subsetUsed to add subset tags, used in conjunction with subset_propertyCL:0000039 germ line cell_upper_levelNoNo
is_inferredThis annotation property is used in some automated pipelines. Do not add manuallyComing soonComing soonNoNo
rdfs:commentUse to add a clarifying comment to a term. This can be useful for adding examples and for clarifying terminological confusions.CL:0007016 adaxial cellIn teleosts, adaxial cells give rise to slow muscle myoblasts.NoYes
rdfs:isDefinedByDo not add manually.Coming soonComing soonNoYes
rdfs:labelPrimary name - used as a display name by Protege (with standard settings) and most downstream consumers. Add only one of these. It must be unique within an ontology.CL:0000418 arcade cellarcade cellMustYes
RO_0002161Coming soonComing soonNoNo
'see also'Used to link to a webpage, such as a GitHub ticket.CL:0000134 mesenchymal stem cellhttps://github.com/obophenotype/cell-ontology/issues/474NoNo
shorthandAdded automatically by some pipelines. Do not add manuallyComing soonComing soonNoNo
subset_propertyA grouping class for subset tags.N/AN/ANoNo
subset_property: added_for_HCAA subset tag for terms that were requested by the Human Cell Atlas.N/AN/ANoNo
subset property: location_groupingA subset tag for cell types from a particular anatomical location.N/AN/ANoNo
synonym_type_propertyA grouping class for synonym tags.N/AN/ANoNo
'term replaced by'To be used on obsolete terms to indicate a term that can be automatically substituted for the obsoleted term.Coming soonComing soonNoNo
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/assets/images/favicon.png b/assets/images/favicon.png new file mode 100644 index 000000000..1cf13b9f9 Binary files /dev/null and b/assets/images/favicon.png differ diff --git a/assets/javascripts/bundle.51d95adb.min.js b/assets/javascripts/bundle.51d95adb.min.js new file mode 100644 index 000000000..b20ec6835 --- /dev/null +++ b/assets/javascripts/bundle.51d95adb.min.js @@ -0,0 +1,29 @@ +"use strict";(()=>{var Hi=Object.create;var xr=Object.defineProperty;var Pi=Object.getOwnPropertyDescriptor;var $i=Object.getOwnPropertyNames,kt=Object.getOwnPropertySymbols,Ii=Object.getPrototypeOf,Er=Object.prototype.hasOwnProperty,an=Object.prototype.propertyIsEnumerable;var on=(e,t,r)=>t in e?xr(e,t,{enumerable:!0,configurable:!0,writable:!0,value:r}):e[t]=r,P=(e,t)=>{for(var r in t||(t={}))Er.call(t,r)&&on(e,r,t[r]);if(kt)for(var r of kt(t))an.call(t,r)&&on(e,r,t[r]);return e};var sn=(e,t)=>{var r={};for(var n in e)Er.call(e,n)&&t.indexOf(n)<0&&(r[n]=e[n]);if(e!=null&&kt)for(var n of kt(e))t.indexOf(n)<0&&an.call(e,n)&&(r[n]=e[n]);return r};var Ht=(e,t)=>()=>(t||e((t={exports:{}}).exports,t),t.exports);var Fi=(e,t,r,n)=>{if(t&&typeof t=="object"||typeof t=="function")for(let o of $i(t))!Er.call(e,o)&&o!==r&&xr(e,o,{get:()=>t[o],enumerable:!(n=Pi(t,o))||n.enumerable});return e};var yt=(e,t,r)=>(r=e!=null?Hi(Ii(e)):{},Fi(t||!e||!e.__esModule?xr(r,"default",{value:e,enumerable:!0}):r,e));var fn=Ht((wr,cn)=>{(function(e,t){typeof wr=="object"&&typeof cn!="undefined"?t():typeof define=="function"&&define.amd?define(t):t()})(wr,function(){"use strict";function e(r){var n=!0,o=!1,i=null,a={text:!0,search:!0,url:!0,tel:!0,email:!0,password:!0,number:!0,date:!0,month:!0,week:!0,time:!0,datetime:!0,"datetime-local":!0};function s(T){return!!(T&&T!==document&&T.nodeName!=="HTML"&&T.nodeName!=="BODY"&&"classList"in T&&"contains"in T.classList)}function f(T){var Ke=T.type,We=T.tagName;return!!(We==="INPUT"&&a[Ke]&&!T.readOnly||We==="TEXTAREA"&&!T.readOnly||T.isContentEditable)}function c(T){T.classList.contains("focus-visible")||(T.classList.add("focus-visible"),T.setAttribute("data-focus-visible-added",""))}function u(T){T.hasAttribute("data-focus-visible-added")&&(T.classList.remove("focus-visible"),T.removeAttribute("data-focus-visible-added"))}function p(T){T.metaKey||T.altKey||T.ctrlKey||(s(r.activeElement)&&c(r.activeElement),n=!0)}function m(T){n=!1}function d(T){s(T.target)&&(n||f(T.target))&&c(T.target)}function h(T){s(T.target)&&(T.target.classList.contains("focus-visible")||T.target.hasAttribute("data-focus-visible-added"))&&(o=!0,window.clearTimeout(i),i=window.setTimeout(function(){o=!1},100),u(T.target))}function v(T){document.visibilityState==="hidden"&&(o&&(n=!0),B())}function B(){document.addEventListener("mousemove",z),document.addEventListener("mousedown",z),document.addEventListener("mouseup",z),document.addEventListener("pointermove",z),document.addEventListener("pointerdown",z),document.addEventListener("pointerup",z),document.addEventListener("touchmove",z),document.addEventListener("touchstart",z),document.addEventListener("touchend",z)}function re(){document.removeEventListener("mousemove",z),document.removeEventListener("mousedown",z),document.removeEventListener("mouseup",z),document.removeEventListener("pointermove",z),document.removeEventListener("pointerdown",z),document.removeEventListener("pointerup",z),document.removeEventListener("touchmove",z),document.removeEventListener("touchstart",z),document.removeEventListener("touchend",z)}function z(T){T.target.nodeName&&T.target.nodeName.toLowerCase()==="html"||(n=!1,re())}document.addEventListener("keydown",p,!0),document.addEventListener("mousedown",m,!0),document.addEventListener("pointerdown",m,!0),document.addEventListener("touchstart",m,!0),document.addEventListener("visibilitychange",v,!0),B(),r.addEventListener("focus",d,!0),r.addEventListener("blur",h,!0),r.nodeType===Node.DOCUMENT_FRAGMENT_NODE&&r.host?r.host.setAttribute("data-js-focus-visible",""):r.nodeType===Node.DOCUMENT_NODE&&(document.documentElement.classList.add("js-focus-visible"),document.documentElement.setAttribute("data-js-focus-visible",""))}if(typeof window!="undefined"&&typeof document!="undefined"){window.applyFocusVisiblePolyfill=e;var t;try{t=new CustomEvent("focus-visible-polyfill-ready")}catch(r){t=document.createEvent("CustomEvent"),t.initCustomEvent("focus-visible-polyfill-ready",!1,!1,{})}window.dispatchEvent(t)}typeof document!="undefined"&&e(document)})});var un=Ht(Sr=>{(function(e){var t=function(){try{return!!Symbol.iterator}catch(c){return!1}},r=t(),n=function(c){var u={next:function(){var p=c.shift();return{done:p===void 0,value:p}}};return r&&(u[Symbol.iterator]=function(){return u}),u},o=function(c){return encodeURIComponent(c).replace(/%20/g,"+")},i=function(c){return decodeURIComponent(String(c).replace(/\+/g," "))},a=function(){var c=function(p){Object.defineProperty(this,"_entries",{writable:!0,value:{}});var m=typeof p;if(m!=="undefined")if(m==="string")p!==""&&this._fromString(p);else if(p instanceof c){var d=this;p.forEach(function(re,z){d.append(z,re)})}else if(p!==null&&m==="object")if(Object.prototype.toString.call(p)==="[object Array]")for(var h=0;hd[0]?1:0}),c._entries&&(c._entries={});for(var p=0;p1?i(d[1]):"")}})})(typeof global!="undefined"?global:typeof window!="undefined"?window:typeof self!="undefined"?self:Sr);(function(e){var t=function(){try{var o=new e.URL("b","http://a");return o.pathname="c d",o.href==="http://a/c%20d"&&o.searchParams}catch(i){return!1}},r=function(){var o=e.URL,i=function(f,c){typeof f!="string"&&(f=String(f)),c&&typeof c!="string"&&(c=String(c));var u=document,p;if(c&&(e.location===void 0||c!==e.location.href)){c=c.toLowerCase(),u=document.implementation.createHTMLDocument(""),p=u.createElement("base"),p.href=c,u.head.appendChild(p);try{if(p.href.indexOf(c)!==0)throw new Error(p.href)}catch(T){throw new Error("URL unable to set base "+c+" due to "+T)}}var m=u.createElement("a");m.href=f,p&&(u.body.appendChild(m),m.href=m.href);var d=u.createElement("input");if(d.type="url",d.value=f,m.protocol===":"||!/:/.test(m.href)||!d.checkValidity()&&!c)throw new TypeError("Invalid URL");Object.defineProperty(this,"_anchorElement",{value:m});var h=new e.URLSearchParams(this.search),v=!0,B=!0,re=this;["append","delete","set"].forEach(function(T){var Ke=h[T];h[T]=function(){Ke.apply(h,arguments),v&&(B=!1,re.search=h.toString(),B=!0)}}),Object.defineProperty(this,"searchParams",{value:h,enumerable:!0});var z=void 0;Object.defineProperty(this,"_updateSearchParams",{enumerable:!1,configurable:!1,writable:!1,value:function(){this.search!==z&&(z=this.search,B&&(v=!1,this.searchParams._fromString(this.search),v=!0))}})},a=i.prototype,s=function(f){Object.defineProperty(a,f,{get:function(){return this._anchorElement[f]},set:function(c){this._anchorElement[f]=c},enumerable:!0})};["hash","host","hostname","port","protocol"].forEach(function(f){s(f)}),Object.defineProperty(a,"search",{get:function(){return this._anchorElement.search},set:function(f){this._anchorElement.search=f,this._updateSearchParams()},enumerable:!0}),Object.defineProperties(a,{toString:{get:function(){var f=this;return function(){return f.href}}},href:{get:function(){return this._anchorElement.href.replace(/\?$/,"")},set:function(f){this._anchorElement.href=f,this._updateSearchParams()},enumerable:!0},pathname:{get:function(){return this._anchorElement.pathname.replace(/(^\/?)/,"/")},set:function(f){this._anchorElement.pathname=f},enumerable:!0},origin:{get:function(){var f={"http:":80,"https:":443,"ftp:":21}[this._anchorElement.protocol],c=this._anchorElement.port!=f&&this._anchorElement.port!=="";return this._anchorElement.protocol+"//"+this._anchorElement.hostname+(c?":"+this._anchorElement.port:"")},enumerable:!0},password:{get:function(){return""},set:function(f){},enumerable:!0},username:{get:function(){return""},set:function(f){},enumerable:!0}}),i.createObjectURL=function(f){return o.createObjectURL.apply(o,arguments)},i.revokeObjectURL=function(f){return o.revokeObjectURL.apply(o,arguments)},e.URL=i};if(t()||r(),e.location!==void 0&&!("origin"in e.location)){var n=function(){return e.location.protocol+"//"+e.location.hostname+(e.location.port?":"+e.location.port:"")};try{Object.defineProperty(e.location,"origin",{get:n,enumerable:!0})}catch(o){setInterval(function(){e.location.origin=n()},100)}}})(typeof global!="undefined"?global:typeof window!="undefined"?window:typeof self!="undefined"?self:Sr)});var Qr=Ht((Lt,Kr)=>{/*! + * clipboard.js v2.0.11 + * https://clipboardjs.com/ + * + * Licensed MIT © Zeno Rocha + */(function(t,r){typeof Lt=="object"&&typeof Kr=="object"?Kr.exports=r():typeof define=="function"&&define.amd?define([],r):typeof Lt=="object"?Lt.ClipboardJS=r():t.ClipboardJS=r()})(Lt,function(){return function(){var e={686:function(n,o,i){"use strict";i.d(o,{default:function(){return ki}});var a=i(279),s=i.n(a),f=i(370),c=i.n(f),u=i(817),p=i.n(u);function m(j){try{return document.execCommand(j)}catch(O){return!1}}var d=function(O){var w=p()(O);return m("cut"),w},h=d;function v(j){var O=document.documentElement.getAttribute("dir")==="rtl",w=document.createElement("textarea");w.style.fontSize="12pt",w.style.border="0",w.style.padding="0",w.style.margin="0",w.style.position="absolute",w.style[O?"right":"left"]="-9999px";var k=window.pageYOffset||document.documentElement.scrollTop;return w.style.top="".concat(k,"px"),w.setAttribute("readonly",""),w.value=j,w}var B=function(O,w){var k=v(O);w.container.appendChild(k);var F=p()(k);return m("copy"),k.remove(),F},re=function(O){var w=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body},k="";return typeof O=="string"?k=B(O,w):O instanceof HTMLInputElement&&!["text","search","url","tel","password"].includes(O==null?void 0:O.type)?k=B(O.value,w):(k=p()(O),m("copy")),k},z=re;function T(j){return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?T=function(w){return typeof w}:T=function(w){return w&&typeof Symbol=="function"&&w.constructor===Symbol&&w!==Symbol.prototype?"symbol":typeof w},T(j)}var Ke=function(){var O=arguments.length>0&&arguments[0]!==void 0?arguments[0]:{},w=O.action,k=w===void 0?"copy":w,F=O.container,q=O.target,Le=O.text;if(k!=="copy"&&k!=="cut")throw new Error('Invalid "action" value, use either "copy" or "cut"');if(q!==void 0)if(q&&T(q)==="object"&&q.nodeType===1){if(k==="copy"&&q.hasAttribute("disabled"))throw new Error('Invalid "target" attribute. Please use "readonly" instead of "disabled" attribute');if(k==="cut"&&(q.hasAttribute("readonly")||q.hasAttribute("disabled")))throw new Error(`Invalid "target" attribute. You can't cut text from elements with "readonly" or "disabled" attributes`)}else throw new Error('Invalid "target" value, use a valid Element');if(Le)return z(Le,{container:F});if(q)return k==="cut"?h(q):z(q,{container:F})},We=Ke;function Ie(j){return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?Ie=function(w){return typeof w}:Ie=function(w){return w&&typeof Symbol=="function"&&w.constructor===Symbol&&w!==Symbol.prototype?"symbol":typeof w},Ie(j)}function Ti(j,O){if(!(j instanceof O))throw new TypeError("Cannot call a class as a function")}function nn(j,O){for(var w=0;w0&&arguments[0]!==void 0?arguments[0]:{};this.action=typeof F.action=="function"?F.action:this.defaultAction,this.target=typeof F.target=="function"?F.target:this.defaultTarget,this.text=typeof F.text=="function"?F.text:this.defaultText,this.container=Ie(F.container)==="object"?F.container:document.body}},{key:"listenClick",value:function(F){var q=this;this.listener=c()(F,"click",function(Le){return q.onClick(Le)})}},{key:"onClick",value:function(F){var q=F.delegateTarget||F.currentTarget,Le=this.action(q)||"copy",Rt=We({action:Le,container:this.container,target:this.target(q),text:this.text(q)});this.emit(Rt?"success":"error",{action:Le,text:Rt,trigger:q,clearSelection:function(){q&&q.focus(),window.getSelection().removeAllRanges()}})}},{key:"defaultAction",value:function(F){return yr("action",F)}},{key:"defaultTarget",value:function(F){var q=yr("target",F);if(q)return document.querySelector(q)}},{key:"defaultText",value:function(F){return yr("text",F)}},{key:"destroy",value:function(){this.listener.destroy()}}],[{key:"copy",value:function(F){var q=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body};return z(F,q)}},{key:"cut",value:function(F){return h(F)}},{key:"isSupported",value:function(){var F=arguments.length>0&&arguments[0]!==void 0?arguments[0]:["copy","cut"],q=typeof F=="string"?[F]:F,Le=!!document.queryCommandSupported;return q.forEach(function(Rt){Le=Le&&!!document.queryCommandSupported(Rt)}),Le}}]),w}(s()),ki=Ri},828:function(n){var o=9;if(typeof Element!="undefined"&&!Element.prototype.matches){var i=Element.prototype;i.matches=i.matchesSelector||i.mozMatchesSelector||i.msMatchesSelector||i.oMatchesSelector||i.webkitMatchesSelector}function a(s,f){for(;s&&s.nodeType!==o;){if(typeof s.matches=="function"&&s.matches(f))return s;s=s.parentNode}}n.exports=a},438:function(n,o,i){var a=i(828);function s(u,p,m,d,h){var v=c.apply(this,arguments);return u.addEventListener(m,v,h),{destroy:function(){u.removeEventListener(m,v,h)}}}function f(u,p,m,d,h){return typeof u.addEventListener=="function"?s.apply(null,arguments):typeof m=="function"?s.bind(null,document).apply(null,arguments):(typeof u=="string"&&(u=document.querySelectorAll(u)),Array.prototype.map.call(u,function(v){return s(v,p,m,d,h)}))}function c(u,p,m,d){return function(h){h.delegateTarget=a(h.target,p),h.delegateTarget&&d.call(u,h)}}n.exports=f},879:function(n,o){o.node=function(i){return i!==void 0&&i instanceof HTMLElement&&i.nodeType===1},o.nodeList=function(i){var a=Object.prototype.toString.call(i);return i!==void 0&&(a==="[object NodeList]"||a==="[object HTMLCollection]")&&"length"in i&&(i.length===0||o.node(i[0]))},o.string=function(i){return typeof i=="string"||i instanceof String},o.fn=function(i){var a=Object.prototype.toString.call(i);return a==="[object Function]"}},370:function(n,o,i){var a=i(879),s=i(438);function f(m,d,h){if(!m&&!d&&!h)throw new Error("Missing required arguments");if(!a.string(d))throw new TypeError("Second argument must be a String");if(!a.fn(h))throw new TypeError("Third argument must be a Function");if(a.node(m))return c(m,d,h);if(a.nodeList(m))return u(m,d,h);if(a.string(m))return p(m,d,h);throw new TypeError("First argument must be a String, HTMLElement, HTMLCollection, or NodeList")}function c(m,d,h){return m.addEventListener(d,h),{destroy:function(){m.removeEventListener(d,h)}}}function u(m,d,h){return Array.prototype.forEach.call(m,function(v){v.addEventListener(d,h)}),{destroy:function(){Array.prototype.forEach.call(m,function(v){v.removeEventListener(d,h)})}}}function p(m,d,h){return s(document.body,m,d,h)}n.exports=f},817:function(n){function o(i){var a;if(i.nodeName==="SELECT")i.focus(),a=i.value;else if(i.nodeName==="INPUT"||i.nodeName==="TEXTAREA"){var s=i.hasAttribute("readonly");s||i.setAttribute("readonly",""),i.select(),i.setSelectionRange(0,i.value.length),s||i.removeAttribute("readonly"),a=i.value}else{i.hasAttribute("contenteditable")&&i.focus();var f=window.getSelection(),c=document.createRange();c.selectNodeContents(i),f.removeAllRanges(),f.addRange(c),a=f.toString()}return a}n.exports=o},279:function(n){function o(){}o.prototype={on:function(i,a,s){var f=this.e||(this.e={});return(f[i]||(f[i]=[])).push({fn:a,ctx:s}),this},once:function(i,a,s){var f=this;function c(){f.off(i,c),a.apply(s,arguments)}return c._=a,this.on(i,c,s)},emit:function(i){var a=[].slice.call(arguments,1),s=((this.e||(this.e={}))[i]||[]).slice(),f=0,c=s.length;for(f;f{"use strict";/*! + * escape-html + * Copyright(c) 2012-2013 TJ Holowaychuk + * Copyright(c) 2015 Andreas Lubbe + * Copyright(c) 2015 Tiancheng "Timothy" Gu + * MIT Licensed + */var is=/["'&<>]/;Jo.exports=as;function as(e){var t=""+e,r=is.exec(t);if(!r)return t;var n,o="",i=0,a=0;for(i=r.index;i0&&i[i.length-1])&&(c[0]===6||c[0]===2)){r=0;continue}if(c[0]===3&&(!i||c[1]>i[0]&&c[1]=e.length&&(e=void 0),{value:e&&e[n++],done:!e}}};throw new TypeError(t?"Object is not iterable.":"Symbol.iterator is not defined.")}function W(e,t){var r=typeof Symbol=="function"&&e[Symbol.iterator];if(!r)return e;var n=r.call(e),o,i=[],a;try{for(;(t===void 0||t-- >0)&&!(o=n.next()).done;)i.push(o.value)}catch(s){a={error:s}}finally{try{o&&!o.done&&(r=n.return)&&r.call(n)}finally{if(a)throw a.error}}return i}function D(e,t,r){if(r||arguments.length===2)for(var n=0,o=t.length,i;n1||s(m,d)})})}function s(m,d){try{f(n[m](d))}catch(h){p(i[0][3],h)}}function f(m){m.value instanceof Xe?Promise.resolve(m.value.v).then(c,u):p(i[0][2],m)}function c(m){s("next",m)}function u(m){s("throw",m)}function p(m,d){m(d),i.shift(),i.length&&s(i[0][0],i[0][1])}}function mn(e){if(!Symbol.asyncIterator)throw new TypeError("Symbol.asyncIterator is not defined.");var t=e[Symbol.asyncIterator],r;return t?t.call(e):(e=typeof xe=="function"?xe(e):e[Symbol.iterator](),r={},n("next"),n("throw"),n("return"),r[Symbol.asyncIterator]=function(){return this},r);function n(i){r[i]=e[i]&&function(a){return new Promise(function(s,f){a=e[i](a),o(s,f,a.done,a.value)})}}function o(i,a,s,f){Promise.resolve(f).then(function(c){i({value:c,done:s})},a)}}function A(e){return typeof e=="function"}function at(e){var t=function(n){Error.call(n),n.stack=new Error().stack},r=e(t);return r.prototype=Object.create(Error.prototype),r.prototype.constructor=r,r}var $t=at(function(e){return function(r){e(this),this.message=r?r.length+` errors occurred during unsubscription: +`+r.map(function(n,o){return o+1+") "+n.toString()}).join(` + `):"",this.name="UnsubscriptionError",this.errors=r}});function De(e,t){if(e){var r=e.indexOf(t);0<=r&&e.splice(r,1)}}var Fe=function(){function e(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}return e.prototype.unsubscribe=function(){var t,r,n,o,i;if(!this.closed){this.closed=!0;var a=this._parentage;if(a)if(this._parentage=null,Array.isArray(a))try{for(var s=xe(a),f=s.next();!f.done;f=s.next()){var c=f.value;c.remove(this)}}catch(v){t={error:v}}finally{try{f&&!f.done&&(r=s.return)&&r.call(s)}finally{if(t)throw t.error}}else a.remove(this);var u=this.initialTeardown;if(A(u))try{u()}catch(v){i=v instanceof $t?v.errors:[v]}var p=this._finalizers;if(p){this._finalizers=null;try{for(var m=xe(p),d=m.next();!d.done;d=m.next()){var h=d.value;try{dn(h)}catch(v){i=i!=null?i:[],v instanceof $t?i=D(D([],W(i)),W(v.errors)):i.push(v)}}}catch(v){n={error:v}}finally{try{d&&!d.done&&(o=m.return)&&o.call(m)}finally{if(n)throw n.error}}}if(i)throw new $t(i)}},e.prototype.add=function(t){var r;if(t&&t!==this)if(this.closed)dn(t);else{if(t instanceof e){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(r=this._finalizers)!==null&&r!==void 0?r:[]).push(t)}},e.prototype._hasParent=function(t){var r=this._parentage;return r===t||Array.isArray(r)&&r.includes(t)},e.prototype._addParent=function(t){var r=this._parentage;this._parentage=Array.isArray(r)?(r.push(t),r):r?[r,t]:t},e.prototype._removeParent=function(t){var r=this._parentage;r===t?this._parentage=null:Array.isArray(r)&&De(r,t)},e.prototype.remove=function(t){var r=this._finalizers;r&&De(r,t),t instanceof e&&t._removeParent(this)},e.EMPTY=function(){var t=new e;return t.closed=!0,t}(),e}();var Or=Fe.EMPTY;function It(e){return e instanceof Fe||e&&"closed"in e&&A(e.remove)&&A(e.add)&&A(e.unsubscribe)}function dn(e){A(e)?e():e.unsubscribe()}var Ae={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var st={setTimeout:function(e,t){for(var r=[],n=2;n0},enumerable:!1,configurable:!0}),t.prototype._trySubscribe=function(r){return this._throwIfClosed(),e.prototype._trySubscribe.call(this,r)},t.prototype._subscribe=function(r){return this._throwIfClosed(),this._checkFinalizedStatuses(r),this._innerSubscribe(r)},t.prototype._innerSubscribe=function(r){var n=this,o=this,i=o.hasError,a=o.isStopped,s=o.observers;return i||a?Or:(this.currentObservers=null,s.push(r),new Fe(function(){n.currentObservers=null,De(s,r)}))},t.prototype._checkFinalizedStatuses=function(r){var n=this,o=n.hasError,i=n.thrownError,a=n.isStopped;o?r.error(i):a&&r.complete()},t.prototype.asObservable=function(){var r=new U;return r.source=this,r},t.create=function(r,n){return new wn(r,n)},t}(U);var wn=function(e){ne(t,e);function t(r,n){var o=e.call(this)||this;return o.destination=r,o.source=n,o}return t.prototype.next=function(r){var n,o;(o=(n=this.destination)===null||n===void 0?void 0:n.next)===null||o===void 0||o.call(n,r)},t.prototype.error=function(r){var n,o;(o=(n=this.destination)===null||n===void 0?void 0:n.error)===null||o===void 0||o.call(n,r)},t.prototype.complete=function(){var r,n;(n=(r=this.destination)===null||r===void 0?void 0:r.complete)===null||n===void 0||n.call(r)},t.prototype._subscribe=function(r){var n,o;return(o=(n=this.source)===null||n===void 0?void 0:n.subscribe(r))!==null&&o!==void 0?o:Or},t}(E);var Et={now:function(){return(Et.delegate||Date).now()},delegate:void 0};var wt=function(e){ne(t,e);function t(r,n,o){r===void 0&&(r=1/0),n===void 0&&(n=1/0),o===void 0&&(o=Et);var i=e.call(this)||this;return i._bufferSize=r,i._windowTime=n,i._timestampProvider=o,i._buffer=[],i._infiniteTimeWindow=!0,i._infiniteTimeWindow=n===1/0,i._bufferSize=Math.max(1,r),i._windowTime=Math.max(1,n),i}return t.prototype.next=function(r){var n=this,o=n.isStopped,i=n._buffer,a=n._infiniteTimeWindow,s=n._timestampProvider,f=n._windowTime;o||(i.push(r),!a&&i.push(s.now()+f)),this._trimBuffer(),e.prototype.next.call(this,r)},t.prototype._subscribe=function(r){this._throwIfClosed(),this._trimBuffer();for(var n=this._innerSubscribe(r),o=this,i=o._infiniteTimeWindow,a=o._buffer,s=a.slice(),f=0;f0?e.prototype.requestAsyncId.call(this,r,n,o):(r.actions.push(this),r._scheduled||(r._scheduled=ut.requestAnimationFrame(function(){return r.flush(void 0)})))},t.prototype.recycleAsyncId=function(r,n,o){var i;if(o===void 0&&(o=0),o!=null?o>0:this.delay>0)return e.prototype.recycleAsyncId.call(this,r,n,o);var a=r.actions;n!=null&&((i=a[a.length-1])===null||i===void 0?void 0:i.id)!==n&&(ut.cancelAnimationFrame(n),r._scheduled=void 0)},t}(Ut);var On=function(e){ne(t,e);function t(){return e!==null&&e.apply(this,arguments)||this}return t.prototype.flush=function(r){this._active=!0;var n=this._scheduled;this._scheduled=void 0;var o=this.actions,i;r=r||o.shift();do if(i=r.execute(r.state,r.delay))break;while((r=o[0])&&r.id===n&&o.shift());if(this._active=!1,i){for(;(r=o[0])&&r.id===n&&o.shift();)r.unsubscribe();throw i}},t}(Wt);var we=new On(Tn);var R=new U(function(e){return e.complete()});function Dt(e){return e&&A(e.schedule)}function kr(e){return e[e.length-1]}function Qe(e){return A(kr(e))?e.pop():void 0}function Se(e){return Dt(kr(e))?e.pop():void 0}function Vt(e,t){return typeof kr(e)=="number"?e.pop():t}var pt=function(e){return e&&typeof e.length=="number"&&typeof e!="function"};function zt(e){return A(e==null?void 0:e.then)}function Nt(e){return A(e[ft])}function qt(e){return Symbol.asyncIterator&&A(e==null?void 0:e[Symbol.asyncIterator])}function Kt(e){return new TypeError("You provided "+(e!==null&&typeof e=="object"?"an invalid object":"'"+e+"'")+" where a stream was expected. You can provide an Observable, Promise, ReadableStream, Array, AsyncIterable, or Iterable.")}function Ki(){return typeof Symbol!="function"||!Symbol.iterator?"@@iterator":Symbol.iterator}var Qt=Ki();function Yt(e){return A(e==null?void 0:e[Qt])}function Gt(e){return ln(this,arguments,function(){var r,n,o,i;return Pt(this,function(a){switch(a.label){case 0:r=e.getReader(),a.label=1;case 1:a.trys.push([1,,9,10]),a.label=2;case 2:return[4,Xe(r.read())];case 3:return n=a.sent(),o=n.value,i=n.done,i?[4,Xe(void 0)]:[3,5];case 4:return[2,a.sent()];case 5:return[4,Xe(o)];case 6:return[4,a.sent()];case 7:return a.sent(),[3,2];case 8:return[3,10];case 9:return r.releaseLock(),[7];case 10:return[2]}})})}function Bt(e){return A(e==null?void 0:e.getReader)}function $(e){if(e instanceof U)return e;if(e!=null){if(Nt(e))return Qi(e);if(pt(e))return Yi(e);if(zt(e))return Gi(e);if(qt(e))return _n(e);if(Yt(e))return Bi(e);if(Bt(e))return Ji(e)}throw Kt(e)}function Qi(e){return new U(function(t){var r=e[ft]();if(A(r.subscribe))return r.subscribe(t);throw new TypeError("Provided object does not correctly implement Symbol.observable")})}function Yi(e){return new U(function(t){for(var r=0;r=2;return function(n){return n.pipe(e?_(function(o,i){return e(o,i,n)}):me,Oe(1),r?He(t):zn(function(){return new Xt}))}}function Nn(){for(var e=[],t=0;t=2,!0))}function fe(e){e===void 0&&(e={});var t=e.connector,r=t===void 0?function(){return new E}:t,n=e.resetOnError,o=n===void 0?!0:n,i=e.resetOnComplete,a=i===void 0?!0:i,s=e.resetOnRefCountZero,f=s===void 0?!0:s;return function(c){var u,p,m,d=0,h=!1,v=!1,B=function(){p==null||p.unsubscribe(),p=void 0},re=function(){B(),u=m=void 0,h=v=!1},z=function(){var T=u;re(),T==null||T.unsubscribe()};return g(function(T,Ke){d++,!v&&!h&&B();var We=m=m!=null?m:r();Ke.add(function(){d--,d===0&&!v&&!h&&(p=jr(z,f))}),We.subscribe(Ke),!u&&d>0&&(u=new et({next:function(Ie){return We.next(Ie)},error:function(Ie){v=!0,B(),p=jr(re,o,Ie),We.error(Ie)},complete:function(){h=!0,B(),p=jr(re,a),We.complete()}}),$(T).subscribe(u))})(c)}}function jr(e,t){for(var r=[],n=2;ne.next(document)),e}function K(e,t=document){return Array.from(t.querySelectorAll(e))}function V(e,t=document){let r=se(e,t);if(typeof r=="undefined")throw new ReferenceError(`Missing element: expected "${e}" to be present`);return r}function se(e,t=document){return t.querySelector(e)||void 0}function _e(){return document.activeElement instanceof HTMLElement&&document.activeElement||void 0}function tr(e){return L(b(document.body,"focusin"),b(document.body,"focusout")).pipe(ke(1),l(()=>{let t=_e();return typeof t!="undefined"?e.contains(t):!1}),N(e===_e()),Y())}function Be(e){return{x:e.offsetLeft,y:e.offsetTop}}function Yn(e){return L(b(window,"load"),b(window,"resize")).pipe(Ce(0,we),l(()=>Be(e)),N(Be(e)))}function rr(e){return{x:e.scrollLeft,y:e.scrollTop}}function dt(e){return L(b(e,"scroll"),b(window,"resize")).pipe(Ce(0,we),l(()=>rr(e)),N(rr(e)))}var Bn=function(){if(typeof Map!="undefined")return Map;function e(t,r){var n=-1;return t.some(function(o,i){return o[0]===r?(n=i,!0):!1}),n}return function(){function t(){this.__entries__=[]}return Object.defineProperty(t.prototype,"size",{get:function(){return this.__entries__.length},enumerable:!0,configurable:!0}),t.prototype.get=function(r){var n=e(this.__entries__,r),o=this.__entries__[n];return o&&o[1]},t.prototype.set=function(r,n){var o=e(this.__entries__,r);~o?this.__entries__[o][1]=n:this.__entries__.push([r,n])},t.prototype.delete=function(r){var n=this.__entries__,o=e(n,r);~o&&n.splice(o,1)},t.prototype.has=function(r){return!!~e(this.__entries__,r)},t.prototype.clear=function(){this.__entries__.splice(0)},t.prototype.forEach=function(r,n){n===void 0&&(n=null);for(var o=0,i=this.__entries__;o0},e.prototype.connect_=function(){!zr||this.connected_||(document.addEventListener("transitionend",this.onTransitionEnd_),window.addEventListener("resize",this.refresh),xa?(this.mutationsObserver_=new MutationObserver(this.refresh),this.mutationsObserver_.observe(document,{attributes:!0,childList:!0,characterData:!0,subtree:!0})):(document.addEventListener("DOMSubtreeModified",this.refresh),this.mutationEventsAdded_=!0),this.connected_=!0)},e.prototype.disconnect_=function(){!zr||!this.connected_||(document.removeEventListener("transitionend",this.onTransitionEnd_),window.removeEventListener("resize",this.refresh),this.mutationsObserver_&&this.mutationsObserver_.disconnect(),this.mutationEventsAdded_&&document.removeEventListener("DOMSubtreeModified",this.refresh),this.mutationsObserver_=null,this.mutationEventsAdded_=!1,this.connected_=!1)},e.prototype.onTransitionEnd_=function(t){var r=t.propertyName,n=r===void 0?"":r,o=ya.some(function(i){return!!~n.indexOf(i)});o&&this.refresh()},e.getInstance=function(){return this.instance_||(this.instance_=new e),this.instance_},e.instance_=null,e}(),Jn=function(e,t){for(var r=0,n=Object.keys(t);r0},e}(),Zn=typeof WeakMap!="undefined"?new WeakMap:new Bn,eo=function(){function e(t){if(!(this instanceof e))throw new TypeError("Cannot call a class as a function.");if(!arguments.length)throw new TypeError("1 argument required, but only 0 present.");var r=Ea.getInstance(),n=new Ra(t,r,this);Zn.set(this,n)}return e}();["observe","unobserve","disconnect"].forEach(function(e){eo.prototype[e]=function(){var t;return(t=Zn.get(this))[e].apply(t,arguments)}});var ka=function(){return typeof nr.ResizeObserver!="undefined"?nr.ResizeObserver:eo}(),to=ka;var ro=new E,Ha=I(()=>H(new to(e=>{for(let t of e)ro.next(t)}))).pipe(x(e=>L(Te,H(e)).pipe(C(()=>e.disconnect()))),J(1));function de(e){return{width:e.offsetWidth,height:e.offsetHeight}}function ge(e){return Ha.pipe(S(t=>t.observe(e)),x(t=>ro.pipe(_(({target:r})=>r===e),C(()=>t.unobserve(e)),l(()=>de(e)))),N(de(e)))}function bt(e){return{width:e.scrollWidth,height:e.scrollHeight}}function ar(e){let t=e.parentElement;for(;t&&(e.scrollWidth<=t.scrollWidth&&e.scrollHeight<=t.scrollHeight);)t=(e=t).parentElement;return t?e:void 0}var no=new E,Pa=I(()=>H(new IntersectionObserver(e=>{for(let t of e)no.next(t)},{threshold:0}))).pipe(x(e=>L(Te,H(e)).pipe(C(()=>e.disconnect()))),J(1));function sr(e){return Pa.pipe(S(t=>t.observe(e)),x(t=>no.pipe(_(({target:r})=>r===e),C(()=>t.unobserve(e)),l(({isIntersecting:r})=>r))))}function oo(e,t=16){return dt(e).pipe(l(({y:r})=>{let n=de(e),o=bt(e);return r>=o.height-n.height-t}),Y())}var cr={drawer:V("[data-md-toggle=drawer]"),search:V("[data-md-toggle=search]")};function io(e){return cr[e].checked}function qe(e,t){cr[e].checked!==t&&cr[e].click()}function je(e){let t=cr[e];return b(t,"change").pipe(l(()=>t.checked),N(t.checked))}function $a(e,t){switch(e.constructor){case HTMLInputElement:return e.type==="radio"?/^Arrow/.test(t):!0;case HTMLSelectElement:case HTMLTextAreaElement:return!0;default:return e.isContentEditable}}function Ia(){return L(b(window,"compositionstart").pipe(l(()=>!0)),b(window,"compositionend").pipe(l(()=>!1))).pipe(N(!1))}function ao(){let e=b(window,"keydown").pipe(_(t=>!(t.metaKey||t.ctrlKey)),l(t=>({mode:io("search")?"search":"global",type:t.key,claim(){t.preventDefault(),t.stopPropagation()}})),_(({mode:t,type:r})=>{if(t==="global"){let n=_e();if(typeof n!="undefined")return!$a(n,r)}return!0}),fe());return Ia().pipe(x(t=>t?R:e))}function Me(){return new URL(location.href)}function ot(e){location.href=e.href}function so(){return new E}function co(e,t){if(typeof t=="string"||typeof t=="number")e.innerHTML+=t.toString();else if(t instanceof Node)e.appendChild(t);else if(Array.isArray(t))for(let r of t)co(e,r)}function M(e,t,...r){let n=document.createElement(e);if(t)for(let o of Object.keys(t))typeof t[o]!="undefined"&&(typeof t[o]!="boolean"?n.setAttribute(o,t[o]):n.setAttribute(o,""));for(let o of r)co(n,o);return n}function fr(e){if(e>999){let t=+((e-950)%1e3>99);return`${((e+1e-6)/1e3).toFixed(t)}k`}else return e.toString()}function fo(){return location.hash.substring(1)}function uo(e){let t=M("a",{href:e});t.addEventListener("click",r=>r.stopPropagation()),t.click()}function Fa(){return b(window,"hashchange").pipe(l(fo),N(fo()),_(e=>e.length>0),J(1))}function po(){return Fa().pipe(l(e=>se(`[id="${e}"]`)),_(e=>typeof e!="undefined"))}function Nr(e){let t=matchMedia(e);return Zt(r=>t.addListener(()=>r(t.matches))).pipe(N(t.matches))}function lo(){let e=matchMedia("print");return L(b(window,"beforeprint").pipe(l(()=>!0)),b(window,"afterprint").pipe(l(()=>!1))).pipe(N(e.matches))}function qr(e,t){return e.pipe(x(r=>r?t():R))}function ur(e,t={credentials:"same-origin"}){return ve(fetch(`${e}`,t)).pipe(ce(()=>R),x(r=>r.status!==200?Tt(()=>new Error(r.statusText)):H(r)))}function Ue(e,t){return ur(e,t).pipe(x(r=>r.json()),J(1))}function mo(e,t){let r=new DOMParser;return ur(e,t).pipe(x(n=>n.text()),l(n=>r.parseFromString(n,"text/xml")),J(1))}function pr(e){let t=M("script",{src:e});return I(()=>(document.head.appendChild(t),L(b(t,"load"),b(t,"error").pipe(x(()=>Tt(()=>new ReferenceError(`Invalid script: ${e}`))))).pipe(l(()=>{}),C(()=>document.head.removeChild(t)),Oe(1))))}function ho(){return{x:Math.max(0,scrollX),y:Math.max(0,scrollY)}}function bo(){return L(b(window,"scroll",{passive:!0}),b(window,"resize",{passive:!0})).pipe(l(ho),N(ho()))}function vo(){return{width:innerWidth,height:innerHeight}}function go(){return b(window,"resize",{passive:!0}).pipe(l(vo),N(vo()))}function yo(){return Q([bo(),go()]).pipe(l(([e,t])=>({offset:e,size:t})),J(1))}function lr(e,{viewport$:t,header$:r}){let n=t.pipe(X("size")),o=Q([n,r]).pipe(l(()=>Be(e)));return Q([r,t,o]).pipe(l(([{height:i},{offset:a,size:s},{x:f,y:c}])=>({offset:{x:a.x-f,y:a.y-c+i},size:s})))}(()=>{function e(n,o){parent.postMessage(n,o||"*")}function t(...n){return n.reduce((o,i)=>o.then(()=>new Promise(a=>{let s=document.createElement("script");s.src=i,s.onload=a,document.body.appendChild(s)})),Promise.resolve())}var r=class{constructor(n){this.url=n,this.onerror=null,this.onmessage=null,this.onmessageerror=null,this.m=a=>{a.source===this.w&&(a.stopImmediatePropagation(),this.dispatchEvent(new MessageEvent("message",{data:a.data})),this.onmessage&&this.onmessage(a))},this.e=(a,s,f,c,u)=>{if(s===this.url.toString()){let p=new ErrorEvent("error",{message:a,filename:s,lineno:f,colno:c,error:u});this.dispatchEvent(p),this.onerror&&this.onerror(p)}};let o=new EventTarget;this.addEventListener=o.addEventListener.bind(o),this.removeEventListener=o.removeEventListener.bind(o),this.dispatchEvent=o.dispatchEvent.bind(o);let i=document.createElement("iframe");i.width=i.height=i.frameBorder="0",document.body.appendChild(this.iframe=i),this.w.document.open(),this.w.document.write(` + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

How to cite CL

+

The Cell Ontology 2016: enhanced content, modularization, and ontology interoperability. +Alexander D Diehl, Terrence F Meehan, Yvonne M Bradford, Matthew H Brush, Wasila M Dahdul, David S Dougall, Yongqun He, David Osumi-Sutherland, Alan Ruttenberg, Sirarat Sarntivijai, Ceri E Van Slyke, Nicole A Vasilevsky, Melissa A Haendel, Judith A Blake, Christopher J Mungall. +J Biomed Semantics. 2016 Jul 4;7(1):44.
+PMID:27377652, PMCID:PMC4932724, DOI:10.1186/s13326-016-0088-7

+

Logical development of the cell ontology. +Terrence F Meehan, Anna Maria Masci, Amina Abdulla, Lindsay G Cowell, Judith A Blake, Christopher J Mungall, Alexander D Diehl. +BMC Bioinformatics. 2011 Jan 5;12:6. +PMID:21208450, PMCID:PMC3024222, DOI:10.1186/1471-2105-12-6

+

An ontology for cell types. +Jonathan Bard, Seung Y Rhee, Michael Ashburner. +Genome Biol. 2005;6(2):R21. +PMID:15693950, PMCID:PMC551541, DOI:10.1186/gb-2005-6-2-r21

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/cl-release/index.html b/cl-release/index.html new file mode 100644 index 000000000..e41a07fa4 --- /dev/null +++ b/cl-release/index.html @@ -0,0 +1,928 @@ + + + + + + + + + + + + + + + + + + + + + + CL release - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

CL Release workflow

+

While CL is an ODK ontology, it has a specific workflow for releases due to its large size and the limitations on standard GitHub releases.

+

Requirements

+

Aside from the standard requirements needed for ODK workflow, GH is required. +Instructions on how to install GH can be found here

+

You will need to log in to your GitHub account on GH before you can uses it. To do this, enter the following in your terminal:

+
gh auth login
+
+

You can then follow instructions below for web browser login (or use your prefer means of logging in).

+
% gh auth login
+? What account do you want to log into? GitHub.com
+? What is your preferred protocol for Git operations? SSH
+? Generate a new SSH key to add to your GitHub account? Yes
+? Enter a passphrase for your new SSH key (Optional) 
+? Title for your SSH key: GitHub SSH
+? How would you like to authenticate GitHub CLI? Login with a web browser
+
+! First copy your one-time code: XXXX-XXXX
+Press Enter to open github.com in your browser... 
+✓ Authentication complete.
+- gh config set -h github.com git_protocol ssh
+✓ Configured git protocol
+✓ Uploaded the SSH key to your GitHub account: /Users/username/.ssh/id_ed25519.pub
+✓ Logged in as username
+
+

Release Process

+

Preparation

+

Preparation:

+
    +
  1. Ensure that all your pull requests are merged into your main (master) branch
    2. +
  2. Make sure that all changes to master are committed to Github (git status should say that there are no modified files)
    3. +
  3. Locally make sure you have the latest changes from master (git pull)
    4. +
  4. Checkout a new branch (e.g. git checkout -b release-2021-01-01)
    5. +
  5. You may or may not want to refresh your imports as part of your release strategy (see here)(Note: in CL we decouple our imports and releases - we hence advice that you do not update imports)
    6. +
  6. Make sure you have the latest ODK installed by running docker pull obolibrary/odkfull
    7. +
+

To actually run the release, you:

+
    +
  1. Open a command line terminal window and navigate to the src/ontology directory (cd cl/src/ontology)
    2. +
  2. Run the release using sh run.sh make cl DEPLOY_GH=false. This will build all files and copy them to the correct place. (Note: the IMP=false is used to decouple imports refresh with release)
    3. +
  3. Review the release as per the review release section in ODK-workflow release document
    4. +
  4. Create a pull request and get a second set of eyes to review it. As CL uses a custom release pipeline, we ask that you get at least one core developer to review it too.
    5. +
  5. Merge to main branch once reviewed and CI checks have passed
    6. +
  6. Deploy release on GitHub by running make deploy_release GHVERSION="v2022-06-20" on the release branch (DO NOTE CHANGE TO MAIN BRANCH!), replacing the date with the date of release (NOTE: no sh run.sh) +Editors note: ODK 1.3.2 will have a feature to run the release from inside the docker container. For now deploy_release has to be run outside.
    7. +
  7. This should end with a GitHub release link that looks something like:
    8. +
+
https://github.com/obophenotype/cl/releases/tag/untagged-8935f3432525b27a0d84
+
+

Copy the link and paste it in your browser, this should show you a draft release. +1. Click the edit button (the pencil button on the top right corner) and change the tag to the GHVERSION you entered above (eg v2022-06-20) +1. Change the TBD. in the main text to a summary of the main changes in the release if needed. +1. Scroll down all the way and click the update release button.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/contact_us/index.html b/contact_us/index.html new file mode 100644 index 000000000..510eafecf --- /dev/null +++ b/contact_us/index.html @@ -0,0 +1,855 @@ + + + + + + + + + + + + + + + + + + + + Contact - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Contact Us

+

GitHub

+

The preferred point of contact for the Cell Ontology is the GitHub Issue Tracker. It is used commonly for requesting new terms or suggesting changes to existing terms. Feel free to open a new issue with any kind of request/question/consideration you might have.

+

If you do not have a GitHub account, you can sign up for a free account here.

+

E-mail and other contacts

+

The Cell Ontology also has a Slack Channel and monthly meetings, with an open agenda.

+

If you want to join the channel or talk about any other topics related to the Cell Ontology, you may reach out to one of the active CL editors. +The official contact for CL listed at the OBO Foundry is Alexander Diehl, but feel free to contact any other individual. A list of active CL editors can be found here.

+

Note that requests on the GitHub issue tracker may be addressed more quickly, as individuals are often busy.

+

For other details on CL, you may see http://obofoundry.org/ontology/cl.html.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/contributing/index.html b/contributing/index.html new file mode 100644 index 000000000..2181b8948 --- /dev/null +++ b/contributing/index.html @@ -0,0 +1,890 @@ + + + + + + + + + + + + + + + + + + + + + + Contributing - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

How to contribute to CL

+

We welcome your contributions to CL! Generally, you can follow the editors workflow instructions here.

+

However, if you aren't confident in directly editing the ontology, you can contribute by writing up an issue and one of our curators/developers will pick it up and address it.

+

Writing up an issue

+

If you want a new term added, or want edits to a current term, or spot any mistakes/issues with CL, or you have any other CL related issues, you can write up a ticket using the following steps:

+
    +
  1. Go to the issues tab in CL.
    2. +
  2. Click the 'New issue' tab on the top right corner and select the most appropriate category for your issue. (Note: blank issues can be created if none of the categories fit, but we recommend using the categories as they are designed to be more comprehensive).
    3. +
  3. Fill up the form as best you can, giving a descriptive title to your issue name and leaving the bracketed [] tag in the title: eg.Add new term is bad name, while [NTR] larval stage X is good name.
    4. +
  4. When writing up more complex issues that include multiple items or steps, make sure you include the use of - [ ] to denote action items. These turn into checkboxes which makes it much faster to assess which comments have been addressed. (Note: it is better to write up multiple issues than one big one with multiple items, e.g. write up one issue for each term you want added rather than an issue with all the terms you want added.)
    5. +
  5. If you know a specific curator/editor that you want handling your ticket, you can assign them to your ticket in the assignee tab on the right, if not, someone from our team will assign an appropriate person to handle your ticket. If, however, your ticket has not been looked at in more than 10 days, and you suspect that it might have been missed, please assign it to gouttegd and they will assign it appropriately.
    6. +
  6. If you know how to edit the ontology directly, please then proceed to making a Pull request with the guidelines below, following the editors workflow instructions here.
    7. +
+

Pull request guidelines

+
    +
  • Give your pull requests good names: Add new terms is bad. Adding larval stage X term #332 is ok.
    • +
  • Make sure pull requests have someone assigned to review them and remind them once in a while. Do not let them go dormant
    • +
  • Assign yourself to be the Assignee
    • +
  • Make sure to use - [ ] to denote action items in issues and pull requests, not just comments. These turn into checkboxes which makes it much faster to assess which comments have been addressed and can be ignored.
    • +
  • Give a short summary of the pull request - that way we can find suitable reviewers much quicker. Say which terms you are adding or what kinds of changes you are proposing.
    • +
  • It is most of the time a good idea to use squash merge rather than merge for your pull request, to keep the git history short and useful.
    • +
+

Pull requests that require imports to be refreshed

+

If your pull request references foreign terms from an external ontology that are not yet present in the import module for that ontology (for example, you’re adding a logical definition that makes use of a GO term for the first time), imports needs to be refreshed for the foreign terms to be available to use.

+

If you have the technical skills and/or the required computer resources (refreshing imports can be a memory-intensive task), you may refresh the imports yourself before submitting the pull request, by following the appropriate procedure.

+

If you can’t apply the imports refreshing procedure for any reason, you may instead opt for using “bare IRIs” when editing the ontology, everywhere you need a reference to a foreign term. Then, when submitting your pull request, label it with the tag update-imports-required to ask that a member of the tech support group refresh the imports before the pull request can be merged.

+

People reviewing pull requests must 1) make sure that if a pull request is referencing bare IRIs, the request is tagged with update-imports-required (adding the label themselves if needed); and 2) make sure that imports have indeed been updated (either by the author of the pull request, or by someone from the tech support group if requested) before allowing the request to be merged.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/editing_guidelines/index.html b/editing_guidelines/index.html new file mode 100644 index 000000000..744bce9b2 --- /dev/null +++ b/editing_guidelines/index.html @@ -0,0 +1,1038 @@ + + + + + + + + + + + + + + + + + + Guidelines for CL editors - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Guidelines for CL editors

+

Naming terms

+

General rules

+
    +
  1. All labels should be singular nouns.
    2. +
  2. Words should not be capitalized, unless they are proper names or are capitalized as a standard (e.g., “Peyer's” and “B” in “Peyer's patch B cell”).
    3. +
  3. Avoid special characters: use only alphanumeric characters, space, dash (-), slash (/), and apostrophe (').
    4. +
+

Advice on writing labels

+

Bear in mind that users will often encounter terms in isolation. Sufficiently descriptive labels are therefore recommended, especially where there is obvious potential for confusion. For example, the label 'peripheral nervous system neuron' is preferred over 'peripheral neuron' as the former has more clarity.

+

For classes that refer to cell types in specific anatomical structures, the label may have the adjectival or noun form of the anatomical structure, for example, 'hepatic oval stem cell' or 'liver dendritic cell' ("hepatic" vs. "liver"). To determine which form of the anatomical term to use, it is recommended for the editor to search PubMed/review literature to determine common usage. If there is no clear preference, the editor can defer to use the adjectival form and add the noun form as an exact synonym.

+

Try to maintain consistent patterns of naming where possible. However, it may make sense to override this in order to conform to common usage.

+

Defining terms

+

This section is about the textual definition of terms. Logically consistent classification is important, but an ontology is only useful (and maintainable) if all humans that interact with it (users, curators and editors) can quickly find the terms they need and understand what they refer to. This requires clear, unambiguous, human-readable definitions.

+

When crafting a definition, editors should aim for a reasonably succinct statement about the class allowing curators and users to easily distinguish it from other, similar classes, and which captures key points of interest about that class. It should capture assertions made in the formal part of the definition (the relationships) as closely as possible without becoming stilted and difficult to read.

+

The basic structure of a definition should be as follows:

+
+

A genus that diff1 and diff2. It also diff3 and gloss...

+
+

where genus is a general classification and diff1, diff2, etc. are the differentia, which state what differentiates this class from others that share the same general classification. The gloss, when present, gives some key points of interest about the class.

+

The first sentence of the definition should refer to the definiendum in singular form. The rest of the definition may then invoke the plural form.

+

Contents of definitions

+

It is difficult to specify, a priori, which assertions should be included in a textual definition. However there are some general guidelines.

+
    +
  1. DO make sure your definition is consistent with the definition of the superclass(es).
    2. +
  2. DO make sure your definition includes the information that is recorded in all the direct formal relationships to the class.
    3. +
  3. AVOID assertions about structures or cell types that are not part of the cell type being described, except when they pertain to some direct relationships with the cell type being described.
    4. +
  4. AVOID including details that could better be included in the definition of subtypes of the cell type being described.
    5. +
  5. LIMIT information that applies to only some members of the class. This should only be used sparingly; when used, it should be made clear that it does not apply to all members of the class.
    6. +
  6. AVOID using gene expression as the differentium.
    7. +
  7. AVOID extensive repetition of assertions made in superclass definitions, unless these assertions are used to provide direct evidence for class membership.
    8. +
  8. DO NOT add informations about what happens in mutant or pathological states backgrounds or other kinds of non-control conditions.
    9. +
  9. DO NOT include reasons for believing the assertions to be true. These should be recorded in comments.
    10. +
  10. DO NOT raise questions in the definition. The definition should have the sense of being definitive, which is undermined if we show doubt. Any doubt should be recorded in comments instead.
    11. +
+

Comments

+

Comments should be used for:

+
    +
  1. Providing evidence. In some cases it is useful to know the type of evidence for an assertion. This should not be recorded in the definition, but can be recorded in a comment.
    2. +
  2. Disambiguation. Sometimes a single term is used in the literature with multiple meanings. In such cases, a comment should be added outlining these different uses and how they relate to the definition set in the ontology.
    3. +
  3. Reporting editorial decisions (or decisions in waiting) about the term. This includes providing a reason for obsoleting a term, or letting users and curators know that the term may be merged or split in the future, e.g., when enough evidence for the merge/split will be available.
    4. +
+

Try to be consistent in how you phrase the various types of comments. For example:

+
    +
  • when giving a reason for obsoletion, use “Obsoleted as ...”;
    • +
  • when indicating a potential future merge, use “Possible equivalence with {other term} ...”.
    • +
+

Synonyms

+

Extensive addition of synonyms helps “findability” of terms when search. Synonyms can and should be added liberally. Of note, the intention of the ontology is not meant to record how a synonym is used in all specific sources in which it appears. Rather an editor, after doing due diligence in researching the terms/synonyms, must determine how a term is used at the present moment in the scientific community.

+

Guidelines on the type of synonyms:

+
    +
  1. DO use has_exact_synonym only when the label and the synonym can be used interchangeably without dispute and refer to the same concept.
    2. +
+

Example: the terms “leukocyte”, “leucocyte” (spelling variation”) and “white blood cell” (layman’s term) all refer to the exact same concept (a specific cell type) and would be considered exact synonyms.

+
    +
  1. Terms that may refer to more than one concept, especially within the biomedical domain, should NOT be annotated as exact synonyms, including abbreviations. A synonym that is an abbreviation should be annotated using has_related_synonymand with property type “abbreviation” (technically: the synonym annotation assertion axiom should itself be annotated with a http://www.geneontology.org/formats/oboInOwl#hasSynonymType property with value http://purl.obolibrary.org/obo/OMO_0003000).
    2. +
+

Example: “WBC” can stand for “white blood cell” and refer to “leukocyte”, but within the biomedical domain it can also represent “white blood cell count” or, perhaps less frequently, “whole-body counting”, two distinct concepts with separate OBO ontology terms.

+
    +
  1. +

    DO check that exact synonyms are unique across the ontology. In other words, if class A has synonym “X”, “X” should NOT be an exact synonym for any other CL term.

    +
    2. +
  2. +

    DO be mindful of the “directionality” of the narrow and broad types of synonyms. They qualify the synonym, not the original term.

    +
    3. +
+

Example: asserting that “peripheral blood mononuclear cell” is a narrow synonym of “mononuclear cell” means that “peripheral blood mononuclear cell“ refers to a narrower (more specific) concept than “mononuclear cell”, not the other way around.

+
    +
  1. +

    DO use has_related_synonym where the overlap between the synonym and the term label may be uncler, disputable or not true in all scenarios or contexts, but do want the term to be findable when using the synonym as a search string. This includes abbreviations, which should be annotated as related synonyms with synonym type “abbreviation” (see point 2 above).

    +
    2. +
  2. +

    If a synonym includes a mix of abbreviations and words, DO use has_related_synonym EXCEPT when there is enough context within the synonym itself to make it clear that the synonym refers only to the concept being annotated.

    +
    3. +
+

Example: “lung TRM CD8-positive, CD103-positive cell” should be an exact synonym of “lung resident memory CD8-positive, CD103-positive, alpha-beta T cell”, even though “TRM” (in this case) is an abbreviation for “tissue resident T cell”. Note that without this context “TRM” should NOT be considered an exact synonym for “tissue resident T cell” as “TRM” could also mean “treatment-related mortality”, another OBO ontology concept. +Compare the previous example to “IMB cell”, which should be a related synonym of “invaginating midget bipolar cell”, as there is not enough context to confidently infer what "IMB" stands for.

+

Considerations on style

+

The following considerations apply both to all human-readable fields (names, textual definitions, comments, synonyms).

+
    +
  • United States (US) English or British English? Where there are differences in the accepted spelling between British and US English, use the US form. British English variants of the labels may be added as synonyms (e.g., a term labelled “epithelial cell of esophagus” may have “epithelial cell of oesophagus” as an exact synonym).
    • +
  • Use of jargon. The aim of the ontology is to provide useable descriptions and links to the reader. Consequently, try to avoid obscure jargon or pretentious Latin/Greek, especially where widely understood, plain alternatives exist. Where it will aid searching, such terms may be added to synonyms and/or as asides in the “gloss” part of definitions.
    • +
  • That or which? These are so interchangeable that there isn’t really a rule anymore. But as a guideline, use “which” after a comma (e.g., “this study, which cost $10,000, was a success”), and “that” when no comma is used (e.g., “the study that cost $10,000 was a success”).
    • +
  • Use of hyphens. Yes to those that help clarity (e.g., “posterior-most”) and those that are regularly used/accepted in the literature. Generally, no hyphens after prefixes such as “sub”, “mid”, “semi”, “hemi”, etc. (e.g., “hemidesmosome” instead of “hemi-desmosome”), unless it helps with clarity (e.g., “multi-innervated”). No hyphens for composed location adjectives (e.g., “posteroanterior”), unless there are more than two compounds (e.g., “ventro-posterolateral”).
    • +
  • Abbreviations. Avoid abbreviations, contractions, and symbols born out of laziness, such as &, +, or vs for “versus”. Avoid abbreviations unless they are self-explanatory, commonly understood, or they really do help to reduce the amount of typing enough to enhance readability. Use full chemical element names, not symbols (e.g., “hydrogen” instead of “H+”, “copper” instead of “Cu”, etc.). For biomolecules, spell out the term wherever practical (e.g., “fibroblast growth factor” instead of “FGF”). Abbreviations are acceptable in synonyms, in cases where the abbreviation is the synonym.
    • +
+

Cross-references to the literature

+

Assertions in textual definitions, evidence provided in comments, and synonyms should be backed up by citing the appropriate literature.

+

Citations are made by cross-references, that is by adding http://www.geneontology.org/formats/oboInOwl#hasDbXref annotations to the definition, comment, and synonym annotations. Add one such annotation per reference, using the CURIE syntax with well-known prefixes:

+
    +
  • PMID:1234567 for a PubMed identifier;
    • +
  • doi:xx.yyyy/... for a DOI;
    • +
  • ISBN:... for a ISBN.
    • +
+

If the main source for an assertion is a term in another ontology, the short identifier for that term may be used as a cross-reference. For example, WBbt:0006799 to cross-reference a term in the C. elegans Gross Anatomy Ontology.

+

If using a MeSH (Medical Subject Heading) term as a cross-reference, add the database_cross_reference annotation using the MeSH Unique ID, NOT MeSH Tree Number. For example, a database_cross_reference can be MESH:D004759, NOT MESH:A03.492.766.440.250.

+

ORCID identifiers may also be used when the only available source for an assertion is an individual researcher. However, this should be AVOIDED.

+

Technical details of adding a cross-reference using Protégé:

+

For CURIEs, ORCIDs: In the "Create Annotation" window, select the annotation property database_cross_reference.

+

For adding URLs to text definitions or synonyms: In the "Create Annotation" window, select the annotation property database_cross_reference.

+

For adding URLs to axioms that are NOT text definitions or synonyms: In the "Create Annotation" window, select the annotation property source.

+

For CURIEs: Enter the CURIE, using the bioregistry OBO context prefix (link to prefixmap), as a Value on the "Literal" tab. Leave Datatype empty.

+

In cases where more than one CURIE is available for a resource, either is acceptable, but using the more semantically specific identifier is recommended. For example, when both a PMID and a doi are available for a resource, using the PMID is recommended since it indicates the cross-reference points to a paper, as opposed to a doi which could point to any digital object.

+

For ORCIDs: Enter the ORCID as an IRI in the IRI field on the "IRI Editor" tab, for example https://orcid.org/0000-0002-7356-1779.

+

For URLs: Enter the URL as a literal string with Datatype xsd:anyURI selected.

+

To restate, in all cases above except ORCIDs, the values are entered as literal strings. An ORCID MUST BE entered as an IRI.

+

Term contributors

+

When adding an ORCID to identify a term contributor, in the "Create Annotation" window, select the annotation property dcterms:contributor. The ORCID is still entered as an IRI on the “IRI Editor” tab.

+

Formal definitions

+

The formal definition of a class is made up of all the logical axioms about the class (as opposed to the annotation assertion axioms). This includes classification assertions, relationship assertions, equivalence assertions, and disjointness assertions.

+
+

Note: In OWL formalism, both classification and relationship assertions are represented using SubClassOf axioms. However, in this document, we make a strict distinction between a classification (where a class is a subclass of a named class), and a relationship (where a class is a subclass of an anonymous class expression).

+

For example, in the formal definition of CL:0000392 (“crystal cell”):

+

Class: 'crystal cell' + SubClassOf: 'hemocyte' + SubClassOf: 'develops from' some 'procrystal cell'

+

The first SubClassOf axiom denotes an actual classification, whereas the second denotes a relationship.

+
+

Asserting classification

+

In order to keep the ontology maintainable, asserted classifications should be limited where possible. Ideally, all terms would have only a single asserted parent (also known as “superclass”). Given the presence of suitable logically defined classes (see below) and sufficient relationships for the term you are making, additional classification can be inferred automatically by reasoning.

+

However, as we are limited in what types of classification we are able to infer, you may need to assert multiple parents. Two asserted classifications are plainly acceptable. If you must assert three or more classifications, then you should make a note with a suggestion for which of the asserted classifications are good candidates for formalisation and inferred classification.

+

Logically defined classes

+

Logically defined classes are terms that have formal definitions that specify complete necessary and sufficient conditions for membership of the class. They can be used by a reasoner to auto-classify the ontology by searching for terms that fulfill these conditions.

+
+

Logically defined classes are variously referred to as “cross-products” (XPs), “genus and differentia definitions”, “equivalent classes”, or “intersections”. In OWL formalism, they are represented using EquivalentClasses axioms.

+
+

Generally, logical definitions follow the structure: “Any X that REL some Y [and REL some Z...]”, where X is the genus and each following clause (“that REL some Y”) is a differentium.

+
+

Strictly speaking, logical definitions can be arbitrarily complex. It is recommended, however, to stick as much as possible to “simple” logical definitions following the structure above. More complex definitions are harder to understand, and may involve OWL constructs that are legal but not fully supported by the available reasoners.

+
+

Good candidates for logically defined classes include classes whose only differentium is:

+
    +
  • what the cell is part of;
    • +
  • what it innervates (or, if more precisely known, what its axon(s) or dendrite(s) innervate);
    • +
  • its function.
    • +
+

Care must be taken not to use logically defined classes too liberally. You should be satisfied that all automatic classifications that result would make sense (or at least be justifiable) to a biologist and would cover most cases of usages with as few edge cases as possible.

+

If you are unsure whether a characteristic of a cell should be expressed as a relationship or as a differentium in a logically defined class: use a relationship.

+

The textual definition of a logically defined class should be a plain English equivalent of the logical definition. However, in some cases it can be useful to add more details in the “gloss” part when relevant.

+

Asserting relationships

+

Please refer to the relations guide for detailled guidelines about which relations to use for most cases.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/history/index.html b/history/index.html new file mode 100644 index 000000000..d230dc75d --- /dev/null +++ b/history/index.html @@ -0,0 +1,789 @@ + + + + + + + + + + + + + + + + + + + + + + History - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

A brief history of CL

+

The Cell Ontology (CL) represents canonical, natural biological cell types in a variety of animal species, with a focus on vertebrates. It works in concert with the Uberon anatomy ontology, and integrates seamlessly with the Gene Ontology (GO). It is part of the Open Biological and Biomedical Ontology (OBO) Foundry. As of April 2022, CL consists of over 2400 classes, and is used by key projects that involve single-cell transcriptomics, such as HuBMAP (HuBMAP Consortium 2019), the Human Cell Atlas (Regev et al. 2017), the EBI Single Cell Expression Atlas (Papatheodorou et al. 2020), and the Brain Data Standards Ontology (Tan et al. 2021).

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 000000000..acf4768aa --- /dev/null +++ b/index.html @@ -0,0 +1,924 @@ + + + + + + + + + + + + + + + + + + + + CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Getting started

+ +

The Cell Ontology

+

logo

+

The Cell Ontology (CL) is an OBO Foundry ontology covering the domain of canonical, natural biological cell types.

+

OLS

+

The recommended way to browse the ontology is through the ontology lookup service (OLS).

+

Accessing CL

+

The main release files can be found in our GitHub repository.

+

The main release file can also be found here.

+

Contributing

+

To see how you can contribute to the cell ontology, please visit our Contributing page.

+

Documentation

+

The documentations here is intended for maintainers and editors of the cell ontology.

+

For general guidance on editing OBO-(ish) ontologies, please see the obook.

+

Cell Ontology Website

+

You can also access our front facing website at https://cell-ontology.github.io/

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/merging-terms/index.html b/merging-terms/index.html new file mode 100644 index 000000000..7fcb1e80a --- /dev/null +++ b/merging-terms/index.html @@ -0,0 +1,805 @@ + + + + + + + + + + + + + + + + + + Merging Terms in CL - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Merging Terms in CL

+

How to merge terms

+

For general instructions on how to merge terms, please see this How-to Guide +In addition to the above, please add the annotation has_alternative_id on the winning term with the ID of the losing term.

+

Considerations on which should be the winning term

+
    +
  1. Check Usage by GO - This can be done by using AmiGO 2
    2. +
  2. Check Usage by other ontologies - This can be done by using Ontobee
    3. +
  3. Check Usage within CL (you can do this in Protege with the usage tab) - this should be lower priority as you can easily change this while obsoleting the "losing" term
    4. +
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/ContinuousIntegration/index.html b/odk-workflows/ContinuousIntegration/index.html new file mode 100644 index 000000000..2a2b6c0f3 --- /dev/null +++ b/odk-workflows/ContinuousIntegration/index.html @@ -0,0 +1,806 @@ + + + + + + + + + + + + + + + + + + + + + + Continuous Integration - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Introduction to Continuous Integration Workflows with ODK

+

Historically, most repos have been using Travis CI for continuous integration testing and building, but due to +runtime restrictions, we recently switched a lot of our repos to GitHub actions. You can set up your repo with CI by adding +this to your configuration file (src/ontology/cl-odk.yaml):

+
ci:
+  - github_actions
+
+

When updateing your repo, you will notice a new file being added: .github/workflows/qc.yml.

+

This file contains your CI logic, so if you need to change, or add anything, this is the place!

+

Alternatively, if your repo is in GitLab instead of GitHub, you can set up your repo with GitLab CI by adding +this to your configuration file (src/ontology/cl-odk.yaml):

+
ci:
+  - gitlab-ci
+
+

This will add a file called .gitlab-ci.yml in the root of your repo.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/EditorsWorkflow/index.html b/odk-workflows/EditorsWorkflow/index.html new file mode 100644 index 000000000..f3e7de483 --- /dev/null +++ b/odk-workflows/EditorsWorkflow/index.html @@ -0,0 +1,1112 @@ + + + + + + + + + + + + + + + + + + + + + + Editors Workflow - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Editors Workflow

+

The editors workflow is one of the formal workflows to ensure that the ontology is developed correctly according to ontology engineering principles. There are a few different editors workflows:

+
    +
  1. Local editing workflow: Editing the ontology in your local environment by hand, using tools such as Protégé, ROBOT templates or DOSDP patterns.
    2. +
  2. Completely automated data pipeline (GitHub Actions)
    3. +
  3. DROID workflow
    4. +
+

This document only covers the first editing workflow, but more will be added in the future

+

Local editing workflow

+

Workflow requirements:

+
    +
  • git
    • +
  • github
    • +
  • docker
    • +
  • editing tool of choice, e.g. Protégé, your favourite text editor, etc
    • +
+

1. Create issue

+

Ensure that there is a ticket on your issue tracker that describes the change you are about to make. While this seems optional, this is a very important part of the social contract of building an ontology - no change to the ontology should be performed without a good ticket, describing the motivation and nature of the intended change.

+

2. Update main branch

+

In your local environment (e.g. your laptop), make sure you are on the main (prev. master) branch and ensure that you have all the upstream changes, for example:

+
git checkout master
+git pull
+
+

3. Create feature branch

+

Create a new branch. Per convention, we try to use meaningful branch names such as: +- issue23removeprocess (where issue 23 is the related issue on GitHub) +- issue26addcontributor +- release20210101 (for releases)

+

On your command line, this looks like this:

+
git checkout -b issue23removeprocess
+
+

4. Perform edit

+

Using your editor of choice, perform the intended edit. For example:

+

Protégé

+
    +
  1. Open src/ontology/cl-edit.owl in Protégé
    2. +
  2. Make the change
    3. +
  3. Save the file
    4. +
+

TextEdit

+
    +
  1. Open src/ontology/cl-edit.owl in TextEdit (or Sublime, Atom, Vim, Nano)
    2. +
  2. Make the change
    3. +
  3. Save the file
    4. +
+

Consider the following when making the edit.

+
    +
  1. According to our development philosophy, the only places that should be manually edited are:
      +
    • src/ontology/cl-edit.owl
      • +
    • Any ROBOT templates you chose to use (the TSV files only)
      • +
    • Any DOSDP data tables you chose to use (the TSV files, and potentially the associated patterns)
      • +
    • components (anything in src/ontology/components), see here.
      • +
    +
    2. +
  2. Imports should not be edited (any edits will be flushed out with the next update). However, refreshing imports is a potentially breaking change - and is discussed elsewhere.
    3. +
  3. Changes should usually be small. Adding or changing 1 term is great. Adding or changing 10 related terms is ok. Adding or changing 100 or more terms at once should be considered very carefully.
    4. +
+

4. Check the Git diff

+

This step is very important. Rather than simply trusting your change had the intended effect, we should always use a git diff as a first pass for sanity checking.

+

In our experience, having a visual git client like GitHub Desktop or sourcetree is really helpful for this part. In case you prefer the command line:

+
git status
+git diff
+
+

5. Quality control

+

Now it's time to run your quality control checks. This can either happen locally (5a) or through your continuous integration system (7/5b).

+

5a. Local testing

+

If you chose to run your test locally:

+
sh run.sh make IMP=false test
+
+

This will run the whole set of configured ODK tests on including your change. If you have a complex DOSDP pattern pipeline you may want to add PAT=false to skip the potentially lengthy process of rebuilding the patterns.

+
sh run.sh make IMP=false PAT=false test
+
+

6. Pull request

+

When you are happy with the changes, you commit your changes to your feature branch, push them upstream (to GitHub) and create a pull request. For example:

+
git add NAMEOFCHANGEDFILES
+git commit -m "Added biological process term #12"
+git push -u origin issue23removeprocess
+
+

Then you go to your project on GitHub, and create a new pull request from the branch, for example: https://github.com/INCATools/ontology-development-kit/pulls

+

There is a lot of great advise on how to write pull requests, but at the very least you should: +- mention the tickets affected: see #23 to link to a related ticket, or fixes #23 if, by merging this pull request, the ticket is fixed. Tickets in the latter case will be closed automatically by GitHub when the pull request is merged. +- summarise the changes in a few sentences. Consider the reviewer: what would they want to know right away. +- If the diff is large, provide instructions on how to review the pull request best (sometimes, there are many changed files, but only one important change).

+

7/5b. Continuous Integration Testing

+

If you didn't run and local quality control checks (see 5a), you should have Continuous Integration (CI) set up, for example: +- Travis +- GitHub Actions

+

More on how to set this up here. Once the pull request is created, the CI will automatically trigger. If all is fine, it will show up green, otherwise red.

+

8. Community review

+

Once all the automatic tests have passed, it is important to put a second set of eyes on the pull request. Ontologies are inherently social - as in that they represent some kind of community consensus on how a domain is organised conceptually. This seems high brow talk, but it is very important that as an ontology editor, you have your work validated by the community you are trying to serve (e.g. your colleagues, other contributors etc.). In our experience, it is hard to get more than one review on a pull request - two is great. You can set up GitHub branch protection to actually require a review before a pull request can be merged! We recommend this.

+

This step seems daunting to some hopefully under-resourced ontologies, but we recommend to put this high up on your list of priorities - train a colleague, reach out!

+

9. Merge and cleanup

+

When the QC is green and the reviews are in (approvals), it is time to merge the pull request. After the pull request is merged, remember to delete the branch as well (this option will show up as a big button right after you have merged the pull request). If you have not done so, close all the associated tickets fixed by the pull request.

+

10. Changelog (Optional)

+

It is sometimes difficult to keep track of changes made to an ontology. Some ontology teams opt to document changes in a changelog (simply a text file in your repository) so that when release day comes, you know everything you have changed. This is advisable at least for major changes (such as a new release system, a new pattern or template etc.).

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/ManageAutomatedTest/index.html b/odk-workflows/ManageAutomatedTest/index.html new file mode 100644 index 000000000..7c474a8b1 --- /dev/null +++ b/odk-workflows/ManageAutomatedTest/index.html @@ -0,0 +1,827 @@ + + + + + + + + + + + + + + + + + + ManageAutomatedTest - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

ManageAutomatedTest

+ +

Constraint violation checks

+

We can define custom checks using SPARQL. SPARQL queries define bad modelling patterns (missing labels, misspelt URIs, and many more) in the ontology. If these queries return any results, then the build will fail. Custom checks are designed to be run as part of GitHub Actions Continuous Integration testing, but they can also run locally.

+

Steps to add a constraint violation check:

+
    +
  1. Add the SPARQL query in src/sparql. The name of the file should end with -violation.sparql. Please give a name that helps to understand which violation the query wants to check.
    2. +
  2. Add the name of the new file to odk configuration file src/ontology/uberon-odk.yaml:
      +
    1. Include the name of the file (without the -violation.sparql part) to the list inside the key custom_sparql_checks that is inside robot_report key.
      2. +
    2. +

      If the robot_report or custom_sparql_checks keys are not available, please add this code block to the end of the file.

      +

      yaml + robot_report: + release_reports: False + fail_on: ERROR + use_labels: False + custom_profile: True + report_on: + - edit + custom_sparql_checks: + - name-of-the-file-check +3. Update the repository so your new SPARQL check will be included in the QC.

      +
      3. +
    +
    3. +
+
sh run.sh make update_repo
+
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/ManageDocumentation/index.html b/odk-workflows/ManageDocumentation/index.html new file mode 100644 index 000000000..78eecb8c2 --- /dev/null +++ b/odk-workflows/ManageDocumentation/index.html @@ -0,0 +1,912 @@ + + + + + + + + + + + + + + + + + + + + + + Managing the documentation - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Updating the Documentation

+

The documentation for CL is managed in two places (relative to the repository root):

+
    +
  1. The docs directory contains all the files that pertain to the content of the documentation (more below)
    2. +
  2. the mkdocs.yaml file contains the documentation config, in particular its navigation bar and theme.
    3. +
+

The documentation is hosted using GitHub pages, on a special branch of the repository (called gh-pages). It is important that this branch is never deleted - it contains all the files GitHub pages needs to render and deploy the site. It is also important to note that the gh-pages branch should never be edited manually. All changes to the docs happen inside the docs directory on the main branch.

+

Editing the docs

+

Changing content

+

All the documentation is contained in the docs directory, and is managed in Markdown. Markdown is a very simple and convenient way to produce text documents with formatting instructions, and is very easy to learn - it is also used, for example, in GitHub issues. This is a normal editing workflow:

+
    +
  1. Open the .md file you want to change in an editor of choice (a simple text editor is often best). IMPORTANT: Do not edit any files in the docs/odk-workflows/ directory. These files are managed by the ODK system and will be overwritten when the repository is upgraded! If you wish to change these files, make an issue on the ODK issue tracker.
    2. +
  2. Perform the edit and save the file
    3. +
  3. Commit the file to a branch, and create a pull request as usual.
    4. +
  4. If your development team likes your changes, merge the docs into master branch.
    5. +
  5. Deploy the documentation (see below)
    6. +
+

Deploy the documentation

+

The documentation is not automatically updated from the Markdown, and needs to be deployed deliberately. To do this, perform the following steps:

+
    +
  1. In your terminal, navigate to the edit directory of your ontology, e.g.: + cd cl/src/ontology
    2. +
  2. Now you are ready to build the docs as follows: + sh run.sh make update_docs + Mkdocs now sets off to build the site from the markdown pages. You will be asked to
      +
    • Enter your username
      • +
    • Enter your password (see here for using GitHub access tokens instead) + IMPORTANT: Using password based authentication will be deprecated this year (2021). Make sure you read up on personal access tokens if that happens!
      • +
    +
    3. +
+

If everything was successful, you will see a message similar to this one:

+

INFO - Your documentation should shortly be available at: https://obophenotype.github.io/cell-ontology/ +3. Just to double check, you can now navigate to your documentation pages (usually https://obophenotype.github.io/cell-ontology/). + Just make sure you give GitHub 2-5 minutes to build the pages!

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/ReleaseWorkflow/index.html b/odk-workflows/ReleaseWorkflow/index.html new file mode 100644 index 000000000..43df696f1 --- /dev/null +++ b/odk-workflows/ReleaseWorkflow/index.html @@ -0,0 +1,998 @@ + + + + + + + + + + + + + + + + + + + + + + Release Workflow - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

The release workflow

+

The release workflow recommended by the ODK is based on GitHub releases and works as follows:

+
    +
  1. Run a release with the ODK
    2. +
  2. Review the release
    3. +
  3. Merge to main branch
    4. +
  4. Create a GitHub release
    5. +
+

These steps are outlined in detail in the following.

+

Run a release with the ODK

+

Preparation:

+
    +
  1. Ensure that all your pull requests are merged into your main (master) branch
    2. +
  2. Make sure that all changes to master are committed to GitHub (git status should say that there are no modified files)
    3. +
  3. Locally make sure you have the latest changes from master (git pull)
    4. +
  4. Checkout a new branch (e.g. git checkout -b release-2021-01-01)
    5. +
  5. You may or may not want to refresh your imports as part of your release strategy (see here)
    6. +
  6. Make sure you have the latest ODK installed by running docker pull obolibrary/odkfull
    7. +
+

To actually run the release, you:

+
    +
  1. Open a command line terminal window and navigate to the src/ontology directory (cd cl/src/ontology)
    2. +
  2. Run release pipeline:sh run.sh make prepare_release -B. Note that for some ontologies, this process can take up to 90 minutes - especially if there are large ontologies you depend on, like PRO or CHEBI.
    3. +
  3. If everything went well, you should see the following output on your machine: Release files are now in ../.. - now you should commit, push and make a release on your git hosting site such as GitHub or GitLab.
    4. +
+

This will create all the specified release targets (OBO, OWL, JSON, and the variants, ont-full and ont-base) and copy them into your release directory (the top level of your repo).

+

Review the release

+
    +
  1. (Optional) Rough check. This step is frequently skipped, but for the more paranoid among us (like the author of this doc), this is a 3 minute additional effort for some peace of mind. Open the main release (cl.owl) in you favourite development environment (i.e. Protégé) and eyeball the hierarchy. We recommend two simple checks:
      +
    1. Does the very top level of the hierarchy look ok? This means that all new terms have been imported/updated correctly.
      2. +
    2. Does at least one change that you know should be in this release appear? For example, a new class. This means that the release was actually based on the recent edit file.
      3. +
    +
    2. +
  2. Commit your changes to the branch and make a pull request
    3. +
  3. In your GitHub pull request, review the following three files in detail (based on our experience):
      +
    1. cl.obo - this reflects a useful subset of the whole ontology (everything that can be covered by OBO format). OBO format has that speaking for it: it is very easy to review!
      2. +
    2. cl-base.owl - this reflects the asserted axioms in your ontology that you have actually edited.
      3. +
    3. Ideally also take a look at cl-full.owl, which may reveal interesting new inferences you did not know about. Note that the diff of this file is sometimes quite large.
      4. +
    +
    4. +
  4. Like with every pull request, we recommend to always employ a second set of eyes when reviewing a PR!
    5. +
+

Merge the main branch

+

Once your CI checks have passed, and your reviews are completed, you can now merge the branch into your main branch (don't forget to delete the branch afterwards - a big button will appear after the merge is finished).

+

Create a GitHub release

+
    +
  1. Go to your releases page on GitHub by navigating to your repository, and then clicking on releases (usually on the right, for example: https://github.com/obophenotype/cell-ontology/releases). Then click "Draft new release"
    2. +
  2. As the tag version you need to choose the date on which your ontologies were build. You can find this, for example, by looking at the cl.obo file and check the data-version: property. The date needs to be prefixed with a v, so, for example v2020-02-06.
    3. +
  3. You can write whatever you want in the release title, but we typically write the date again. The description underneath should contain a concise list of changes or term additions.
    4. +
  4. Click "Publish release". Done.
    5. +
+

Debugging typical ontology release problems

+

Problems with memory

+

When you are dealing with large ontologies, you need a lot of memory. When you see error messages relating to large ontologies such as CHEBI, PRO, NCBITAXON, or Uberon, you should think of memory first, see here.

+

Problems when using OBO format based tools

+

Sometimes you will get cryptic error messages when using legacy tools using OBO format, such as the ontology release tool (OORT), which is also available as part of the ODK docker container. In these cases, you need to track down what axiom or annotation actually caused the breakdown. In our experience (in about 60% of the cases) the problem lies with duplicate annotations (def, comment) which are illegal in OBO. Here is an example recipe of how to deal with such a problem:

+
    +
  1. If you get a message like make: *** [cl.Makefile:84: oort] Error 255 you might have a OORT error.
    2. +
  2. To debug this, in your terminal enter sh run.sh make IMP=false PAT=false oort -B (assuming you are already in the ontology folder in your directory)
    3. +
  3. This should show you where the error is in the log (eg multiple different definitions) +WARNING: THE FIX BELOW IS NOT IDEAL, YOU SHOULD ALWAYS TRY TO FIX UPSTREAM IF POSSIBLE
    4. +
  4. Open cl-edit.owl in Protégé and find the offending term and delete all offending issue (e.g. delete ALL definition, if the problem was "multiple def tags not allowed") and save. +*While this is not idea, as it will remove all definitions from that term, it will be added back again when the term is fixed in the ontology it was imported from and added back in.
    5. +
  5. Rerun sh run.sh make IMP=false PAT=false oort -B and if it all passes, commit your changes to a branch and make a pull request as usual.
    6. +
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/RepoManagement/index.html b/odk-workflows/RepoManagement/index.html new file mode 100644 index 000000000..b87099da2 --- /dev/null +++ b/odk-workflows/RepoManagement/index.html @@ -0,0 +1,1115 @@ + + + + + + + + + + + + + + + + + + + + + + Manage your ODK Repository - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Managing your ODK repository

+

Updating your ODK repository

+

Your ODK repositories configuration is managed in src/ontology/cl-odk.yaml. Once you have made your changes, you can run the following to apply your changes to the repository:

+
sh run.sh make update_repo
+
+

There are a large number of options that can be set to configure your ODK, but we will only discuss a few of them here.

+

NOTE for Windows users:

+

You may get a cryptic failure such as Set Illegal Option - if the update script located in src/scripts/update_repo.sh +was saved using Windows Line endings. These need to change to unix line endings. In Notepad++, for example, you can +click on Edit->EOL Conversion->Unix LF to change this.

+

Managing imports

+

You can use the update repository workflow described on this page to perform the following operations to your imports:

+
    +
  1. Add a new import
    2. +
  2. Modify an existing import
    3. +
  3. Remove an import you no longer want
    4. +
  4. Customise an import
    5. +
+

We will discuss all these workflows in the following.

+

Add new import

+

To add a new import, you first edit your odk config as described above, adding an id to the product list in the import_group section (for the sake of this example, we assume you already import RO, and your goal is to also import GO):

+
import_group:
+  products:
+    - id: ro
+    - id: go
+
+

Note: our ODK file should only have one import_group which can contain multiple imports (in the products section). Next, you run the update repo workflow to apply these changes. Note that by default, this module is going to be a SLME Bottom module, see here. To change that or customise your module, see section "Customise an import". To finalise the addition of your import, perform the following steps:

+
    +
  1. Add an import statement to your src/ontology/cl-edit.owl file. We suggest to do this using a text editor, by simply copying an existing import declaration and renaming it to the new ontology import, for example as follows: + ... + Ontology(<http://purl.obolibrary.org/obo/cl.owl> + Import(<http://purl.obolibrary.org/obo/cl/imports/ro_import.owl>) + Import(<http://purl.obolibrary.org/obo/cl/imports/go_import.owl>) + ...
    2. +
  2. Add your imports redirect to your catalog file src/ontology/catalog-v001.xml, for example: + <uri name="http://purl.obolibrary.org/obo/cl/imports/go_import.owl" uri="imports/go_import.owl"/>
    3. +
  3. Test whether everything is in order:
      +
    1. Refresh your import
      2. +
    2. Open in your Ontology Editor of choice (Protege) and ensure that the expected terms are imported.
      3. +
    +
    4. +
+

Note: The catalog file src/ontology/catalog-v001.xml has one purpose: redirecting +imports from URLs to local files. For example, if you have

+
Import(<http://purl.obolibrary.org/obo/cl/imports/go_import.owl>)
+
+

in your editors file (the ontology) and

+
<uri name="http://purl.obolibrary.org/obo/cl/imports/go_import.owl" uri="imports/go_import.owl"/>
+
+

in your catalog, tools like robot or Protégé will recognize the statement +in the catalog file to redirect the URL http://purl.obolibrary.org/obo/cl/imports/go_import.owl +to the local file imports/go_import.owl (which is in your src/ontology directory).

+

Modify an existing import

+

If you simply wish to refresh your import in light of new terms, see here. If you wish to change the type of your module see section "Customise an import".

+

Remove an existing import

+

To remove an existing import, perform the following steps:

+
    +
  1. remove the import declaration from your src/ontology/cl-edit.owl.
    2. +
  2. remove the id from your src/ontology/cl-odk.yaml, eg. - id: go from the list of products in the import_group.
    3. +
  3. run update repo workflow
    4. +
  4. delete the associated files manually:
      +
    • src/imports/go_import.owl
      • +
    • src/imports/go_terms.txt
      • +
    +
    5. +
  5. Remove the respective entry from the src/ontology/catalog-v001.xml file.
    6. +
+

Customise an import

+

By default, an import module extracted from a source ontology will be a SLME module, see here. There are various options to change the default.

+

The following change to your repo config (src/ontology/cl-odk.yaml) will switch the go import from an SLME module to a simple ROBOT filter module:

+
import_group:
+  products:
+    - id: ro
+    - id: go
+      module_type: filter
+
+

A ROBOT filter module is, essentially, importing all external terms declared by your ontology (see here on how to declare external terms to be imported). Note that the filter module does +not consider terms/annotations from namespaces other than the base-namespace of the ontology itself. For example, in the +example of GO above, only annotations / axioms related to the GO base IRI (http://purl.obolibrary.org/obo/GO_) would be considered. This +behaviour can be changed by adding additional base IRIs as follows:

+
import_group:
+  products:
+    - id: go
+      module_type: filter
+      base_iris:
+        - http://purl.obolibrary.org/obo/GO_
+        - http://purl.obolibrary.org/obo/CL_
+        - http://purl.obolibrary.org/obo/BFO
+
+

If you wish to customise your import entirely, you can specify your own ROBOT command to do so. To do that, add the following to your repo config (src/ontology/cl-odk.yaml):

+
import_group:
+  products:
+    - id: ro
+    - id: go
+      module_type: custom
+
+

Now add a new goal in your custom Makefile (src/ontology/cl.Makefile, not src/ontology/Makefile).

+
imports/go_import.owl: mirror/ro.owl imports/ro_terms_combined.txt
+    if [ $(IMP) = true ]; then $(ROBOT) query  -i $< --update ../sparql/preprocess-module.ru \
+        extract -T imports/ro_terms_combined.txt --force true --individuals exclude --method BOT \
+        query --update ../sparql/inject-subset-declaration.ru --update ../sparql/postprocess-module.ru \
+        annotate --ontology-iri $(ONTBASE)/$@ $(ANNOTATE_ONTOLOGY_VERSION) --output $@.tmp.owl && mv $@.tmp.owl $@; fi
+
+

Now feel free to change this goal to do whatever you wish it to do! It probably makes some sense (albeit not being a strict necessity), to leave most of the goal instead and replace only:

+
extract -T imports/ro_terms_combined.txt --force true --individuals exclude --method BOT \
+
+

to another ROBOT pipeline.

+

Add a component

+

A component is an import which belongs to your ontology, e.g. is managed by +you and your team.

+
    +
  1. Open src/ontology/cl-odk.yaml
    2. +
  2. If you dont have it yet, add a new top level section components
    3. +
  3. Under the components section, add a new section called products. +This is where all your components are specified
    4. +
  4. Under the products section, add a new component, e.g. - filename: mycomp.owl
    5. +
+

Example

+
components:
+  products:
+    - filename: mycomp.owl
+
+

When running sh run.sh make update_repo, a new file src/ontology/components/mycomp.owl will +be created which you can edit as you see fit. Typical ways to edit:

+
    +
  1. Using a ROBOT template to generate the component (see below)
    2. +
  2. Manually curating the component separately with Protégé or any other editor
    3. +
  3. Providing a components/mycomp.owl: make target in src/ontology/cl.Makefile +and provide a custom command to generate the component
      +
    • WARNING: Note that the custom rule to generate the component MUST NOT depend on any other ODK-generated file such as seed files and the like (see issue).
      • +
    +
    4. +
  4. Providing an additional attribute for the component in src/ontology/cl-odk.yaml, source, +to specify that this component should simply be downloaded from somewhere on the web.
    5. +
+

Adding a new component based on a ROBOT template

+

Since ODK 1.3.2, it is possible to simply link a ROBOT template to a component without having to specify any of the import logic. In order to add a new component that is connected to one or more template files, follow these steps:

+
    +
  1. Open src/ontology/cl-odk.yaml.
    2. +
  2. Make sure that use_templates: TRUE is set in the global project options. You should also make sure that use_context: TRUE is set in case you are using prefixes in your templates that are not known to robot, such as OMOP:, CPONT: and more. All non-standard prefixes you are using should be added to config/context.json.
    3. +
  3. Add another component to the products section.
    4. +
  4. To activate this component to be template-driven, simply say: use_template: TRUE. This will create an empty template for you in the templates directory, which will automatically be processed when recreating the component (e.g. run.bat make recreate-mycomp).
    5. +
  5. If you want to use more than one component, use the templates field to add as many template names as you wish. ODK will look for them in the src/templates directory.
    6. +
  6. Advanced: If you want to provide additional processing options, you can use the template_options field. This should be a string with option from robot template. One typical example for additional options you may want to provide is --add-prefixes config/context.json to ensure the prefix map of your context is provided to robot, see above.
    7. +
+

Example:

+
components:
+  products:
+    - filename: mycomp.owl
+      use_template: TRUE
+      template_options: --add-prefixes config/context.json
+      templates:
+        - template1.tsv
+        - template2.tsv
+
+

Note: if your mirror is particularly large and complex, read this ODK recommendation.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/RepositoryFileStructure/index.html b/odk-workflows/RepositoryFileStructure/index.html new file mode 100644 index 000000000..8d52faadf --- /dev/null +++ b/odk-workflows/RepositoryFileStructure/index.html @@ -0,0 +1,961 @@ + + + + + + + + + + + + + + + + + + + + + + Your ODK Repository Overview - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Repository structure

+

The main kinds of files in the repository:

+
    +
  1. Release files
    2. +
  2. Imports
    3. +
  3. Components
    4. +
+

Release files

+

Release file are the file that are considered part of the official ontology release and to be used by the community. A detailed description of the release artefacts can be found here.

+

Imports

+

Imports are subsets of external ontologies that contain terms and axioms you would like to re-use in your ontology. These are considered "external", like dependencies in software development, and are not included in your "base" product, which is the release artefact which contains only those axioms that you personally maintain.

+

These are the current imports in CL

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ImportURLType
prhttps://raw.githubusercontent.com/obophenotype/pro_obo_slim/master/pr_slim.owlNone
gohttp://purl.obolibrary.org/obo/go.owlNone
uberonhttp://purl.obolibrary.org/obo/uberon.owlNone
rohttp://purl.obolibrary.org/obo/ro.owlNone
patohttp://purl.obolibrary.org/obo/pato.owlNone
ncbitaxonhttp://purl.obolibrary.org/obo/ncbitaxon/subsets/taxslim.owlNone
omohttp://purl.obolibrary.org/obo/omo.owlmirror
+

Components

+

Components, in contrast to imports, are considered full members of the ontology. This means that any axiom in a component is also included in the ontology base - which means it is considered native to the ontology. While this sounds complicated, consider this: conceptually, no component should be part of more than one ontology. If that seems to be the case, we are most likely talking about an import. Components are often not needed for ontologies, but there are some use cases:

+
    +
  1. There is an automated process that generates and re-generates a part of the ontology
    2. +
  2. A part of the ontology is managed in ROBOT templates
    3. +
  3. The expressivity of the component is higher than the format of the edit file. For example, people still choose to manage their ontology in OBO format (they should not) missing out on a lot of owl features. They may choose to manage logic that is beyond OBO in a specific OWL component.
    4. +
+

These are the components in CL

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FilenameURL
hra_subset.owlNone
mappings.owlNone
blood_and_immune_upper_slim.owlNone
eye_upper_slim.owlNone
general_cell_types_upper_slim.owlNone
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/SettingUpDockerForODK/index.html b/odk-workflows/SettingUpDockerForODK/index.html new file mode 100644 index 000000000..8b007701c --- /dev/null +++ b/odk-workflows/SettingUpDockerForODK/index.html @@ -0,0 +1,801 @@ + + + + + + + + + + + + + + + + + + + + + + Setting up Docker for ODK - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Setting up your Docker environment for ODK use

+

One of the most frequent problems with running the ODK for the first time is failure because of lack of memory. This can look like a Java OutOfMemory exception, +but more often than not it will appear as something like an Error 137. There are two places you need to consider to set your memory:

+
    +
  1. Your src/ontology/run.sh (or run.bat) file. You can set the memory in there by adding +robot_java_args: '-Xmx8G' to your src/ontology/cl-odk.yaml file, see for example here.
    2. +
  2. Set your docker memory. By default, it should be about 10-20% more than your robot_java_args variable. You can manage your memory settings +by right-clicking on the docker whale in your system bar-->Preferences-->Resources-->Advanced, see picture below.
    3. +
+

dockermemory

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/UpdateImports/index.html b/odk-workflows/UpdateImports/index.html new file mode 100644 index 000000000..9f46e6197 --- /dev/null +++ b/odk-workflows/UpdateImports/index.html @@ -0,0 +1,1055 @@ + + + + + + + + + + + + + + + + + + + + + + Imports management - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Update Imports Workflow

+

This page discusses how to update the contents of your imports, like adding or removing terms. If you are looking to customise imports, like changing the module type, see here.

+

Importing a new term

+

Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title "Using the Base Module approach".

+

Importing a new term is split into two sub-phases:

+
    +
  1. Declaring the terms to be imported
    2. +
  2. Refreshing imports dynamically
    3. +
+

Declaring terms to be imported

+

There are three ways to declare terms that are to be imported from an external ontology. Choose the appropriate one for your particular scenario (all three can be used in parallel if need be):

+
    +
  1. Protégé-based declaration
    2. +
  2. Using term files
    3. +
  3. Using the custom import template
    4. +
+

Protégé-based declaration

+

This workflow is to be avoided, but may be appropriate if the editor does not have access to the ODK docker container. +This approach also applies to ontologies that use base module import approach.

+
    +
  1. Open your ontology (edit file) in Protégé (5.5+).
    2. +
  2. Select 'owl:Thing'
    3. +
  3. Add a new class as usual.
    4. +
  4. Paste the full iri in the 'Name:' field, for example, http://purl.obolibrary.org/obo/CHEBI_50906.
    5. +
  5. Click 'OK'
    6. +
+

Adding Classes

+

Now you can use this term for example to construct logical definitions. The next time the imports are refreshed (see how to refresh here), the metadata (labels, definitions, etc.) for this term are imported from the respective external source ontology and becomes visible in your ontology.

+

Using term files

+

Every import has, by default a term file associated with it, which can be found in the imports directory. For example, if you have a GO import in src/ontology/go_import.owl, you will also have an associated term file src/ontology/go_terms.txt. You can add terms in there simply as a list:

+
GO:0008150
+GO:0008151
+
+

Now you can run the refresh imports workflow) and the two terms will be imported.

+

Using the custom import template

+

This workflow is appropriate if:

+
    +
  1. You prefer to manage all your imported terms in a single file (rather than multiple files like in the "Using term files" workflow above).
    2. +
  2. You wish to augment your imported ontologies with additional information. This requires a cautionary discussion.
    3. +
+

To enable this workflow, you add the following to your ODK config file (src/ontology/cl-odk.yaml), and update the repository:

+
use_custom_import_module: TRUE
+
+

Now you can manage your imported terms directly in the custom external terms template, which is located at src/templates/external_import.owl. Note that this file is a ROBOT template, and can, in principle, be extended to include any axioms you like. Before extending the template, however, read the following carefully.

+

The main purpose of the custom import template is to enable the management off all terms to be imported in a centralised place. To enable that, you do not have to do anything other than maintaining the template. So if you, say currently import APOLLO_SV:00000480, and you wish to import APOLLO_SV:00000532, you simply add a row like this:

+
ID  Entity Type
+ID  TYPE
+APOLLO_SV:00000480  owl:Class
+APOLLO_SV:00000532  owl:Class
+
+

When the imports are refreshed see imports refresh workflow, the term(s) will simply be imported from the configured ontologies.

+

Now, if you wish to extend the Makefile (which is beyond these instructions) and add, say, synonyms to the imported terms, you can do that, but you need to (a) preserve the ID and ENTITY columns and (b) ensure that the ROBOT template is valid otherwise, see here.

+

WARNING. Note that doing this is a widespread antipattern (see related issue). You should not change the axioms of terms that do not belong into your ontology unless necessary - such changes should always be pushed into the ontology where they belong. However, since people are doing it, whether the OBO Foundry likes it or not, at least using the custom imports module as described here localises the changes to a single simple template and ensures that none of the annotations added this way are merged into the base file.

+

Refresh imports

+

If you want to refresh the import yourself (this may be necessary to pass the travis tests), and you have the ODK installed, you can do the following (using go as an example):

+

First, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory).

+
cd src/ontology
+
+

Then, you regenerate the import that will now include any new terms you have added. Note: You must have docker installed.

+
sh run.sh make PAT=false imports/go_import.owl -B
+
+

Since ODK 1.2.27, it is also possible to simply run the following, which is the same as the above:

+
sh run.sh make refresh-go
+
+

Note that in case you changed the defaults, you need to add IMP=true and/or MIR=true to the command below:

+
sh run.sh make IMP=true MIR=true PAT=false imports/go_import.owl -B
+
+

If you wish to skip refreshing the mirror, i.e. skip downloading the latest version of the source ontology for your import (e.g. go.owl for your go import) you can set MIR=false instead, which will do the exact same thing as the above, but is easier to remember:

+
sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B
+
+

Using the Base Module approach

+

Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files. +The idea is to only import axioms from ontologies that actually belong to it. +A base file is a subset of the ontology that only contains those axioms that nominally +belong there. In other words, the base file does not contain any axioms that belong +to another ontology. An example would be this:

+

Imagine this being the full Uberon ontology:

+
Axiom 1: BFO:123 SubClassOf BFO:124
+Axiom 1: UBERON:123 SubClassOf BFO:123
+Axiom 1: UBERON:124 SubClassOf UBERON 123
+
+

The base file is the set of all axioms that are about UBERON terms:

+
Axiom 1: UBERON:123 SubClassOf BFO:123
+Axiom 1: UBERON:124 SubClassOf UBERON 123
+
+

I.e.

+
Axiom 1: BFO:123 SubClassOf BFO:124
+
+

Gets removed.

+

The base file pipeline is a bit more complex than the normal pipelines, because +of the logical interactions between the imported ontologies. This is solved by _first +merging all mirrors into one huge file and then extracting one mega module from it.

+

Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies. +When we use the base pipelines, we

+

1) First obtain the base (usually by simply downloading it, but there is also an option now to create it with ROBOT) +2) We merge all base files into one big pile +3) Then we extract a single module imports/merged_import.owl

+

The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml.

+

To check if your ontology uses this method, check src/ontology/cl-odk.yaml to see if use_base_merging: TRUE is declared under import_group

+

If your ontology uses Base Module approach, please use the following steps:

+

First, add the term to be imported to the term file associated with it (see above "Using term files" section if this is not clear to you)

+

Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory).

+
cd src/ontology
+
+

Then refresh imports by running

+
sh run.sh make imports/merged_import.owl
+
+

Note: if your mirrors are updated, you can run sh run.sh make no-mirror-refresh-merged

+

This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and one of the local devs should pick this up and run the import for you.

+

Lastly, restart Protégé, and the term should be imported in ready to be used.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/components/index.html b/odk-workflows/components/index.html new file mode 100644 index 000000000..b7c29771f --- /dev/null +++ b/odk-workflows/components/index.html @@ -0,0 +1,800 @@ + + + + + + + + + + + + + + + + + + Adding components to an ODK repo - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Adding components to an ODK repo

+

For details on what components are, please see component section of repository file structure document.

+

To add custom components to an ODK repo, please follow the following steps:

+

1) Locate your odk yaml file and open it with your favourite text editor (src/ontology/cl-odk.yaml) +2) Search if there is already a component section to the yaml file, if not add it accordingly, adding the name of your component:

+
components:
+  products:
+    - filename: your-component-name.owl
+
+

3) Add the component to your catalog file (src/ontology/catalog-v001.xml)

+
  <uri name="http://purl.obolibrary.org/obo/cl/components/your-component-name.owl" uri="components/your-component-name.owl"/>
+
+

4) Add the component to the edit file (src/ontology/cl-edit.obo) +for .obo formats:

+
import: http://purl.obolibrary.org/obo/cl/components/your-component-name.owl
+
+

for .owl formats:

+
Import(<http://purl.obolibrary.org/obo/cl/components/your-component-name.owl>)
+
+

5) Refresh your repo by running sh run.sh make update_repo - this should create a new file in src/ontology/components. +6) In your custom makefile (src/ontology/cl.Makefile) add a goal for your custom make file. In this example, the goal is a ROBOT template.

+
$(COMPONENTSDIR)/your-component-name.owl: $(SRC) ../templates/your-component-template.tsv 
+    $(ROBOT) template --template ../templates/your-component-template.tsv \
+  annotate --ontology-iri $(ONTBASE)/$@ --output $(COMPONENTSDIR)/your-component-name.owl
+
+

(If using a ROBOT template, do not forget to add your template tsv in src/templates/)

+

7) Make the file by running sh run.sh make components/your-component-name.owl

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/odk-workflows/index.html b/odk-workflows/index.html new file mode 100644 index 000000000..01b427e82 --- /dev/null +++ b/odk-workflows/index.html @@ -0,0 +1,801 @@ + + + + + + + + + + + + + + + + + + + + + + Overview - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+ +
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/patterns/cellBearerOfQuality/index.html b/patterns/cellBearerOfQuality/index.html new file mode 100644 index 000000000..d47b57cc5 --- /dev/null +++ b/patterns/cellBearerOfQuality/index.html @@ -0,0 +1,994 @@ + + + + + + + + + + + + + + + + + + + + + + Cell, bearer of quality - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

cellBearerOfQuality

+

http://purl.obolibrary.org/obo/cl/cellBearerOfQuality

+

Description

+

A cell that has a specific quality, such as binucleate.

+

Contributors

+ +

Name

+

{quality} {cell}

+

Annotations

+ +

Definition

+

Any {cell} that is {quality}

+

Equivalent to

+

{cell} and (bearer of some {quality})

+

Data preview

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
defined_classdefined_class_labelcellcell_labelqualityquality_label
CL:0001061abnormal cellCL:0000000cellPATO:0000460abnormal
CL:0000225anucleate cellCL:0000003native cellPATO:0001405anucleate
CL:0000227binucleate cellCL:0000003native cellPATO:0001406binucleate
CL:0000103bipolar neuronCL:0000099interneuronPATO:0070006bipolar morphology
CL:4023077bitufted neuronCL:0000099interneuronPATO:0070012bitufted cell morphology
+

See full table here

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/patterns/cellCapableOfBiologicalProcess/index.html b/patterns/cellCapableOfBiologicalProcess/index.html new file mode 100644 index 000000000..3827422af --- /dev/null +++ b/patterns/cellCapableOfBiologicalProcess/index.html @@ -0,0 +1,957 @@ + + + + + + + + + + + + + + + + + + + + + + Cell, capable of Biological Process - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

cellCapableOfBiologicalProcess

+

http://purl.obolibrary.org/obo/cl/cellCapableOfBiologicalProcess

+

Description

+

Any cell that is involved in/capable of a particular biological process, such as acid secretion.

+

Name

+

{biological_process} {cell}

+

Definition

+

A {cell} that is capable of {biological_process}.

+

Equivalent to

+

{cell} and (capable of some {biological_process})

+

Data preview

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
defined_classdefined_class_labelbiological_processbiological_process_labelcellcell_label
CL:0000236B cellGO:0019724B cell mediated immunityCL:0000945lymphocyte of B lineage
CL:0000492CD4-positive helper T cellGO:0001816cytokine productionCL:0000624CD4-positive, alpha-beta T cell
CL:0000795CD8-positive, alpha-beta regulatory T cellGO:0050777negative regulation of immune responseCL:0000625CD8-positive, alpha-beta T cell
CL:0011005GABAergic interneuronGO:0061534gamma-aminobutyric acid secretion, neurotransmissionCL:0000099interneuron
CL:0000617GABAergic neuronGO:0061534gamma-aminobutyric acid secretion, neurotransmissionCL:0000540neuron
+

See full table here

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/patterns/cellHasPlasmaMembranePartX/index.html b/patterns/cellHasPlasmaMembranePartX/index.html new file mode 100644 index 000000000..b12a69263 --- /dev/null +++ b/patterns/cellHasPlasmaMembranePartX/index.html @@ -0,0 +1,924 @@ + + + + + + + + + + + + + + + + + + + + + + Cell, has plasme membrane - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

cellHasPlasmaMembranePartX

+

http://purl.obolibrary.org/obo/cl/cellHasPlasmaMembranePartX

+

Description

+

A cell type that is characterized by a plasma membrane part, such as a cilium or receptor. Note - that this is only good for cells defined by a single plasma membrane receptor.

+

Contributors

+ +

Name

+

{cell} {plasma_membrane}

+

Annotations

+ +

Definition

+

A {cell} that has a {plasma_membrane}.

+

Equivalent to

+

{cell} and (has plasma membrane part some {plasma_membrane})

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/patterns/cellPartOfAnatomicalEntity/index.html b/patterns/cellPartOfAnatomicalEntity/index.html new file mode 100644 index 000000000..6fcc286a3 --- /dev/null +++ b/patterns/cellPartOfAnatomicalEntity/index.html @@ -0,0 +1,976 @@ + + + + + + + + + + + + + + + + + + + + + + Cell, part of anatomical entity - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

cellPartOfAnatomicalEntity

+

http://purl.obolibrary.org/obo/cl/cellPartOfAnatomicalEntity

+

Description

+

A cell that is part of an anatomical entity.

+

Contributors

+ +

Name

+

{cell} {anatomical_entity}

+

Definition

+

Any {cell} that is part of a {anatomical_entity}.

+

Equivalent to

+

{cell} and (part of some {anatomical_entity})

+

Data preview

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
defined_classdefined_class_labelanatomical_entityanatomical_entity_labelcellcell_label
CL:0009032B cell of appendixUBERON:0001154vermiform appendixCL:0000236B cell
CL:0009045B cell of medullary sinus of lymph nodeUBERON:0009744lymph node medullary sinusCL:0000236B cell
CL:0010007His-Purkinje system cellUBERON:0004146His-Purkinje systemCL:0000003native cell
CL:0002680PP cell of intestineUBERON:0000160intestineCL:0000696PP cell
CL:0009015Peyer's patch follicular dendritic cellUBERON:0001211Peyer's patchCL:0000442follicular dendritic cell
+

See full table here

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/patterns/overview/index.html b/patterns/overview/index.html new file mode 100644 index 000000000..982acedfc --- /dev/null +++ b/patterns/overview/index.html @@ -0,0 +1,1346 @@ + + + + + + + + + + + + + + + + + + + + + + Design Patterns Overview - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Pattern directory

+

This is a listing of all the patterns hosted as part of this directory

+

Patterns in

+

Cell bearer of quality

+

A cell that has a specific quality, such as binucleate.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AttributeInfo
IRIhttp://purl.obolibrary.org/obo/cl/cellBearerOfQuality
NamecellBearerOfQuality
ClassesCL:0000000, PATO:0000001,
Variablescell (CL:0000000), quality (PATO:0000001),
Contributors0000-0001-5208-3432, 0000-0002-6601-2165,
Examplesmondo
+

Data preview:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
defined:classdefined:class:labelcellcell:labelqualityquality:label
CL:0001061abnormal cellCL:0000000cellPATO:0000460abnormal
CL:0000225anucleate cellCL:0000003native cellPATO:0001405anucleate
CL:0000227binucleate cellCL:0000003native cellPATO:0001406binucleate
CL:0000103bipolar neuronCL:0000099interneuronPATO:0070006bipolar morphology
CL:4023077bitufted neuronCL:0000099interneuronPATO:0070012bitufted cell morphology
+

See full table here

+

Cell capable of biological process

+

Any cell that is involved in/capable of a particular biological process, such as acid secretion.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AttributeInfo
IRIhttp://purl.obolibrary.org/obo/cl/cellCapableOfBiologicalProcess
NamecellCapableOfBiologicalProcess
ClassesCL:0000000, GO:0008150,
Variablescell (CL:0000000), biological_process (GO:0008150),
Contributors
Examplesmondo
+

Data preview:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
defined:classdefined:class:labelbiological:processbiological:process:labelcellcell:label
CL:0000236B cellGO:0019724B cell mediated immunityCL:0000945lymphocyte of B lineage
CL:0000492CD4-positive helper T cellGO:0001816cytokine productionCL:0000624CD4-positive, alpha-beta T cell
CL:0000795CD8-positive, alpha-beta regulatory T cellGO:0050777negative regulation of immune responseCL:0000625CD8-positive, alpha-beta T cell
CL:0011005GABAergic interneuronGO:0061534gamma-aminobutyric acid secretion, neurotransmissionCL:0000099interneuron
CL:0000617GABAergic neuronGO:0061534gamma-aminobutyric acid secretion, neurotransmissionCL:0000540neuron
+

See full table here

+

Cell has plasma membrane part x

+

A cell type that is characterized by a plasma membrane part, such as a cilium or receptor. Note - that this is only good for cells defined by a single plasma membrane receptor.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AttributeInfo
IRIhttp://purl.obolibrary.org/obo/cl/cellHasPlasmaMembranePartX
NamecellHasPlasmaMembranePartX
ClassesCL:0000000, CL:0000003, GO:0005886,
Variablescell (CL:0000003), plasma_membrane (GO:0005886),
Contributors0000-0001-5208-3432, 0000-0002-6601-2165,
Examples
+

Cell part of anatomical entity

+

A cell that is part of an anatomical entity.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AttributeInfo
IRIhttp://purl.obolibrary.org/obo/cl/cellPartOfAnatomicalEntity
NamecellPartOfAnatomicalEntity
ClassesCL:0000000, UBERON:0001062,
Variablescell (CL:0000000), anatomical_entity (UBERON:0001062),
Contributors0000-0001-5208-3432, 0000-0002-6601-2165,
Examplesmondo
+

Data preview:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
defined:classdefined:class:labelanatomical:entityanatomical:entity:labelcellcell:label
CL:0009032B cell of appendixUBERON:0001154vermiform appendixCL:0000236B cell
CL:0009045B cell of medullary sinus of lymph nodeUBERON:0009744lymph node medullary sinusCL:0000236B cell
CL:0010007His-Purkinje system cellUBERON:0004146His-Purkinje systemCL:0000003native cell
CL:0002680PP cell of intestineUBERON:0000160intestineCL:0000696PP cell
CL:0009015Peyer's patch follicular dendritic cellUBERON:0001211Peyer's patchCL:0000442follicular dendritic cell
+

See full table here

+

Taxon specific

+

A cell that is restricted to a specific taxon, such as CL:0001200 'lymphocyte of B lineage, CD19-positive' are only in mammals. Note - this is not to be used for any cell that is restricted to a taxon, this is for taxon-specific subclasses of existing cell types. This should hardly ever be used.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
AttributeInfo
IRIhttp://purl.obolibrary.org/obo/cl/taxonSpecific
NametaxonSpecific
ClassesCL:0000000, NCBITaxon:1,
Variablescell (CL:0000000), taxon (NCBITaxon:1),
Contributors0000-0001-5208-3432, 0000-0002-6601-2165,
Examples
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/patterns/taxonSpecific/index.html b/patterns/taxonSpecific/index.html new file mode 100644 index 000000000..c3e6e1a37 --- /dev/null +++ b/patterns/taxonSpecific/index.html @@ -0,0 +1,906 @@ + + + + + + + + + + + + + + + + + + + + + + Cell, taxon specific - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

taxonSpecific

+

http://purl.obolibrary.org/obo/cl/taxonSpecific

+

Description

+

A cell that is restricted to a specific taxon, such as CL:0001200 'lymphocyte of B lineage, CD19-positive' are only in mammals. Note - this is not to be used for any cell that is restricted to a taxon, this is for taxon-specific subclasses of existing cell types. This should hardly ever be used.

+

Contributors

+ +

Name

+

{cell}, {taxon}

+

Definition

+

Any {cell} that is part of a {taxon}.

+

Equivalent to

+

{cell} and (in taxon some {taxon})

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/presentations/index.html b/presentations/index.html new file mode 100644 index 000000000..25d26c8c2 --- /dev/null +++ b/presentations/index.html @@ -0,0 +1,793 @@ + + + + + + + + + + + + + + + + + + + + + + CL presentations - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Presentations

+

Cell Ontology (Genentech presentation) by Nicole Vasilevsky. +Presented on 2021-02-09.

+ + + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/relations_guide/index.html b/relations_guide/index.html new file mode 100644 index 000000000..fe92a7058 --- /dev/null +++ b/relations_guide/index.html @@ -0,0 +1,1404 @@ + + + + + + + + + + + + + + + + + + + + + + CL relations - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Cell Ontology (CL) relations guide.

+

Intro

+

The aim of this document is to provide an accessible guide to how to use relations to record the properties that define cell types including location, lineage, function, morphology and marker genes. The term 'relations' here refers principally to OWL object properties, but also includes annoation properties used as shortcuts for more expressive logical axioms that can be programatically generated from them.

+

Relations in this guide are grouped by general use case (e.g. recording location) and each is illustrated by an example e.g.-

+
    +
  • melanocyte subClassOf ‘has part’ some melanosome.
    • +
+

This should be read as ‘all melanocytes have some type of melanosome as a part’ as should all axioms of this form. The examples should all be correct, but may not reflect the full complexity of axioms in the ontology. Where no example is currently present in CL, examples are taken from the Drosophila Anatomy Ontology, which follows the same schema.

+

Recording location

+

Location of cell types is recorded by relating a cell type to a term in an anatomical ontology. For the Cell Ontology this means a term from Uberon.

+

'part of'

+

Use part_of for cases where the location is a material anatomical structure (rather than a space, such as a sinus) and all of the cell is within the anatomical structure.

+

‘epithelial cell' +subClassOf 'part of' +some epithelium

+

‘part of’ is transitive, which means that it applies across chains of relationships. For example,

+

‘ileal goblet cell’ part_of some ileum

+

ilium ‘part of’ some ‘small intestine’

+

‘small intestine’ ‘part of’ some intestine

+

=>

+

’ileal goblet cell’ ‘part of’ some ‘small intestine’ +& +‘ileal goblet cell’ ‘part of’ some intestine

+

located_in

+

To record the location of a cell in an anatomical space (e.g., a sinus), +'located in' is used.

+

For example:

+

‘lymph node marginal reticular cell’ subClassOf 'located in' some 'subcapsular sinus of lymph node'

+

overlaps

+

'part of' applies in cases where an entire cell is within an anatomical structure, but some cells have parts in multiple anatomical structures. For example, many neurons span multiple regions of the central nervous system. The general relation for this is overlaps (has some part in).

+

overlaps is not currently used directly in the cell ontology (time of writing 05/2023), but more specific relationships exist for recording the location of neurons and their parts. These are described in the next section.

+

Recording the location of neurons

+

has soma location

+

When neurobiologists talk about the location of vertebrate neurons, they are typically referring to soma location. The importance of soma location to identify is underscored by how commonly cell types are named, in part, by soma location. We therefore have a dedicated relation for recording this: 'has soma location'.

+

For example, anterior horn motor neuron has the following subclass axiom:

+

'has soma location' some 'ventral horn of spinal cord'

+

axiomatization of ‘has soma location’

+
    +
  • +

    subPropertyOf: overlaps # if X has_soma_location some Y, then X overlaps some Y)

    +
    • +
  • +

    domain: neuron # X has_soma_location some Y => X is inferred to be a subClassOf neuron

    +
    • +
  • +

    property chain: has_soma_location o part_of --> has_soma_location # If x has soma location y and y is part_of z, then x has_soma_location_z

    +
    • +
+

Example of reasoning with the property chain:

+

'cortical interneuron' equivalentTo 'interneuron' that has_soma_location some 'cerebral cortex'

+

'rosehip neuron' subClassOf interneuron and has_soma_location some 'cortical layer 1'

+

'cortical layer 1' subClassOf part_of some 'cerebral cortex

+

=> 'rosehip neuron' subClassOf 'cortical interneuron'

+

sends synaptic output to region

+

A relationship between a neuron and a region, where the neuron has a functionally relevant number of output synapses in that region.

+

'adult basket subesophageal neuron' SubClassOf sends synaptic output to region +some inferior posterior slope

+

receives synaptic input in region

+

A relationship between a neuron and a region, where the neuron has a functionally relevant number of input synapses:

+

e.g. 'adult basket subesophageal neuron' SubClassOf ‘receives synaptic input in region’ +some ‘superior posterior slope’

+

fasciculates_with

+

Use this to record the tracts or nerves that a neuron’s projections fasciculate with.

+

e.g. ‘Betz cell’ subClasssOf ‘fasciculates with’ some ‘corticospinal tract’.

+

subPropertyOf: overlaps

+

domain: neuron

+

range: neuron projection bundle

+

Recording synaptic connectivity (neurons)

+

To record neuron-to-neuron or motor neuron-to-target muscle connectivity, consider the following object properties. These properties should be used when connectivity is key to the definition, for example, in cases where a motor neuron type is defined by the type of muscle fiber on which it synapses.

+

synapsed to

+

For example, 'alpha motor neuron' SubClassOf synapsed to some 'extrafusal muscle fiber'

+

synapsed by

+ +

For example, 'extrafusal muscle fiber' SubClassOf synapsed by some 'alpha motor neuron'

+

Recording function

+

Cellular function is recorded by linking GO biological process terms with the object properties 'capable of' and ‘capable of part of’

+

'capable of'

+

Use this relationships where the cell is capable of carrying out the entirety of the process

+

For example, 'hilus cell of ovary' has the following subclass:

+

'capable of' some 'androgen secretion'

+

‘capable of part of’

+

Use this relationship where only part of the process occurs in the cell type.

+

e.g. 'retinal bipolar neuron' 'capable of part of' some 'visual perception'

+

Recording developmental lineage

+

Developmental lineage is recorded between cell types with the object property develops from (a transitive property), or in the case where there are no intermediates between the cells, 'directly develops from' (a non-transitive subproperty of develops_from)

+

For example, 'leukocyte' subClassOf develops from some 'hematopoietic stem cell'

+

Recording cell markers

+

Only markers that are necessary to define a cell type should be recorded.

+

cell surface (protein) markers

+

The cell ontology has a set of terms for recording cell surface markers.

+

The most commonly used relation for recording markers is 'has plasma +membrane part'. This +object property is used to record cell surface markers, especially in +immune cells. There are also more specific properties, 'has low +plasma membrane amount' +and 'has high plasma membrane +amount', that can be used +at an editor's discretion. In each case, a term from the PRotein +Ontology (PRO) or a +protein complex term from the Gene Ontology +(GO) is used as the object +of the relation.

+

For example, 'alpha-beta T cell' has the following +equivalence axiom:

+

'T cell' and 'has +plasma membrane part' +some 'alpha-beta T cell receptor complex'

+

Absence of a marker can be recorded using +lacks_plasma_membrane_part

+

Warning - this is used in place of the more accurate OWL expression "NOT has_part some X*** in order to keep within the EL profile of OWL. It's use with a general class as a target can potentially lead to reasoning errors.

+

recording gene markers

+

‘expresses’

+

Use this to link a cell type to a gene or gene product that defines it:

+

For example:

+

'lamp5 GABAergic cortical interneuron' EquivalentTo: +interneuron and ('has soma location' some 'cerebral cortex') +and ('capable of' some 'gamma-aminobutyric acid secretion, neurotransmission') +and (expresses some 'lysosome-associated membrane glycoprotein 5')

+

In FBbt, FlyBase Gene IDs are permitted here (using standard resolvable URL pattern). In CL currenly only PRO IDs are permitted. In PCL, a broader range of IDs have been used (depending on data sources used).

+

Recording cell parts

+

To record parts above the granularity of proteins and complexes, use a 'has part' relationship with an object from the Gene Ontology cellular_component branch.

+

e.g. 'melanocyte' subClassOf 'has part' some 'melanosome'

+

This GO term can be combined with a PATO quality term (e.g. for shape) where necessary, e.g.

+

For example:

+

'mature basophil' subClassOf ('has part' some (nucleus and ('has characteristic' some lobed)))

+

Recording general cellular characteristics

+

The ontology PATO, +has a rich set of terms that can be used to record the general +characteristics of cells, such as their morphology. These are recorded +using 'has characteristic'.

+

In choosing PATO terms, avoid those referring to some change in +characteristic (e.g,.’ increased branchiness’). The following list of +examples is not exhaustive:

+

Recording Morphology

+

PATO has a set of general morphology terms which may be applicable to +cells.

+

For example, erythrocyte +subClassOf 'has characteristic' +some biconcave

+

PATO also has a set of terms for specific cell morphologies (mostly +neuronal), e.g.

+

‘Betz cell’ subClassOf ‘has characteristic’ some ‘standard pyramidal morphology’

+

Recording nuclear number

+

To record the number of nuclei in a cell, use a PATO subclass +under the term 'nucleate quality' with the 'has +characteristic' relation.

+

image

+

For example,

+

platelet subClassOf ('has_characteristic' some anucleate)

+

Note - that PATO includes bridging axioms that infer part relationships +based on these characteristics.

+

e.g.

+

cell and ('has characteristic' some multinucleate) SubClassOf 'has part' +some nucleus

+

Taxon constraints

+

See +https://oboacademy.github.io/obook/explanation/taxon-constraints-explainer/.

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/resolving_merge_conflicts/index.html b/resolving_merge_conflicts/index.html new file mode 100644 index 000000000..5c9828ffc --- /dev/null +++ b/resolving_merge_conflicts/index.html @@ -0,0 +1,909 @@ + + + + + + + + + + + + + + + + + + + + + + Resolving merge conflicts - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

Resolving merge conflicts

+ +

A guide to resolving merge conflicts

+

When you come to merge your pull request, you may find that conflicts prevent automated merging back into the master. In some cases, GitHub supports resolution of these through its web interface. However, probably due to file size, this is not currently supported for cl-edit.owl.

+

The majority of the time, the conflict is trivial - due to addition of new terms to the same point in the file. Because terms are ordered in the file by ID, this happens whenever two edits add terms without any intervening IDs. Trivial clashes are easy to spot - they involve whole term stanzas + declarations.

+

Occassionally non-trivial clashes will happen when two pull requests include edits to the same term or even the same axiom. Ask an editor for help if you don't feel confident resolving these.

+

SOP.

+
    +
  1. +

    Reserialise the Master file using the Ontology Development Kit (ODK). This requires setting up Docker and ODK. If not already set up, follow the instructions here.

    +
    2. +
  2. +

    Open Docker.

    +
    3. +
  3. +

    At the line command (PC) or Terminal (Mac), use the cd (change directory) command to navigate to the repository's src/ontology/ directory. + For example,

    +
    4. +
+

''' + cd PATH_TO_ONTOLOGY/src/ontology/ + '''

+

Replace "PATH_TO_ONTOLOGY" with the actual file path to the ontology. If you need to orient yourself, use the '''pwd''' (present working directory) or '''ls''' (list) line commands.

+
    +
  1. If you are resolving a conflict in an .owl file, run:
    2. +
+

''' +sh run.sh make normalize_src + '''

+

If you are resolving a conflict in an .obo file, run:

+

''' +sh run.sh make normalize_obo_src + '''

+
    +
  1. +

    In CL, edits sometimes result in creating a large amount of uninteded differences involving ^^xsd:string. If you see these differences after running the command above, they can be resolved by following the instructions here.

    +
    2. +
  2. +

    In GitHub Desktop:

    +
    3. +
  3. +

    Checkout Master and pull to make sure your Master branch is up to date.

    +
    4. +
  4. Checkout the branch for the pull request and make sure it is up to date.
    5. +
  5. +

    Choose Branch > Update from master: image

    +
    6. +
  6. +

    GitHub desktop should detect the clash and ask you if you want to open in your text editor of choice (e.g., Atom).

    +
    7. +
  7. If clashes are due to trivial ordering problems, delete the conflict marks (<<<<<<<, =======, >>>>>>>), commit and push back to GitHub.
    8. +
  8. Check the resulting diffs on the Pull Request on GitHub.
    9. +
  9. Once the checks have run and are successful, merge and delete the branch.
    10. +
+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file diff --git a/search/lunr.js b/search/lunr.js new file mode 100644 index 000000000..aca0a167f --- /dev/null +++ b/search/lunr.js @@ -0,0 +1,3475 @@ +/** + * lunr - http://lunrjs.com - A bit like Solr, but much smaller and not as bright - 2.3.9 + * Copyright (C) 2020 Oliver Nightingale + * @license MIT + */ + +;(function(){ + +/** + * A convenience function for configuring and constructing + * a new lunr Index. + * + * A lunr.Builder instance is created and the pipeline setup + * with a trimmer, stop word filter and stemmer. + * + * This builder object is yielded to the configuration function + * that is passed as a parameter, allowing the list of fields + * and other builder parameters to be customised. + * + * All documents _must_ be added within the passed config function. + * + * @example + * var idx = lunr(function () { + * this.field('title') + * this.field('body') + * this.ref('id') + * + * documents.forEach(function (doc) { + * this.add(doc) + * }, this) + * }) + * + * @see {@link lunr.Builder} + * @see {@link lunr.Pipeline} + * @see {@link lunr.trimmer} + * @see {@link lunr.stopWordFilter} + * @see {@link lunr.stemmer} + * @namespace {function} lunr + */ +var lunr = function (config) { + var builder = new lunr.Builder + + builder.pipeline.add( + lunr.trimmer, + lunr.stopWordFilter, + lunr.stemmer + ) + + builder.searchPipeline.add( + lunr.stemmer + ) + + config.call(builder, builder) + return builder.build() +} + +lunr.version = "2.3.9" +/*! + * lunr.utils + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * A namespace containing utils for the rest of the lunr library + * @namespace lunr.utils + */ +lunr.utils = {} + +/** + * Print a warning message to the console. + * + * @param {String} message The message to be printed. + * @memberOf lunr.utils + * @function + */ +lunr.utils.warn = (function (global) { + /* eslint-disable no-console */ + return function (message) { + if (global.console && console.warn) { + console.warn(message) + } + } + /* eslint-enable no-console */ +})(this) + +/** + * Convert an object to a string. + * + * In the case of `null` and `undefined` the function returns + * the empty string, in all other cases the result of calling + * `toString` on the passed object is returned. + * + * @param {Any} obj The object to convert to a string. + * @return {String} string representation of the passed object. + * @memberOf lunr.utils + */ +lunr.utils.asString = function (obj) { + if (obj === void 0 || obj === null) { + return "" + } else { + return obj.toString() + } +} + +/** + * Clones an object. + * + * Will create a copy of an existing object such that any mutations + * on the copy cannot affect the original. + * + * Only shallow objects are supported, passing a nested object to this + * function will cause a TypeError. + * + * Objects with primitives, and arrays of primitives are supported. + * + * @param {Object} obj The object to clone. + * @return {Object} a clone of the passed object. + * @throws {TypeError} when a nested object is passed. + * @memberOf Utils + */ +lunr.utils.clone = function (obj) { + if (obj === null || obj === undefined) { + return obj + } + + var clone = Object.create(null), + keys = Object.keys(obj) + + for (var i = 0; i < keys.length; i++) { + var key = keys[i], + val = obj[key] + + if (Array.isArray(val)) { + clone[key] = val.slice() + continue + } + + if (typeof val === 'string' || + typeof val === 'number' || + typeof val === 'boolean') { + clone[key] = val + continue + } + + throw new TypeError("clone is not deep and does not support nested objects") + } + + return clone +} +lunr.FieldRef = function (docRef, fieldName, stringValue) { + this.docRef = docRef + this.fieldName = fieldName + this._stringValue = stringValue +} + +lunr.FieldRef.joiner = "/" + +lunr.FieldRef.fromString = function (s) { + var n = s.indexOf(lunr.FieldRef.joiner) + + if (n === -1) { + throw "malformed field ref string" + } + + var fieldRef = s.slice(0, n), + docRef = s.slice(n + 1) + + return new lunr.FieldRef (docRef, fieldRef, s) +} + +lunr.FieldRef.prototype.toString = function () { + if (this._stringValue == undefined) { + this._stringValue = this.fieldName + lunr.FieldRef.joiner + this.docRef + } + + return this._stringValue +} +/*! + * lunr.Set + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * A lunr set. + * + * @constructor + */ +lunr.Set = function (elements) { + this.elements = Object.create(null) + + if (elements) { + this.length = elements.length + + for (var i = 0; i < this.length; i++) { + this.elements[elements[i]] = true + } + } else { + this.length = 0 + } +} + +/** + * A complete set that contains all elements. + * + * @static + * @readonly + * @type {lunr.Set} + */ +lunr.Set.complete = { + intersect: function (other) { + return other + }, + + union: function () { + return this + }, + + contains: function () { + return true + } +} + +/** + * An empty set that contains no elements. + * + * @static + * @readonly + * @type {lunr.Set} + */ +lunr.Set.empty = { + intersect: function () { + return this + }, + + union: function (other) { + return other + }, + + contains: function () { + return false + } +} + +/** + * Returns true if this set contains the specified object. + * + * @param {object} object - Object whose presence in this set is to be tested. + * @returns {boolean} - True if this set contains the specified object. + */ +lunr.Set.prototype.contains = function (object) { + return !!this.elements[object] +} + +/** + * Returns a new set containing only the elements that are present in both + * this set and the specified set. + * + * @param {lunr.Set} other - set to intersect with this set. + * @returns {lunr.Set} a new set that is the intersection of this and the specified set. + */ + +lunr.Set.prototype.intersect = function (other) { + var a, b, elements, intersection = [] + + if (other === lunr.Set.complete) { + return this + } + + if (other === lunr.Set.empty) { + return other + } + + if (this.length < other.length) { + a = this + b = other + } else { + a = other + b = this + } + + elements = Object.keys(a.elements) + + for (var i = 0; i < elements.length; i++) { + var element = elements[i] + if (element in b.elements) { + intersection.push(element) + } + } + + return new lunr.Set (intersection) +} + +/** + * Returns a new set combining the elements of this and the specified set. + * + * @param {lunr.Set} other - set to union with this set. + * @return {lunr.Set} a new set that is the union of this and the specified set. + */ + +lunr.Set.prototype.union = function (other) { + if (other === lunr.Set.complete) { + return lunr.Set.complete + } + + if (other === lunr.Set.empty) { + return this + } + + return new lunr.Set(Object.keys(this.elements).concat(Object.keys(other.elements))) +} +/** + * A function to calculate the inverse document frequency for + * a posting. This is shared between the builder and the index + * + * @private + * @param {object} posting - The posting for a given term + * @param {number} documentCount - The total number of documents. + */ +lunr.idf = function (posting, documentCount) { + var documentsWithTerm = 0 + + for (var fieldName in posting) { + if (fieldName == '_index') continue // Ignore the term index, its not a field + documentsWithTerm += Object.keys(posting[fieldName]).length + } + + var x = (documentCount - documentsWithTerm + 0.5) / (documentsWithTerm + 0.5) + + return Math.log(1 + Math.abs(x)) +} + +/** + * A token wraps a string representation of a token + * as it is passed through the text processing pipeline. + * + * @constructor + * @param {string} [str=''] - The string token being wrapped. + * @param {object} [metadata={}] - Metadata associated with this token. + */ +lunr.Token = function (str, metadata) { + this.str = str || "" + this.metadata = metadata || {} +} + +/** + * Returns the token string that is being wrapped by this object. + * + * @returns {string} + */ +lunr.Token.prototype.toString = function () { + return this.str +} + +/** + * A token update function is used when updating or optionally + * when cloning a token. + * + * @callback lunr.Token~updateFunction + * @param {string} str - The string representation of the token. + * @param {Object} metadata - All metadata associated with this token. + */ + +/** + * Applies the given function to the wrapped string token. + * + * @example + * token.update(function (str, metadata) { + * return str.toUpperCase() + * }) + * + * @param {lunr.Token~updateFunction} fn - A function to apply to the token string. + * @returns {lunr.Token} + */ +lunr.Token.prototype.update = function (fn) { + this.str = fn(this.str, this.metadata) + return this +} + +/** + * Creates a clone of this token. Optionally a function can be + * applied to the cloned token. + * + * @param {lunr.Token~updateFunction} [fn] - An optional function to apply to the cloned token. + * @returns {lunr.Token} + */ +lunr.Token.prototype.clone = function (fn) { + fn = fn || function (s) { return s } + return new lunr.Token (fn(this.str, this.metadata), this.metadata) +} +/*! + * lunr.tokenizer + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * A function for splitting a string into tokens ready to be inserted into + * the search index. Uses `lunr.tokenizer.separator` to split strings, change + * the value of this property to change how strings are split into tokens. + * + * This tokenizer will convert its parameter to a string by calling `toString` and + * then will split this string on the character in `lunr.tokenizer.separator`. + * Arrays will have their elements converted to strings and wrapped in a lunr.Token. + * + * Optional metadata can be passed to the tokenizer, this metadata will be cloned and + * added as metadata to every token that is created from the object to be tokenized. + * + * @static + * @param {?(string|object|object[])} obj - The object to convert into tokens + * @param {?object} metadata - Optional metadata to associate with every token + * @returns {lunr.Token[]} + * @see {@link lunr.Pipeline} + */ +lunr.tokenizer = function (obj, metadata) { + if (obj == null || obj == undefined) { + return [] + } + + if (Array.isArray(obj)) { + return obj.map(function (t) { + return new lunr.Token( + lunr.utils.asString(t).toLowerCase(), + lunr.utils.clone(metadata) + ) + }) + } + + var str = obj.toString().toLowerCase(), + len = str.length, + tokens = [] + + for (var sliceEnd = 0, sliceStart = 0; sliceEnd <= len; sliceEnd++) { + var char = str.charAt(sliceEnd), + sliceLength = sliceEnd - sliceStart + + if ((char.match(lunr.tokenizer.separator) || sliceEnd == len)) { + + if (sliceLength > 0) { + var tokenMetadata = lunr.utils.clone(metadata) || {} + tokenMetadata["position"] = [sliceStart, sliceLength] + tokenMetadata["index"] = tokens.length + + tokens.push( + new lunr.Token ( + str.slice(sliceStart, sliceEnd), + tokenMetadata + ) + ) + } + + sliceStart = sliceEnd + 1 + } + + } + + return tokens +} + +/** + * The separator used to split a string into tokens. Override this property to change the behaviour of + * `lunr.tokenizer` behaviour when tokenizing strings. By default this splits on whitespace and hyphens. + * + * @static + * @see lunr.tokenizer + */ +lunr.tokenizer.separator = /[\s\-]+/ +/*! + * lunr.Pipeline + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * lunr.Pipelines maintain an ordered list of functions to be applied to all + * tokens in documents entering the search index and queries being ran against + * the index. + * + * An instance of lunr.Index created with the lunr shortcut will contain a + * pipeline with a stop word filter and an English language stemmer. Extra + * functions can be added before or after either of these functions or these + * default functions can be removed. + * + * When run the pipeline will call each function in turn, passing a token, the + * index of that token in the original list of all tokens and finally a list of + * all the original tokens. + * + * The output of functions in the pipeline will be passed to the next function + * in the pipeline. To exclude a token from entering the index the function + * should return undefined, the rest of the pipeline will not be called with + * this token. + * + * For serialisation of pipelines to work, all functions used in an instance of + * a pipeline should be registered with lunr.Pipeline. Registered functions can + * then be loaded. If trying to load a serialised pipeline that uses functions + * that are not registered an error will be thrown. + * + * If not planning on serialising the pipeline then registering pipeline functions + * is not necessary. + * + * @constructor + */ +lunr.Pipeline = function () { + this._stack = [] +} + +lunr.Pipeline.registeredFunctions = Object.create(null) + +/** + * A pipeline function maps lunr.Token to lunr.Token. A lunr.Token contains the token + * string as well as all known metadata. A pipeline function can mutate the token string + * or mutate (or add) metadata for a given token. + * + * A pipeline function can indicate that the passed token should be discarded by returning + * null, undefined or an empty string. This token will not be passed to any downstream pipeline + * functions and will not be added to the index. + * + * Multiple tokens can be returned by returning an array of tokens. Each token will be passed + * to any downstream pipeline functions and all will returned tokens will be added to the index. + * + * Any number of pipeline functions may be chained together using a lunr.Pipeline. + * + * @interface lunr.PipelineFunction + * @param {lunr.Token} token - A token from the document being processed. + * @param {number} i - The index of this token in the complete list of tokens for this document/field. + * @param {lunr.Token[]} tokens - All tokens for this document/field. + * @returns {(?lunr.Token|lunr.Token[])} + */ + +/** + * Register a function with the pipeline. + * + * Functions that are used in the pipeline should be registered if the pipeline + * needs to be serialised, or a serialised pipeline needs to be loaded. + * + * Registering a function does not add it to a pipeline, functions must still be + * added to instances of the pipeline for them to be used when running a pipeline. + * + * @param {lunr.PipelineFunction} fn - The function to check for. + * @param {String} label - The label to register this function with + */ +lunr.Pipeline.registerFunction = function (fn, label) { + if (label in this.registeredFunctions) { + lunr.utils.warn('Overwriting existing registered function: ' + label) + } + + fn.label = label + lunr.Pipeline.registeredFunctions[fn.label] = fn +} + +/** + * Warns if the function is not registered as a Pipeline function. + * + * @param {lunr.PipelineFunction} fn - The function to check for. + * @private + */ +lunr.Pipeline.warnIfFunctionNotRegistered = function (fn) { + var isRegistered = fn.label && (fn.label in this.registeredFunctions) + + if (!isRegistered) { + lunr.utils.warn('Function is not registered with pipeline. This may cause problems when serialising the index.\n', fn) + } +} + +/** + * Loads a previously serialised pipeline. + * + * All functions to be loaded must already be registered with lunr.Pipeline. + * If any function from the serialised data has not been registered then an + * error will be thrown. + * + * @param {Object} serialised - The serialised pipeline to load. + * @returns {lunr.Pipeline} + */ +lunr.Pipeline.load = function (serialised) { + var pipeline = new lunr.Pipeline + + serialised.forEach(function (fnName) { + var fn = lunr.Pipeline.registeredFunctions[fnName] + + if (fn) { + pipeline.add(fn) + } else { + throw new Error('Cannot load unregistered function: ' + fnName) + } + }) + + return pipeline +} + +/** + * Adds new functions to the end of the pipeline. + * + * Logs a warning if the function has not been registered. + * + * @param {lunr.PipelineFunction[]} functions - Any number of functions to add to the pipeline. + */ +lunr.Pipeline.prototype.add = function () { + var fns = Array.prototype.slice.call(arguments) + + fns.forEach(function (fn) { + lunr.Pipeline.warnIfFunctionNotRegistered(fn) + this._stack.push(fn) + }, this) +} + +/** + * Adds a single function after a function that already exists in the + * pipeline. + * + * Logs a warning if the function has not been registered. + * + * @param {lunr.PipelineFunction} existingFn - A function that already exists in the pipeline. + * @param {lunr.PipelineFunction} newFn - The new function to add to the pipeline. + */ +lunr.Pipeline.prototype.after = function (existingFn, newFn) { + lunr.Pipeline.warnIfFunctionNotRegistered(newFn) + + var pos = this._stack.indexOf(existingFn) + if (pos == -1) { + throw new Error('Cannot find existingFn') + } + + pos = pos + 1 + this._stack.splice(pos, 0, newFn) +} + +/** + * Adds a single function before a function that already exists in the + * pipeline. + * + * Logs a warning if the function has not been registered. + * + * @param {lunr.PipelineFunction} existingFn - A function that already exists in the pipeline. + * @param {lunr.PipelineFunction} newFn - The new function to add to the pipeline. + */ +lunr.Pipeline.prototype.before = function (existingFn, newFn) { + lunr.Pipeline.warnIfFunctionNotRegistered(newFn) + + var pos = this._stack.indexOf(existingFn) + if (pos == -1) { + throw new Error('Cannot find existingFn') + } + + this._stack.splice(pos, 0, newFn) +} + +/** + * Removes a function from the pipeline. + * + * @param {lunr.PipelineFunction} fn The function to remove from the pipeline. + */ +lunr.Pipeline.prototype.remove = function (fn) { + var pos = this._stack.indexOf(fn) + if (pos == -1) { + return + } + + this._stack.splice(pos, 1) +} + +/** + * Runs the current list of functions that make up the pipeline against the + * passed tokens. + * + * @param {Array} tokens The tokens to run through the pipeline. + * @returns {Array} + */ +lunr.Pipeline.prototype.run = function (tokens) { + var stackLength = this._stack.length + + for (var i = 0; i < stackLength; i++) { + var fn = this._stack[i] + var memo = [] + + for (var j = 0; j < tokens.length; j++) { + var result = fn(tokens[j], j, tokens) + + if (result === null || result === void 0 || result === '') continue + + if (Array.isArray(result)) { + for (var k = 0; k < result.length; k++) { + memo.push(result[k]) + } + } else { + memo.push(result) + } + } + + tokens = memo + } + + return tokens +} + +/** + * Convenience method for passing a string through a pipeline and getting + * strings out. This method takes care of wrapping the passed string in a + * token and mapping the resulting tokens back to strings. + * + * @param {string} str - The string to pass through the pipeline. + * @param {?object} metadata - Optional metadata to associate with the token + * passed to the pipeline. + * @returns {string[]} + */ +lunr.Pipeline.prototype.runString = function (str, metadata) { + var token = new lunr.Token (str, metadata) + + return this.run([token]).map(function (t) { + return t.toString() + }) +} + +/** + * Resets the pipeline by removing any existing processors. + * + */ +lunr.Pipeline.prototype.reset = function () { + this._stack = [] +} + +/** + * Returns a representation of the pipeline ready for serialisation. + * + * Logs a warning if the function has not been registered. + * + * @returns {Array} + */ +lunr.Pipeline.prototype.toJSON = function () { + return this._stack.map(function (fn) { + lunr.Pipeline.warnIfFunctionNotRegistered(fn) + + return fn.label + }) +} +/*! + * lunr.Vector + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * A vector is used to construct the vector space of documents and queries. These + * vectors support operations to determine the similarity between two documents or + * a document and a query. + * + * Normally no parameters are required for initializing a vector, but in the case of + * loading a previously dumped vector the raw elements can be provided to the constructor. + * + * For performance reasons vectors are implemented with a flat array, where an elements + * index is immediately followed by its value. E.g. [index, value, index, value]. This + * allows the underlying array to be as sparse as possible and still offer decent + * performance when being used for vector calculations. + * + * @constructor + * @param {Number[]} [elements] - The flat list of element index and element value pairs. + */ +lunr.Vector = function (elements) { + this._magnitude = 0 + this.elements = elements || [] +} + + +/** + * Calculates the position within the vector to insert a given index. + * + * This is used internally by insert and upsert. If there are duplicate indexes then + * the position is returned as if the value for that index were to be updated, but it + * is the callers responsibility to check whether there is a duplicate at that index + * + * @param {Number} insertIdx - The index at which the element should be inserted. + * @returns {Number} + */ +lunr.Vector.prototype.positionForIndex = function (index) { + // For an empty vector the tuple can be inserted at the beginning + if (this.elements.length == 0) { + return 0 + } + + var start = 0, + end = this.elements.length / 2, + sliceLength = end - start, + pivotPoint = Math.floor(sliceLength / 2), + pivotIndex = this.elements[pivotPoint * 2] + + while (sliceLength > 1) { + if (pivotIndex < index) { + start = pivotPoint + } + + if (pivotIndex > index) { + end = pivotPoint + } + + if (pivotIndex == index) { + break + } + + sliceLength = end - start + pivotPoint = start + Math.floor(sliceLength / 2) + pivotIndex = this.elements[pivotPoint * 2] + } + + if (pivotIndex == index) { + return pivotPoint * 2 + } + + if (pivotIndex > index) { + return pivotPoint * 2 + } + + if (pivotIndex < index) { + return (pivotPoint + 1) * 2 + } +} + +/** + * Inserts an element at an index within the vector. + * + * Does not allow duplicates, will throw an error if there is already an entry + * for this index. + * + * @param {Number} insertIdx - The index at which the element should be inserted. + * @param {Number} val - The value to be inserted into the vector. + */ +lunr.Vector.prototype.insert = function (insertIdx, val) { + this.upsert(insertIdx, val, function () { + throw "duplicate index" + }) +} + +/** + * Inserts or updates an existing index within the vector. + * + * @param {Number} insertIdx - The index at which the element should be inserted. + * @param {Number} val - The value to be inserted into the vector. + * @param {function} fn - A function that is called for updates, the existing value and the + * requested value are passed as arguments + */ +lunr.Vector.prototype.upsert = function (insertIdx, val, fn) { + this._magnitude = 0 + var position = this.positionForIndex(insertIdx) + + if (this.elements[position] == insertIdx) { + this.elements[position + 1] = fn(this.elements[position + 1], val) + } else { + this.elements.splice(position, 0, insertIdx, val) + } +} + +/** + * Calculates the magnitude of this vector. + * + * @returns {Number} + */ +lunr.Vector.prototype.magnitude = function () { + if (this._magnitude) return this._magnitude + + var sumOfSquares = 0, + elementsLength = this.elements.length + + for (var i = 1; i < elementsLength; i += 2) { + var val = this.elements[i] + sumOfSquares += val * val + } + + return this._magnitude = Math.sqrt(sumOfSquares) +} + +/** + * Calculates the dot product of this vector and another vector. + * + * @param {lunr.Vector} otherVector - The vector to compute the dot product with. + * @returns {Number} + */ +lunr.Vector.prototype.dot = function (otherVector) { + var dotProduct = 0, + a = this.elements, b = otherVector.elements, + aLen = a.length, bLen = b.length, + aVal = 0, bVal = 0, + i = 0, j = 0 + + while (i < aLen && j < bLen) { + aVal = a[i], bVal = b[j] + if (aVal < bVal) { + i += 2 + } else if (aVal > bVal) { + j += 2 + } else if (aVal == bVal) { + dotProduct += a[i + 1] * b[j + 1] + i += 2 + j += 2 + } + } + + return dotProduct +} + +/** + * Calculates the similarity between this vector and another vector. + * + * @param {lunr.Vector} otherVector - The other vector to calculate the + * similarity with. + * @returns {Number} + */ +lunr.Vector.prototype.similarity = function (otherVector) { + return this.dot(otherVector) / this.magnitude() || 0 +} + +/** + * Converts the vector to an array of the elements within the vector. + * + * @returns {Number[]} + */ +lunr.Vector.prototype.toArray = function () { + var output = new Array (this.elements.length / 2) + + for (var i = 1, j = 0; i < this.elements.length; i += 2, j++) { + output[j] = this.elements[i] + } + + return output +} + +/** + * A JSON serializable representation of the vector. + * + * @returns {Number[]} + */ +lunr.Vector.prototype.toJSON = function () { + return this.elements +} +/* eslint-disable */ +/*! + * lunr.stemmer + * Copyright (C) 2020 Oliver Nightingale + * Includes code from - http://tartarus.org/~martin/PorterStemmer/js.txt + */ + +/** + * lunr.stemmer is an english language stemmer, this is a JavaScript + * implementation of the PorterStemmer taken from http://tartarus.org/~martin + * + * @static + * @implements {lunr.PipelineFunction} + * @param {lunr.Token} token - The string to stem + * @returns {lunr.Token} + * @see {@link lunr.Pipeline} + * @function + */ +lunr.stemmer = (function(){ + var step2list = { + "ational" : "ate", + "tional" : "tion", + "enci" : "ence", + "anci" : "ance", + "izer" : "ize", + "bli" : "ble", + "alli" : "al", + "entli" : "ent", + "eli" : "e", + "ousli" : "ous", + "ization" : "ize", + "ation" : "ate", + "ator" : "ate", + "alism" : "al", + "iveness" : "ive", + "fulness" : "ful", + "ousness" : "ous", + "aliti" : "al", + "iviti" : "ive", + "biliti" : "ble", + "logi" : "log" + }, + + step3list = { + "icate" : "ic", + "ative" : "", + "alize" : "al", + "iciti" : "ic", + "ical" : "ic", + "ful" : "", + "ness" : "" + }, + + c = "[^aeiou]", // consonant + v = "[aeiouy]", // vowel + C = c + "[^aeiouy]*", // consonant sequence + V = v + "[aeiou]*", // vowel sequence + + mgr0 = "^(" + C + ")?" + V + C, // [C]VC... is m>0 + meq1 = "^(" + C + ")?" + V + C + "(" + V + ")?$", // [C]VC[V] is m=1 + mgr1 = "^(" + C + ")?" + V + C + V + C, // [C]VCVC... is m>1 + s_v = "^(" + C + ")?" + v; // vowel in stem + + var re_mgr0 = new RegExp(mgr0); + var re_mgr1 = new RegExp(mgr1); + var re_meq1 = new RegExp(meq1); + var re_s_v = new RegExp(s_v); + + var re_1a = /^(.+?)(ss|i)es$/; + var re2_1a = /^(.+?)([^s])s$/; + var re_1b = /^(.+?)eed$/; + var re2_1b = /^(.+?)(ed|ing)$/; + var re_1b_2 = /.$/; + var re2_1b_2 = /(at|bl|iz)$/; + var re3_1b_2 = new RegExp("([^aeiouylsz])\\1$"); + var re4_1b_2 = new RegExp("^" + C + v + "[^aeiouwxy]$"); + + var re_1c = /^(.+?[^aeiou])y$/; + var re_2 = /^(.+?)(ational|tional|enci|anci|izer|bli|alli|entli|eli|ousli|ization|ation|ator|alism|iveness|fulness|ousness|aliti|iviti|biliti|logi)$/; + + var re_3 = /^(.+?)(icate|ative|alize|iciti|ical|ful|ness)$/; + + var re_4 = /^(.+?)(al|ance|ence|er|ic|able|ible|ant|ement|ment|ent|ou|ism|ate|iti|ous|ive|ize)$/; + var re2_4 = /^(.+?)(s|t)(ion)$/; + + var re_5 = /^(.+?)e$/; + var re_5_1 = /ll$/; + var re3_5 = new RegExp("^" + C + v + "[^aeiouwxy]$"); + + var porterStemmer = function porterStemmer(w) { + var stem, + suffix, + firstch, + re, + re2, + re3, + re4; + + if (w.length < 3) { return w; } + + firstch = w.substr(0,1); + if (firstch == "y") { + w = firstch.toUpperCase() + w.substr(1); + } + + // Step 1a + re = re_1a + re2 = re2_1a; + + if (re.test(w)) { w = w.replace(re,"$1$2"); } + else if (re2.test(w)) { w = w.replace(re2,"$1$2"); } + + // Step 1b + re = re_1b; + re2 = re2_1b; + if (re.test(w)) { + var fp = re.exec(w); + re = re_mgr0; + if (re.test(fp[1])) { + re = re_1b_2; + w = w.replace(re,""); + } + } else if (re2.test(w)) { + var fp = re2.exec(w); + stem = fp[1]; + re2 = re_s_v; + if (re2.test(stem)) { + w = stem; + re2 = re2_1b_2; + re3 = re3_1b_2; + re4 = re4_1b_2; + if (re2.test(w)) { w = w + "e"; } + else if (re3.test(w)) { re = re_1b_2; w = w.replace(re,""); } + else if (re4.test(w)) { w = w + "e"; } + } + } + + // Step 1c - replace suffix y or Y by i if preceded by a non-vowel which is not the first letter of the word (so cry -> cri, by -> by, say -> say) + re = re_1c; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + w = stem + "i"; + } + + // Step 2 + re = re_2; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + suffix = fp[2]; + re = re_mgr0; + if (re.test(stem)) { + w = stem + step2list[suffix]; + } + } + + // Step 3 + re = re_3; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + suffix = fp[2]; + re = re_mgr0; + if (re.test(stem)) { + w = stem + step3list[suffix]; + } + } + + // Step 4 + re = re_4; + re2 = re2_4; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + re = re_mgr1; + if (re.test(stem)) { + w = stem; + } + } else if (re2.test(w)) { + var fp = re2.exec(w); + stem = fp[1] + fp[2]; + re2 = re_mgr1; + if (re2.test(stem)) { + w = stem; + } + } + + // Step 5 + re = re_5; + if (re.test(w)) { + var fp = re.exec(w); + stem = fp[1]; + re = re_mgr1; + re2 = re_meq1; + re3 = re3_5; + if (re.test(stem) || (re2.test(stem) && !(re3.test(stem)))) { + w = stem; + } + } + + re = re_5_1; + re2 = re_mgr1; + if (re.test(w) && re2.test(w)) { + re = re_1b_2; + w = w.replace(re,""); + } + + // and turn initial Y back to y + + if (firstch == "y") { + w = firstch.toLowerCase() + w.substr(1); + } + + return w; + }; + + return function (token) { + return token.update(porterStemmer); + } +})(); + +lunr.Pipeline.registerFunction(lunr.stemmer, 'stemmer') +/*! + * lunr.stopWordFilter + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * lunr.generateStopWordFilter builds a stopWordFilter function from the provided + * list of stop words. + * + * The built in lunr.stopWordFilter is built using this generator and can be used + * to generate custom stopWordFilters for applications or non English languages. + * + * @function + * @param {Array} token The token to pass through the filter + * @returns {lunr.PipelineFunction} + * @see lunr.Pipeline + * @see lunr.stopWordFilter + */ +lunr.generateStopWordFilter = function (stopWords) { + var words = stopWords.reduce(function (memo, stopWord) { + memo[stopWord] = stopWord + return memo + }, {}) + + return function (token) { + if (token && words[token.toString()] !== token.toString()) return token + } +} + +/** + * lunr.stopWordFilter is an English language stop word list filter, any words + * contained in the list will not be passed through the filter. + * + * This is intended to be used in the Pipeline. If the token does not pass the + * filter then undefined will be returned. + * + * @function + * @implements {lunr.PipelineFunction} + * @params {lunr.Token} token - A token to check for being a stop word. + * @returns {lunr.Token} + * @see {@link lunr.Pipeline} + */ +lunr.stopWordFilter = lunr.generateStopWordFilter([ + 'a', + 'able', + 'about', + 'across', + 'after', + 'all', + 'almost', + 'also', + 'am', + 'among', + 'an', + 'and', + 'any', + 'are', + 'as', + 'at', + 'be', + 'because', + 'been', + 'but', + 'by', + 'can', + 'cannot', + 'could', + 'dear', + 'did', + 'do', + 'does', + 'either', + 'else', + 'ever', + 'every', + 'for', + 'from', + 'get', + 'got', + 'had', + 'has', + 'have', + 'he', + 'her', + 'hers', + 'him', + 'his', + 'how', + 'however', + 'i', + 'if', + 'in', + 'into', + 'is', + 'it', + 'its', + 'just', + 'least', + 'let', + 'like', + 'likely', + 'may', + 'me', + 'might', + 'most', + 'must', + 'my', + 'neither', + 'no', + 'nor', + 'not', + 'of', + 'off', + 'often', + 'on', + 'only', + 'or', + 'other', + 'our', + 'own', + 'rather', + 'said', + 'say', + 'says', + 'she', + 'should', + 'since', + 'so', + 'some', + 'than', + 'that', + 'the', + 'their', + 'them', + 'then', + 'there', + 'these', + 'they', + 'this', + 'tis', + 'to', + 'too', + 'twas', + 'us', + 'wants', + 'was', + 'we', + 'were', + 'what', + 'when', + 'where', + 'which', + 'while', + 'who', + 'whom', + 'why', + 'will', + 'with', + 'would', + 'yet', + 'you', + 'your' +]) + +lunr.Pipeline.registerFunction(lunr.stopWordFilter, 'stopWordFilter') +/*! + * lunr.trimmer + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * lunr.trimmer is a pipeline function for trimming non word + * characters from the beginning and end of tokens before they + * enter the index. + * + * This implementation may not work correctly for non latin + * characters and should either be removed or adapted for use + * with languages with non-latin characters. + * + * @static + * @implements {lunr.PipelineFunction} + * @param {lunr.Token} token The token to pass through the filter + * @returns {lunr.Token} + * @see lunr.Pipeline + */ +lunr.trimmer = function (token) { + return token.update(function (s) { + return s.replace(/^\W+/, '').replace(/\W+$/, '') + }) +} + +lunr.Pipeline.registerFunction(lunr.trimmer, 'trimmer') +/*! + * lunr.TokenSet + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * A token set is used to store the unique list of all tokens + * within an index. Token sets are also used to represent an + * incoming query to the index, this query token set and index + * token set are then intersected to find which tokens to look + * up in the inverted index. + * + * A token set can hold multiple tokens, as in the case of the + * index token set, or it can hold a single token as in the + * case of a simple query token set. + * + * Additionally token sets are used to perform wildcard matching. + * Leading, contained and trailing wildcards are supported, and + * from this edit distance matching can also be provided. + * + * Token sets are implemented as a minimal finite state automata, + * where both common prefixes and suffixes are shared between tokens. + * This helps to reduce the space used for storing the token set. + * + * @constructor + */ +lunr.TokenSet = function () { + this.final = false + this.edges = {} + this.id = lunr.TokenSet._nextId + lunr.TokenSet._nextId += 1 +} + +/** + * Keeps track of the next, auto increment, identifier to assign + * to a new tokenSet. + * + * TokenSets require a unique identifier to be correctly minimised. + * + * @private + */ +lunr.TokenSet._nextId = 1 + +/** + * Creates a TokenSet instance from the given sorted array of words. + * + * @param {String[]} arr - A sorted array of strings to create the set from. + * @returns {lunr.TokenSet} + * @throws Will throw an error if the input array is not sorted. + */ +lunr.TokenSet.fromArray = function (arr) { + var builder = new lunr.TokenSet.Builder + + for (var i = 0, len = arr.length; i < len; i++) { + builder.insert(arr[i]) + } + + builder.finish() + return builder.root +} + +/** + * Creates a token set from a query clause. + * + * @private + * @param {Object} clause - A single clause from lunr.Query. + * @param {string} clause.term - The query clause term. + * @param {number} [clause.editDistance] - The optional edit distance for the term. + * @returns {lunr.TokenSet} + */ +lunr.TokenSet.fromClause = function (clause) { + if ('editDistance' in clause) { + return lunr.TokenSet.fromFuzzyString(clause.term, clause.editDistance) + } else { + return lunr.TokenSet.fromString(clause.term) + } +} + +/** + * Creates a token set representing a single string with a specified + * edit distance. + * + * Insertions, deletions, substitutions and transpositions are each + * treated as an edit distance of 1. + * + * Increasing the allowed edit distance will have a dramatic impact + * on the performance of both creating and intersecting these TokenSets. + * It is advised to keep the edit distance less than 3. + * + * @param {string} str - The string to create the token set from. + * @param {number} editDistance - The allowed edit distance to match. + * @returns {lunr.Vector} + */ +lunr.TokenSet.fromFuzzyString = function (str, editDistance) { + var root = new lunr.TokenSet + + var stack = [{ + node: root, + editsRemaining: editDistance, + str: str + }] + + while (stack.length) { + var frame = stack.pop() + + // no edit + if (frame.str.length > 0) { + var char = frame.str.charAt(0), + noEditNode + + if (char in frame.node.edges) { + noEditNode = frame.node.edges[char] + } else { + noEditNode = new lunr.TokenSet + frame.node.edges[char] = noEditNode + } + + if (frame.str.length == 1) { + noEditNode.final = true + } + + stack.push({ + node: noEditNode, + editsRemaining: frame.editsRemaining, + str: frame.str.slice(1) + }) + } + + if (frame.editsRemaining == 0) { + continue + } + + // insertion + if ("*" in frame.node.edges) { + var insertionNode = frame.node.edges["*"] + } else { + var insertionNode = new lunr.TokenSet + frame.node.edges["*"] = insertionNode + } + + if (frame.str.length == 0) { + insertionNode.final = true + } + + stack.push({ + node: insertionNode, + editsRemaining: frame.editsRemaining - 1, + str: frame.str + }) + + // deletion + // can only do a deletion if we have enough edits remaining + // and if there are characters left to delete in the string + if (frame.str.length > 1) { + stack.push({ + node: frame.node, + editsRemaining: frame.editsRemaining - 1, + str: frame.str.slice(1) + }) + } + + // deletion + // just removing the last character from the str + if (frame.str.length == 1) { + frame.node.final = true + } + + // substitution + // can only do a substitution if we have enough edits remaining + // and if there are characters left to substitute + if (frame.str.length >= 1) { + if ("*" in frame.node.edges) { + var substitutionNode = frame.node.edges["*"] + } else { + var substitutionNode = new lunr.TokenSet + frame.node.edges["*"] = substitutionNode + } + + if (frame.str.length == 1) { + substitutionNode.final = true + } + + stack.push({ + node: substitutionNode, + editsRemaining: frame.editsRemaining - 1, + str: frame.str.slice(1) + }) + } + + // transposition + // can only do a transposition if there are edits remaining + // and there are enough characters to transpose + if (frame.str.length > 1) { + var charA = frame.str.charAt(0), + charB = frame.str.charAt(1), + transposeNode + + if (charB in frame.node.edges) { + transposeNode = frame.node.edges[charB] + } else { + transposeNode = new lunr.TokenSet + frame.node.edges[charB] = transposeNode + } + + if (frame.str.length == 1) { + transposeNode.final = true + } + + stack.push({ + node: transposeNode, + editsRemaining: frame.editsRemaining - 1, + str: charA + frame.str.slice(2) + }) + } + } + + return root +} + +/** + * Creates a TokenSet from a string. + * + * The string may contain one or more wildcard characters (*) + * that will allow wildcard matching when intersecting with + * another TokenSet. + * + * @param {string} str - The string to create a TokenSet from. + * @returns {lunr.TokenSet} + */ +lunr.TokenSet.fromString = function (str) { + var node = new lunr.TokenSet, + root = node + + /* + * Iterates through all characters within the passed string + * appending a node for each character. + * + * When a wildcard character is found then a self + * referencing edge is introduced to continually match + * any number of any characters. + */ + for (var i = 0, len = str.length; i < len; i++) { + var char = str[i], + final = (i == len - 1) + + if (char == "*") { + node.edges[char] = node + node.final = final + + } else { + var next = new lunr.TokenSet + next.final = final + + node.edges[char] = next + node = next + } + } + + return root +} + +/** + * Converts this TokenSet into an array of strings + * contained within the TokenSet. + * + * This is not intended to be used on a TokenSet that + * contains wildcards, in these cases the results are + * undefined and are likely to cause an infinite loop. + * + * @returns {string[]} + */ +lunr.TokenSet.prototype.toArray = function () { + var words = [] + + var stack = [{ + prefix: "", + node: this + }] + + while (stack.length) { + var frame = stack.pop(), + edges = Object.keys(frame.node.edges), + len = edges.length + + if (frame.node.final) { + /* In Safari, at this point the prefix is sometimes corrupted, see: + * https://github.com/olivernn/lunr.js/issues/279 Calling any + * String.prototype method forces Safari to "cast" this string to what + * it's supposed to be, fixing the bug. */ + frame.prefix.charAt(0) + words.push(frame.prefix) + } + + for (var i = 0; i < len; i++) { + var edge = edges[i] + + stack.push({ + prefix: frame.prefix.concat(edge), + node: frame.node.edges[edge] + }) + } + } + + return words +} + +/** + * Generates a string representation of a TokenSet. + * + * This is intended to allow TokenSets to be used as keys + * in objects, largely to aid the construction and minimisation + * of a TokenSet. As such it is not designed to be a human + * friendly representation of the TokenSet. + * + * @returns {string} + */ +lunr.TokenSet.prototype.toString = function () { + // NOTE: Using Object.keys here as this.edges is very likely + // to enter 'hash-mode' with many keys being added + // + // avoiding a for-in loop here as it leads to the function + // being de-optimised (at least in V8). From some simple + // benchmarks the performance is comparable, but allowing + // V8 to optimize may mean easy performance wins in the future. + + if (this._str) { + return this._str + } + + var str = this.final ? '1' : '0', + labels = Object.keys(this.edges).sort(), + len = labels.length + + for (var i = 0; i < len; i++) { + var label = labels[i], + node = this.edges[label] + + str = str + label + node.id + } + + return str +} + +/** + * Returns a new TokenSet that is the intersection of + * this TokenSet and the passed TokenSet. + * + * This intersection will take into account any wildcards + * contained within the TokenSet. + * + * @param {lunr.TokenSet} b - An other TokenSet to intersect with. + * @returns {lunr.TokenSet} + */ +lunr.TokenSet.prototype.intersect = function (b) { + var output = new lunr.TokenSet, + frame = undefined + + var stack = [{ + qNode: b, + output: output, + node: this + }] + + while (stack.length) { + frame = stack.pop() + + // NOTE: As with the #toString method, we are using + // Object.keys and a for loop instead of a for-in loop + // as both of these objects enter 'hash' mode, causing + // the function to be de-optimised in V8 + var qEdges = Object.keys(frame.qNode.edges), + qLen = qEdges.length, + nEdges = Object.keys(frame.node.edges), + nLen = nEdges.length + + for (var q = 0; q < qLen; q++) { + var qEdge = qEdges[q] + + for (var n = 0; n < nLen; n++) { + var nEdge = nEdges[n] + + if (nEdge == qEdge || qEdge == '*') { + var node = frame.node.edges[nEdge], + qNode = frame.qNode.edges[qEdge], + final = node.final && qNode.final, + next = undefined + + if (nEdge in frame.output.edges) { + // an edge already exists for this character + // no need to create a new node, just set the finality + // bit unless this node is already final + next = frame.output.edges[nEdge] + next.final = next.final || final + + } else { + // no edge exists yet, must create one + // set the finality bit and insert it + // into the output + next = new lunr.TokenSet + next.final = final + frame.output.edges[nEdge] = next + } + + stack.push({ + qNode: qNode, + output: next, + node: node + }) + } + } + } + } + + return output +} +lunr.TokenSet.Builder = function () { + this.previousWord = "" + this.root = new lunr.TokenSet + this.uncheckedNodes = [] + this.minimizedNodes = {} +} + +lunr.TokenSet.Builder.prototype.insert = function (word) { + var node, + commonPrefix = 0 + + if (word < this.previousWord) { + throw new Error ("Out of order word insertion") + } + + for (var i = 0; i < word.length && i < this.previousWord.length; i++) { + if (word[i] != this.previousWord[i]) break + commonPrefix++ + } + + this.minimize(commonPrefix) + + if (this.uncheckedNodes.length == 0) { + node = this.root + } else { + node = this.uncheckedNodes[this.uncheckedNodes.length - 1].child + } + + for (var i = commonPrefix; i < word.length; i++) { + var nextNode = new lunr.TokenSet, + char = word[i] + + node.edges[char] = nextNode + + this.uncheckedNodes.push({ + parent: node, + char: char, + child: nextNode + }) + + node = nextNode + } + + node.final = true + this.previousWord = word +} + +lunr.TokenSet.Builder.prototype.finish = function () { + this.minimize(0) +} + +lunr.TokenSet.Builder.prototype.minimize = function (downTo) { + for (var i = this.uncheckedNodes.length - 1; i >= downTo; i--) { + var node = this.uncheckedNodes[i], + childKey = node.child.toString() + + if (childKey in this.minimizedNodes) { + node.parent.edges[node.char] = this.minimizedNodes[childKey] + } else { + // Cache the key for this node since + // we know it can't change anymore + node.child._str = childKey + + this.minimizedNodes[childKey] = node.child + } + + this.uncheckedNodes.pop() + } +} +/*! + * lunr.Index + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * An index contains the built index of all documents and provides a query interface + * to the index. + * + * Usually instances of lunr.Index will not be created using this constructor, instead + * lunr.Builder should be used to construct new indexes, or lunr.Index.load should be + * used to load previously built and serialized indexes. + * + * @constructor + * @param {Object} attrs - The attributes of the built search index. + * @param {Object} attrs.invertedIndex - An index of term/field to document reference. + * @param {Object} attrs.fieldVectors - Field vectors + * @param {lunr.TokenSet} attrs.tokenSet - An set of all corpus tokens. + * @param {string[]} attrs.fields - The names of indexed document fields. + * @param {lunr.Pipeline} attrs.pipeline - The pipeline to use for search terms. + */ +lunr.Index = function (attrs) { + this.invertedIndex = attrs.invertedIndex + this.fieldVectors = attrs.fieldVectors + this.tokenSet = attrs.tokenSet + this.fields = attrs.fields + this.pipeline = attrs.pipeline +} + +/** + * A result contains details of a document matching a search query. + * @typedef {Object} lunr.Index~Result + * @property {string} ref - The reference of the document this result represents. + * @property {number} score - A number between 0 and 1 representing how similar this document is to the query. + * @property {lunr.MatchData} matchData - Contains metadata about this match including which term(s) caused the match. + */ + +/** + * Although lunr provides the ability to create queries using lunr.Query, it also provides a simple + * query language which itself is parsed into an instance of lunr.Query. + * + * For programmatically building queries it is advised to directly use lunr.Query, the query language + * is best used for human entered text rather than program generated text. + * + * At its simplest queries can just be a single term, e.g. `hello`, multiple terms are also supported + * and will be combined with OR, e.g `hello world` will match documents that contain either 'hello' + * or 'world', though those that contain both will rank higher in the results. + * + * Wildcards can be included in terms to match one or more unspecified characters, these wildcards can + * be inserted anywhere within the term, and more than one wildcard can exist in a single term. Adding + * wildcards will increase the number of documents that will be found but can also have a negative + * impact on query performance, especially with wildcards at the beginning of a term. + * + * Terms can be restricted to specific fields, e.g. `title:hello`, only documents with the term + * hello in the title field will match this query. Using a field not present in the index will lead + * to an error being thrown. + * + * Modifiers can also be added to terms, lunr supports edit distance and boost modifiers on terms. A term + * boost will make documents matching that term score higher, e.g. `foo^5`. Edit distance is also supported + * to provide fuzzy matching, e.g. 'hello~2' will match documents with hello with an edit distance of 2. + * Avoid large values for edit distance to improve query performance. + * + * Each term also supports a presence modifier. By default a term's presence in document is optional, however + * this can be changed to either required or prohibited. For a term's presence to be required in a document the + * term should be prefixed with a '+', e.g. `+foo bar` is a search for documents that must contain 'foo' and + * optionally contain 'bar'. Conversely a leading '-' sets the terms presence to prohibited, i.e. it must not + * appear in a document, e.g. `-foo bar` is a search for documents that do not contain 'foo' but may contain 'bar'. + * + * To escape special characters the backslash character '\' can be used, this allows searches to include + * characters that would normally be considered modifiers, e.g. `foo\~2` will search for a term "foo~2" instead + * of attempting to apply a boost of 2 to the search term "foo". + * + * @typedef {string} lunr.Index~QueryString + * @example Simple single term query + * hello + * @example Multiple term query + * hello world + * @example term scoped to a field + * title:hello + * @example term with a boost of 10 + * hello^10 + * @example term with an edit distance of 2 + * hello~2 + * @example terms with presence modifiers + * -foo +bar baz + */ + +/** + * Performs a search against the index using lunr query syntax. + * + * Results will be returned sorted by their score, the most relevant results + * will be returned first. For details on how the score is calculated, please see + * the {@link https://lunrjs.com/guides/searching.html#scoring|guide}. + * + * For more programmatic querying use lunr.Index#query. + * + * @param {lunr.Index~QueryString} queryString - A string containing a lunr query. + * @throws {lunr.QueryParseError} If the passed query string cannot be parsed. + * @returns {lunr.Index~Result[]} + */ +lunr.Index.prototype.search = function (queryString) { + return this.query(function (query) { + var parser = new lunr.QueryParser(queryString, query) + parser.parse() + }) +} + +/** + * A query builder callback provides a query object to be used to express + * the query to perform on the index. + * + * @callback lunr.Index~queryBuilder + * @param {lunr.Query} query - The query object to build up. + * @this lunr.Query + */ + +/** + * Performs a query against the index using the yielded lunr.Query object. + * + * If performing programmatic queries against the index, this method is preferred + * over lunr.Index#search so as to avoid the additional query parsing overhead. + * + * A query object is yielded to the supplied function which should be used to + * express the query to be run against the index. + * + * Note that although this function takes a callback parameter it is _not_ an + * asynchronous operation, the callback is just yielded a query object to be + * customized. + * + * @param {lunr.Index~queryBuilder} fn - A function that is used to build the query. + * @returns {lunr.Index~Result[]} + */ +lunr.Index.prototype.query = function (fn) { + // for each query clause + // * process terms + // * expand terms from token set + // * find matching documents and metadata + // * get document vectors + // * score documents + + var query = new lunr.Query(this.fields), + matchingFields = Object.create(null), + queryVectors = Object.create(null), + termFieldCache = Object.create(null), + requiredMatches = Object.create(null), + prohibitedMatches = Object.create(null) + + /* + * To support field level boosts a query vector is created per + * field. An empty vector is eagerly created to support negated + * queries. + */ + for (var i = 0; i < this.fields.length; i++) { + queryVectors[this.fields[i]] = new lunr.Vector + } + + fn.call(query, query) + + for (var i = 0; i < query.clauses.length; i++) { + /* + * Unless the pipeline has been disabled for this term, which is + * the case for terms with wildcards, we need to pass the clause + * term through the search pipeline. A pipeline returns an array + * of processed terms. Pipeline functions may expand the passed + * term, which means we may end up performing multiple index lookups + * for a single query term. + */ + var clause = query.clauses[i], + terms = null, + clauseMatches = lunr.Set.empty + + if (clause.usePipeline) { + terms = this.pipeline.runString(clause.term, { + fields: clause.fields + }) + } else { + terms = [clause.term] + } + + for (var m = 0; m < terms.length; m++) { + var term = terms[m] + + /* + * Each term returned from the pipeline needs to use the same query + * clause object, e.g. the same boost and or edit distance. The + * simplest way to do this is to re-use the clause object but mutate + * its term property. + */ + clause.term = term + + /* + * From the term in the clause we create a token set which will then + * be used to intersect the indexes token set to get a list of terms + * to lookup in the inverted index + */ + var termTokenSet = lunr.TokenSet.fromClause(clause), + expandedTerms = this.tokenSet.intersect(termTokenSet).toArray() + + /* + * If a term marked as required does not exist in the tokenSet it is + * impossible for the search to return any matches. We set all the field + * scoped required matches set to empty and stop examining any further + * clauses. + */ + if (expandedTerms.length === 0 && clause.presence === lunr.Query.presence.REQUIRED) { + for (var k = 0; k < clause.fields.length; k++) { + var field = clause.fields[k] + requiredMatches[field] = lunr.Set.empty + } + + break + } + + for (var j = 0; j < expandedTerms.length; j++) { + /* + * For each term get the posting and termIndex, this is required for + * building the query vector. + */ + var expandedTerm = expandedTerms[j], + posting = this.invertedIndex[expandedTerm], + termIndex = posting._index + + for (var k = 0; k < clause.fields.length; k++) { + /* + * For each field that this query term is scoped by (by default + * all fields are in scope) we need to get all the document refs + * that have this term in that field. + * + * The posting is the entry in the invertedIndex for the matching + * term from above. + */ + var field = clause.fields[k], + fieldPosting = posting[field], + matchingDocumentRefs = Object.keys(fieldPosting), + termField = expandedTerm + "/" + field, + matchingDocumentsSet = new lunr.Set(matchingDocumentRefs) + + /* + * if the presence of this term is required ensure that the matching + * documents are added to the set of required matches for this clause. + * + */ + if (clause.presence == lunr.Query.presence.REQUIRED) { + clauseMatches = clauseMatches.union(matchingDocumentsSet) + + if (requiredMatches[field] === undefined) { + requiredMatches[field] = lunr.Set.complete + } + } + + /* + * if the presence of this term is prohibited ensure that the matching + * documents are added to the set of prohibited matches for this field, + * creating that set if it does not yet exist. + */ + if (clause.presence == lunr.Query.presence.PROHIBITED) { + if (prohibitedMatches[field] === undefined) { + prohibitedMatches[field] = lunr.Set.empty + } + + prohibitedMatches[field] = prohibitedMatches[field].union(matchingDocumentsSet) + + /* + * Prohibited matches should not be part of the query vector used for + * similarity scoring and no metadata should be extracted so we continue + * to the next field + */ + continue + } + + /* + * The query field vector is populated using the termIndex found for + * the term and a unit value with the appropriate boost applied. + * Using upsert because there could already be an entry in the vector + * for the term we are working with. In that case we just add the scores + * together. + */ + queryVectors[field].upsert(termIndex, clause.boost, function (a, b) { return a + b }) + + /** + * If we've already seen this term, field combo then we've already collected + * the matching documents and metadata, no need to go through all that again + */ + if (termFieldCache[termField]) { + continue + } + + for (var l = 0; l < matchingDocumentRefs.length; l++) { + /* + * All metadata for this term/field/document triple + * are then extracted and collected into an instance + * of lunr.MatchData ready to be returned in the query + * results + */ + var matchingDocumentRef = matchingDocumentRefs[l], + matchingFieldRef = new lunr.FieldRef (matchingDocumentRef, field), + metadata = fieldPosting[matchingDocumentRef], + fieldMatch + + if ((fieldMatch = matchingFields[matchingFieldRef]) === undefined) { + matchingFields[matchingFieldRef] = new lunr.MatchData (expandedTerm, field, metadata) + } else { + fieldMatch.add(expandedTerm, field, metadata) + } + + } + + termFieldCache[termField] = true + } + } + } + + /** + * If the presence was required we need to update the requiredMatches field sets. + * We do this after all fields for the term have collected their matches because + * the clause terms presence is required in _any_ of the fields not _all_ of the + * fields. + */ + if (clause.presence === lunr.Query.presence.REQUIRED) { + for (var k = 0; k < clause.fields.length; k++) { + var field = clause.fields[k] + requiredMatches[field] = requiredMatches[field].intersect(clauseMatches) + } + } + } + + /** + * Need to combine the field scoped required and prohibited + * matching documents into a global set of required and prohibited + * matches + */ + var allRequiredMatches = lunr.Set.complete, + allProhibitedMatches = lunr.Set.empty + + for (var i = 0; i < this.fields.length; i++) { + var field = this.fields[i] + + if (requiredMatches[field]) { + allRequiredMatches = allRequiredMatches.intersect(requiredMatches[field]) + } + + if (prohibitedMatches[field]) { + allProhibitedMatches = allProhibitedMatches.union(prohibitedMatches[field]) + } + } + + var matchingFieldRefs = Object.keys(matchingFields), + results = [], + matches = Object.create(null) + + /* + * If the query is negated (contains only prohibited terms) + * we need to get _all_ fieldRefs currently existing in the + * index. This is only done when we know that the query is + * entirely prohibited terms to avoid any cost of getting all + * fieldRefs unnecessarily. + * + * Additionally, blank MatchData must be created to correctly + * populate the results. + */ + if (query.isNegated()) { + matchingFieldRefs = Object.keys(this.fieldVectors) + + for (var i = 0; i < matchingFieldRefs.length; i++) { + var matchingFieldRef = matchingFieldRefs[i] + var fieldRef = lunr.FieldRef.fromString(matchingFieldRef) + matchingFields[matchingFieldRef] = new lunr.MatchData + } + } + + for (var i = 0; i < matchingFieldRefs.length; i++) { + /* + * Currently we have document fields that match the query, but we + * need to return documents. The matchData and scores are combined + * from multiple fields belonging to the same document. + * + * Scores are calculated by field, using the query vectors created + * above, and combined into a final document score using addition. + */ + var fieldRef = lunr.FieldRef.fromString(matchingFieldRefs[i]), + docRef = fieldRef.docRef + + if (!allRequiredMatches.contains(docRef)) { + continue + } + + if (allProhibitedMatches.contains(docRef)) { + continue + } + + var fieldVector = this.fieldVectors[fieldRef], + score = queryVectors[fieldRef.fieldName].similarity(fieldVector), + docMatch + + if ((docMatch = matches[docRef]) !== undefined) { + docMatch.score += score + docMatch.matchData.combine(matchingFields[fieldRef]) + } else { + var match = { + ref: docRef, + score: score, + matchData: matchingFields[fieldRef] + } + matches[docRef] = match + results.push(match) + } + } + + /* + * Sort the results objects by score, highest first. + */ + return results.sort(function (a, b) { + return b.score - a.score + }) +} + +/** + * Prepares the index for JSON serialization. + * + * The schema for this JSON blob will be described in a + * separate JSON schema file. + * + * @returns {Object} + */ +lunr.Index.prototype.toJSON = function () { + var invertedIndex = Object.keys(this.invertedIndex) + .sort() + .map(function (term) { + return [term, this.invertedIndex[term]] + }, this) + + var fieldVectors = Object.keys(this.fieldVectors) + .map(function (ref) { + return [ref, this.fieldVectors[ref].toJSON()] + }, this) + + return { + version: lunr.version, + fields: this.fields, + fieldVectors: fieldVectors, + invertedIndex: invertedIndex, + pipeline: this.pipeline.toJSON() + } +} + +/** + * Loads a previously serialized lunr.Index + * + * @param {Object} serializedIndex - A previously serialized lunr.Index + * @returns {lunr.Index} + */ +lunr.Index.load = function (serializedIndex) { + var attrs = {}, + fieldVectors = {}, + serializedVectors = serializedIndex.fieldVectors, + invertedIndex = Object.create(null), + serializedInvertedIndex = serializedIndex.invertedIndex, + tokenSetBuilder = new lunr.TokenSet.Builder, + pipeline = lunr.Pipeline.load(serializedIndex.pipeline) + + if (serializedIndex.version != lunr.version) { + lunr.utils.warn("Version mismatch when loading serialised index. Current version of lunr '" + lunr.version + "' does not match serialized index '" + serializedIndex.version + "'") + } + + for (var i = 0; i < serializedVectors.length; i++) { + var tuple = serializedVectors[i], + ref = tuple[0], + elements = tuple[1] + + fieldVectors[ref] = new lunr.Vector(elements) + } + + for (var i = 0; i < serializedInvertedIndex.length; i++) { + var tuple = serializedInvertedIndex[i], + term = tuple[0], + posting = tuple[1] + + tokenSetBuilder.insert(term) + invertedIndex[term] = posting + } + + tokenSetBuilder.finish() + + attrs.fields = serializedIndex.fields + + attrs.fieldVectors = fieldVectors + attrs.invertedIndex = invertedIndex + attrs.tokenSet = tokenSetBuilder.root + attrs.pipeline = pipeline + + return new lunr.Index(attrs) +} +/*! + * lunr.Builder + * Copyright (C) 2020 Oliver Nightingale + */ + +/** + * lunr.Builder performs indexing on a set of documents and + * returns instances of lunr.Index ready for querying. + * + * All configuration of the index is done via the builder, the + * fields to index, the document reference, the text processing + * pipeline and document scoring parameters are all set on the + * builder before indexing. + * + * @constructor + * @property {string} _ref - Internal reference to the document reference field. + * @property {string[]} _fields - Internal reference to the document fields to index. + * @property {object} invertedIndex - The inverted index maps terms to document fields. + * @property {object} documentTermFrequencies - Keeps track of document term frequencies. + * @property {object} documentLengths - Keeps track of the length of documents added to the index. + * @property {lunr.tokenizer} tokenizer - Function for splitting strings into tokens for indexing. + * @property {lunr.Pipeline} pipeline - The pipeline performs text processing on tokens before indexing. + * @property {lunr.Pipeline} searchPipeline - A pipeline for processing search terms before querying the index. + * @property {number} documentCount - Keeps track of the total number of documents indexed. + * @property {number} _b - A parameter to control field length normalization, setting this to 0 disabled normalization, 1 fully normalizes field lengths, the default value is 0.75. + * @property {number} _k1 - A parameter to control how quickly an increase in term frequency results in term frequency saturation, the default value is 1.2. + * @property {number} termIndex - A counter incremented for each unique term, used to identify a terms position in the vector space. + * @property {array} metadataWhitelist - A list of metadata keys that have been whitelisted for entry in the index. + */ +lunr.Builder = function () { + this._ref = "id" + this._fields = Object.create(null) + this._documents = Object.create(null) + this.invertedIndex = Object.create(null) + this.fieldTermFrequencies = {} + this.fieldLengths = {} + this.tokenizer = lunr.tokenizer + this.pipeline = new lunr.Pipeline + this.searchPipeline = new lunr.Pipeline + this.documentCount = 0 + this._b = 0.75 + this._k1 = 1.2 + this.termIndex = 0 + this.metadataWhitelist = [] +} + +/** + * Sets the document field used as the document reference. Every document must have this field. + * The type of this field in the document should be a string, if it is not a string it will be + * coerced into a string by calling toString. + * + * The default ref is 'id'. + * + * The ref should _not_ be changed during indexing, it should be set before any documents are + * added to the index. Changing it during indexing can lead to inconsistent results. + * + * @param {string} ref - The name of the reference field in the document. + */ +lunr.Builder.prototype.ref = function (ref) { + this._ref = ref +} + +/** + * A function that is used to extract a field from a document. + * + * Lunr expects a field to be at the top level of a document, if however the field + * is deeply nested within a document an extractor function can be used to extract + * the right field for indexing. + * + * @callback fieldExtractor + * @param {object} doc - The document being added to the index. + * @returns {?(string|object|object[])} obj - The object that will be indexed for this field. + * @example Extracting a nested field + * function (doc) { return doc.nested.field } + */ + +/** + * Adds a field to the list of document fields that will be indexed. Every document being + * indexed should have this field. Null values for this field in indexed documents will + * not cause errors but will limit the chance of that document being retrieved by searches. + * + * All fields should be added before adding documents to the index. Adding fields after + * a document has been indexed will have no effect on already indexed documents. + * + * Fields can be boosted at build time. This allows terms within that field to have more + * importance when ranking search results. Use a field boost to specify that matches within + * one field are more important than other fields. + * + * @param {string} fieldName - The name of a field to index in all documents. + * @param {object} attributes - Optional attributes associated with this field. + * @param {number} [attributes.boost=1] - Boost applied to all terms within this field. + * @param {fieldExtractor} [attributes.extractor] - Function to extract a field from a document. + * @throws {RangeError} fieldName cannot contain unsupported characters '/' + */ +lunr.Builder.prototype.field = function (fieldName, attributes) { + if (/\//.test(fieldName)) { + throw new RangeError ("Field '" + fieldName + "' contains illegal character '/'") + } + + this._fields[fieldName] = attributes || {} +} + +/** + * A parameter to tune the amount of field length normalisation that is applied when + * calculating relevance scores. A value of 0 will completely disable any normalisation + * and a value of 1 will fully normalise field lengths. The default is 0.75. Values of b + * will be clamped to the range 0 - 1. + * + * @param {number} number - The value to set for this tuning parameter. + */ +lunr.Builder.prototype.b = function (number) { + if (number < 0) { + this._b = 0 + } else if (number > 1) { + this._b = 1 + } else { + this._b = number + } +} + +/** + * A parameter that controls the speed at which a rise in term frequency results in term + * frequency saturation. The default value is 1.2. Setting this to a higher value will give + * slower saturation levels, a lower value will result in quicker saturation. + * + * @param {number} number - The value to set for this tuning parameter. + */ +lunr.Builder.prototype.k1 = function (number) { + this._k1 = number +} + +/** + * Adds a document to the index. + * + * Before adding fields to the index the index should have been fully setup, with the document + * ref and all fields to index already having been specified. + * + * The document must have a field name as specified by the ref (by default this is 'id') and + * it should have all fields defined for indexing, though null or undefined values will not + * cause errors. + * + * Entire documents can be boosted at build time. Applying a boost to a document indicates that + * this document should rank higher in search results than other documents. + * + * @param {object} doc - The document to add to the index. + * @param {object} attributes - Optional attributes associated with this document. + * @param {number} [attributes.boost=1] - Boost applied to all terms within this document. + */ +lunr.Builder.prototype.add = function (doc, attributes) { + var docRef = doc[this._ref], + fields = Object.keys(this._fields) + + this._documents[docRef] = attributes || {} + this.documentCount += 1 + + for (var i = 0; i < fields.length; i++) { + var fieldName = fields[i], + extractor = this._fields[fieldName].extractor, + field = extractor ? extractor(doc) : doc[fieldName], + tokens = this.tokenizer(field, { + fields: [fieldName] + }), + terms = this.pipeline.run(tokens), + fieldRef = new lunr.FieldRef (docRef, fieldName), + fieldTerms = Object.create(null) + + this.fieldTermFrequencies[fieldRef] = fieldTerms + this.fieldLengths[fieldRef] = 0 + + // store the length of this field for this document + this.fieldLengths[fieldRef] += terms.length + + // calculate term frequencies for this field + for (var j = 0; j < terms.length; j++) { + var term = terms[j] + + if (fieldTerms[term] == undefined) { + fieldTerms[term] = 0 + } + + fieldTerms[term] += 1 + + // add to inverted index + // create an initial posting if one doesn't exist + if (this.invertedIndex[term] == undefined) { + var posting = Object.create(null) + posting["_index"] = this.termIndex + this.termIndex += 1 + + for (var k = 0; k < fields.length; k++) { + posting[fields[k]] = Object.create(null) + } + + this.invertedIndex[term] = posting + } + + // add an entry for this term/fieldName/docRef to the invertedIndex + if (this.invertedIndex[term][fieldName][docRef] == undefined) { + this.invertedIndex[term][fieldName][docRef] = Object.create(null) + } + + // store all whitelisted metadata about this token in the + // inverted index + for (var l = 0; l < this.metadataWhitelist.length; l++) { + var metadataKey = this.metadataWhitelist[l], + metadata = term.metadata[metadataKey] + + if (this.invertedIndex[term][fieldName][docRef][metadataKey] == undefined) { + this.invertedIndex[term][fieldName][docRef][metadataKey] = [] + } + + this.invertedIndex[term][fieldName][docRef][metadataKey].push(metadata) + } + } + + } +} + +/** + * Calculates the average document length for this index + * + * @private + */ +lunr.Builder.prototype.calculateAverageFieldLengths = function () { + + var fieldRefs = Object.keys(this.fieldLengths), + numberOfFields = fieldRefs.length, + accumulator = {}, + documentsWithField = {} + + for (var i = 0; i < numberOfFields; i++) { + var fieldRef = lunr.FieldRef.fromString(fieldRefs[i]), + field = fieldRef.fieldName + + documentsWithField[field] || (documentsWithField[field] = 0) + documentsWithField[field] += 1 + + accumulator[field] || (accumulator[field] = 0) + accumulator[field] += this.fieldLengths[fieldRef] + } + + var fields = Object.keys(this._fields) + + for (var i = 0; i < fields.length; i++) { + var fieldName = fields[i] + accumulator[fieldName] = accumulator[fieldName] / documentsWithField[fieldName] + } + + this.averageFieldLength = accumulator +} + +/** + * Builds a vector space model of every document using lunr.Vector + * + * @private + */ +lunr.Builder.prototype.createFieldVectors = function () { + var fieldVectors = {}, + fieldRefs = Object.keys(this.fieldTermFrequencies), + fieldRefsLength = fieldRefs.length, + termIdfCache = Object.create(null) + + for (var i = 0; i < fieldRefsLength; i++) { + var fieldRef = lunr.FieldRef.fromString(fieldRefs[i]), + fieldName = fieldRef.fieldName, + fieldLength = this.fieldLengths[fieldRef], + fieldVector = new lunr.Vector, + termFrequencies = this.fieldTermFrequencies[fieldRef], + terms = Object.keys(termFrequencies), + termsLength = terms.length + + + var fieldBoost = this._fields[fieldName].boost || 1, + docBoost = this._documents[fieldRef.docRef].boost || 1 + + for (var j = 0; j < termsLength; j++) { + var term = terms[j], + tf = termFrequencies[term], + termIndex = this.invertedIndex[term]._index, + idf, score, scoreWithPrecision + + if (termIdfCache[term] === undefined) { + idf = lunr.idf(this.invertedIndex[term], this.documentCount) + termIdfCache[term] = idf + } else { + idf = termIdfCache[term] + } + + score = idf * ((this._k1 + 1) * tf) / (this._k1 * (1 - this._b + this._b * (fieldLength / this.averageFieldLength[fieldName])) + tf) + score *= fieldBoost + score *= docBoost + scoreWithPrecision = Math.round(score * 1000) / 1000 + // Converts 1.23456789 to 1.234. + // Reducing the precision so that the vectors take up less + // space when serialised. Doing it now so that they behave + // the same before and after serialisation. Also, this is + // the fastest approach to reducing a number's precision in + // JavaScript. + + fieldVector.insert(termIndex, scoreWithPrecision) + } + + fieldVectors[fieldRef] = fieldVector + } + + this.fieldVectors = fieldVectors +} + +/** + * Creates a token set of all tokens in the index using lunr.TokenSet + * + * @private + */ +lunr.Builder.prototype.createTokenSet = function () { + this.tokenSet = lunr.TokenSet.fromArray( + Object.keys(this.invertedIndex).sort() + ) +} + +/** + * Builds the index, creating an instance of lunr.Index. + * + * This completes the indexing process and should only be called + * once all documents have been added to the index. + * + * @returns {lunr.Index} + */ +lunr.Builder.prototype.build = function () { + this.calculateAverageFieldLengths() + this.createFieldVectors() + this.createTokenSet() + + return new lunr.Index({ + invertedIndex: this.invertedIndex, + fieldVectors: this.fieldVectors, + tokenSet: this.tokenSet, + fields: Object.keys(this._fields), + pipeline: this.searchPipeline + }) +} + +/** + * Applies a plugin to the index builder. + * + * A plugin is a function that is called with the index builder as its context. + * Plugins can be used to customise or extend the behaviour of the index + * in some way. A plugin is just a function, that encapsulated the custom + * behaviour that should be applied when building the index. + * + * The plugin function will be called with the index builder as its argument, additional + * arguments can also be passed when calling use. The function will be called + * with the index builder as its context. + * + * @param {Function} plugin The plugin to apply. + */ +lunr.Builder.prototype.use = function (fn) { + var args = Array.prototype.slice.call(arguments, 1) + args.unshift(this) + fn.apply(this, args) +} +/** + * Contains and collects metadata about a matching document. + * A single instance of lunr.MatchData is returned as part of every + * lunr.Index~Result. + * + * @constructor + * @param {string} term - The term this match data is associated with + * @param {string} field - The field in which the term was found + * @param {object} metadata - The metadata recorded about this term in this field + * @property {object} metadata - A cloned collection of metadata associated with this document. + * @see {@link lunr.Index~Result} + */ +lunr.MatchData = function (term, field, metadata) { + var clonedMetadata = Object.create(null), + metadataKeys = Object.keys(metadata || {}) + + // Cloning the metadata to prevent the original + // being mutated during match data combination. + // Metadata is kept in an array within the inverted + // index so cloning the data can be done with + // Array#slice + for (var i = 0; i < metadataKeys.length; i++) { + var key = metadataKeys[i] + clonedMetadata[key] = metadata[key].slice() + } + + this.metadata = Object.create(null) + + if (term !== undefined) { + this.metadata[term] = Object.create(null) + this.metadata[term][field] = clonedMetadata + } +} + +/** + * An instance of lunr.MatchData will be created for every term that matches a + * document. However only one instance is required in a lunr.Index~Result. This + * method combines metadata from another instance of lunr.MatchData with this + * objects metadata. + * + * @param {lunr.MatchData} otherMatchData - Another instance of match data to merge with this one. + * @see {@link lunr.Index~Result} + */ +lunr.MatchData.prototype.combine = function (otherMatchData) { + var terms = Object.keys(otherMatchData.metadata) + + for (var i = 0; i < terms.length; i++) { + var term = terms[i], + fields = Object.keys(otherMatchData.metadata[term]) + + if (this.metadata[term] == undefined) { + this.metadata[term] = Object.create(null) + } + + for (var j = 0; j < fields.length; j++) { + var field = fields[j], + keys = Object.keys(otherMatchData.metadata[term][field]) + + if (this.metadata[term][field] == undefined) { + this.metadata[term][field] = Object.create(null) + } + + for (var k = 0; k < keys.length; k++) { + var key = keys[k] + + if (this.metadata[term][field][key] == undefined) { + this.metadata[term][field][key] = otherMatchData.metadata[term][field][key] + } else { + this.metadata[term][field][key] = this.metadata[term][field][key].concat(otherMatchData.metadata[term][field][key]) + } + + } + } + } +} + +/** + * Add metadata for a term/field pair to this instance of match data. + * + * @param {string} term - The term this match data is associated with + * @param {string} field - The field in which the term was found + * @param {object} metadata - The metadata recorded about this term in this field + */ +lunr.MatchData.prototype.add = function (term, field, metadata) { + if (!(term in this.metadata)) { + this.metadata[term] = Object.create(null) + this.metadata[term][field] = metadata + return + } + + if (!(field in this.metadata[term])) { + this.metadata[term][field] = metadata + return + } + + var metadataKeys = Object.keys(metadata) + + for (var i = 0; i < metadataKeys.length; i++) { + var key = metadataKeys[i] + + if (key in this.metadata[term][field]) { + this.metadata[term][field][key] = this.metadata[term][field][key].concat(metadata[key]) + } else { + this.metadata[term][field][key] = metadata[key] + } + } +} +/** + * A lunr.Query provides a programmatic way of defining queries to be performed + * against a {@link lunr.Index}. + * + * Prefer constructing a lunr.Query using the {@link lunr.Index#query} method + * so the query object is pre-initialized with the right index fields. + * + * @constructor + * @property {lunr.Query~Clause[]} clauses - An array of query clauses. + * @property {string[]} allFields - An array of all available fields in a lunr.Index. + */ +lunr.Query = function (allFields) { + this.clauses = [] + this.allFields = allFields +} + +/** + * Constants for indicating what kind of automatic wildcard insertion will be used when constructing a query clause. + * + * This allows wildcards to be added to the beginning and end of a term without having to manually do any string + * concatenation. + * + * The wildcard constants can be bitwise combined to select both leading and trailing wildcards. + * + * @constant + * @default + * @property {number} wildcard.NONE - The term will have no wildcards inserted, this is the default behaviour + * @property {number} wildcard.LEADING - Prepend the term with a wildcard, unless a leading wildcard already exists + * @property {number} wildcard.TRAILING - Append a wildcard to the term, unless a trailing wildcard already exists + * @see lunr.Query~Clause + * @see lunr.Query#clause + * @see lunr.Query#term + * @example query term with trailing wildcard + * query.term('foo', { wildcard: lunr.Query.wildcard.TRAILING }) + * @example query term with leading and trailing wildcard + * query.term('foo', { + * wildcard: lunr.Query.wildcard.LEADING | lunr.Query.wildcard.TRAILING + * }) + */ + +lunr.Query.wildcard = new String ("*") +lunr.Query.wildcard.NONE = 0 +lunr.Query.wildcard.LEADING = 1 +lunr.Query.wildcard.TRAILING = 2 + +/** + * Constants for indicating what kind of presence a term must have in matching documents. + * + * @constant + * @enum {number} + * @see lunr.Query~Clause + * @see lunr.Query#clause + * @see lunr.Query#term + * @example query term with required presence + * query.term('foo', { presence: lunr.Query.presence.REQUIRED }) + */ +lunr.Query.presence = { + /** + * Term's presence in a document is optional, this is the default value. + */ + OPTIONAL: 1, + + /** + * Term's presence in a document is required, documents that do not contain + * this term will not be returned. + */ + REQUIRED: 2, + + /** + * Term's presence in a document is prohibited, documents that do contain + * this term will not be returned. + */ + PROHIBITED: 3 +} + +/** + * A single clause in a {@link lunr.Query} contains a term and details on how to + * match that term against a {@link lunr.Index}. + * + * @typedef {Object} lunr.Query~Clause + * @property {string[]} fields - The fields in an index this clause should be matched against. + * @property {number} [boost=1] - Any boost that should be applied when matching this clause. + * @property {number} [editDistance] - Whether the term should have fuzzy matching applied, and how fuzzy the match should be. + * @property {boolean} [usePipeline] - Whether the term should be passed through the search pipeline. + * @property {number} [wildcard=lunr.Query.wildcard.NONE] - Whether the term should have wildcards appended or prepended. + * @property {number} [presence=lunr.Query.presence.OPTIONAL] - The terms presence in any matching documents. + */ + +/** + * Adds a {@link lunr.Query~Clause} to this query. + * + * Unless the clause contains the fields to be matched all fields will be matched. In addition + * a default boost of 1 is applied to the clause. + * + * @param {lunr.Query~Clause} clause - The clause to add to this query. + * @see lunr.Query~Clause + * @returns {lunr.Query} + */ +lunr.Query.prototype.clause = function (clause) { + if (!('fields' in clause)) { + clause.fields = this.allFields + } + + if (!('boost' in clause)) { + clause.boost = 1 + } + + if (!('usePipeline' in clause)) { + clause.usePipeline = true + } + + if (!('wildcard' in clause)) { + clause.wildcard = lunr.Query.wildcard.NONE + } + + if ((clause.wildcard & lunr.Query.wildcard.LEADING) && (clause.term.charAt(0) != lunr.Query.wildcard)) { + clause.term = "*" + clause.term + } + + if ((clause.wildcard & lunr.Query.wildcard.TRAILING) && (clause.term.slice(-1) != lunr.Query.wildcard)) { + clause.term = "" + clause.term + "*" + } + + if (!('presence' in clause)) { + clause.presence = lunr.Query.presence.OPTIONAL + } + + this.clauses.push(clause) + + return this +} + +/** + * A negated query is one in which every clause has a presence of + * prohibited. These queries require some special processing to return + * the expected results. + * + * @returns boolean + */ +lunr.Query.prototype.isNegated = function () { + for (var i = 0; i < this.clauses.length; i++) { + if (this.clauses[i].presence != lunr.Query.presence.PROHIBITED) { + return false + } + } + + return true +} + +/** + * Adds a term to the current query, under the covers this will create a {@link lunr.Query~Clause} + * to the list of clauses that make up this query. + * + * The term is used as is, i.e. no tokenization will be performed by this method. Instead conversion + * to a token or token-like string should be done before calling this method. + * + * The term will be converted to a string by calling `toString`. Multiple terms can be passed as an + * array, each term in the array will share the same options. + * + * @param {object|object[]} term - The term(s) to add to the query. + * @param {object} [options] - Any additional properties to add to the query clause. + * @returns {lunr.Query} + * @see lunr.Query#clause + * @see lunr.Query~Clause + * @example adding a single term to a query + * query.term("foo") + * @example adding a single term to a query and specifying search fields, term boost and automatic trailing wildcard + * query.term("foo", { + * fields: ["title"], + * boost: 10, + * wildcard: lunr.Query.wildcard.TRAILING + * }) + * @example using lunr.tokenizer to convert a string to tokens before using them as terms + * query.term(lunr.tokenizer("foo bar")) + */ +lunr.Query.prototype.term = function (term, options) { + if (Array.isArray(term)) { + term.forEach(function (t) { this.term(t, lunr.utils.clone(options)) }, this) + return this + } + + var clause = options || {} + clause.term = term.toString() + + this.clause(clause) + + return this +} +lunr.QueryParseError = function (message, start, end) { + this.name = "QueryParseError" + this.message = message + this.start = start + this.end = end +} + +lunr.QueryParseError.prototype = new Error +lunr.QueryLexer = function (str) { + this.lexemes = [] + this.str = str + this.length = str.length + this.pos = 0 + this.start = 0 + this.escapeCharPositions = [] +} + +lunr.QueryLexer.prototype.run = function () { + var state = lunr.QueryLexer.lexText + + while (state) { + state = state(this) + } +} + +lunr.QueryLexer.prototype.sliceString = function () { + var subSlices = [], + sliceStart = this.start, + sliceEnd = this.pos + + for (var i = 0; i < this.escapeCharPositions.length; i++) { + sliceEnd = this.escapeCharPositions[i] + subSlices.push(this.str.slice(sliceStart, sliceEnd)) + sliceStart = sliceEnd + 1 + } + + subSlices.push(this.str.slice(sliceStart, this.pos)) + this.escapeCharPositions.length = 0 + + return subSlices.join('') +} + +lunr.QueryLexer.prototype.emit = function (type) { + this.lexemes.push({ + type: type, + str: this.sliceString(), + start: this.start, + end: this.pos + }) + + this.start = this.pos +} + +lunr.QueryLexer.prototype.escapeCharacter = function () { + this.escapeCharPositions.push(this.pos - 1) + this.pos += 1 +} + +lunr.QueryLexer.prototype.next = function () { + if (this.pos >= this.length) { + return lunr.QueryLexer.EOS + } + + var char = this.str.charAt(this.pos) + this.pos += 1 + return char +} + +lunr.QueryLexer.prototype.width = function () { + return this.pos - this.start +} + +lunr.QueryLexer.prototype.ignore = function () { + if (this.start == this.pos) { + this.pos += 1 + } + + this.start = this.pos +} + +lunr.QueryLexer.prototype.backup = function () { + this.pos -= 1 +} + +lunr.QueryLexer.prototype.acceptDigitRun = function () { + var char, charCode + + do { + char = this.next() + charCode = char.charCodeAt(0) + } while (charCode > 47 && charCode < 58) + + if (char != lunr.QueryLexer.EOS) { + this.backup() + } +} + +lunr.QueryLexer.prototype.more = function () { + return this.pos < this.length +} + +lunr.QueryLexer.EOS = 'EOS' +lunr.QueryLexer.FIELD = 'FIELD' +lunr.QueryLexer.TERM = 'TERM' +lunr.QueryLexer.EDIT_DISTANCE = 'EDIT_DISTANCE' +lunr.QueryLexer.BOOST = 'BOOST' +lunr.QueryLexer.PRESENCE = 'PRESENCE' + +lunr.QueryLexer.lexField = function (lexer) { + lexer.backup() + lexer.emit(lunr.QueryLexer.FIELD) + lexer.ignore() + return lunr.QueryLexer.lexText +} + +lunr.QueryLexer.lexTerm = function (lexer) { + if (lexer.width() > 1) { + lexer.backup() + lexer.emit(lunr.QueryLexer.TERM) + } + + lexer.ignore() + + if (lexer.more()) { + return lunr.QueryLexer.lexText + } +} + +lunr.QueryLexer.lexEditDistance = function (lexer) { + lexer.ignore() + lexer.acceptDigitRun() + lexer.emit(lunr.QueryLexer.EDIT_DISTANCE) + return lunr.QueryLexer.lexText +} + +lunr.QueryLexer.lexBoost = function (lexer) { + lexer.ignore() + lexer.acceptDigitRun() + lexer.emit(lunr.QueryLexer.BOOST) + return lunr.QueryLexer.lexText +} + +lunr.QueryLexer.lexEOS = function (lexer) { + if (lexer.width() > 0) { + lexer.emit(lunr.QueryLexer.TERM) + } +} + +// This matches the separator used when tokenising fields +// within a document. These should match otherwise it is +// not possible to search for some tokens within a document. +// +// It is possible for the user to change the separator on the +// tokenizer so it _might_ clash with any other of the special +// characters already used within the search string, e.g. :. +// +// This means that it is possible to change the separator in +// such a way that makes some words unsearchable using a search +// string. +lunr.QueryLexer.termSeparator = lunr.tokenizer.separator + +lunr.QueryLexer.lexText = function (lexer) { + while (true) { + var char = lexer.next() + + if (char == lunr.QueryLexer.EOS) { + return lunr.QueryLexer.lexEOS + } + + // Escape character is '\' + if (char.charCodeAt(0) == 92) { + lexer.escapeCharacter() + continue + } + + if (char == ":") { + return lunr.QueryLexer.lexField + } + + if (char == "~") { + lexer.backup() + if (lexer.width() > 0) { + lexer.emit(lunr.QueryLexer.TERM) + } + return lunr.QueryLexer.lexEditDistance + } + + if (char == "^") { + lexer.backup() + if (lexer.width() > 0) { + lexer.emit(lunr.QueryLexer.TERM) + } + return lunr.QueryLexer.lexBoost + } + + // "+" indicates term presence is required + // checking for length to ensure that only + // leading "+" are considered + if (char == "+" && lexer.width() === 1) { + lexer.emit(lunr.QueryLexer.PRESENCE) + return lunr.QueryLexer.lexText + } + + // "-" indicates term presence is prohibited + // checking for length to ensure that only + // leading "-" are considered + if (char == "-" && lexer.width() === 1) { + lexer.emit(lunr.QueryLexer.PRESENCE) + return lunr.QueryLexer.lexText + } + + if (char.match(lunr.QueryLexer.termSeparator)) { + return lunr.QueryLexer.lexTerm + } + } +} + +lunr.QueryParser = function (str, query) { + this.lexer = new lunr.QueryLexer (str) + this.query = query + this.currentClause = {} + this.lexemeIdx = 0 +} + +lunr.QueryParser.prototype.parse = function () { + this.lexer.run() + this.lexemes = this.lexer.lexemes + + var state = lunr.QueryParser.parseClause + + while (state) { + state = state(this) + } + + return this.query +} + +lunr.QueryParser.prototype.peekLexeme = function () { + return this.lexemes[this.lexemeIdx] +} + +lunr.QueryParser.prototype.consumeLexeme = function () { + var lexeme = this.peekLexeme() + this.lexemeIdx += 1 + return lexeme +} + +lunr.QueryParser.prototype.nextClause = function () { + var completedClause = this.currentClause + this.query.clause(completedClause) + this.currentClause = {} +} + +lunr.QueryParser.parseClause = function (parser) { + var lexeme = parser.peekLexeme() + + if (lexeme == undefined) { + return + } + + switch (lexeme.type) { + case lunr.QueryLexer.PRESENCE: + return lunr.QueryParser.parsePresence + case lunr.QueryLexer.FIELD: + return lunr.QueryParser.parseField + case lunr.QueryLexer.TERM: + return lunr.QueryParser.parseTerm + default: + var errorMessage = "expected either a field or a term, found " + lexeme.type + + if (lexeme.str.length >= 1) { + errorMessage += " with value '" + lexeme.str + "'" + } + + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } +} + +lunr.QueryParser.parsePresence = function (parser) { + var lexeme = parser.consumeLexeme() + + if (lexeme == undefined) { + return + } + + switch (lexeme.str) { + case "-": + parser.currentClause.presence = lunr.Query.presence.PROHIBITED + break + case "+": + parser.currentClause.presence = lunr.Query.presence.REQUIRED + break + default: + var errorMessage = "unrecognised presence operator'" + lexeme.str + "'" + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } + + var nextLexeme = parser.peekLexeme() + + if (nextLexeme == undefined) { + var errorMessage = "expecting term or field, found nothing" + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } + + switch (nextLexeme.type) { + case lunr.QueryLexer.FIELD: + return lunr.QueryParser.parseField + case lunr.QueryLexer.TERM: + return lunr.QueryParser.parseTerm + default: + var errorMessage = "expecting term or field, found '" + nextLexeme.type + "'" + throw new lunr.QueryParseError (errorMessage, nextLexeme.start, nextLexeme.end) + } +} + +lunr.QueryParser.parseField = function (parser) { + var lexeme = parser.consumeLexeme() + + if (lexeme == undefined) { + return + } + + if (parser.query.allFields.indexOf(lexeme.str) == -1) { + var possibleFields = parser.query.allFields.map(function (f) { return "'" + f + "'" }).join(', '), + errorMessage = "unrecognised field '" + lexeme.str + "', possible fields: " + possibleFields + + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } + + parser.currentClause.fields = [lexeme.str] + + var nextLexeme = parser.peekLexeme() + + if (nextLexeme == undefined) { + var errorMessage = "expecting term, found nothing" + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } + + switch (nextLexeme.type) { + case lunr.QueryLexer.TERM: + return lunr.QueryParser.parseTerm + default: + var errorMessage = "expecting term, found '" + nextLexeme.type + "'" + throw new lunr.QueryParseError (errorMessage, nextLexeme.start, nextLexeme.end) + } +} + +lunr.QueryParser.parseTerm = function (parser) { + var lexeme = parser.consumeLexeme() + + if (lexeme == undefined) { + return + } + + parser.currentClause.term = lexeme.str.toLowerCase() + + if (lexeme.str.indexOf("*") != -1) { + parser.currentClause.usePipeline = false + } + + var nextLexeme = parser.peekLexeme() + + if (nextLexeme == undefined) { + parser.nextClause() + return + } + + switch (nextLexeme.type) { + case lunr.QueryLexer.TERM: + parser.nextClause() + return lunr.QueryParser.parseTerm + case lunr.QueryLexer.FIELD: + parser.nextClause() + return lunr.QueryParser.parseField + case lunr.QueryLexer.EDIT_DISTANCE: + return lunr.QueryParser.parseEditDistance + case lunr.QueryLexer.BOOST: + return lunr.QueryParser.parseBoost + case lunr.QueryLexer.PRESENCE: + parser.nextClause() + return lunr.QueryParser.parsePresence + default: + var errorMessage = "Unexpected lexeme type '" + nextLexeme.type + "'" + throw new lunr.QueryParseError (errorMessage, nextLexeme.start, nextLexeme.end) + } +} + +lunr.QueryParser.parseEditDistance = function (parser) { + var lexeme = parser.consumeLexeme() + + if (lexeme == undefined) { + return + } + + var editDistance = parseInt(lexeme.str, 10) + + if (isNaN(editDistance)) { + var errorMessage = "edit distance must be numeric" + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } + + parser.currentClause.editDistance = editDistance + + var nextLexeme = parser.peekLexeme() + + if (nextLexeme == undefined) { + parser.nextClause() + return + } + + switch (nextLexeme.type) { + case lunr.QueryLexer.TERM: + parser.nextClause() + return lunr.QueryParser.parseTerm + case lunr.QueryLexer.FIELD: + parser.nextClause() + return lunr.QueryParser.parseField + case lunr.QueryLexer.EDIT_DISTANCE: + return lunr.QueryParser.parseEditDistance + case lunr.QueryLexer.BOOST: + return lunr.QueryParser.parseBoost + case lunr.QueryLexer.PRESENCE: + parser.nextClause() + return lunr.QueryParser.parsePresence + default: + var errorMessage = "Unexpected lexeme type '" + nextLexeme.type + "'" + throw new lunr.QueryParseError (errorMessage, nextLexeme.start, nextLexeme.end) + } +} + +lunr.QueryParser.parseBoost = function (parser) { + var lexeme = parser.consumeLexeme() + + if (lexeme == undefined) { + return + } + + var boost = parseInt(lexeme.str, 10) + + if (isNaN(boost)) { + var errorMessage = "boost must be numeric" + throw new lunr.QueryParseError (errorMessage, lexeme.start, lexeme.end) + } + + parser.currentClause.boost = boost + + var nextLexeme = parser.peekLexeme() + + if (nextLexeme == undefined) { + parser.nextClause() + return + } + + switch (nextLexeme.type) { + case lunr.QueryLexer.TERM: + parser.nextClause() + return lunr.QueryParser.parseTerm + case lunr.QueryLexer.FIELD: + parser.nextClause() + return lunr.QueryParser.parseField + case lunr.QueryLexer.EDIT_DISTANCE: + return lunr.QueryParser.parseEditDistance + case lunr.QueryLexer.BOOST: + return lunr.QueryParser.parseBoost + case lunr.QueryLexer.PRESENCE: + parser.nextClause() + return lunr.QueryParser.parsePresence + default: + var errorMessage = "Unexpected lexeme type '" + nextLexeme.type + "'" + throw new lunr.QueryParseError (errorMessage, nextLexeme.start, nextLexeme.end) + } +} + + /** + * export the module via AMD, CommonJS or as a browser global + * Export code from https://github.com/umdjs/umd/blob/master/returnExports.js + */ + ;(function (root, factory) { + if (typeof define === 'function' && define.amd) { + // AMD. Register as an anonymous module. + define(factory) + } else if (typeof exports === 'object') { + /** + * Node. Does not work with strict CommonJS, but + * only CommonJS-like environments that support module.exports, + * like Node. + */ + module.exports = factory() + } else { + // Browser globals (root is window) + root.lunr = factory() + } + }(this, function () { + /** + * Just return a value to define the module export. + * This example returns an object, but the module + * can return a function as the exported value. + */ + return lunr + })) +})(); diff --git a/search/main.js b/search/main.js new file mode 100644 index 000000000..a5e469d7c --- /dev/null +++ b/search/main.js @@ -0,0 +1,109 @@ +function getSearchTermFromLocation() { + var sPageURL = window.location.search.substring(1); + var sURLVariables = sPageURL.split('&'); + for (var i = 0; i < sURLVariables.length; i++) { + var sParameterName = sURLVariables[i].split('='); + if (sParameterName[0] == 'q') { + return decodeURIComponent(sParameterName[1].replace(/\+/g, '%20')); + } + } +} + +function joinUrl (base, path) { + if (path.substring(0, 1) === "/") { + // path starts with `/`. Thus it is absolute. + return path; + } + if (base.substring(base.length-1) === "/") { + // base ends with `/` + return base + path; + } + return base + "/" + path; +} + +function escapeHtml (value) { + return value.replace(/&/g, '&') + .replace(/"/g, '"') + .replace(//g, '>'); +} + +function formatResult (location, title, summary) { + return ''; +} + +function displayResults (results) { + var search_results = document.getElementById("mkdocs-search-results"); + while (search_results.firstChild) { + search_results.removeChild(search_results.firstChild); + } + if (results.length > 0){ + for (var i=0; i < results.length; i++){ + var result = results[i]; + var html = formatResult(result.location, result.title, result.summary); + search_results.insertAdjacentHTML('beforeend', html); + } + } else { + var noResultsText = search_results.getAttribute('data-no-results-text'); + if (!noResultsText) { + noResultsText = "No results found"; + } + search_results.insertAdjacentHTML('beforeend', '

' + noResultsText + '

'); + } +} + +function doSearch () { + var query = document.getElementById('mkdocs-search-query').value; + if (query.length > min_search_length) { + if (!window.Worker) { + displayResults(search(query)); + } else { + searchWorker.postMessage({query: query}); + } + } else { + // Clear results for short queries + displayResults([]); + } +} + +function initSearch () { + var search_input = document.getElementById('mkdocs-search-query'); + if (search_input) { + search_input.addEventListener("keyup", doSearch); + } + var term = getSearchTermFromLocation(); + if (term) { + search_input.value = term; + doSearch(); + } +} + +function onWorkerMessage (e) { + if (e.data.allowSearch) { + initSearch(); + } else if (e.data.results) { + var results = e.data.results; + displayResults(results); + } else if (e.data.config) { + min_search_length = e.data.config.min_search_length-1; + } +} + +if (!window.Worker) { + console.log('Web Worker API not supported'); + // load index in main thread + $.getScript(joinUrl(base_url, "search/worker.js")).done(function () { + console.log('Loaded worker'); + init(); + window.postMessage = function (msg) { + onWorkerMessage({data: msg}); + }; + }).fail(function (jqxhr, settings, exception) { + console.error('Could not load worker.js'); + }); +} else { + // Wrap search in a web worker + var searchWorker = new Worker(joinUrl(base_url, "search/worker.js")); + searchWorker.postMessage({init: true}); + searchWorker.onmessage = onWorkerMessage; +} diff --git a/search/search_index.json b/search/search_index.json new file mode 100644 index 000000000..c2450a67c --- /dev/null +++ b/search/search_index.json @@ -0,0 +1 @@ +{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"The Cell Ontology \u00b6 The Cell Ontology (CL) is an OBO Foundry ontology covering the domain of canonical, natural biological cell types. OLS \u00b6 The recommended way to browse the ontology is through the ontology lookup service (OLS) . Accessing CL \u00b6 The main release files can be found in our GitHub repository . The main release file can also be found here . Contributing \u00b6 To see how you can contribute to the cell ontology, please visit our Contributing page . Documentation \u00b6 The documentations here is intended for maintainers and editors of the cell ontology. For general guidance on editing OBO-(ish) ontologies, please see the obook . Cell Ontology Website \u00b6 You can also access our front facing website at https://cell-ontology.github.io/","title":"Getting started"},{"location":"#the-cell-ontology","text":"The Cell Ontology (CL) is an OBO Foundry ontology covering the domain of canonical, natural biological cell types.","title":"The Cell Ontology"},{"location":"#ols","text":"The recommended way to browse the ontology is through the ontology lookup service (OLS) .","title":"OLS"},{"location":"#accessing-cl","text":"The main release files can be found in our GitHub repository . The main release file can also be found here .","title":"Accessing CL"},{"location":"#contributing","text":"To see how you can contribute to the cell ontology, please visit our Contributing page .","title":"Contributing"},{"location":"#documentation","text":"The documentations here is intended for maintainers and editors of the cell ontology. For general guidance on editing OBO-(ish) ontologies, please see the obook .","title":"Documentation"},{"location":"#cell-ontology-website","text":"You can also access our front facing website at https://cell-ontology.github.io/","title":"Cell Ontology Website"},{"location":"Adding_classes_from_another_ontology/","text":"How to add (import) classes to the Cell Ontology (CL) from another ontology \u00b6 1. Follow steps 1 - 5 under the heading Protege-based declaration. \u00b6 NB: Even though the instructions state that this workflow is to be avoided, the other solutions in the current documentation are out of date. 2. Refresh the imports \u00b6 To refresh the imports, open Docker so it is running in the background. Then open Terminal, navigate to src/ontology directory in the cell-ontology repository and run: sh run.sh make imports/merged_import.owl Running the above command requires > 8GB RAM and sufficient computational power. If the refresh fails to complete due to hardware limitations, create a new issue in GitHub detailing which class(es) need to be imported and another editor can add it on your behalf. Once the imports are refreshed, return to Prot\u00e9g\u00e9, add the logical axioms that include the newly imported class(es) and create a pull request per standard procedure. Note that the import refresh process seems to be quite laborious/computationally expensive as-is, and a centralised database approach may be an improved longterm solution.","title":"Importing classes from another ontology"},{"location":"Adding_classes_from_another_ontology/#how-to-add-import-classes-to-the-cell-ontology-cl-from-another-ontology","text":"","title":"How to add (import) classes to the Cell Ontology (CL) from another ontology"},{"location":"Adding_classes_from_another_ontology/#1-follow-steps-1-5-under-the-heading-protege-based-declaration","text":"NB: Even though the instructions state that this workflow is to be avoided, the other solutions in the current documentation are out of date.","title":"1. Follow steps 1 - 5 under the heading Protege-based declaration."},{"location":"Adding_classes_from_another_ontology/#2-refresh-the-imports","text":"To refresh the imports, open Docker so it is running in the background. Then open Terminal, navigate to src/ontology directory in the cell-ontology repository and run: sh run.sh make imports/merged_import.owl Running the above command requires > 8GB RAM and sufficient computational power. If the refresh fails to complete due to hardware limitations, create a new issue in GitHub detailing which class(es) need to be imported and another editor can add it on your behalf. Once the imports are refreshed, return to Prot\u00e9g\u00e9, add the logical axioms that include the newly imported class(es) and create a pull request per standard procedure. Note that the import refresh process seems to be quite laborious/computationally expensive as-is, and a centralised database approach may be an improved longterm solution.","title":"2. Refresh the imports"},{"location":"Create_upper_level_slim/","text":"SOP for adding a new slim: \u00b6 Intro \u00b6 An upper slim is a set of terms for summarising annotations. CL has both a general slim and domain-specific slims allowing for the generation of general summaries or for domain specific ones. All slims have a context that covers all cells in the domain of interest. In order to fullfil the summarising use-case, a slim should have good % coverage of the domain and, if possible, avoid excluding major cell types. The following are potentially a problem for the grouping use-case: Classes in the slim with very small numbers of subclasses: Having 0-1 subclass => no capacity to summarise. Classes with disproportionately large numbers of subclasses. Overlapping classes - these clash with some types of summary - e.g. pie charts. Potential clashing concerns: It seems reasonable to want to make sure that very important cell types are covered and are not obscured in generating summaries, but this desire can clash with the considerations above. Some judgement is needed to balance these concerns. There is no perfect solution, but some solutions are better (at fulfilling the use case) than others. Name and definition \u00b6 It is important that the name of the slim accurately reflects the content it covers. Specifically, if the intention is to cover all cell types that are specific for an organ, the slim's name should be in the format of \"organ_upper_slim\". This ensures clarity and helps users understand the scope of the slim. Furthermore, the description of the slim should follow a consistent pattern. It is recommended to use the following structure: \"a subset of general classes related to specific cell types in the [organ or specific context]\". This format provides a concise and informative description of the slim, helping users identify its purpose and content. What files to create and edit \u00b6 1. Preparing the subset: \u00b6 Create XXX_upper_slim in Protege (change \"XXX\" to the subset label). See Adding a new Subset . Create a CSV table with the following characteristics (Find examples in src/templates ): 3 columns ID subset label ID AI oboInOwl:inSubset CL:####### http://purl.obolibrary.org/obo/cl#XXX_upper_slim CL term ... http://purl.obolibrary.org/obo/cl#XXX_upper_slim ... Save the CSV file with the name 'XXX_upper_slim.csv' in the src/templates directory. Modify src/ontology/cl-odk.yaml introducing new lines for the new slim (change \"XXX\" to the subset label): - id: XXX_upper_slim - filename: XXX_upper_slim.owl use_template: True templates: - XXX_upper_slim.csv ![image](https://github.com/obophenotype/cell-ontology/assets/94959119/254ad25f-7bf2-4ac2-afe2-9ad067d9c1ea) 2. Generating the Slim OWL file: \u00b6 Navigate to the src/ontology file in the terminal. Make sure Docker is running. Run the command: sh run.sh make update_repo 3. Modifying the Catalog: \u00b6 Open the src/ontology/catalog-v001.xml file. Add the following line (change \"XXX\" to the subset label): ` - 4. Preparing the Upper Level Slim import to CL: \u00b6 Open src/ontology/cl-edit.owl . Add the following import statement (change \"XXX\" to the subset label): Import() 5. Updating the slim owl file: \u00b6 Run the command: sh run.sh make all_subsets -B - Alternatively, run the following command to run it anyway even if it says it is up to date (change \"XXX\" to the subset label): sh run.sh make components/XXX_upper_slim.owl -B 6. Testing Slim Coverage: \u00b6 Open src/ontology/cl.Makefile . Add the subset label to SLIM_TEMPLATES (without _upper_slim!!!). Add the term that will be used to test coverage Add: $(REPORTDIR)/XXX_upper_slim.csv: $(TEMPLATEDIR)/XXX_upper_slim.csv $(eval TERM_ID := $(YYY)) $(COVERAGECMD) (substitute \"XXX\" to the subset label and YYY for the tested label) ![image](https://github.com/obophenotype/cell-ontology/assets/94959119/7eb18255-0ef7-4fbc-9f7f-e582372165bf) Using the terminal, navigate to src/ontology . Run the command: sh run.sh make slim_coverage Understanding reports \u00b6 The reports can be accessed at src/ontology/reports/XXX_upper_slim.csv . First, the coverage percentage is displayed, indicating the proportion of cells covered. Secondly, the number of cells covered by each term of the subset is provided. Finally, a list is presented, indicating all the terms that were expected to be covered but are not currently included. Ideally, terms would have more than 1 cell covered. Furthermore, a term covering hundreds of cells might indicate that it is too general, and a more specific term (or multiple) should be evaluated, specially if it overlaps with other terms of the subset. Example: For the eye_upper_slim, 'retinal cell' is a (too) general term that overlaps other grouping terms such as 'retinal bipolar neuron', 'retina horizontal cell' or 'amacrine cell'. In the case that there is overlapping of terms (term A in the subset covers term B of the subset), a coverage file will be created and it can be accessed at src/ontology/reports/overlapping_terms_XXX_upper_slim.csv .","title":"SOP for adding a new slim:"},{"location":"Create_upper_level_slim/#sop-for-adding-a-new-slim","text":"","title":"SOP for adding a new slim:"},{"location":"Create_upper_level_slim/#intro","text":"An upper slim is a set of terms for summarising annotations. CL has both a general slim and domain-specific slims allowing for the generation of general summaries or for domain specific ones. All slims have a context that covers all cells in the domain of interest. In order to fullfil the summarising use-case, a slim should have good % coverage of the domain and, if possible, avoid excluding major cell types. The following are potentially a problem for the grouping use-case: Classes in the slim with very small numbers of subclasses: Having 0-1 subclass => no capacity to summarise. Classes with disproportionately large numbers of subclasses. Overlapping classes - these clash with some types of summary - e.g. pie charts. Potential clashing concerns: It seems reasonable to want to make sure that very important cell types are covered and are not obscured in generating summaries, but this desire can clash with the considerations above. Some judgement is needed to balance these concerns. There is no perfect solution, but some solutions are better (at fulfilling the use case) than others.","title":"Intro"},{"location":"Create_upper_level_slim/#name-and-definition","text":"It is important that the name of the slim accurately reflects the content it covers. Specifically, if the intention is to cover all cell types that are specific for an organ, the slim's name should be in the format of \"organ_upper_slim\". This ensures clarity and helps users understand the scope of the slim. Furthermore, the description of the slim should follow a consistent pattern. It is recommended to use the following structure: \"a subset of general classes related to specific cell types in the [organ or specific context]\". This format provides a concise and informative description of the slim, helping users identify its purpose and content.","title":"Name and definition"},{"location":"Create_upper_level_slim/#what-files-to-create-and-edit","text":"","title":"What files to create and edit"},{"location":"Create_upper_level_slim/#1-preparing-the-subset","text":"Create XXX_upper_slim in Protege (change \"XXX\" to the subset label). See Adding a new Subset . Create a CSV table with the following characteristics (Find examples in src/templates ): 3 columns ID subset label ID AI oboInOwl:inSubset CL:####### http://purl.obolibrary.org/obo/cl#XXX_upper_slim CL term ... http://purl.obolibrary.org/obo/cl#XXX_upper_slim ... Save the CSV file with the name 'XXX_upper_slim.csv' in the src/templates directory. Modify src/ontology/cl-odk.yaml introducing new lines for the new slim (change \"XXX\" to the subset label): - id: XXX_upper_slim - filename: XXX_upper_slim.owl use_template: True templates: - XXX_upper_slim.csv ![image](https://github.com/obophenotype/cell-ontology/assets/94959119/254ad25f-7bf2-4ac2-afe2-9ad067d9c1ea)","title":"1. Preparing the subset:"},{"location":"Create_upper_level_slim/#2-generating-the-slim-owl-file","text":"Navigate to the src/ontology file in the terminal. Make sure Docker is running. Run the command: sh run.sh make update_repo","title":"2. Generating the Slim OWL file:"},{"location":"Create_upper_level_slim/#3-modifying-the-catalog","text":"Open the src/ontology/catalog-v001.xml file. Add the following line (change \"XXX\" to the subset label): ` -","title":"3. Modifying the Catalog:"},{"location":"Create_upper_level_slim/#4-preparing-the-upper-level-slim-import-to-cl","text":"Open src/ontology/cl-edit.owl . Add the following import statement (change \"XXX\" to the subset label): Import()","title":"4. Preparing the Upper Level Slim import to CL:"},{"location":"Create_upper_level_slim/#5-updating-the-slim-owl-file","text":"Run the command: sh run.sh make all_subsets -B - Alternatively, run the following command to run it anyway even if it says it is up to date (change \"XXX\" to the subset label): sh run.sh make components/XXX_upper_slim.owl -B","title":"5. Updating the slim owl file:"},{"location":"Create_upper_level_slim/#6-testing-slim-coverage","text":"Open src/ontology/cl.Makefile . Add the subset label to SLIM_TEMPLATES (without _upper_slim!!!). Add the term that will be used to test coverage Add: $(REPORTDIR)/XXX_upper_slim.csv: $(TEMPLATEDIR)/XXX_upper_slim.csv $(eval TERM_ID := $(YYY)) $(COVERAGECMD) (substitute \"XXX\" to the subset label and YYY for the tested label) ![image](https://github.com/obophenotype/cell-ontology/assets/94959119/7eb18255-0ef7-4fbc-9f7f-e582372165bf) Using the terminal, navigate to src/ontology . Run the command: sh run.sh make slim_coverage","title":"6. Testing Slim Coverage:"},{"location":"Create_upper_level_slim/#understanding-reports","text":"The reports can be accessed at src/ontology/reports/XXX_upper_slim.csv . First, the coverage percentage is displayed, indicating the proportion of cells covered. Secondly, the number of cells covered by each term of the subset is provided. Finally, a list is presented, indicating all the terms that were expected to be covered but are not currently included. Ideally, terms would have more than 1 cell covered. Furthermore, a term covering hundreds of cells might indicate that it is too general, and a more specific term (or multiple) should be evaluated, specially if it overlaps with other terms of the subset. Example: For the eye_upper_slim, 'retinal cell' is a (too) general term that overlaps other grouping terms such as 'retinal bipolar neuron', 'retina horizontal cell' or 'amacrine cell'. In the case that there is overlapping of terms (term A in the subset covers term B of the subset), a coverage file will be created and it can be accessed at src/ontology/reports/overlapping_terms_XXX_upper_slim.csv .","title":"Understanding reports"},{"location":"Fixing_xsdstring_diffs/","text":"Fixing ^^xsd:string Diffs \u00b6 When you make edits, sometimes there will be large amounts of unintended differences that show up that involves the removal of ^^xsd:string . If so, you can resolve them by following normalising your cl-edit.owl file. SOP \u00b6 Update your file from Master (see 'How to resolve merge conflicts' for instructions on how to do this including how to resolve clashes while doing this). in the terminal, set directory to the ontology folder in CL: cd .../GitHub/cell-ontology/src/ontology Run the normaliser in terminal: If you have docker installed: sh run.sh make normalise_xsd_string If you do not have docker installed: make normalise_xsd_string If make is not installed, on MAC: sed -i '' -E \"s/Annotation[(](oboInOwl[:]hasDbXref [\\\"][^\\\"]*[\\\"])[)]/Annotation(\\1^^xsd:string)/g\" cl-edit.owl This should resolve your ^^xsd:string issue, after which, you can handle your pull request as per usual.","title":"Fixing unintended ^^xsd:string diffs"},{"location":"Fixing_xsdstring_diffs/#fixing-xsdstring-diffs","text":"When you make edits, sometimes there will be large amounts of unintended differences that show up that involves the removal of ^^xsd:string . If so, you can resolve them by following normalising your cl-edit.owl file.","title":"Fixing ^^xsd:string Diffs"},{"location":"Fixing_xsdstring_diffs/#sop","text":"Update your file from Master (see 'How to resolve merge conflicts' for instructions on how to do this including how to resolve clashes while doing this). in the terminal, set directory to the ontology folder in CL: cd .../GitHub/cell-ontology/src/ontology Run the normaliser in terminal: If you have docker installed: sh run.sh make normalise_xsd_string If you do not have docker installed: make normalise_xsd_string If make is not installed, on MAC: sed -i '' -E \"s/Annotation[(](oboInOwl[:]hasDbXref [\\\"][^\\\"]*[\\\"])[)]/Annotation(\\1^^xsd:string)/g\" cl-edit.owl This should resolve your ^^xsd:string issue, after which, you can handle your pull request as per usual.","title":"SOP"},{"location":"Keeping_ontology_terms_up_to_date/","text":"Keeping cell ontology annotation up to date \u00b6 Cell ontology identifiers (IRIs) are never lost, but they are occasionally deprecated. On the rare occasions that this happens, all logical links to other ontology terms (e.g. recording classification or partonomy) are removed and term is tagged with the annotation owl:deprecated True . To aid migration of annotations to the latest standard, these terms are also annotated with either a term_replaced_by or a consider tag. A term_replaced_by annotation is used to record the ID of a term it is safe to auto-migrate annotations to. More rarely, consider is used to record multiple potential replacement terms requiring human consideration to map. In these cases, a comment will be present to provide guidance for mapping. The Ontology Lookup Service API provides a convenient way to check for deprecated terms & find replacements. The term http://purl.obolibrary.org/obo/CL_0000375 has been deprecated and has that tag term_replaced_by Querying the OLS API for this: https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375. (Note the query IRI must be double encoded) Returns: { \"iri\" : \"http://purl.obolibrary.org/obo/CL_0000375\", \"label\" : \"obsolete osteoprogenitor cell\", \"description\" : null, \"annotation\" : { \"database_cross_reference\" : [ \"BTO:0002051\" ], \"has_obo_namespace\" : [ \"cell\" ], \"term replaced by\" : [ \"CL:0007010\" ] }, \"synonyms\" : null, \"ontology_name\" : \"cl\", \"ontology_prefix\" : \"CL\", \"ontology_iri\" : \"http://purl.obolibrary.org/obo/cl.owl\", \"is_obsolete\" : true, \"term_replaced_by\" : \"CL:0007010\", \"is_defining_ontology\" : true, \"has_children\" : false, \"is_root\" : true, \"short_form\" : \"CL_0000375\", \"obo_id\" : \"CL:0000375\", \"in_subset\" : null, \"obo_definition_citation\" : null, \"obo_xref\" : [{\"database\":\"BTO\",\"id\":\"0002051\",\"description\":null,\"url\":\"http://purl.obolibrary.org/obo/BTO_0002051\"}], \"obo_synonym\" : null, \"is_preferred_root\" : false, \"_links\" : { \"self\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375\" }, \"graph\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375/graph\" } } } The term_replaced_by key points to the ID of a safe replacement term: CL:0007010. This is a CURIE for http://purl.obolibrary.org/obo/CL_0007010 * consider The term http://purl.obolibrary.org/obo/CL_0000144 has been deprecated and has a consider tag pointing to multiple possible replacement terms, along with a comment for guidance. Querying the OLS API for this: https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144 Returns { \"iri\" : \"http://purl.obolibrary.org/obo/CL_0000144\", \"label\" : \"obsolete cell by function\", \"description\" : [ \"OBSOLETE: A classification of cells by their primary end goal or behavior.\" ], \"annotation\" : { \"comment\" : [ \"This term was made obsolete because there is no difference in meaning between it and 'cell', as any cell can be classified by its function or behavior. If you have used this term in annotation, please replace it with cell (CL:0000000), native cell (CL:0000003), or cell in vitro (CL:0001034) as appropriate.\" ], \"consider\" : [ \"CL:0001034\", \"CL:0000000\", \"CL:0000003\" ], \"has_obo_namespace\" : [ \"cell\" ] }, \"synonyms\" : null, \"ontology_name\" : \"cl\", \"ontology_prefix\" : \"CL\", \"ontology_iri\" : \"http://purl.obolibrary.org/obo/cl.owl\", \"is_obsolete\" : true, \"term_replaced_by\" : null, \"is_defining_ontology\" : true, \"has_children\" : false, \"is_root\" : true, \"short_form\" : \"CL_0000144\", \"obo_id\" : \"CL:0000144\", \"in_subset\" : null, \"obo_definition_citation\" : [{\"definition\":\"OBSOLETE: A classification of cells by their primary end goal or behavior.\",\"oboXrefs\":[{\"database\":\"FB\",\"id\":\"ma\",\"description\":null,\"url\":\"http://flybase.org/reports/ma.html\"}]}], \"obo_xref\" : null, \"obo_synonym\" : null, \"is_preferred_root\" : false, \"_links\" : { \"self\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144\" }, \"graph\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144/graph\" } } } * Warning - due to legacy issues, the values of these tags are either a curie (CL:0000123) or short_form ID (CL_0000123) rather than an iri. Handling code needs to deal with both of these formats.","title":"Keep terms up-to-date"},{"location":"Keeping_ontology_terms_up_to_date/#keeping-cell-ontology-annotation-up-to-date","text":"Cell ontology identifiers (IRIs) are never lost, but they are occasionally deprecated. On the rare occasions that this happens, all logical links to other ontology terms (e.g. recording classification or partonomy) are removed and term is tagged with the annotation owl:deprecated True . To aid migration of annotations to the latest standard, these terms are also annotated with either a term_replaced_by or a consider tag. A term_replaced_by annotation is used to record the ID of a term it is safe to auto-migrate annotations to. More rarely, consider is used to record multiple potential replacement terms requiring human consideration to map. In these cases, a comment will be present to provide guidance for mapping. The Ontology Lookup Service API provides a convenient way to check for deprecated terms & find replacements. The term http://purl.obolibrary.org/obo/CL_0000375 has been deprecated and has that tag term_replaced_by Querying the OLS API for this: https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375. (Note the query IRI must be double encoded) Returns: { \"iri\" : \"http://purl.obolibrary.org/obo/CL_0000375\", \"label\" : \"obsolete osteoprogenitor cell\", \"description\" : null, \"annotation\" : { \"database_cross_reference\" : [ \"BTO:0002051\" ], \"has_obo_namespace\" : [ \"cell\" ], \"term replaced by\" : [ \"CL:0007010\" ] }, \"synonyms\" : null, \"ontology_name\" : \"cl\", \"ontology_prefix\" : \"CL\", \"ontology_iri\" : \"http://purl.obolibrary.org/obo/cl.owl\", \"is_obsolete\" : true, \"term_replaced_by\" : \"CL:0007010\", \"is_defining_ontology\" : true, \"has_children\" : false, \"is_root\" : true, \"short_form\" : \"CL_0000375\", \"obo_id\" : \"CL:0000375\", \"in_subset\" : null, \"obo_definition_citation\" : null, \"obo_xref\" : [{\"database\":\"BTO\",\"id\":\"0002051\",\"description\":null,\"url\":\"http://purl.obolibrary.org/obo/BTO_0002051\"}], \"obo_synonym\" : null, \"is_preferred_root\" : false, \"_links\" : { \"self\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375\" }, \"graph\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000375/graph\" } } } The term_replaced_by key points to the ID of a safe replacement term: CL:0007010. This is a CURIE for http://purl.obolibrary.org/obo/CL_0007010 * consider The term http://purl.obolibrary.org/obo/CL_0000144 has been deprecated and has a consider tag pointing to multiple possible replacement terms, along with a comment for guidance. Querying the OLS API for this: https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144 Returns { \"iri\" : \"http://purl.obolibrary.org/obo/CL_0000144\", \"label\" : \"obsolete cell by function\", \"description\" : [ \"OBSOLETE: A classification of cells by their primary end goal or behavior.\" ], \"annotation\" : { \"comment\" : [ \"This term was made obsolete because there is no difference in meaning between it and 'cell', as any cell can be classified by its function or behavior. If you have used this term in annotation, please replace it with cell (CL:0000000), native cell (CL:0000003), or cell in vitro (CL:0001034) as appropriate.\" ], \"consider\" : [ \"CL:0001034\", \"CL:0000000\", \"CL:0000003\" ], \"has_obo_namespace\" : [ \"cell\" ] }, \"synonyms\" : null, \"ontology_name\" : \"cl\", \"ontology_prefix\" : \"CL\", \"ontology_iri\" : \"http://purl.obolibrary.org/obo/cl.owl\", \"is_obsolete\" : true, \"term_replaced_by\" : null, \"is_defining_ontology\" : true, \"has_children\" : false, \"is_root\" : true, \"short_form\" : \"CL_0000144\", \"obo_id\" : \"CL:0000144\", \"in_subset\" : null, \"obo_definition_citation\" : [{\"definition\":\"OBSOLETE: A classification of cells by their primary end goal or behavior.\",\"oboXrefs\":[{\"database\":\"FB\",\"id\":\"ma\",\"description\":null,\"url\":\"http://flybase.org/reports/ma.html\"}]}], \"obo_xref\" : null, \"obo_synonym\" : null, \"is_preferred_root\" : false, \"_links\" : { \"self\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144\" }, \"graph\" : { \"href\" : \"https://www.ebi.ac.uk/ols/api/ontologies/cl/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FCL_0000144/graph\" } } } * Warning - due to legacy issues, the values of these tags are either a curie (CL:0000123) or short_form ID (CL_0000123) rather than an iri. Handling code needs to deal with both of these formats.","title":"Keeping cell ontology annotation up to date"},{"location":"annotation_properties/","text":"Annotation Properties \u00b6 Note- this page is currently under development. The Cell Ontology has the following annotation properties: Annotation property Description Example term Example annotation Must have? Only one use per term is allowed? consider To be used on obsoleted classes, to point to a term that should be considered by curators for use in place of the obsoleted term. Multiple consider terms are allowed. It can be useful to combine this with a comment to indicate when replacement would be appropriate. CL:0000610 obsolete plant cell PO:0009002 No No created_by Added automatically on term creation with standard Protege settings. Ideally, this should use the \"supplied user name\" in the Protege User Details preference pane. This has been inconsistently applied in the past. CL:0002518 tmeehan Should Yes creation_date Added automatically on term creation with standard Protege settings. CL:0002518 2011-02-08T10:46:34Z Should Yes database_cross_reference Citable references that have helped generate the term and term's definition. Includes PubMed IDs (in the format PMID:XXXXXXXX). CL:0011005 GABAergic interneuron PMID:29724907 Should No dc:contributor Use this to annotate a whole ontology file with the identifier of a contributor. ORCID preferred. N/A https://orcid.org/0000-0001-9990-8331 Nice to have, if applicable No dc:creator Coming Soon CL:0001201 B cell, CD19-positive https://orcid.org/0000-0001-9990-8331 No Yes dc:date Coming soon CL:0001065 innate lymphoid cell 2017-01-30T20:20:48Z No Yes dc:description Use this to annotate a whole ontology file with a brief description of the ontology. N/A An ontology of cell types. No No dc:title Use this to annotate an ontology, giving it a human readable title. N/A Cell Ontology No No dcterms:license Use to attach a license to the whole ontology file. N/A http://creativecommons.org/licenses/by/4.0/ No No definition The textual definition for the ontology class. CL:0000946 antibody secreting cell A lymphocyte of B lineage that is devoted to secreting large amounts of immunoglobulin. Must Yes 'expand expression to' Coming soon Coming soon Coming soon No No foaf:depicted_by Use this to add a link to an image that depicts an example of an entity referred to by the term Coming soon Coming soon No No has_alternative_id In CL this is a legacy property. Do not use. CL:0000059 ameloblast CL:0000053 No No has_broad_synonym Used for synonyms where the primary definition accurately describes the synonym, but the definition of the synonym may encompass other structures as well. In some cases where a broad synonym is given, it will be a broad synonym for more than one ontology term. You are encouraged to add a reference that uses the term in this way. CL:0000365 animal zygote zygote No No has_exact_synonym Used for synonyms where the definition of the synonym is exactly the same as primary term definition. This is used when the same class can have more than one name. You are encouraged to add a reference that uses the term in this way. CL:0000622 acinar cell acinic cell Nice to have, if applicable No has_narrow_synonym Used for synonyms where the definition of the synonym is the same as the primary definition, but has additional qualifiers. You are encouraged to add a reference that uses the term in this way. CL:0000362 epidermal cell epithelial cell of skin No No has_obo_namespace This is a legacy annotation property. Do not add this manually. CL:0001061 abnormal cell cell No No has_related_synonym This scope is applied when a word of phrase has been used synonymously with the primary term name in the literature, but the usage is not strictly correct. That is, the synonym in fact has a slightly different meaning than the primary term name. Since users may not be aware that the synonym was being used incorrectly when searching for a term, related synonyms are included. CL:0000902 induced T-regulatory cell adaptive Treg No No has_synonym_type The target of this relation must be an annotation property of type 'synonym_type_property'. N/A N/A No No IAO_0000116 Coming soon Coming soon Coming soon No No id Automatically added by some pathways. Do not add manually. If duplicating a term (with the duplicate getting a new ID), it should be deleted. CL:2000074 splenocyte CL:2000074 Yes Yes in_subset Used to add subset tags, used in conjunction with subset_property CL:0000039 germ line cell _upper_level No No is_inferred This annotation property is used in some automated pipelines. Do not add manually Coming soon Coming soon No No rdfs:comment Use to add a clarifying comment to a term. This can be useful for adding examples and for clarifying terminological confusions. CL:0007016 adaxial cell In teleosts, adaxial cells give rise to slow muscle myoblasts. No Yes rdfs:isDefinedBy Do not add manually. Coming soon Coming soon No Yes rdfs:label Primary name - used as a display name by Protege (with standard settings) and most downstream consumers. Add only one of these. It must be unique within an ontology. CL:0000418 arcade cell arcade cell Must Yes RO_0002161 Coming soon Coming soon No No 'see also' Used to link to a webpage, such as a GitHub ticket. CL:0000134 mesenchymal stem cell https://github.com/obophenotype/cell-ontology/issues/474 No No shorthand Added automatically by some pipelines. Do not add manually Coming soon Coming soon No No subset_property A grouping class for subset tags. N/A N/A No No subset_property: added_for_HCA A subset tag for terms that were requested by the Human Cell Atlas. N/A N/A No No subset property: location_grouping A subset tag for cell types from a particular anatomical location. N/A N/A No No synonym_type_property A grouping class for synonym tags. N/A N/A No No 'term replaced by' To be used on obsolete terms to indicate a term that can be automatically substituted for the obsoleted term. Coming soon Coming soon No No","title":"CL annotation properties"},{"location":"annotation_properties/#annotation-properties","text":"Note- this page is currently under development. The Cell Ontology has the following annotation properties: Annotation property Description Example term Example annotation Must have? Only one use per term is allowed? consider To be used on obsoleted classes, to point to a term that should be considered by curators for use in place of the obsoleted term. Multiple consider terms are allowed. It can be useful to combine this with a comment to indicate when replacement would be appropriate. CL:0000610 obsolete plant cell PO:0009002 No No created_by Added automatically on term creation with standard Protege settings. Ideally, this should use the \"supplied user name\" in the Protege User Details preference pane. This has been inconsistently applied in the past. CL:0002518 tmeehan Should Yes creation_date Added automatically on term creation with standard Protege settings. CL:0002518 2011-02-08T10:46:34Z Should Yes database_cross_reference Citable references that have helped generate the term and term's definition. Includes PubMed IDs (in the format PMID:XXXXXXXX). CL:0011005 GABAergic interneuron PMID:29724907 Should No dc:contributor Use this to annotate a whole ontology file with the identifier of a contributor. ORCID preferred. N/A https://orcid.org/0000-0001-9990-8331 Nice to have, if applicable No dc:creator Coming Soon CL:0001201 B cell, CD19-positive https://orcid.org/0000-0001-9990-8331 No Yes dc:date Coming soon CL:0001065 innate lymphoid cell 2017-01-30T20:20:48Z No Yes dc:description Use this to annotate a whole ontology file with a brief description of the ontology. N/A An ontology of cell types. No No dc:title Use this to annotate an ontology, giving it a human readable title. N/A Cell Ontology No No dcterms:license Use to attach a license to the whole ontology file. N/A http://creativecommons.org/licenses/by/4.0/ No No definition The textual definition for the ontology class. CL:0000946 antibody secreting cell A lymphocyte of B lineage that is devoted to secreting large amounts of immunoglobulin. Must Yes 'expand expression to' Coming soon Coming soon Coming soon No No foaf:depicted_by Use this to add a link to an image that depicts an example of an entity referred to by the term Coming soon Coming soon No No has_alternative_id In CL this is a legacy property. Do not use. CL:0000059 ameloblast CL:0000053 No No has_broad_synonym Used for synonyms where the primary definition accurately describes the synonym, but the definition of the synonym may encompass other structures as well. In some cases where a broad synonym is given, it will be a broad synonym for more than one ontology term. You are encouraged to add a reference that uses the term in this way. CL:0000365 animal zygote zygote No No has_exact_synonym Used for synonyms where the definition of the synonym is exactly the same as primary term definition. This is used when the same class can have more than one name. You are encouraged to add a reference that uses the term in this way. CL:0000622 acinar cell acinic cell Nice to have, if applicable No has_narrow_synonym Used for synonyms where the definition of the synonym is the same as the primary definition, but has additional qualifiers. You are encouraged to add a reference that uses the term in this way. CL:0000362 epidermal cell epithelial cell of skin No No has_obo_namespace This is a legacy annotation property. Do not add this manually. CL:0001061 abnormal cell cell No No has_related_synonym This scope is applied when a word of phrase has been used synonymously with the primary term name in the literature, but the usage is not strictly correct. That is, the synonym in fact has a slightly different meaning than the primary term name. Since users may not be aware that the synonym was being used incorrectly when searching for a term, related synonyms are included. CL:0000902 induced T-regulatory cell adaptive Treg No No has_synonym_type The target of this relation must be an annotation property of type 'synonym_type_property'. N/A N/A No No IAO_0000116 Coming soon Coming soon Coming soon No No id Automatically added by some pathways. Do not add manually. If duplicating a term (with the duplicate getting a new ID), it should be deleted. CL:2000074 splenocyte CL:2000074 Yes Yes in_subset Used to add subset tags, used in conjunction with subset_property CL:0000039 germ line cell _upper_level No No is_inferred This annotation property is used in some automated pipelines. Do not add manually Coming soon Coming soon No No rdfs:comment Use to add a clarifying comment to a term. This can be useful for adding examples and for clarifying terminological confusions. CL:0007016 adaxial cell In teleosts, adaxial cells give rise to slow muscle myoblasts. No Yes rdfs:isDefinedBy Do not add manually. Coming soon Coming soon No Yes rdfs:label Primary name - used as a display name by Protege (with standard settings) and most downstream consumers. Add only one of these. It must be unique within an ontology. CL:0000418 arcade cell arcade cell Must Yes RO_0002161 Coming soon Coming soon No No 'see also' Used to link to a webpage, such as a GitHub ticket. CL:0000134 mesenchymal stem cell https://github.com/obophenotype/cell-ontology/issues/474 No No shorthand Added automatically by some pipelines. Do not add manually Coming soon Coming soon No No subset_property A grouping class for subset tags. N/A N/A No No subset_property: added_for_HCA A subset tag for terms that were requested by the Human Cell Atlas. N/A N/A No No subset property: location_grouping A subset tag for cell types from a particular anatomical location. N/A N/A No No synonym_type_property A grouping class for synonym tags. N/A N/A No No 'term replaced by' To be used on obsolete terms to indicate a term that can be automatically substituted for the obsoleted term. Coming soon Coming soon No No","title":"Annotation Properties"},{"location":"cite/","text":"How to cite CL \u00b6 The Cell Ontology 2016: enhanced content, modularization, and ontology interoperability. Alexander D Diehl, Terrence F Meehan, Yvonne M Bradford, Matthew H Brush, Wasila M Dahdul, David S Dougall, Yongqun He, David Osumi-Sutherland, Alan Ruttenberg, Sirarat Sarntivijai, Ceri E Van Slyke, Nicole A Vasilevsky, Melissa A Haendel, Judith A Blake, Christopher J Mungall. J Biomed Semantics. 2016 Jul 4;7(1):44. PMID:27377652, PMCID:PMC4932724, DOI:10.1186/s13326-016-0088-7 Logical development of the cell ontology. Terrence F Meehan, Anna Maria Masci, Amina Abdulla, Lindsay G Cowell, Judith A Blake, Christopher J Mungall, Alexander D Diehl. BMC Bioinformatics. 2011 Jan 5;12:6. PMID:21208450, PMCID:PMC3024222, DOI:10.1186/1471-2105-12-6 An ontology for cell types. Jonathan Bard, Seung Y Rhee, Michael Ashburner. Genome Biol. 2005;6(2):R21. PMID:15693950, PMCID:PMC551541, DOI:10.1186/gb-2005-6-2-r21","title":"Cite"},{"location":"cite/#how-to-cite-cl","text":"The Cell Ontology 2016: enhanced content, modularization, and ontology interoperability. Alexander D Diehl, Terrence F Meehan, Yvonne M Bradford, Matthew H Brush, Wasila M Dahdul, David S Dougall, Yongqun He, David Osumi-Sutherland, Alan Ruttenberg, Sirarat Sarntivijai, Ceri E Van Slyke, Nicole A Vasilevsky, Melissa A Haendel, Judith A Blake, Christopher J Mungall. J Biomed Semantics. 2016 Jul 4;7(1):44. PMID:27377652, PMCID:PMC4932724, DOI:10.1186/s13326-016-0088-7 Logical development of the cell ontology. Terrence F Meehan, Anna Maria Masci, Amina Abdulla, Lindsay G Cowell, Judith A Blake, Christopher J Mungall, Alexander D Diehl. BMC Bioinformatics. 2011 Jan 5;12:6. PMID:21208450, PMCID:PMC3024222, DOI:10.1186/1471-2105-12-6 An ontology for cell types. Jonathan Bard, Seung Y Rhee, Michael Ashburner. Genome Biol. 2005;6(2):R21. PMID:15693950, PMCID:PMC551541, DOI:10.1186/gb-2005-6-2-r21","title":"How to cite CL"},{"location":"cl-release/","text":"CL Release workflow \u00b6 While CL is an ODK ontology, it has a specific workflow for releases due to its large size and the limitations on standard GitHub releases. Requirements \u00b6 Aside from the standard requirements needed for ODK workflow, GH is required. Instructions on how to install GH can be found here You will need to log in to your GitHub account on GH before you can uses it. To do this, enter the following in your terminal: gh auth login You can then follow instructions below for web browser login (or use your prefer means of logging in). % gh auth login ? What account do you want to log into? GitHub.com ? What is your preferred protocol for Git operations? SSH ? Generate a new SSH key to add to your GitHub account? Yes ? Enter a passphrase for your new SSH key (Optional) ? Title for your SSH key: GitHub SSH ? How would you like to authenticate GitHub CLI? Login with a web browser ! First copy your one-time code: XXXX-XXXX Press Enter to open github.com in your browser... \u2713 Authentication complete. - gh config set -h github.com git_protocol ssh \u2713 Configured git protocol \u2713 Uploaded the SSH key to your GitHub account: /Users/username/.ssh/id_ed25519.pub \u2713 Logged in as username Release Process \u00b6 Preparation \u00b6 Preparation: Ensure that all your pull requests are merged into your main (master) branch Make sure that all changes to master are committed to Github ( git status should say that there are no modified files) Locally make sure you have the latest changes from master ( git pull ) Checkout a new branch (e.g. git checkout -b release-2021-01-01 ) You may or may not want to refresh your imports as part of your release strategy (see here )(Note: in CL we decouple our imports and releases - we hence advice that you do not update imports) Make sure you have the latest ODK installed by running docker pull obolibrary/odkfull To actually run the release, you: Open a command line terminal window and navigate to the src/ontology directory ( cd cl/src/ontology ) Run the release using sh run.sh make cl DEPLOY_GH=false . This will build all files and copy them to the correct place. (Note: the IMP=false is used to decouple imports refresh with release) Review the release as per the review release section in ODK-workflow release document Create a pull request and get a second set of eyes to review it. As CL uses a custom release pipeline, we ask that you get at least one core developer to review it too. Merge to main branch once reviewed and CI checks have passed Deploy release on GitHub by running make deploy_release GHVERSION=\"v2022-06-20\" on the release branch (DO NOTE CHANGE TO MAIN BRANCH!), replacing the date with the date of release (NOTE: no sh run.sh ) Editors note: ODK 1.3.2 will have a feature to run the release from inside the docker container. For now deploy_release has to be run outside. This should end with a GitHub release link that looks something like: https://github.com/obophenotype/cl/releases/tag/untagged-8935f3432525b27a0d84 Copy the link and paste it in your browser, this should show you a draft release. 1. Click the edit button (the pencil button on the top right corner) and change the tag to the GHVERSION you entered above (eg v2022-06-20) 1. Change the TBD. in the main text to a summary of the main changes in the release if needed. 1. Scroll down all the way and click the update release button.","title":"CL release"},{"location":"cl-release/#cl-release-workflow","text":"While CL is an ODK ontology, it has a specific workflow for releases due to its large size and the limitations on standard GitHub releases.","title":"CL Release workflow"},{"location":"cl-release/#requirements","text":"Aside from the standard requirements needed for ODK workflow, GH is required. Instructions on how to install GH can be found here You will need to log in to your GitHub account on GH before you can uses it. To do this, enter the following in your terminal: gh auth login You can then follow instructions below for web browser login (or use your prefer means of logging in). % gh auth login ? What account do you want to log into? GitHub.com ? What is your preferred protocol for Git operations? SSH ? Generate a new SSH key to add to your GitHub account? Yes ? Enter a passphrase for your new SSH key (Optional) ? Title for your SSH key: GitHub SSH ? How would you like to authenticate GitHub CLI? Login with a web browser ! First copy your one-time code: XXXX-XXXX Press Enter to open github.com in your browser... \u2713 Authentication complete. - gh config set -h github.com git_protocol ssh \u2713 Configured git protocol \u2713 Uploaded the SSH key to your GitHub account: /Users/username/.ssh/id_ed25519.pub \u2713 Logged in as username","title":"Requirements"},{"location":"cl-release/#release-process","text":"","title":"Release Process"},{"location":"cl-release/#preparation","text":"Preparation: Ensure that all your pull requests are merged into your main (master) branch Make sure that all changes to master are committed to Github ( git status should say that there are no modified files) Locally make sure you have the latest changes from master ( git pull ) Checkout a new branch (e.g. git checkout -b release-2021-01-01 ) You may or may not want to refresh your imports as part of your release strategy (see here )(Note: in CL we decouple our imports and releases - we hence advice that you do not update imports) Make sure you have the latest ODK installed by running docker pull obolibrary/odkfull To actually run the release, you: Open a command line terminal window and navigate to the src/ontology directory ( cd cl/src/ontology ) Run the release using sh run.sh make cl DEPLOY_GH=false . This will build all files and copy them to the correct place. (Note: the IMP=false is used to decouple imports refresh with release) Review the release as per the review release section in ODK-workflow release document Create a pull request and get a second set of eyes to review it. As CL uses a custom release pipeline, we ask that you get at least one core developer to review it too. Merge to main branch once reviewed and CI checks have passed Deploy release on GitHub by running make deploy_release GHVERSION=\"v2022-06-20\" on the release branch (DO NOTE CHANGE TO MAIN BRANCH!), replacing the date with the date of release (NOTE: no sh run.sh ) Editors note: ODK 1.3.2 will have a feature to run the release from inside the docker container. For now deploy_release has to be run outside. This should end with a GitHub release link that looks something like: https://github.com/obophenotype/cl/releases/tag/untagged-8935f3432525b27a0d84 Copy the link and paste it in your browser, this should show you a draft release. 1. Click the edit button (the pencil button on the top right corner) and change the tag to the GHVERSION you entered above (eg v2022-06-20) 1. Change the TBD. in the main text to a summary of the main changes in the release if needed. 1. Scroll down all the way and click the update release button.","title":"Preparation"},{"location":"contact_us/","text":"Contact Us \u00b6 GitHub \u00b6 The preferred point of contact for the Cell Ontology is the GitHub Issue Tracker . It is used commonly for requesting new terms or suggesting changes to existing terms. Feel free to open a new issue with any kind of request/question/consideration you might have. If you do not have a GitHub account, you can sign up for a free account here . E-mail and other contacts \u00b6 The Cell Ontology also has a Slack Channel and monthly meetings, with an open agenda . If you want to join the channel or talk about any other topics related to the Cell Ontology, you may reach out to one of the active CL editors. The official contact for CL listed at the OBO Foundry is Alexander Diehl , but feel free to contact any other individual. A list of active CL editors can be found here . Note that requests on the GitHub issue tracker may be addressed more quickly, as individuals are often busy. For other details on CL, you may see http://obofoundry.org/ontology/cl.html .","title":"Contact"},{"location":"contact_us/#contact-us","text":"","title":"Contact Us"},{"location":"contact_us/#github","text":"The preferred point of contact for the Cell Ontology is the GitHub Issue Tracker . It is used commonly for requesting new terms or suggesting changes to existing terms. Feel free to open a new issue with any kind of request/question/consideration you might have. If you do not have a GitHub account, you can sign up for a free account here .","title":"GitHub"},{"location":"contact_us/#e-mail-and-other-contacts","text":"The Cell Ontology also has a Slack Channel and monthly meetings, with an open agenda . If you want to join the channel or talk about any other topics related to the Cell Ontology, you may reach out to one of the active CL editors. The official contact for CL listed at the OBO Foundry is Alexander Diehl , but feel free to contact any other individual. A list of active CL editors can be found here . Note that requests on the GitHub issue tracker may be addressed more quickly, as individuals are often busy. For other details on CL, you may see http://obofoundry.org/ontology/cl.html .","title":"E-mail and other contacts"},{"location":"contributing/","text":"How to contribute to CL \u00b6 We welcome your contributions to CL! Generally, you can follow the editors workflow instructions here . However, if you aren't confident in directly editing the ontology, you can contribute by writing up an issue and one of our curators/developers will pick it up and address it. Writing up an issue \u00b6 If you want a new term added, or want edits to a current term, or spot any mistakes/issues with CL, or you have any other CL related issues, you can write up a ticket using the following steps: Go to the issues tab in CL. Click the 'New issue' tab on the top right corner and select the most appropriate category for your issue. (Note: blank issues can be created if none of the categories fit, but we recommend using the categories as they are designed to be more comprehensive). Fill up the form as best you can, giving a descriptive title to your issue name and leaving the bracketed [] tag in the title: eg. Add new term is bad name, while [NTR] larval stage X is good name. When writing up more complex issues that include multiple items or steps, make sure you include the use of - [ ] to denote action items. These turn into checkboxes which makes it much faster to assess which comments have been addressed. (Note: it is better to write up multiple issues than one big one with multiple items, e.g. write up one issue for each term you want added rather than an issue with all the terms you want added.) If you know a specific curator/editor that you want handling your ticket, you can assign them to your ticket in the assignee tab on the right, if not, someone from our team will assign an appropriate person to handle your ticket. If, however, your ticket has not been looked at in more than 10 days, and you suspect that it might have been missed, please assign it to gouttegd and they will assign it appropriately. If you know how to edit the ontology directly, please then proceed to making a Pull request with the guidelines below, following the editors workflow instructions here . Pull request guidelines \u00b6 Give your pull requests good names: Add new terms is bad. Adding larval stage X term #332 is ok. Make sure pull requests have someone assigned to review them and remind them once in a while. Do not let them go dormant Assign yourself to be the Assignee Make sure to use - [ ] to denote action items in issues and pull requests, not just comments. These turn into checkboxes which makes it much faster to assess which comments have been addressed and can be ignored. Give a short summary of the pull request - that way we can find suitable reviewers much quicker. Say which terms you are adding or what kinds of changes you are proposing. It is most of the time a good idea to use squash merge rather than merge for your pull request, to keep the git history short and useful. Pull requests that require imports to be refreshed \u00b6 If your pull request references foreign terms from an external ontology that are not yet present in the import module for that ontology (for example, you\u2019re adding a logical definition that makes use of a GO term for the first time), imports needs to be refreshed for the foreign terms to be available to use. If you have the technical skills and/or the required computer resources (refreshing imports can be a memory-intensive task), you may refresh the imports yourself before submitting the pull request, by following the appropriate procedure . If you can\u2019t apply the imports refreshing procedure for any reason, you may instead opt for using \u201cbare IRIs\u201d when editing the ontology, everywhere you need a reference to a foreign term. Then, when submitting your pull request, label it with the tag update-imports-required to ask that a member of the tech support group refresh the imports before the pull request can be merged. People reviewing pull requests must 1) make sure that if a pull request is referencing bare IRIs, the request is tagged with update-imports-required (adding the label themselves if needed); and 2) make sure that imports have indeed been updated (either by the author of the pull request, or by someone from the tech support group if requested) before allowing the request to be merged.","title":"Contributing"},{"location":"contributing/#how-to-contribute-to-cl","text":"We welcome your contributions to CL! Generally, you can follow the editors workflow instructions here . However, if you aren't confident in directly editing the ontology, you can contribute by writing up an issue and one of our curators/developers will pick it up and address it.","title":"How to contribute to CL"},{"location":"contributing/#writing-up-an-issue","text":"If you want a new term added, or want edits to a current term, or spot any mistakes/issues with CL, or you have any other CL related issues, you can write up a ticket using the following steps: Go to the issues tab in CL. Click the 'New issue' tab on the top right corner and select the most appropriate category for your issue. (Note: blank issues can be created if none of the categories fit, but we recommend using the categories as they are designed to be more comprehensive). Fill up the form as best you can, giving a descriptive title to your issue name and leaving the bracketed [] tag in the title: eg. Add new term is bad name, while [NTR] larval stage X is good name. When writing up more complex issues that include multiple items or steps, make sure you include the use of - [ ] to denote action items. These turn into checkboxes which makes it much faster to assess which comments have been addressed. (Note: it is better to write up multiple issues than one big one with multiple items, e.g. write up one issue for each term you want added rather than an issue with all the terms you want added.) If you know a specific curator/editor that you want handling your ticket, you can assign them to your ticket in the assignee tab on the right, if not, someone from our team will assign an appropriate person to handle your ticket. If, however, your ticket has not been looked at in more than 10 days, and you suspect that it might have been missed, please assign it to gouttegd and they will assign it appropriately. If you know how to edit the ontology directly, please then proceed to making a Pull request with the guidelines below, following the editors workflow instructions here .","title":"Writing up an issue"},{"location":"contributing/#pull-request-guidelines","text":"Give your pull requests good names: Add new terms is bad. Adding larval stage X term #332 is ok. Make sure pull requests have someone assigned to review them and remind them once in a while. Do not let them go dormant Assign yourself to be the Assignee Make sure to use - [ ] to denote action items in issues and pull requests, not just comments. These turn into checkboxes which makes it much faster to assess which comments have been addressed and can be ignored. Give a short summary of the pull request - that way we can find suitable reviewers much quicker. Say which terms you are adding or what kinds of changes you are proposing. It is most of the time a good idea to use squash merge rather than merge for your pull request, to keep the git history short and useful.","title":"Pull request guidelines"},{"location":"contributing/#pull-requests-that-require-imports-to-be-refreshed","text":"If your pull request references foreign terms from an external ontology that are not yet present in the import module for that ontology (for example, you\u2019re adding a logical definition that makes use of a GO term for the first time), imports needs to be refreshed for the foreign terms to be available to use. If you have the technical skills and/or the required computer resources (refreshing imports can be a memory-intensive task), you may refresh the imports yourself before submitting the pull request, by following the appropriate procedure . If you can\u2019t apply the imports refreshing procedure for any reason, you may instead opt for using \u201cbare IRIs\u201d when editing the ontology, everywhere you need a reference to a foreign term. Then, when submitting your pull request, label it with the tag update-imports-required to ask that a member of the tech support group refresh the imports before the pull request can be merged. People reviewing pull requests must 1) make sure that if a pull request is referencing bare IRIs, the request is tagged with update-imports-required (adding the label themselves if needed); and 2) make sure that imports have indeed been updated (either by the author of the pull request, or by someone from the tech support group if requested) before allowing the request to be merged.","title":"Pull requests that require imports to be refreshed"},{"location":"editing_guidelines/","text":"Guidelines for CL editors \u00b6 Naming terms \u00b6 General rules \u00b6 All labels should be singular nouns. Words should not be capitalized, unless they are proper names or are capitalized as a standard (e.g., \u201cPeyer's\u201d and \u201cB\u201d in \u201cPeyer's patch B cell\u201d). Avoid special characters: use only alphanumeric characters, space, dash ( - ), slash ( / ), and apostrophe ( ' ). Advice on writing labels \u00b6 Bear in mind that users will often encounter terms in isolation. Sufficiently descriptive labels are therefore recommended, especially where there is obvious potential for confusion. For example, the label 'peripheral nervous system neuron' is preferred over 'peripheral neuron' as the former has more clarity. For classes that refer to cell types in specific anatomical structures, the label may have the adjectival or noun form of the anatomical structure, for example, 'hepatic oval stem cell' or 'liver dendritic cell' (\"hepatic\" vs. \"liver\"). To determine which form of the anatomical term to use, it is recommended for the editor to search PubMed/review literature to determine common usage. If there is no clear preference, the editor can defer to use the adjectival form and add the noun form as an exact synonym. Try to maintain consistent patterns of naming where possible. However, it may make sense to override this in order to conform to common usage. Defining terms \u00b6 This section is about the textual definition of terms. Logically consistent classification is important, but an ontology is only useful (and maintainable) if all humans that interact with it (users, curators and editors) can quickly find the terms they need and understand what they refer to. This requires clear, unambiguous, human-readable definitions. When crafting a definition, editors should aim for a reasonably succinct statement about the class allowing curators and users to easily distinguish it from other, similar classes, and which captures key points of interest about that class. It should capture assertions made in the formal part of the definition (the relationships) as closely as possible without becoming stilted and difficult to read. The basic structure of a definition should be as follows: A genus that diff1 and diff2 . It also diff3 and gloss ... where genus is a general classification and diff1 , diff2 , etc. are the differentia , which state what differentiates this class from others that share the same general classification. The gloss, when present, gives some key points of interest about the class. The first sentence of the definition should refer to the definiendum in singular form. The rest of the definition may then invoke the plural form. Contents of definitions \u00b6 It is difficult to specify, a priori , which assertions should be included in a textual definition. However there are some general guidelines. DO make sure your definition is consistent with the definition of the superclass(es). DO make sure your definition includes the information that is recorded in all the direct formal relationships to the class. AVOID assertions about structures or cell types that are not part of the cell type being described, except when they pertain to some direct relationships with the cell type being described. AVOID including details that could better be included in the definition of subtypes of the cell type being described. LIMIT information that applies to only some members of the class. This should only be used sparingly; when used, it should be made clear that it does not apply to all members of the class. AVOID using gene expression as the differentium. AVOID extensive repetition of assertions made in superclass definitions, unless these assertions are used to provide direct evidence for class membership. DO NOT add informations about what happens in mutant or pathological states backgrounds or other kinds of non-control conditions. DO NOT include reasons for believing the assertions to be true. These should be recorded in comments. DO NOT raise questions in the definition. The definition should have the sense of being definitive, which is undermined if we show doubt. Any doubt should be recorded in comments instead. Comments \u00b6 Comments should be used for: Providing evidence. In some cases it is useful to know the type of evidence for an assertion. This should not be recorded in the definition, but can be recorded in a comment. Disambiguation. Sometimes a single term is used in the literature with multiple meanings. In such cases, a comment should be added outlining these different uses and how they relate to the definition set in the ontology. Reporting editorial decisions (or decisions in waiting) about the term. This includes providing a reason for obsoleting a term, or letting users and curators know that the term may be merged or split in the future, e.g., when enough evidence for the merge/split will be available. Try to be consistent in how you phrase the various types of comments. For example: when giving a reason for obsoletion, use \u201cObsoleted as ...\u201d; when indicating a potential future merge, use \u201cPossible equivalence with {other term} ...\u201d. Synonyms \u00b6 Extensive addition of synonyms helps \u201cfindability\u201d of terms when search. Synonyms can and should be added liberally. Of note, the intention of the ontology is not meant to record how a synonym is used in all specific sources in which it appears. Rather an editor, after doing due diligence in researching the terms/synonyms, must determine how a term is used at the present moment in the scientific community. Guidelines on the type of synonyms: DO use has_exact_synonym only when the label and the synonym can be used interchangeably without dispute and refer to the same concept. Example: the terms \u201cleukocyte\u201d, \u201cleucocyte\u201d (spelling variation\u201d) and \u201cwhite blood cell\u201d (layman\u2019s term) all refer to the exact same concept (a specific cell type) and would be considered exact synonyms. Terms that may refer to more than one concept, especially within the biomedical domain, should NOT be annotated as exact synonyms, including abbreviations. A synonym that is an abbreviation should be annotated using has_related_synonym and with property type \u201cabbreviation\u201d (technically: the synonym annotation assertion axiom should itself be annotated with a http://www.geneontology.org/formats/oboInOwl#hasSynonymType property with value http://purl.obolibrary.org/obo/OMO_0003000 ). Example: \u201cWBC\u201d can stand for \u201cwhite blood cell\u201d and refer to \u201cleukocyte\u201d, but within the biomedical domain it can also represent \u201cwhite blood cell count\u201d or, perhaps less frequently, \u201cwhole-body counting\u201d, two distinct concepts with separate OBO ontology terms. DO check that exact synonyms are unique across the ontology. In other words, if class A has synonym \u201cX\u201d, \u201cX\u201d should NOT be an exact synonym for any other CL term. DO be mindful of the \u201cdirectionality\u201d of the narrow and broad types of synonyms. They qualify the synonym , not the original term. Example: asserting that \u201cperipheral blood mononuclear cell\u201d is a narrow synonym of \u201cmononuclear cell\u201d means that \u201cperipheral blood mononuclear cell\u201c refers to a narrower (more specific) concept than \u201cmononuclear cell\u201d, not the other way around. DO use has_related_synonym where the overlap between the synonym and the term label may be uncler, disputable or not true in all scenarios or contexts, but do want the term to be findable when using the synonym as a search string. This includes abbreviations, which should be annotated as related synonyms with synonym type \u201cabbreviation\u201d (see point 2 above). If a synonym includes a mix of abbreviations and words, DO use has_related_synonym EXCEPT when there is enough context within the synonym itself to make it clear that the synonym refers only to the concept being annotated. Example: \u201clung TRM CD8-positive, CD103-positive cell\u201d should be an exact synonym of \u201clung resident memory CD8-positive, CD103-positive, alpha-beta T cell\u201d, even though \u201cTRM\u201d (in this case) is an abbreviation for \u201ctissue resident T cell\u201d. Note that without this context \u201cTRM\u201d should NOT be considered an exact synonym for \u201ctissue resident T cell\u201d as \u201cTRM\u201d could also mean \u201ctreatment-related mortality\u201d, another OBO ontology concept. Compare the previous example to \u201cIMB cell\u201d, which should be a related synonym of \u201cinvaginating midget bipolar cell\u201d, as there is not enough context to confidently infer what \"IMB\" stands for. Considerations on style \u00b6 The following considerations apply both to all human-readable fields (names, textual definitions, comments, synonyms). United States (US) English or British English? Where there are differences in the accepted spelling between British and US English, use the US form. British English variants of the labels may be added as synonyms (e.g., a term labelled \u201cepithelial cell of esophagus\u201d may have \u201cepithelial cell of oesophagus\u201d as an exact synonym). Use of jargon. The aim of the ontology is to provide useable descriptions and links to the reader. Consequently, try to avoid obscure jargon or pretentious Latin/Greek, especially where widely understood, plain alternatives exist. Where it will aid searching, such terms may be added to synonyms and/or as asides in the \u201cgloss\u201d part of definitions. That or which? These are so interchangeable that there isn\u2019t really a rule anymore. But as a guideline, use \u201cwhich\u201d after a comma (e.g., \u201cthis study, which cost $10,000, was a success\u201d), and \u201cthat\u201d when no comma is used (e.g., \u201cthe study that cost $10,000 was a success\u201d). Use of hyphens. Yes to those that help clarity (e.g., \u201cposterior-most\u201d) and those that are regularly used/accepted in the literature. Generally, no hyphens after prefixes such as \u201csub\u201d, \u201cmid\u201d, \u201csemi\u201d, \u201chemi\u201d, etc. (e.g., \u201chemidesmosome\u201d instead of \u201chemi-desmosome\u201d), unless it helps with clarity (e.g., \u201cmulti-innervated\u201d). No hyphens for composed location adjectives (e.g., \u201cposteroanterior\u201d), unless there are more than two compounds (e.g., \u201cventro-posterolateral\u201d). Abbreviations. Avoid abbreviations, contractions, and symbols born out of laziness, such as & , + , or vs for \u201cversus\u201d. Avoid abbreviations unless they are self-explanatory, commonly understood, or they really do help to reduce the amount of typing enough to enhance readability. Use full chemical element names, not symbols (e.g., \u201chydrogen\u201d instead of \u201cH+\u201d, \u201ccopper\u201d instead of \u201cCu\u201d, etc.). For biomolecules, spell out the term wherever practical (e.g., \u201cfibroblast growth factor\u201d instead of \u201cFGF\u201d). Abbreviations are acceptable in synonyms, in cases where the abbreviation is the synonym. Cross-references to the literature \u00b6 Assertions in textual definitions, evidence provided in comments, and synonyms should be backed up by citing the appropriate literature. Citations are made by cross-references, that is by adding http://www.geneontology.org/formats/oboInOwl#hasDbXref annotations to the definition, comment, and synonym annotations. Add one such annotation per reference, using the CURIE syntax with well-known prefixes: PMID:1234567 for a PubMed identifier; doi:xx.yyyy/... for a DOI; ISBN:... for a ISBN. If the main source for an assertion is a term in another ontology, the short identifier for that term may be used as a cross-reference. For example, WBbt:0006799 to cross-reference a term in the C. elegans Gross Anatomy Ontology. If using a MeSH (Medical Subject Heading) term as a cross-reference, add the database_cross_reference annotation using the MeSH Unique ID, NOT MeSH Tree Number. For example, a database_cross_reference can be MESH:D004759, NOT MESH:A03.492.766.440.250. ORCID identifiers may also be used when the only available source for an assertion is an individual researcher. However, this should be AVOIDED. Technical details of adding a cross-reference using Prot\u00e9g\u00e9 : For CURIEs, ORCIDs: In the \"Create Annotation\" window, select the annotation property database_cross_reference . For adding URLs to text definitions or synonyms: In the \"Create Annotation\" window, select the annotation property database_cross_reference . For adding URLs to axioms that are NOT text definitions or synonyms: In the \"Create Annotation\" window, select the annotation property source . For CURIEs: Enter the CURIE, using the bioregistry OBO context prefix ( link to prefixmap ), as a Value on the \"Literal\" tab. Leave Datatype empty. In cases where more than one CURIE is available for a resource, either is acceptable, but using the more semantically specific identifier is recommended. For example, when both a PMID and a doi are available for a resource, using the PMID is recommended since it indicates the cross-reference points to a paper, as opposed to a doi which could point to any digital object. For ORCIDs: Enter the ORCID as an IRI in the IRI field on the \"IRI Editor\" tab, for example https://orcid.org/0000-0002-7356-1779 . For URLs: Enter the URL as a literal string with Datatype xsd:anyURI selected. To restate, in all cases above except ORCIDs, the values are entered as literal strings. An ORCID MUST BE entered as an IRI. Term contributors \u00b6 When adding an ORCID to identify a term contributor, in the \"Create Annotation\" window, select the annotation property dcterms:contributor . The ORCID is still entered as an IRI on the \u201cIRI Editor\u201d tab. Formal definitions \u00b6 The formal definition of a class is made up of all the logical axioms about the class (as opposed to the annotation assertion axioms). This includes classification assertions, relationship assertions, equivalence assertions, and disjointness assertions. Note: In OWL formalism, both classification and relationship assertions are represented using SubClassOf axioms. However, in this document, we make a strict distinction between a classification (where a class is a subclass of a named class ), and a relationship (where a class is a subclass of an anonymous class expression ). For example, in the formal definition of CL:0000392 (\u201ccrystal cell\u201d): Class: 'crystal cell' SubClassOf: 'hemocyte' SubClassOf: 'develops from' some 'procrystal cell' The first SubClassOf axiom denotes an actual classification, whereas the second denotes a relationship. Asserting classification \u00b6 In order to keep the ontology maintainable, asserted classifications should be limited where possible. Ideally, all terms would have only a single asserted parent (also known as \u201csuperclass\u201d). Given the presence of suitable logically defined classes (see below) and sufficient relationships for the term you are making, additional classification can be inferred automatically by reasoning. However, as we are limited in what types of classification we are able to infer, you may need to assert multiple parents. Two asserted classifications are plainly acceptable. If you must assert three or more classifications, then you should make a note with a suggestion for which of the asserted classifications are good candidates for formalisation and inferred classification. Logically defined classes \u00b6 Logically defined classes are terms that have formal definitions that specify complete necessary and sufficient conditions for membership of the class. They can be used by a reasoner to auto-classify the ontology by searching for terms that fulfill these conditions. Logically defined classes are variously referred to as \u201ccross-products\u201d (XPs), \u201cgenus and differentia definitions\u201d, \u201cequivalent classes\u201d, or \u201cintersections\u201d. In OWL formalism, they are represented using EquivalentClasses axioms. Generally, logical definitions follow the structure: \u201cAny X that REL some Y [and REL some Z ...]\u201d, where X is the genus and each following clause (\u201cthat REL some Y \u201d) is a differentium. Strictly speaking, logical definitions can be arbitrarily complex. It is recommended, however, to stick as much as possible to \u201csimple\u201d logical definitions following the structure above. More complex definitions are harder to understand, and may involve OWL constructs that are legal but not fully supported by the available reasoners. Good candidates for logically defined classes include classes whose only differentium is: what the cell is part of; what it innervates (or, if more precisely known, what its axon(s) or dendrite(s) innervate); its function. Care must be taken not to use logically defined classes too liberally. You should be satisfied that all automatic classifications that result would make sense (or at least be justifiable) to a biologist and would cover most cases of usages with as few edge cases as possible. If you are unsure whether a characteristic of a cell should be expressed as a relationship or as a differentium in a logically defined class: use a relationship. The textual definition of a logically defined class should be a plain English equivalent of the logical definition. However, in some cases it can be useful to add more details in the \u201cgloss\u201d part when relevant. Asserting relationships \u00b6 Please refer to the relations guide for detailled guidelines about which relations to use for most cases.","title":"Guidelines for CL editors"},{"location":"editing_guidelines/#guidelines-for-cl-editors","text":"","title":"Guidelines for CL editors"},{"location":"editing_guidelines/#naming-terms","text":"","title":"Naming terms"},{"location":"editing_guidelines/#general-rules","text":"All labels should be singular nouns. Words should not be capitalized, unless they are proper names or are capitalized as a standard (e.g., \u201cPeyer's\u201d and \u201cB\u201d in \u201cPeyer's patch B cell\u201d). Avoid special characters: use only alphanumeric characters, space, dash ( - ), slash ( / ), and apostrophe ( ' ).","title":"General rules"},{"location":"editing_guidelines/#advice-on-writing-labels","text":"Bear in mind that users will often encounter terms in isolation. Sufficiently descriptive labels are therefore recommended, especially where there is obvious potential for confusion. For example, the label 'peripheral nervous system neuron' is preferred over 'peripheral neuron' as the former has more clarity. For classes that refer to cell types in specific anatomical structures, the label may have the adjectival or noun form of the anatomical structure, for example, 'hepatic oval stem cell' or 'liver dendritic cell' (\"hepatic\" vs. \"liver\"). To determine which form of the anatomical term to use, it is recommended for the editor to search PubMed/review literature to determine common usage. If there is no clear preference, the editor can defer to use the adjectival form and add the noun form as an exact synonym. Try to maintain consistent patterns of naming where possible. However, it may make sense to override this in order to conform to common usage.","title":"Advice on writing labels"},{"location":"editing_guidelines/#defining-terms","text":"This section is about the textual definition of terms. Logically consistent classification is important, but an ontology is only useful (and maintainable) if all humans that interact with it (users, curators and editors) can quickly find the terms they need and understand what they refer to. This requires clear, unambiguous, human-readable definitions. When crafting a definition, editors should aim for a reasonably succinct statement about the class allowing curators and users to easily distinguish it from other, similar classes, and which captures key points of interest about that class. It should capture assertions made in the formal part of the definition (the relationships) as closely as possible without becoming stilted and difficult to read. The basic structure of a definition should be as follows: A genus that diff1 and diff2 . It also diff3 and gloss ... where genus is a general classification and diff1 , diff2 , etc. are the differentia , which state what differentiates this class from others that share the same general classification. The gloss, when present, gives some key points of interest about the class. The first sentence of the definition should refer to the definiendum in singular form. The rest of the definition may then invoke the plural form.","title":"Defining terms"},{"location":"editing_guidelines/#contents-of-definitions","text":"It is difficult to specify, a priori , which assertions should be included in a textual definition. However there are some general guidelines. DO make sure your definition is consistent with the definition of the superclass(es). DO make sure your definition includes the information that is recorded in all the direct formal relationships to the class. AVOID assertions about structures or cell types that are not part of the cell type being described, except when they pertain to some direct relationships with the cell type being described. AVOID including details that could better be included in the definition of subtypes of the cell type being described. LIMIT information that applies to only some members of the class. This should only be used sparingly; when used, it should be made clear that it does not apply to all members of the class. AVOID using gene expression as the differentium. AVOID extensive repetition of assertions made in superclass definitions, unless these assertions are used to provide direct evidence for class membership. DO NOT add informations about what happens in mutant or pathological states backgrounds or other kinds of non-control conditions. DO NOT include reasons for believing the assertions to be true. These should be recorded in comments. DO NOT raise questions in the definition. The definition should have the sense of being definitive, which is undermined if we show doubt. Any doubt should be recorded in comments instead.","title":"Contents of definitions"},{"location":"editing_guidelines/#comments","text":"Comments should be used for: Providing evidence. In some cases it is useful to know the type of evidence for an assertion. This should not be recorded in the definition, but can be recorded in a comment. Disambiguation. Sometimes a single term is used in the literature with multiple meanings. In such cases, a comment should be added outlining these different uses and how they relate to the definition set in the ontology. Reporting editorial decisions (or decisions in waiting) about the term. This includes providing a reason for obsoleting a term, or letting users and curators know that the term may be merged or split in the future, e.g., when enough evidence for the merge/split will be available. Try to be consistent in how you phrase the various types of comments. For example: when giving a reason for obsoletion, use \u201cObsoleted as ...\u201d; when indicating a potential future merge, use \u201cPossible equivalence with {other term} ...\u201d.","title":"Comments"},{"location":"editing_guidelines/#synonyms","text":"Extensive addition of synonyms helps \u201cfindability\u201d of terms when search. Synonyms can and should be added liberally. Of note, the intention of the ontology is not meant to record how a synonym is used in all specific sources in which it appears. Rather an editor, after doing due diligence in researching the terms/synonyms, must determine how a term is used at the present moment in the scientific community. Guidelines on the type of synonyms: DO use has_exact_synonym only when the label and the synonym can be used interchangeably without dispute and refer to the same concept. Example: the terms \u201cleukocyte\u201d, \u201cleucocyte\u201d (spelling variation\u201d) and \u201cwhite blood cell\u201d (layman\u2019s term) all refer to the exact same concept (a specific cell type) and would be considered exact synonyms. Terms that may refer to more than one concept, especially within the biomedical domain, should NOT be annotated as exact synonyms, including abbreviations. A synonym that is an abbreviation should be annotated using has_related_synonym and with property type \u201cabbreviation\u201d (technically: the synonym annotation assertion axiom should itself be annotated with a http://www.geneontology.org/formats/oboInOwl#hasSynonymType property with value http://purl.obolibrary.org/obo/OMO_0003000 ). Example: \u201cWBC\u201d can stand for \u201cwhite blood cell\u201d and refer to \u201cleukocyte\u201d, but within the biomedical domain it can also represent \u201cwhite blood cell count\u201d or, perhaps less frequently, \u201cwhole-body counting\u201d, two distinct concepts with separate OBO ontology terms. DO check that exact synonyms are unique across the ontology. In other words, if class A has synonym \u201cX\u201d, \u201cX\u201d should NOT be an exact synonym for any other CL term. DO be mindful of the \u201cdirectionality\u201d of the narrow and broad types of synonyms. They qualify the synonym , not the original term. Example: asserting that \u201cperipheral blood mononuclear cell\u201d is a narrow synonym of \u201cmononuclear cell\u201d means that \u201cperipheral blood mononuclear cell\u201c refers to a narrower (more specific) concept than \u201cmononuclear cell\u201d, not the other way around. DO use has_related_synonym where the overlap between the synonym and the term label may be uncler, disputable or not true in all scenarios or contexts, but do want the term to be findable when using the synonym as a search string. This includes abbreviations, which should be annotated as related synonyms with synonym type \u201cabbreviation\u201d (see point 2 above). If a synonym includes a mix of abbreviations and words, DO use has_related_synonym EXCEPT when there is enough context within the synonym itself to make it clear that the synonym refers only to the concept being annotated. Example: \u201clung TRM CD8-positive, CD103-positive cell\u201d should be an exact synonym of \u201clung resident memory CD8-positive, CD103-positive, alpha-beta T cell\u201d, even though \u201cTRM\u201d (in this case) is an abbreviation for \u201ctissue resident T cell\u201d. Note that without this context \u201cTRM\u201d should NOT be considered an exact synonym for \u201ctissue resident T cell\u201d as \u201cTRM\u201d could also mean \u201ctreatment-related mortality\u201d, another OBO ontology concept. Compare the previous example to \u201cIMB cell\u201d, which should be a related synonym of \u201cinvaginating midget bipolar cell\u201d, as there is not enough context to confidently infer what \"IMB\" stands for.","title":"Synonyms"},{"location":"editing_guidelines/#considerations-on-style","text":"The following considerations apply both to all human-readable fields (names, textual definitions, comments, synonyms). United States (US) English or British English? Where there are differences in the accepted spelling between British and US English, use the US form. British English variants of the labels may be added as synonyms (e.g., a term labelled \u201cepithelial cell of esophagus\u201d may have \u201cepithelial cell of oesophagus\u201d as an exact synonym). Use of jargon. The aim of the ontology is to provide useable descriptions and links to the reader. Consequently, try to avoid obscure jargon or pretentious Latin/Greek, especially where widely understood, plain alternatives exist. Where it will aid searching, such terms may be added to synonyms and/or as asides in the \u201cgloss\u201d part of definitions. That or which? These are so interchangeable that there isn\u2019t really a rule anymore. But as a guideline, use \u201cwhich\u201d after a comma (e.g., \u201cthis study, which cost $10,000, was a success\u201d), and \u201cthat\u201d when no comma is used (e.g., \u201cthe study that cost $10,000 was a success\u201d). Use of hyphens. Yes to those that help clarity (e.g., \u201cposterior-most\u201d) and those that are regularly used/accepted in the literature. Generally, no hyphens after prefixes such as \u201csub\u201d, \u201cmid\u201d, \u201csemi\u201d, \u201chemi\u201d, etc. (e.g., \u201chemidesmosome\u201d instead of \u201chemi-desmosome\u201d), unless it helps with clarity (e.g., \u201cmulti-innervated\u201d). No hyphens for composed location adjectives (e.g., \u201cposteroanterior\u201d), unless there are more than two compounds (e.g., \u201cventro-posterolateral\u201d). Abbreviations. Avoid abbreviations, contractions, and symbols born out of laziness, such as & , + , or vs for \u201cversus\u201d. Avoid abbreviations unless they are self-explanatory, commonly understood, or they really do help to reduce the amount of typing enough to enhance readability. Use full chemical element names, not symbols (e.g., \u201chydrogen\u201d instead of \u201cH+\u201d, \u201ccopper\u201d instead of \u201cCu\u201d, etc.). For biomolecules, spell out the term wherever practical (e.g., \u201cfibroblast growth factor\u201d instead of \u201cFGF\u201d). Abbreviations are acceptable in synonyms, in cases where the abbreviation is the synonym.","title":"Considerations on style"},{"location":"editing_guidelines/#cross-references-to-the-literature","text":"Assertions in textual definitions, evidence provided in comments, and synonyms should be backed up by citing the appropriate literature. Citations are made by cross-references, that is by adding http://www.geneontology.org/formats/oboInOwl#hasDbXref annotations to the definition, comment, and synonym annotations. Add one such annotation per reference, using the CURIE syntax with well-known prefixes: PMID:1234567 for a PubMed identifier; doi:xx.yyyy/... for a DOI; ISBN:... for a ISBN. If the main source for an assertion is a term in another ontology, the short identifier for that term may be used as a cross-reference. For example, WBbt:0006799 to cross-reference a term in the C. elegans Gross Anatomy Ontology. If using a MeSH (Medical Subject Heading) term as a cross-reference, add the database_cross_reference annotation using the MeSH Unique ID, NOT MeSH Tree Number. For example, a database_cross_reference can be MESH:D004759, NOT MESH:A03.492.766.440.250. ORCID identifiers may also be used when the only available source for an assertion is an individual researcher. However, this should be AVOIDED. Technical details of adding a cross-reference using Prot\u00e9g\u00e9 : For CURIEs, ORCIDs: In the \"Create Annotation\" window, select the annotation property database_cross_reference . For adding URLs to text definitions or synonyms: In the \"Create Annotation\" window, select the annotation property database_cross_reference . For adding URLs to axioms that are NOT text definitions or synonyms: In the \"Create Annotation\" window, select the annotation property source . For CURIEs: Enter the CURIE, using the bioregistry OBO context prefix ( link to prefixmap ), as a Value on the \"Literal\" tab. Leave Datatype empty. In cases where more than one CURIE is available for a resource, either is acceptable, but using the more semantically specific identifier is recommended. For example, when both a PMID and a doi are available for a resource, using the PMID is recommended since it indicates the cross-reference points to a paper, as opposed to a doi which could point to any digital object. For ORCIDs: Enter the ORCID as an IRI in the IRI field on the \"IRI Editor\" tab, for example https://orcid.org/0000-0002-7356-1779 . For URLs: Enter the URL as a literal string with Datatype xsd:anyURI selected. To restate, in all cases above except ORCIDs, the values are entered as literal strings. An ORCID MUST BE entered as an IRI.","title":"Cross-references to the literature"},{"location":"editing_guidelines/#term-contributors","text":"When adding an ORCID to identify a term contributor, in the \"Create Annotation\" window, select the annotation property dcterms:contributor . The ORCID is still entered as an IRI on the \u201cIRI Editor\u201d tab.","title":"Term contributors"},{"location":"editing_guidelines/#formal-definitions","text":"The formal definition of a class is made up of all the logical axioms about the class (as opposed to the annotation assertion axioms). This includes classification assertions, relationship assertions, equivalence assertions, and disjointness assertions. Note: In OWL formalism, both classification and relationship assertions are represented using SubClassOf axioms. However, in this document, we make a strict distinction between a classification (where a class is a subclass of a named class ), and a relationship (where a class is a subclass of an anonymous class expression ). For example, in the formal definition of CL:0000392 (\u201ccrystal cell\u201d): Class: 'crystal cell' SubClassOf: 'hemocyte' SubClassOf: 'develops from' some 'procrystal cell' The first SubClassOf axiom denotes an actual classification, whereas the second denotes a relationship.","title":"Formal definitions"},{"location":"editing_guidelines/#asserting-classification","text":"In order to keep the ontology maintainable, asserted classifications should be limited where possible. Ideally, all terms would have only a single asserted parent (also known as \u201csuperclass\u201d). Given the presence of suitable logically defined classes (see below) and sufficient relationships for the term you are making, additional classification can be inferred automatically by reasoning. However, as we are limited in what types of classification we are able to infer, you may need to assert multiple parents. Two asserted classifications are plainly acceptable. If you must assert three or more classifications, then you should make a note with a suggestion for which of the asserted classifications are good candidates for formalisation and inferred classification.","title":"Asserting classification"},{"location":"editing_guidelines/#logically-defined-classes","text":"Logically defined classes are terms that have formal definitions that specify complete necessary and sufficient conditions for membership of the class. They can be used by a reasoner to auto-classify the ontology by searching for terms that fulfill these conditions. Logically defined classes are variously referred to as \u201ccross-products\u201d (XPs), \u201cgenus and differentia definitions\u201d, \u201cequivalent classes\u201d, or \u201cintersections\u201d. In OWL formalism, they are represented using EquivalentClasses axioms. Generally, logical definitions follow the structure: \u201cAny X that REL some Y [and REL some Z ...]\u201d, where X is the genus and each following clause (\u201cthat REL some Y \u201d) is a differentium. Strictly speaking, logical definitions can be arbitrarily complex. It is recommended, however, to stick as much as possible to \u201csimple\u201d logical definitions following the structure above. More complex definitions are harder to understand, and may involve OWL constructs that are legal but not fully supported by the available reasoners. Good candidates for logically defined classes include classes whose only differentium is: what the cell is part of; what it innervates (or, if more precisely known, what its axon(s) or dendrite(s) innervate); its function. Care must be taken not to use logically defined classes too liberally. You should be satisfied that all automatic classifications that result would make sense (or at least be justifiable) to a biologist and would cover most cases of usages with as few edge cases as possible. If you are unsure whether a characteristic of a cell should be expressed as a relationship or as a differentium in a logically defined class: use a relationship. The textual definition of a logically defined class should be a plain English equivalent of the logical definition. However, in some cases it can be useful to add more details in the \u201cgloss\u201d part when relevant.","title":"Logically defined classes"},{"location":"editing_guidelines/#asserting-relationships","text":"Please refer to the relations guide for detailled guidelines about which relations to use for most cases.","title":"Asserting relationships"},{"location":"history/","text":"A brief history of CL \u00b6 The Cell Ontology (CL) represents canonical, natural biological cell types in a variety of animal species, with a focus on vertebrates. It works in concert with the Uberon anatomy ontology, and integrates seamlessly with the Gene Ontology (GO). It is part of the Open Biological and Biomedical Ontology (OBO) Foundry. As of April 2022, CL consists of over 2400 classes, and is used by key projects that involve single-cell transcriptomics, such as HuBMAP (HuBMAP Consortium 2019), the Human Cell Atlas (Regev et al. 2017), the EBI Single Cell Expression Atlas (Papatheodorou et al. 2020), and the Brain Data Standards Ontology (Tan et al. 2021).","title":"History"},{"location":"history/#a-brief-history-of-cl","text":"The Cell Ontology (CL) represents canonical, natural biological cell types in a variety of animal species, with a focus on vertebrates. It works in concert with the Uberon anatomy ontology, and integrates seamlessly with the Gene Ontology (GO). It is part of the Open Biological and Biomedical Ontology (OBO) Foundry. As of April 2022, CL consists of over 2400 classes, and is used by key projects that involve single-cell transcriptomics, such as HuBMAP (HuBMAP Consortium 2019), the Human Cell Atlas (Regev et al. 2017), the EBI Single Cell Expression Atlas (Papatheodorou et al. 2020), and the Brain Data Standards Ontology (Tan et al. 2021).","title":"A brief history of CL"},{"location":"merging-terms/","text":"Merging Terms in CL \u00b6 How to merge terms \u00b6 For general instructions on how to merge terms, please see this How-to Guide In addition to the above, please add the annotation has_alternative_id on the winning term with the ID of the losing term. Considerations on which should be the winning term \u00b6 Check Usage by GO - This can be done by using AmiGO 2 Check Usage by other ontologies - This can be done by using Ontobee Check Usage within CL (you can do this in Protege with the usage tab) - this should be lower priority as you can easily change this while obsoleting the \"losing\" term","title":"Merging Terms in CL"},{"location":"merging-terms/#merging-terms-in-cl","text":"","title":"Merging Terms in CL"},{"location":"merging-terms/#how-to-merge-terms","text":"For general instructions on how to merge terms, please see this How-to Guide In addition to the above, please add the annotation has_alternative_id on the winning term with the ID of the losing term.","title":"How to merge terms"},{"location":"merging-terms/#considerations-on-which-should-be-the-winning-term","text":"Check Usage by GO - This can be done by using AmiGO 2 Check Usage by other ontologies - This can be done by using Ontobee Check Usage within CL (you can do this in Protege with the usage tab) - this should be lower priority as you can easily change this while obsoleting the \"losing\" term","title":"Considerations on which should be the winning term"},{"location":"presentations/","text":"Presentations \u00b6 Cell Ontology (Genentech presentation) by Nicole Vasilevsky. Presented on 2021-02-09.","title":"CL presentations"},{"location":"presentations/#presentations","text":"Cell Ontology (Genentech presentation) by Nicole Vasilevsky. Presented on 2021-02-09.","title":"Presentations"},{"location":"relations_guide/","text":"Cell Ontology (CL) relations guide. \u00b6 Intro \u00b6 The aim of this document is to provide an accessible guide to how to use relations to record the properties that define cell types including location, lineage, function, morphology and marker genes. The term 'relations' here refers principally to OWL object properties, but also includes annoation properties used as shortcuts for more expressive logical axioms that can be programatically generated from them. Relations in this guide are grouped by general use case (e.g. recording location) and each is illustrated by an example e.g.- melanocyte subClassOf \u2018has part\u2019 some melanosome. This should be read as \u2018 all melanocytes have some type of melanosome as a part\u2019 as should all axioms of this form. The examples should all be correct, but may not reflect the full complexity of axioms in the ontology. Where no example is currently present in CL, examples are taken from the Drosophila Anatomy Ontology, which follows the same schema. Recording location \u00b6 Location of cell types is recorded by relating a cell type to a term in an anatomical ontology. For the Cell Ontology this means a term from Uberon. 'part of' \u00b6 Use part_of for cases where the location is a material anatomical structure (rather than a space, such as a sinus) and all of the cell is within the anatomical structure. \u2018 epithelial cell' subClassOf 'part of' some epithelium \u2018part of\u2019 is transitive, which means that it applies across chains of relationships. For example, \u2018ileal goblet cell\u2019 part_of some ileum ilium \u2018part of\u2019 some \u2018small intestine\u2019 \u2018small intestine\u2019 \u2018part of\u2019 some intestine => \u2019ileal goblet cell\u2019 \u2018part of\u2019 some \u2018small intestine\u2019 & \u2018ileal goblet cell\u2019 \u2018part of\u2019 some intestine located_in \u00b6 To record the location of a cell in an anatomical space (e.g., a sinus), 'located in' is used. For example: \u2018 lymph node marginal reticular cell \u2019 subClassOf 'located in' some 'subcapsular sinus of lymph node' overlaps \u00b6 'part of' applies in cases where an entire cell is within an anatomical structure, but some cells have parts in multiple anatomical structures. For example, many neurons span multiple regions of the central nervous system. The general relation for this is overlaps (has some part in). overlaps is not currently used directly in the cell ontology (time of writing 05/2023), but more specific relationships exist for recording the location of neurons and their parts. These are described in the next section. Recording the location of neurons \u00b6 has soma location \u00b6 When neurobiologists talk about the location of vertebrate neurons, they are typically referring to soma location. The importance of soma location to identify is underscored by how commonly cell types are named, in part, by soma location. We therefore have a dedicated relation for recording this: 'has soma location' . For example, anterior horn motor neuron has the following subclass axiom: 'has soma location' some 'ventral horn of spinal cord' axiomatization of \u2018has soma location\u2019 subPropertyOf : overlaps # if X has_soma_location some Y, then X overlaps some Y) domain : neuron # X has_soma_location some Y => X is inferred to be a subClassOf neuron property chain : has_soma_location o part_of --> has_soma_location # If x has soma location y and y is part_of z, then x has_soma_location_z Example of reasoning with the property chain: 'cortical interneuron' equivalentTo 'interneuron' that has_soma_location some 'cerebral cortex' 'rosehip neuron' subClassOf interneuron and has_soma_location some 'cortical layer 1' 'cortical layer 1' subClassOf part_of some 'cerebral cortex => 'rosehip neuron' subClassOf 'cortical interneuron' sends synaptic output to region \u00b6 A relationship between a neuron and a region, where the neuron has a functionally relevant number of output synapses in that region. ' adult basket subesophageal neuron ' SubClassOf sends synaptic output to region some inferior posterior slope receives synaptic input in region \u00b6 A relationship between a neuron and a region, where the neuron has a functionally relevant number of input synapses: e.g. ' adult basket subesophageal neuron ' SubClassOf \u2018 receives synaptic input in region \u2019 some \u2018 superior posterior slope \u2019 fasciculates_with \u00b6 Use this to record the tracts or nerves that a neuron\u2019s projections fasciculate with. e.g. \u2018Betz cell\u2019 subClasssOf \u2018fasciculates with\u2019 some \u2018corticospinal tract\u2019. subPropertyOf: overlaps domain: neuron range: neuron projection bundle Recording synaptic connectivity (neurons) \u00b6 To record neuron-to-neuron or motor neuron-to-target muscle connectivity, consider the following object properties. These properties should be used when connectivity is key to the definition, for example, in cases where a motor neuron type is defined by the type of muscle fiber on which it synapses. synapsed to \u00b6 For example, 'alpha motor neuron' SubClassOf synapsed to some 'extrafusal muscle fiber' synapsed by \u00b6 This is the the inverse of synapsed to For example, 'extrafusal muscle fiber' SubClassOf synapsed by some 'alpha motor neuron' Recording function \u00b6 Cellular function is recorded by linking GO biological process terms with the object properties 'capable of' and \u2018capable of part of\u2019 'capable of' \u00b6 Use this relationships where the cell is capable of carrying out the entirety of the process For example, 'hilus cell of ovary' has the following subclass: 'capable of' some 'androgen secretion' \u2018capable of part of\u2019 \u00b6 Use this relationship where only part of the process occurs in the cell type. e.g. 'retinal bipolar neuron' 'capable of part of' some 'visual perception' Recording developmental lineage \u00b6 Developmental lineage is recorded between cell types with the object property develops from (a transitive property), or in the case where there are no intermediates between the cells, 'directly develops from' (a non-transitive subproperty of develops_from ) For example, 'leukocyte' subClassOf develops from some 'hematopoietic stem cell' Recording cell markers \u00b6 Only markers that are necessary to define a cell type should be recorded. cell surface (protein) markers \u00b6 The cell ontology has a set of terms for recording cell surface markers. The most commonly used relation for recording markers is 'has plasma membrane part' . This object property is used to record cell surface markers, especially in immune cells. There are also more specific properties, 'has low plasma membrane amount' and 'has high plasma membrane amount' , that can be used at an editor's discretion. In each case, a term from the PRotein Ontology (PRO) or a protein complex term from the Gene Ontology (GO) is used as the object of the relation. For example, 'alpha-beta T cell' has the following equivalence axiom: 'T cell' and 'has plasma membrane part' some 'alpha-beta T cell receptor complex' Absence of a marker can be recorded using lacks_plasma_membrane_part Warning - this is used in place of the more accurate OWL expression \"NOT has_part some X*** in order to keep within the EL profile of OWL. It's use with a general class as a target can potentially lead to reasoning errors. recording gene markers \u00b6 \u2018expresses\u2019 \u00b6 Use this to link a cell type to a gene or gene product that defines it: For example: 'lamp5 GABAergic cortical interneuron' EquivalentTo: interneuron and ('has soma location' some 'cerebral cortex') and ('capable of' some 'gamma-aminobutyric acid secretion, neurotransmission') and (expresses some 'lysosome-associated membrane glycoprotein 5') In FBbt, FlyBase Gene IDs are permitted here (using standard resolvable URL pattern). In CL currenly only PRO IDs are permitted. In PCL, a broader range of IDs have been used (depending on data sources used). Recording cell parts \u00b6 To record parts above the granularity of proteins and complexes, use a 'has part' relationship with an object from the Gene Ontology cellular_component branch. e.g. 'melanocyte' subClassOf 'has part' some 'melanosome' This GO term can be combined with a PATO quality term (e.g. for shape) where necessary, e.g. For example: 'mature basophil' subClassOf ('has part' some (nucleus and ('has characteristic' some lobed))) Recording general cellular characteristics \u00b6 The ontology PATO , has a rich set of terms that can be used to record the general characteristics of cells, such as their morphology. These are recorded using 'has characteristic' . In choosing PATO terms, avoid those referring to some change in characteristic (e.g,.\u2019 increased branchiness\u2019). The following list of examples is not exhaustive: Recording Morphology \u00b6 PATO has a set of general morphology terms which may be applicable to cells. For example, erythrocyte subClassOf 'has characteristic' some biconcave PATO also has a set of terms for specific cell morphologies (mostly neuronal), e.g. \u2018Betz cell\u2019 subClassOf \u2018has characteristic\u2019 some \u2018standard pyramidal morphology\u2019 Recording nuclear number \u00b6 To record the number of nuclei in a cell, use a PATO subclass under the term 'nucleate quality' with the 'has characteristic' relation. For example, platelet subClassOf ( 'has_characteristic' some anucleate ) Note - that PATO includes bridging axioms that infer part relationships based on these characteristics. e.g. cell and ('has characteristic' some multinucleate) SubClassOf 'has part' some nucleus Taxon constraints \u00b6 See https://oboacademy.github.io/obook/explanation/taxon-constraints-explainer/ .","title":"CL relations"},{"location":"relations_guide/#cell-ontology-cl-relations-guide","text":"","title":"Cell Ontology (CL) relations guide."},{"location":"relations_guide/#intro","text":"The aim of this document is to provide an accessible guide to how to use relations to record the properties that define cell types including location, lineage, function, morphology and marker genes. The term 'relations' here refers principally to OWL object properties, but also includes annoation properties used as shortcuts for more expressive logical axioms that can be programatically generated from them. Relations in this guide are grouped by general use case (e.g. recording location) and each is illustrated by an example e.g.- melanocyte subClassOf \u2018has part\u2019 some melanosome. This should be read as \u2018 all melanocytes have some type of melanosome as a part\u2019 as should all axioms of this form. The examples should all be correct, but may not reflect the full complexity of axioms in the ontology. Where no example is currently present in CL, examples are taken from the Drosophila Anatomy Ontology, which follows the same schema.","title":"Intro"},{"location":"relations_guide/#recording-location","text":"Location of cell types is recorded by relating a cell type to a term in an anatomical ontology. For the Cell Ontology this means a term from Uberon.","title":"Recording location"},{"location":"relations_guide/#part-of","text":"Use part_of for cases where the location is a material anatomical structure (rather than a space, such as a sinus) and all of the cell is within the anatomical structure. \u2018 epithelial cell' subClassOf 'part of' some epithelium \u2018part of\u2019 is transitive, which means that it applies across chains of relationships. For example, \u2018ileal goblet cell\u2019 part_of some ileum ilium \u2018part of\u2019 some \u2018small intestine\u2019 \u2018small intestine\u2019 \u2018part of\u2019 some intestine => \u2019ileal goblet cell\u2019 \u2018part of\u2019 some \u2018small intestine\u2019 & \u2018ileal goblet cell\u2019 \u2018part of\u2019 some intestine","title":"'part of'"},{"location":"relations_guide/#located_in","text":"To record the location of a cell in an anatomical space (e.g., a sinus), 'located in' is used. For example: \u2018 lymph node marginal reticular cell \u2019 subClassOf 'located in' some 'subcapsular sinus of lymph node'","title":"located_in"},{"location":"relations_guide/#overlaps","text":"'part of' applies in cases where an entire cell is within an anatomical structure, but some cells have parts in multiple anatomical structures. For example, many neurons span multiple regions of the central nervous system. The general relation for this is overlaps (has some part in). overlaps is not currently used directly in the cell ontology (time of writing 05/2023), but more specific relationships exist for recording the location of neurons and their parts. These are described in the next section.","title":"overlaps"},{"location":"relations_guide/#recording-the-location-of-neurons","text":"","title":"Recording the location of neurons"},{"location":"relations_guide/#has-soma-location","text":"When neurobiologists talk about the location of vertebrate neurons, they are typically referring to soma location. The importance of soma location to identify is underscored by how commonly cell types are named, in part, by soma location. We therefore have a dedicated relation for recording this: 'has soma location' . For example, anterior horn motor neuron has the following subclass axiom: 'has soma location' some 'ventral horn of spinal cord' axiomatization of \u2018has soma location\u2019 subPropertyOf : overlaps # if X has_soma_location some Y, then X overlaps some Y) domain : neuron # X has_soma_location some Y => X is inferred to be a subClassOf neuron property chain : has_soma_location o part_of --> has_soma_location # If x has soma location y and y is part_of z, then x has_soma_location_z Example of reasoning with the property chain: 'cortical interneuron' equivalentTo 'interneuron' that has_soma_location some 'cerebral cortex' 'rosehip neuron' subClassOf interneuron and has_soma_location some 'cortical layer 1' 'cortical layer 1' subClassOf part_of some 'cerebral cortex => 'rosehip neuron' subClassOf 'cortical interneuron'","title":"has soma location"},{"location":"relations_guide/#sends-synaptic-output-to-region","text":"A relationship between a neuron and a region, where the neuron has a functionally relevant number of output synapses in that region. ' adult basket subesophageal neuron ' SubClassOf sends synaptic output to region some inferior posterior slope","title":"sends synaptic output to region"},{"location":"relations_guide/#receives-synaptic-input-in-region","text":"A relationship between a neuron and a region, where the neuron has a functionally relevant number of input synapses: e.g. ' adult basket subesophageal neuron ' SubClassOf \u2018 receives synaptic input in region \u2019 some \u2018 superior posterior slope \u2019","title":"receives synaptic input in region"},{"location":"relations_guide/#fasciculates_with","text":"Use this to record the tracts or nerves that a neuron\u2019s projections fasciculate with. e.g. \u2018Betz cell\u2019 subClasssOf \u2018fasciculates with\u2019 some \u2018corticospinal tract\u2019. subPropertyOf: overlaps domain: neuron range: neuron projection bundle","title":"fasciculates_with"},{"location":"relations_guide/#recording-synaptic-connectivity-neurons","text":"To record neuron-to-neuron or motor neuron-to-target muscle connectivity, consider the following object properties. These properties should be used when connectivity is key to the definition, for example, in cases where a motor neuron type is defined by the type of muscle fiber on which it synapses.","title":"Recording synaptic connectivity (neurons)"},{"location":"relations_guide/#synapsed-to","text":"For example, 'alpha motor neuron' SubClassOf synapsed to some 'extrafusal muscle fiber'","title":"synapsed to"},{"location":"relations_guide/#synapsed-by","text":"This is the the inverse of synapsed to For example, 'extrafusal muscle fiber' SubClassOf synapsed by some 'alpha motor neuron'","title":"synapsed by"},{"location":"relations_guide/#recording-function","text":"Cellular function is recorded by linking GO biological process terms with the object properties 'capable of' and \u2018capable of part of\u2019","title":"Recording function"},{"location":"relations_guide/#capable-of","text":"Use this relationships where the cell is capable of carrying out the entirety of the process For example, 'hilus cell of ovary' has the following subclass: 'capable of' some 'androgen secretion'","title":"'capable of'"},{"location":"relations_guide/#capable-of-part-of","text":"Use this relationship where only part of the process occurs in the cell type. e.g. 'retinal bipolar neuron' 'capable of part of' some 'visual perception'","title":"\u2018capable of part of\u2019"},{"location":"relations_guide/#recording-developmental-lineage","text":"Developmental lineage is recorded between cell types with the object property develops from (a transitive property), or in the case where there are no intermediates between the cells, 'directly develops from' (a non-transitive subproperty of develops_from ) For example, 'leukocyte' subClassOf develops from some 'hematopoietic stem cell'","title":"Recording developmental lineage"},{"location":"relations_guide/#recording-cell-markers","text":"Only markers that are necessary to define a cell type should be recorded.","title":"Recording cell markers"},{"location":"relations_guide/#cell-surface-protein-markers","text":"The cell ontology has a set of terms for recording cell surface markers. The most commonly used relation for recording markers is 'has plasma membrane part' . This object property is used to record cell surface markers, especially in immune cells. There are also more specific properties, 'has low plasma membrane amount' and 'has high plasma membrane amount' , that can be used at an editor's discretion. In each case, a term from the PRotein Ontology (PRO) or a protein complex term from the Gene Ontology (GO) is used as the object of the relation. For example, 'alpha-beta T cell' has the following equivalence axiom: 'T cell' and 'has plasma membrane part' some 'alpha-beta T cell receptor complex' Absence of a marker can be recorded using lacks_plasma_membrane_part Warning - this is used in place of the more accurate OWL expression \"NOT has_part some X*** in order to keep within the EL profile of OWL. It's use with a general class as a target can potentially lead to reasoning errors.","title":"cell surface (protein) markers"},{"location":"relations_guide/#recording-gene-markers","text":"","title":"recording gene markers"},{"location":"relations_guide/#expresses","text":"Use this to link a cell type to a gene or gene product that defines it: For example: 'lamp5 GABAergic cortical interneuron' EquivalentTo: interneuron and ('has soma location' some 'cerebral cortex') and ('capable of' some 'gamma-aminobutyric acid secretion, neurotransmission') and (expresses some 'lysosome-associated membrane glycoprotein 5') In FBbt, FlyBase Gene IDs are permitted here (using standard resolvable URL pattern). In CL currenly only PRO IDs are permitted. In PCL, a broader range of IDs have been used (depending on data sources used).","title":"\u2018expresses\u2019"},{"location":"relations_guide/#recording-cell-parts","text":"To record parts above the granularity of proteins and complexes, use a 'has part' relationship with an object from the Gene Ontology cellular_component branch. e.g. 'melanocyte' subClassOf 'has part' some 'melanosome' This GO term can be combined with a PATO quality term (e.g. for shape) where necessary, e.g. For example: 'mature basophil' subClassOf ('has part' some (nucleus and ('has characteristic' some lobed)))","title":"Recording cell parts"},{"location":"relations_guide/#recording-general-cellular-characteristics","text":"The ontology PATO , has a rich set of terms that can be used to record the general characteristics of cells, such as their morphology. These are recorded using 'has characteristic' . In choosing PATO terms, avoid those referring to some change in characteristic (e.g,.\u2019 increased branchiness\u2019). The following list of examples is not exhaustive:","title":"Recording general cellular characteristics"},{"location":"relations_guide/#recording-morphology","text":"PATO has a set of general morphology terms which may be applicable to cells. For example, erythrocyte subClassOf 'has characteristic' some biconcave PATO also has a set of terms for specific cell morphologies (mostly neuronal), e.g. \u2018Betz cell\u2019 subClassOf \u2018has characteristic\u2019 some \u2018standard pyramidal morphology\u2019","title":"Recording Morphology"},{"location":"relations_guide/#recording-nuclear-number","text":"To record the number of nuclei in a cell, use a PATO subclass under the term 'nucleate quality' with the 'has characteristic' relation. For example, platelet subClassOf ( 'has_characteristic' some anucleate ) Note - that PATO includes bridging axioms that infer part relationships based on these characteristics. e.g. cell and ('has characteristic' some multinucleate) SubClassOf 'has part' some nucleus","title":"Recording nuclear number"},{"location":"relations_guide/#taxon-constraints","text":"See https://oboacademy.github.io/obook/explanation/taxon-constraints-explainer/ .","title":"Taxon constraints"},{"location":"resolving_merge_conflicts/","text":"A guide to resolving merge conflicts \u00b6 When you come to merge your pull request, you may find that conflicts prevent automated merging back into the master. In some cases, GitHub supports resolution of these through its web interface. However, probably due to file size, this is not currently supported for cl-edit.owl. The majority of the time, the conflict is trivial - due to addition of new terms to the same point in the file. Because terms are ordered in the file by ID, this happens whenever two edits add terms without any intervening IDs. Trivial clashes are easy to spot - they involve whole term stanzas + declarations. Occassionally non-trivial clashes will happen when two pull requests include edits to the same term or even the same axiom. Ask an editor for help if you don't feel confident resolving these. SOP. \u00b6 Reserialise the Master file using the Ontology Development Kit (ODK). This requires setting up Docker and ODK. If not already set up, follow the instructions here . Open Docker. At the line command (PC) or Terminal (Mac), use the cd (change directory) command to navigate to the repository's src/ontology/ directory. For example, ''' cd PATH_TO_ONTOLOGY/src/ontology/ ''' Replace \"PATH_TO_ONTOLOGY\" with the actual file path to the ontology. If you need to orient yourself, use the '''pwd''' (present working directory) or '''ls''' (list) line commands. If you are resolving a conflict in an .owl file, run: ''' sh run.sh make normalize_src ''' If you are resolving a conflict in an .obo file, run: ''' sh run.sh make normalize_obo_src ''' In CL, edits sometimes result in creating a large amount of uninteded differences involving ^^xsd:string. If you see these differences after running the command above, they can be resolved by following the instructions here . In GitHub Desktop: Checkout Master and pull to make sure your Master branch is up to date. Checkout the branch for the pull request and make sure it is up to date. Choose Branch > Update from master: GitHub desktop should detect the clash and ask you if you want to open in your text editor of choice (e.g., Atom). If clashes are due to trivial ordering problems, delete the conflict marks (<<<<<<<, =======, >>>>>>>), commit and push back to GitHub. Check the resulting diffs on the Pull Request on GitHub. Once the checks have run and are successful, merge and delete the branch.","title":"Resolving merge conflicts"},{"location":"resolving_merge_conflicts/#a-guide-to-resolving-merge-conflicts","text":"When you come to merge your pull request, you may find that conflicts prevent automated merging back into the master. In some cases, GitHub supports resolution of these through its web interface. However, probably due to file size, this is not currently supported for cl-edit.owl. The majority of the time, the conflict is trivial - due to addition of new terms to the same point in the file. Because terms are ordered in the file by ID, this happens whenever two edits add terms without any intervening IDs. Trivial clashes are easy to spot - they involve whole term stanzas + declarations. Occassionally non-trivial clashes will happen when two pull requests include edits to the same term or even the same axiom. Ask an editor for help if you don't feel confident resolving these.","title":"A guide to resolving merge conflicts"},{"location":"resolving_merge_conflicts/#sop","text":"Reserialise the Master file using the Ontology Development Kit (ODK). This requires setting up Docker and ODK. If not already set up, follow the instructions here . Open Docker. At the line command (PC) or Terminal (Mac), use the cd (change directory) command to navigate to the repository's src/ontology/ directory. For example, ''' cd PATH_TO_ONTOLOGY/src/ontology/ ''' Replace \"PATH_TO_ONTOLOGY\" with the actual file path to the ontology. If you need to orient yourself, use the '''pwd''' (present working directory) or '''ls''' (list) line commands. If you are resolving a conflict in an .owl file, run: ''' sh run.sh make normalize_src ''' If you are resolving a conflict in an .obo file, run: ''' sh run.sh make normalize_obo_src ''' In CL, edits sometimes result in creating a large amount of uninteded differences involving ^^xsd:string. If you see these differences after running the command above, they can be resolved by following the instructions here . In GitHub Desktop: Checkout Master and pull to make sure your Master branch is up to date. Checkout the branch for the pull request and make sure it is up to date. Choose Branch > Update from master: GitHub desktop should detect the clash and ask you if you want to open in your text editor of choice (e.g., Atom). If clashes are due to trivial ordering problems, delete the conflict marks (<<<<<<<, =======, >>>>>>>), commit and push back to GitHub. Check the resulting diffs on the Pull Request on GitHub. Once the checks have run and are successful, merge and delete the branch.","title":"SOP."},{"location":"taxon-restrictions/","text":"The cell ontology (CL) follows certain conventions regarding taxon constraints. Please review the following pages before adding taxon constraints to CL terms. Explanation of taxon constraints can be found here: https://oboacademy.github.io/obook/explanation/taxon-constraints-explainer/ A how-to guide can be found here: https://oboacademy.github.io/obook/howto/add-taxon-restrictions/","title":"Taxon constraints"},{"location":"odk-workflows/","text":"Default ODK Workflows \u00b6 Daily Editors Workflow Release Workflow Manage your ODK Repository Setting up Docker for ODK Imports management Managing the documentation Managing your Automated Testing","title":"Overview"},{"location":"odk-workflows/#default-odk-workflows","text":"Daily Editors Workflow Release Workflow Manage your ODK Repository Setting up Docker for ODK Imports management Managing the documentation Managing your Automated Testing","title":"Default ODK Workflows"},{"location":"odk-workflows/ContinuousIntegration/","text":"Introduction to Continuous Integration Workflows with ODK \u00b6 Historically, most repos have been using Travis CI for continuous integration testing and building, but due to runtime restrictions, we recently switched a lot of our repos to GitHub actions. You can set up your repo with CI by adding this to your configuration file (src/ontology/cl-odk.yaml): ci: - github_actions When updateing your repo , you will notice a new file being added: .github/workflows/qc.yml . This file contains your CI logic, so if you need to change, or add anything, this is the place! Alternatively, if your repo is in GitLab instead of GitHub, you can set up your repo with GitLab CI by adding this to your configuration file (src/ontology/cl-odk.yaml): ci: - gitlab-ci This will add a file called .gitlab-ci.yml in the root of your repo.","title":"Continuous Integration"},{"location":"odk-workflows/ContinuousIntegration/#introduction-to-continuous-integration-workflows-with-odk","text":"Historically, most repos have been using Travis CI for continuous integration testing and building, but due to runtime restrictions, we recently switched a lot of our repos to GitHub actions. You can set up your repo with CI by adding this to your configuration file (src/ontology/cl-odk.yaml): ci: - github_actions When updateing your repo , you will notice a new file being added: .github/workflows/qc.yml . This file contains your CI logic, so if you need to change, or add anything, this is the place! Alternatively, if your repo is in GitLab instead of GitHub, you can set up your repo with GitLab CI by adding this to your configuration file (src/ontology/cl-odk.yaml): ci: - gitlab-ci This will add a file called .gitlab-ci.yml in the root of your repo.","title":"Introduction to Continuous Integration Workflows with ODK"},{"location":"odk-workflows/EditorsWorkflow/","text":"Editors Workflow \u00b6 The editors workflow is one of the formal workflows to ensure that the ontology is developed correctly according to ontology engineering principles. There are a few different editors workflows: Local editing workflow: Editing the ontology in your local environment by hand, using tools such as Prot\u00e9g\u00e9, ROBOT templates or DOSDP patterns. Completely automated data pipeline (GitHub Actions) DROID workflow This document only covers the first editing workflow, but more will be added in the future Local editing workflow \u00b6 Workflow requirements: git github docker editing tool of choice, e.g. Prot\u00e9g\u00e9, your favourite text editor, etc 1. Create issue \u00b6 Ensure that there is a ticket on your issue tracker that describes the change you are about to make. While this seems optional, this is a very important part of the social contract of building an ontology - no change to the ontology should be performed without a good ticket, describing the motivation and nature of the intended change. 2. Update main branch \u00b6 In your local environment (e.g. your laptop), make sure you are on the main (prev. master ) branch and ensure that you have all the upstream changes, for example: git checkout master git pull 3. Create feature branch \u00b6 Create a new branch. Per convention, we try to use meaningful branch names such as: - issue23removeprocess (where issue 23 is the related issue on GitHub) - issue26addcontributor - release20210101 (for releases) On your command line, this looks like this: git checkout -b issue23removeprocess 4. Perform edit \u00b6 Using your editor of choice, perform the intended edit. For example: Prot\u00e9g\u00e9 Open src/ontology/cl-edit.owl in Prot\u00e9g\u00e9 Make the change Save the file TextEdit Open src/ontology/cl-edit.owl in TextEdit (or Sublime, Atom, Vim, Nano) Make the change Save the file Consider the following when making the edit. According to our development philosophy, the only places that should be manually edited are: src/ontology/cl-edit.owl Any ROBOT templates you chose to use (the TSV files only) Any DOSDP data tables you chose to use (the TSV files, and potentially the associated patterns) components (anything in src/ontology/components ), see here . Imports should not be edited (any edits will be flushed out with the next update). However, refreshing imports is a potentially breaking change - and is discussed elsewhere . Changes should usually be small. Adding or changing 1 term is great. Adding or changing 10 related terms is ok. Adding or changing 100 or more terms at once should be considered very carefully. 4. Check the Git diff \u00b6 This step is very important. Rather than simply trusting your change had the intended effect, we should always use a git diff as a first pass for sanity checking. In our experience, having a visual git client like GitHub Desktop or sourcetree is really helpful for this part. In case you prefer the command line: git status git diff 5. Quality control \u00b6 Now it's time to run your quality control checks. This can either happen locally ( 5a ) or through your continuous integration system ( 7/5b ). 5a. Local testing \u00b6 If you chose to run your test locally: sh run.sh make IMP=false test This will run the whole set of configured ODK tests on including your change. If you have a complex DOSDP pattern pipeline you may want to add PAT=false to skip the potentially lengthy process of rebuilding the patterns. sh run.sh make IMP=false PAT=false test 6. Pull request \u00b6 When you are happy with the changes, you commit your changes to your feature branch, push them upstream (to GitHub) and create a pull request. For example: git add NAMEOFCHANGEDFILES git commit -m \"Added biological process term #12\" git push -u origin issue23removeprocess Then you go to your project on GitHub, and create a new pull request from the branch, for example: https://github.com/INCATools/ontology-development-kit/pulls There is a lot of great advise on how to write pull requests, but at the very least you should: - mention the tickets affected: see #23 to link to a related ticket, or fixes #23 if, by merging this pull request, the ticket is fixed. Tickets in the latter case will be closed automatically by GitHub when the pull request is merged. - summarise the changes in a few sentences. Consider the reviewer: what would they want to know right away. - If the diff is large, provide instructions on how to review the pull request best (sometimes, there are many changed files, but only one important change). 7/5b. Continuous Integration Testing \u00b6 If you didn't run and local quality control checks (see 5a ), you should have Continuous Integration (CI) set up, for example: - Travis - GitHub Actions More on how to set this up here . Once the pull request is created, the CI will automatically trigger. If all is fine, it will show up green, otherwise red. 8. Community review \u00b6 Once all the automatic tests have passed, it is important to put a second set of eyes on the pull request. Ontologies are inherently social - as in that they represent some kind of community consensus on how a domain is organised conceptually. This seems high brow talk, but it is very important that as an ontology editor, you have your work validated by the community you are trying to serve (e.g. your colleagues, other contributors etc.). In our experience, it is hard to get more than one review on a pull request - two is great. You can set up GitHub branch protection to actually require a review before a pull request can be merged! We recommend this. This step seems daunting to some hopefully under-resourced ontologies, but we recommend to put this high up on your list of priorities - train a colleague, reach out! 9. Merge and cleanup \u00b6 When the QC is green and the reviews are in (approvals), it is time to merge the pull request. After the pull request is merged, remember to delete the branch as well (this option will show up as a big button right after you have merged the pull request). If you have not done so, close all the associated tickets fixed by the pull request. 10. Changelog (Optional) \u00b6 It is sometimes difficult to keep track of changes made to an ontology. Some ontology teams opt to document changes in a changelog (simply a text file in your repository) so that when release day comes, you know everything you have changed. This is advisable at least for major changes (such as a new release system, a new pattern or template etc.).","title":"Editors Workflow"},{"location":"odk-workflows/EditorsWorkflow/#editors-workflow","text":"The editors workflow is one of the formal workflows to ensure that the ontology is developed correctly according to ontology engineering principles. There are a few different editors workflows: Local editing workflow: Editing the ontology in your local environment by hand, using tools such as Prot\u00e9g\u00e9, ROBOT templates or DOSDP patterns. Completely automated data pipeline (GitHub Actions) DROID workflow This document only covers the first editing workflow, but more will be added in the future","title":"Editors Workflow"},{"location":"odk-workflows/EditorsWorkflow/#local-editing-workflow","text":"Workflow requirements: git github docker editing tool of choice, e.g. Prot\u00e9g\u00e9, your favourite text editor, etc","title":"Local editing workflow"},{"location":"odk-workflows/EditorsWorkflow/#1-create-issue","text":"Ensure that there is a ticket on your issue tracker that describes the change you are about to make. While this seems optional, this is a very important part of the social contract of building an ontology - no change to the ontology should be performed without a good ticket, describing the motivation and nature of the intended change.","title":"1. Create issue"},{"location":"odk-workflows/EditorsWorkflow/#2-update-main-branch","text":"In your local environment (e.g. your laptop), make sure you are on the main (prev. master ) branch and ensure that you have all the upstream changes, for example: git checkout master git pull","title":"2. Update main branch"},{"location":"odk-workflows/EditorsWorkflow/#3-create-feature-branch","text":"Create a new branch. Per convention, we try to use meaningful branch names such as: - issue23removeprocess (where issue 23 is the related issue on GitHub) - issue26addcontributor - release20210101 (for releases) On your command line, this looks like this: git checkout -b issue23removeprocess","title":"3. Create feature branch"},{"location":"odk-workflows/EditorsWorkflow/#4-perform-edit","text":"Using your editor of choice, perform the intended edit. For example: Prot\u00e9g\u00e9 Open src/ontology/cl-edit.owl in Prot\u00e9g\u00e9 Make the change Save the file TextEdit Open src/ontology/cl-edit.owl in TextEdit (or Sublime, Atom, Vim, Nano) Make the change Save the file Consider the following when making the edit. According to our development philosophy, the only places that should be manually edited are: src/ontology/cl-edit.owl Any ROBOT templates you chose to use (the TSV files only) Any DOSDP data tables you chose to use (the TSV files, and potentially the associated patterns) components (anything in src/ontology/components ), see here . Imports should not be edited (any edits will be flushed out with the next update). However, refreshing imports is a potentially breaking change - and is discussed elsewhere . Changes should usually be small. Adding or changing 1 term is great. Adding or changing 10 related terms is ok. Adding or changing 100 or more terms at once should be considered very carefully.","title":"4. Perform edit"},{"location":"odk-workflows/EditorsWorkflow/#4-check-the-git-diff","text":"This step is very important. Rather than simply trusting your change had the intended effect, we should always use a git diff as a first pass for sanity checking. In our experience, having a visual git client like GitHub Desktop or sourcetree is really helpful for this part. In case you prefer the command line: git status git diff","title":"4. Check the Git diff"},{"location":"odk-workflows/EditorsWorkflow/#5-quality-control","text":"Now it's time to run your quality control checks. This can either happen locally ( 5a ) or through your continuous integration system ( 7/5b ).","title":"5. Quality control"},{"location":"odk-workflows/EditorsWorkflow/#5a-local-testing","text":"If you chose to run your test locally: sh run.sh make IMP=false test This will run the whole set of configured ODK tests on including your change. If you have a complex DOSDP pattern pipeline you may want to add PAT=false to skip the potentially lengthy process of rebuilding the patterns. sh run.sh make IMP=false PAT=false test","title":"5a. Local testing"},{"location":"odk-workflows/EditorsWorkflow/#6-pull-request","text":"When you are happy with the changes, you commit your changes to your feature branch, push them upstream (to GitHub) and create a pull request. For example: git add NAMEOFCHANGEDFILES git commit -m \"Added biological process term #12\" git push -u origin issue23removeprocess Then you go to your project on GitHub, and create a new pull request from the branch, for example: https://github.com/INCATools/ontology-development-kit/pulls There is a lot of great advise on how to write pull requests, but at the very least you should: - mention the tickets affected: see #23 to link to a related ticket, or fixes #23 if, by merging this pull request, the ticket is fixed. Tickets in the latter case will be closed automatically by GitHub when the pull request is merged. - summarise the changes in a few sentences. Consider the reviewer: what would they want to know right away. - If the diff is large, provide instructions on how to review the pull request best (sometimes, there are many changed files, but only one important change).","title":"6. Pull request"},{"location":"odk-workflows/EditorsWorkflow/#75b-continuous-integration-testing","text":"If you didn't run and local quality control checks (see 5a ), you should have Continuous Integration (CI) set up, for example: - Travis - GitHub Actions More on how to set this up here . Once the pull request is created, the CI will automatically trigger. If all is fine, it will show up green, otherwise red.","title":"7/5b. Continuous Integration Testing"},{"location":"odk-workflows/EditorsWorkflow/#8-community-review","text":"Once all the automatic tests have passed, it is important to put a second set of eyes on the pull request. Ontologies are inherently social - as in that they represent some kind of community consensus on how a domain is organised conceptually. This seems high brow talk, but it is very important that as an ontology editor, you have your work validated by the community you are trying to serve (e.g. your colleagues, other contributors etc.). In our experience, it is hard to get more than one review on a pull request - two is great. You can set up GitHub branch protection to actually require a review before a pull request can be merged! We recommend this. This step seems daunting to some hopefully under-resourced ontologies, but we recommend to put this high up on your list of priorities - train a colleague, reach out!","title":"8. Community review"},{"location":"odk-workflows/EditorsWorkflow/#9-merge-and-cleanup","text":"When the QC is green and the reviews are in (approvals), it is time to merge the pull request. After the pull request is merged, remember to delete the branch as well (this option will show up as a big button right after you have merged the pull request). If you have not done so, close all the associated tickets fixed by the pull request.","title":"9. Merge and cleanup"},{"location":"odk-workflows/EditorsWorkflow/#10-changelog-optional","text":"It is sometimes difficult to keep track of changes made to an ontology. Some ontology teams opt to document changes in a changelog (simply a text file in your repository) so that when release day comes, you know everything you have changed. This is advisable at least for major changes (such as a new release system, a new pattern or template etc.).","title":"10. Changelog (Optional)"},{"location":"odk-workflows/ManageAutomatedTest/","text":"Constraint violation checks \u00b6 We can define custom checks using SPARQL . SPARQL queries define bad modelling patterns (missing labels, misspelt URIs, and many more) in the ontology. If these queries return any results, then the build will fail. Custom checks are designed to be run as part of GitHub Actions Continuous Integration testing, but they can also run locally. Steps to add a constraint violation check: \u00b6 Add the SPARQL query in src/sparql . The name of the file should end with -violation.sparql . Please give a name that helps to understand which violation the query wants to check. Add the name of the new file to odk configuration file src/ontology/uberon-odk.yaml : Include the name of the file (without the -violation.sparql part) to the list inside the key custom_sparql_checks that is inside robot_report key. If the robot_report or custom_sparql_checks keys are not available, please add this code block to the end of the file. yaml robot_report: release_reports: False fail_on: ERROR use_labels: False custom_profile: True report_on: - edit custom_sparql_checks: - name-of-the-file-check 3. Update the repository so your new SPARQL check will be included in the QC. sh run.sh make update_repo","title":"ManageAutomatedTest"},{"location":"odk-workflows/ManageAutomatedTest/#constraint-violation-checks","text":"We can define custom checks using SPARQL . SPARQL queries define bad modelling patterns (missing labels, misspelt URIs, and many more) in the ontology. If these queries return any results, then the build will fail. Custom checks are designed to be run as part of GitHub Actions Continuous Integration testing, but they can also run locally.","title":"Constraint violation checks"},{"location":"odk-workflows/ManageAutomatedTest/#steps-to-add-a-constraint-violation-check","text":"Add the SPARQL query in src/sparql . The name of the file should end with -violation.sparql . Please give a name that helps to understand which violation the query wants to check. Add the name of the new file to odk configuration file src/ontology/uberon-odk.yaml : Include the name of the file (without the -violation.sparql part) to the list inside the key custom_sparql_checks that is inside robot_report key. If the robot_report or custom_sparql_checks keys are not available, please add this code block to the end of the file. yaml robot_report: release_reports: False fail_on: ERROR use_labels: False custom_profile: True report_on: - edit custom_sparql_checks: - name-of-the-file-check 3. Update the repository so your new SPARQL check will be included in the QC. sh run.sh make update_repo","title":"Steps to add a constraint violation check:"},{"location":"odk-workflows/ManageDocumentation/","text":"Updating the Documentation \u00b6 The documentation for CL is managed in two places (relative to the repository root): The docs directory contains all the files that pertain to the content of the documentation (more below) the mkdocs.yaml file contains the documentation config, in particular its navigation bar and theme. The documentation is hosted using GitHub pages, on a special branch of the repository (called gh-pages ). It is important that this branch is never deleted - it contains all the files GitHub pages needs to render and deploy the site. It is also important to note that the gh-pages branch should never be edited manually . All changes to the docs happen inside the docs directory on the main branch. Editing the docs \u00b6 Changing content \u00b6 All the documentation is contained in the docs directory, and is managed in Markdown . Markdown is a very simple and convenient way to produce text documents with formatting instructions, and is very easy to learn - it is also used, for example, in GitHub issues. This is a normal editing workflow: Open the .md file you want to change in an editor of choice (a simple text editor is often best). IMPORTANT : Do not edit any files in the docs/odk-workflows/ directory. These files are managed by the ODK system and will be overwritten when the repository is upgraded! If you wish to change these files, make an issue on the ODK issue tracker . Perform the edit and save the file Commit the file to a branch, and create a pull request as usual. If your development team likes your changes, merge the docs into master branch. Deploy the documentation (see below) Deploy the documentation \u00b6 The documentation is not automatically updated from the Markdown, and needs to be deployed deliberately. To do this, perform the following steps: In your terminal, navigate to the edit directory of your ontology, e.g.: cd cl/src/ontology Now you are ready to build the docs as follows: sh run.sh make update_docs Mkdocs now sets off to build the site from the markdown pages. You will be asked to Enter your username Enter your password (see here for using GitHub access tokens instead) IMPORTANT : Using password based authentication will be deprecated this year (2021). Make sure you read up on personal access tokens if that happens! If everything was successful, you will see a message similar to this one: INFO - Your documentation should shortly be available at: https://obophenotype.github.io/cell-ontology/ 3. Just to double check, you can now navigate to your documentation pages (usually https://obophenotype.github.io/cell-ontology/). Just make sure you give GitHub 2-5 minutes to build the pages!","title":"Managing the documentation"},{"location":"odk-workflows/ManageDocumentation/#updating-the-documentation","text":"The documentation for CL is managed in two places (relative to the repository root): The docs directory contains all the files that pertain to the content of the documentation (more below) the mkdocs.yaml file contains the documentation config, in particular its navigation bar and theme. The documentation is hosted using GitHub pages, on a special branch of the repository (called gh-pages ). It is important that this branch is never deleted - it contains all the files GitHub pages needs to render and deploy the site. It is also important to note that the gh-pages branch should never be edited manually . All changes to the docs happen inside the docs directory on the main branch.","title":"Updating the Documentation"},{"location":"odk-workflows/ManageDocumentation/#editing-the-docs","text":"","title":"Editing the docs"},{"location":"odk-workflows/ManageDocumentation/#changing-content","text":"All the documentation is contained in the docs directory, and is managed in Markdown . Markdown is a very simple and convenient way to produce text documents with formatting instructions, and is very easy to learn - it is also used, for example, in GitHub issues. This is a normal editing workflow: Open the .md file you want to change in an editor of choice (a simple text editor is often best). IMPORTANT : Do not edit any files in the docs/odk-workflows/ directory. These files are managed by the ODK system and will be overwritten when the repository is upgraded! If you wish to change these files, make an issue on the ODK issue tracker . Perform the edit and save the file Commit the file to a branch, and create a pull request as usual. If your development team likes your changes, merge the docs into master branch. Deploy the documentation (see below)","title":"Changing content"},{"location":"odk-workflows/ManageDocumentation/#deploy-the-documentation","text":"The documentation is not automatically updated from the Markdown, and needs to be deployed deliberately. To do this, perform the following steps: In your terminal, navigate to the edit directory of your ontology, e.g.: cd cl/src/ontology Now you are ready to build the docs as follows: sh run.sh make update_docs Mkdocs now sets off to build the site from the markdown pages. You will be asked to Enter your username Enter your password (see here for using GitHub access tokens instead) IMPORTANT : Using password based authentication will be deprecated this year (2021). Make sure you read up on personal access tokens if that happens! If everything was successful, you will see a message similar to this one: INFO - Your documentation should shortly be available at: https://obophenotype.github.io/cell-ontology/ 3. Just to double check, you can now navigate to your documentation pages (usually https://obophenotype.github.io/cell-ontology/). Just make sure you give GitHub 2-5 minutes to build the pages!","title":"Deploy the documentation"},{"location":"odk-workflows/ReleaseWorkflow/","text":"The release workflow \u00b6 The release workflow recommended by the ODK is based on GitHub releases and works as follows: Run a release with the ODK Review the release Merge to main branch Create a GitHub release These steps are outlined in detail in the following. Run a release with the ODK \u00b6 Preparation: Ensure that all your pull requests are merged into your main (master) branch Make sure that all changes to master are committed to GitHub ( git status should say that there are no modified files) Locally make sure you have the latest changes from master ( git pull ) Checkout a new branch (e.g. git checkout -b release-2021-01-01 ) You may or may not want to refresh your imports as part of your release strategy (see here ) Make sure you have the latest ODK installed by running docker pull obolibrary/odkfull To actually run the release, you: Open a command line terminal window and navigate to the src/ontology directory ( cd cl/src/ontology ) Run release pipeline: sh run.sh make prepare_release -B . Note that for some ontologies, this process can take up to 90 minutes - especially if there are large ontologies you depend on, like PRO or CHEBI. If everything went well, you should see the following output on your machine: Release files are now in ../.. - now you should commit, push and make a release on your git hosting site such as GitHub or GitLab . This will create all the specified release targets (OBO, OWL, JSON, and the variants, ont-full and ont-base) and copy them into your release directory (the top level of your repo). Review the release \u00b6 (Optional) Rough check. This step is frequently skipped, but for the more paranoid among us (like the author of this doc), this is a 3 minute additional effort for some peace of mind. Open the main release (cl.owl) in you favourite development environment (i.e. Prot\u00e9g\u00e9) and eyeball the hierarchy. We recommend two simple checks: Does the very top level of the hierarchy look ok? This means that all new terms have been imported/updated correctly. Does at least one change that you know should be in this release appear? For example, a new class. This means that the release was actually based on the recent edit file. Commit your changes to the branch and make a pull request In your GitHub pull request, review the following three files in detail (based on our experience): cl.obo - this reflects a useful subset of the whole ontology (everything that can be covered by OBO format). OBO format has that speaking for it: it is very easy to review! cl-base.owl - this reflects the asserted axioms in your ontology that you have actually edited. Ideally also take a look at cl-full.owl , which may reveal interesting new inferences you did not know about. Note that the diff of this file is sometimes quite large. Like with every pull request, we recommend to always employ a second set of eyes when reviewing a PR! Merge the main branch \u00b6 Once your CI checks have passed, and your reviews are completed, you can now merge the branch into your main branch (don't forget to delete the branch afterwards - a big button will appear after the merge is finished). Create a GitHub release \u00b6 Go to your releases page on GitHub by navigating to your repository, and then clicking on releases (usually on the right, for example: https://github.com/obophenotype/cell-ontology/releases). Then click \"Draft new release\" As the tag version you need to choose the date on which your ontologies were build. You can find this, for example, by looking at the cl.obo file and check the data-version: property. The date needs to be prefixed with a v , so, for example v2020-02-06 . You can write whatever you want in the release title, but we typically write the date again. The description underneath should contain a concise list of changes or term additions. Click \"Publish release\". Done. Debugging typical ontology release problems \u00b6 Problems with memory \u00b6 When you are dealing with large ontologies, you need a lot of memory. When you see error messages relating to large ontologies such as CHEBI, PRO, NCBITAXON, or Uberon, you should think of memory first, see here . Problems when using OBO format based tools \u00b6 Sometimes you will get cryptic error messages when using legacy tools using OBO format, such as the ontology release tool (OORT), which is also available as part of the ODK docker container. In these cases, you need to track down what axiom or annotation actually caused the breakdown. In our experience (in about 60% of the cases) the problem lies with duplicate annotations ( def , comment ) which are illegal in OBO. Here is an example recipe of how to deal with such a problem: If you get a message like make: *** [cl.Makefile:84: oort] Error 255 you might have a OORT error. To debug this, in your terminal enter sh run.sh make IMP=false PAT=false oort -B (assuming you are already in the ontology folder in your directory) This should show you where the error is in the log (eg multiple different definitions) WARNING: THE FIX BELOW IS NOT IDEAL, YOU SHOULD ALWAYS TRY TO FIX UPSTREAM IF POSSIBLE Open cl-edit.owl in Prot\u00e9g\u00e9 and find the offending term and delete all offending issue (e.g. delete ALL definition, if the problem was \"multiple def tags not allowed\") and save. *While this is not idea, as it will remove all definitions from that term, it will be added back again when the term is fixed in the ontology it was imported from and added back in. Rerun sh run.sh make IMP=false PAT=false oort -B and if it all passes, commit your changes to a branch and make a pull request as usual.","title":"Release Workflow"},{"location":"odk-workflows/ReleaseWorkflow/#the-release-workflow","text":"The release workflow recommended by the ODK is based on GitHub releases and works as follows: Run a release with the ODK Review the release Merge to main branch Create a GitHub release These steps are outlined in detail in the following.","title":"The release workflow"},{"location":"odk-workflows/ReleaseWorkflow/#run-a-release-with-the-odk","text":"Preparation: Ensure that all your pull requests are merged into your main (master) branch Make sure that all changes to master are committed to GitHub ( git status should say that there are no modified files) Locally make sure you have the latest changes from master ( git pull ) Checkout a new branch (e.g. git checkout -b release-2021-01-01 ) You may or may not want to refresh your imports as part of your release strategy (see here ) Make sure you have the latest ODK installed by running docker pull obolibrary/odkfull To actually run the release, you: Open a command line terminal window and navigate to the src/ontology directory ( cd cl/src/ontology ) Run release pipeline: sh run.sh make prepare_release -B . Note that for some ontologies, this process can take up to 90 minutes - especially if there are large ontologies you depend on, like PRO or CHEBI. If everything went well, you should see the following output on your machine: Release files are now in ../.. - now you should commit, push and make a release on your git hosting site such as GitHub or GitLab . This will create all the specified release targets (OBO, OWL, JSON, and the variants, ont-full and ont-base) and copy them into your release directory (the top level of your repo).","title":"Run a release with the ODK"},{"location":"odk-workflows/ReleaseWorkflow/#review-the-release","text":"(Optional) Rough check. This step is frequently skipped, but for the more paranoid among us (like the author of this doc), this is a 3 minute additional effort for some peace of mind. Open the main release (cl.owl) in you favourite development environment (i.e. Prot\u00e9g\u00e9) and eyeball the hierarchy. We recommend two simple checks: Does the very top level of the hierarchy look ok? This means that all new terms have been imported/updated correctly. Does at least one change that you know should be in this release appear? For example, a new class. This means that the release was actually based on the recent edit file. Commit your changes to the branch and make a pull request In your GitHub pull request, review the following three files in detail (based on our experience): cl.obo - this reflects a useful subset of the whole ontology (everything that can be covered by OBO format). OBO format has that speaking for it: it is very easy to review! cl-base.owl - this reflects the asserted axioms in your ontology that you have actually edited. Ideally also take a look at cl-full.owl , which may reveal interesting new inferences you did not know about. Note that the diff of this file is sometimes quite large. Like with every pull request, we recommend to always employ a second set of eyes when reviewing a PR!","title":"Review the release"},{"location":"odk-workflows/ReleaseWorkflow/#merge-the-main-branch","text":"Once your CI checks have passed, and your reviews are completed, you can now merge the branch into your main branch (don't forget to delete the branch afterwards - a big button will appear after the merge is finished).","title":"Merge the main branch"},{"location":"odk-workflows/ReleaseWorkflow/#create-a-github-release","text":"Go to your releases page on GitHub by navigating to your repository, and then clicking on releases (usually on the right, for example: https://github.com/obophenotype/cell-ontology/releases). Then click \"Draft new release\" As the tag version you need to choose the date on which your ontologies were build. You can find this, for example, by looking at the cl.obo file and check the data-version: property. The date needs to be prefixed with a v , so, for example v2020-02-06 . You can write whatever you want in the release title, but we typically write the date again. The description underneath should contain a concise list of changes or term additions. Click \"Publish release\". Done.","title":"Create a GitHub release"},{"location":"odk-workflows/ReleaseWorkflow/#debugging-typical-ontology-release-problems","text":"","title":"Debugging typical ontology release problems"},{"location":"odk-workflows/ReleaseWorkflow/#problems-with-memory","text":"When you are dealing with large ontologies, you need a lot of memory. When you see error messages relating to large ontologies such as CHEBI, PRO, NCBITAXON, or Uberon, you should think of memory first, see here .","title":"Problems with memory"},{"location":"odk-workflows/ReleaseWorkflow/#problems-when-using-obo-format-based-tools","text":"Sometimes you will get cryptic error messages when using legacy tools using OBO format, such as the ontology release tool (OORT), which is also available as part of the ODK docker container. In these cases, you need to track down what axiom or annotation actually caused the breakdown. In our experience (in about 60% of the cases) the problem lies with duplicate annotations ( def , comment ) which are illegal in OBO. Here is an example recipe of how to deal with such a problem: If you get a message like make: *** [cl.Makefile:84: oort] Error 255 you might have a OORT error. To debug this, in your terminal enter sh run.sh make IMP=false PAT=false oort -B (assuming you are already in the ontology folder in your directory) This should show you where the error is in the log (eg multiple different definitions) WARNING: THE FIX BELOW IS NOT IDEAL, YOU SHOULD ALWAYS TRY TO FIX UPSTREAM IF POSSIBLE Open cl-edit.owl in Prot\u00e9g\u00e9 and find the offending term and delete all offending issue (e.g. delete ALL definition, if the problem was \"multiple def tags not allowed\") and save. *While this is not idea, as it will remove all definitions from that term, it will be added back again when the term is fixed in the ontology it was imported from and added back in. Rerun sh run.sh make IMP=false PAT=false oort -B and if it all passes, commit your changes to a branch and make a pull request as usual.","title":"Problems when using OBO format based tools"},{"location":"odk-workflows/RepoManagement/","text":"Managing your ODK repository \u00b6 Updating your ODK repository \u00b6 Your ODK repositories configuration is managed in src/ontology/cl-odk.yaml . Once you have made your changes, you can run the following to apply your changes to the repository: sh run.sh make update_repo There are a large number of options that can be set to configure your ODK, but we will only discuss a few of them here. NOTE for Windows users: You may get a cryptic failure such as Set Illegal Option - if the update script located in src/scripts/update_repo.sh was saved using Windows Line endings. These need to change to unix line endings. In Notepad++, for example, you can click on Edit->EOL Conversion->Unix LF to change this. Managing imports \u00b6 You can use the update repository workflow described on this page to perform the following operations to your imports: Add a new import Modify an existing import Remove an import you no longer want Customise an import We will discuss all these workflows in the following. Add new import \u00b6 To add a new import, you first edit your odk config as described above , adding an id to the product list in the import_group section (for the sake of this example, we assume you already import RO, and your goal is to also import GO): import_group: products: - id: ro - id: go Note: our ODK file should only have one import_group which can contain multiple imports (in the products section). Next, you run the update repo workflow to apply these changes. Note that by default, this module is going to be a SLME Bottom module, see here . To change that or customise your module, see section \"Customise an import\". To finalise the addition of your import, perform the following steps: Add an import statement to your src/ontology/cl-edit.owl file. We suggest to do this using a text editor, by simply copying an existing import declaration and renaming it to the new ontology import, for example as follows: ... Ontology( Import() Import() ... Add your imports redirect to your catalog file src/ontology/catalog-v001.xml , for example: Test whether everything is in order: Refresh your import Open in your Ontology Editor of choice (Protege) and ensure that the expected terms are imported. Note: The catalog file src/ontology/catalog-v001.xml has one purpose: redirecting imports from URLs to local files. For example, if you have Import() in your editors file (the ontology) and in your catalog, tools like robot or Prot\u00e9g\u00e9 will recognize the statement in the catalog file to redirect the URL http://purl.obolibrary.org/obo/cl/imports/go_import.owl to the local file imports/go_import.owl (which is in your src/ontology directory). Modify an existing import \u00b6 If you simply wish to refresh your import in light of new terms, see here . If you wish to change the type of your module see section \"Customise an import\". Remove an existing import \u00b6 To remove an existing import, perform the following steps: remove the import declaration from your src/ontology/cl-edit.owl . remove the id from your src/ontology/cl-odk.yaml , eg. - id: go from the list of products in the import_group . run update repo workflow delete the associated files manually: src/imports/go_import.owl src/imports/go_terms.txt Remove the respective entry from the src/ontology/catalog-v001.xml file. Customise an import \u00b6 By default, an import module extracted from a source ontology will be a SLME module, see here . There are various options to change the default. The following change to your repo config ( src/ontology/cl-odk.yaml ) will switch the go import from an SLME module to a simple ROBOT filter module: import_group: products: - id: ro - id: go module_type: filter A ROBOT filter module is, essentially, importing all external terms declared by your ontology (see here on how to declare external terms to be imported). Note that the filter module does not consider terms/annotations from namespaces other than the base-namespace of the ontology itself. For example, in the example of GO above, only annotations / axioms related to the GO base IRI (http://purl.obolibrary.org/obo/GO_) would be considered. This behaviour can be changed by adding additional base IRIs as follows: import_group: products: - id: go module_type: filter base_iris: - http://purl.obolibrary.org/obo/GO_ - http://purl.obolibrary.org/obo/CL_ - http://purl.obolibrary.org/obo/BFO If you wish to customise your import entirely, you can specify your own ROBOT command to do so. To do that, add the following to your repo config ( src/ontology/cl-odk.yaml ): import_group: products: - id: ro - id: go module_type: custom Now add a new goal in your custom Makefile ( src/ontology/cl.Makefile , not src/ontology/Makefile ). imports/go_import.owl: mirror/ro.owl imports/ro_terms_combined.txt if [ $(IMP) = true ]; then $(ROBOT) query -i $< --update ../sparql/preprocess-module.ru \\ extract -T imports/ro_terms_combined.txt --force true --individuals exclude --method BOT \\ query --update ../sparql/inject-subset-declaration.ru --update ../sparql/postprocess-module.ru \\ annotate --ontology-iri $(ONTBASE)/$@ $(ANNOTATE_ONTOLOGY_VERSION) --output $@.tmp.owl && mv $@.tmp.owl $@; fi Now feel free to change this goal to do whatever you wish it to do! It probably makes some sense (albeit not being a strict necessity), to leave most of the goal instead and replace only: extract -T imports/ro_terms_combined.txt --force true --individuals exclude --method BOT \\ to another ROBOT pipeline. Add a component \u00b6 A component is an import which belongs to your ontology, e.g. is managed by you and your team. Open src/ontology/cl-odk.yaml If you dont have it yet, add a new top level section components Under the components section, add a new section called products . This is where all your components are specified Under the products section, add a new component, e.g. - filename: mycomp.owl Example components: products: - filename: mycomp.owl When running sh run.sh make update_repo , a new file src/ontology/components/mycomp.owl will be created which you can edit as you see fit. Typical ways to edit: Using a ROBOT template to generate the component (see below) Manually curating the component separately with Prot\u00e9g\u00e9 or any other editor Providing a components/mycomp.owl: make target in src/ontology/cl.Makefile and provide a custom command to generate the component WARNING : Note that the custom rule to generate the component MUST NOT depend on any other ODK-generated file such as seed files and the like (see issue ). Providing an additional attribute for the component in src/ontology/cl-odk.yaml , source , to specify that this component should simply be downloaded from somewhere on the web. Adding a new component based on a ROBOT template \u00b6 Since ODK 1.3.2, it is possible to simply link a ROBOT template to a component without having to specify any of the import logic. In order to add a new component that is connected to one or more template files, follow these steps: Open src/ontology/cl-odk.yaml . Make sure that use_templates: TRUE is set in the global project options. You should also make sure that use_context: TRUE is set in case you are using prefixes in your templates that are not known to robot , such as OMOP: , CPONT: and more. All non-standard prefixes you are using should be added to config/context.json . Add another component to the products section. To activate this component to be template-driven, simply say: use_template: TRUE . This will create an empty template for you in the templates directory, which will automatically be processed when recreating the component (e.g. run.bat make recreate-mycomp ). If you want to use more than one component, use the templates field to add as many template names as you wish. ODK will look for them in the src/templates directory. Advanced: If you want to provide additional processing options, you can use the template_options field. This should be a string with option from robot template . One typical example for additional options you may want to provide is --add-prefixes config/context.json to ensure the prefix map of your context is provided to robot , see above. Example : components: products: - filename: mycomp.owl use_template: TRUE template_options: --add-prefixes config/context.json templates: - template1.tsv - template2.tsv Note : if your mirror is particularly large and complex, read this ODK recommendation .","title":"Manage your ODK Repository"},{"location":"odk-workflows/RepoManagement/#managing-your-odk-repository","text":"","title":"Managing your ODK repository"},{"location":"odk-workflows/RepoManagement/#updating-your-odk-repository","text":"Your ODK repositories configuration is managed in src/ontology/cl-odk.yaml . Once you have made your changes, you can run the following to apply your changes to the repository: sh run.sh make update_repo There are a large number of options that can be set to configure your ODK, but we will only discuss a few of them here. NOTE for Windows users: You may get a cryptic failure such as Set Illegal Option - if the update script located in src/scripts/update_repo.sh was saved using Windows Line endings. These need to change to unix line endings. In Notepad++, for example, you can click on Edit->EOL Conversion->Unix LF to change this.","title":"Updating your ODK repository"},{"location":"odk-workflows/RepoManagement/#managing-imports","text":"You can use the update repository workflow described on this page to perform the following operations to your imports: Add a new import Modify an existing import Remove an import you no longer want Customise an import We will discuss all these workflows in the following.","title":"Managing imports"},{"location":"odk-workflows/RepoManagement/#add-new-import","text":"To add a new import, you first edit your odk config as described above , adding an id to the product list in the import_group section (for the sake of this example, we assume you already import RO, and your goal is to also import GO): import_group: products: - id: ro - id: go Note: our ODK file should only have one import_group which can contain multiple imports (in the products section). Next, you run the update repo workflow to apply these changes. Note that by default, this module is going to be a SLME Bottom module, see here . To change that or customise your module, see section \"Customise an import\". To finalise the addition of your import, perform the following steps: Add an import statement to your src/ontology/cl-edit.owl file. We suggest to do this using a text editor, by simply copying an existing import declaration and renaming it to the new ontology import, for example as follows: ... Ontology( Import() Import() ... Add your imports redirect to your catalog file src/ontology/catalog-v001.xml , for example: Test whether everything is in order: Refresh your import Open in your Ontology Editor of choice (Protege) and ensure that the expected terms are imported. Note: The catalog file src/ontology/catalog-v001.xml has one purpose: redirecting imports from URLs to local files. For example, if you have Import() in your editors file (the ontology) and in your catalog, tools like robot or Prot\u00e9g\u00e9 will recognize the statement in the catalog file to redirect the URL http://purl.obolibrary.org/obo/cl/imports/go_import.owl to the local file imports/go_import.owl (which is in your src/ontology directory).","title":"Add new import"},{"location":"odk-workflows/RepoManagement/#modify-an-existing-import","text":"If you simply wish to refresh your import in light of new terms, see here . If you wish to change the type of your module see section \"Customise an import\".","title":"Modify an existing import"},{"location":"odk-workflows/RepoManagement/#remove-an-existing-import","text":"To remove an existing import, perform the following steps: remove the import declaration from your src/ontology/cl-edit.owl . remove the id from your src/ontology/cl-odk.yaml , eg. - id: go from the list of products in the import_group . run update repo workflow delete the associated files manually: src/imports/go_import.owl src/imports/go_terms.txt Remove the respective entry from the src/ontology/catalog-v001.xml file.","title":"Remove an existing import"},{"location":"odk-workflows/RepoManagement/#customise-an-import","text":"By default, an import module extracted from a source ontology will be a SLME module, see here . There are various options to change the default. The following change to your repo config ( src/ontology/cl-odk.yaml ) will switch the go import from an SLME module to a simple ROBOT filter module: import_group: products: - id: ro - id: go module_type: filter A ROBOT filter module is, essentially, importing all external terms declared by your ontology (see here on how to declare external terms to be imported). Note that the filter module does not consider terms/annotations from namespaces other than the base-namespace of the ontology itself. For example, in the example of GO above, only annotations / axioms related to the GO base IRI (http://purl.obolibrary.org/obo/GO_) would be considered. This behaviour can be changed by adding additional base IRIs as follows: import_group: products: - id: go module_type: filter base_iris: - http://purl.obolibrary.org/obo/GO_ - http://purl.obolibrary.org/obo/CL_ - http://purl.obolibrary.org/obo/BFO If you wish to customise your import entirely, you can specify your own ROBOT command to do so. To do that, add the following to your repo config ( src/ontology/cl-odk.yaml ): import_group: products: - id: ro - id: go module_type: custom Now add a new goal in your custom Makefile ( src/ontology/cl.Makefile , not src/ontology/Makefile ). imports/go_import.owl: mirror/ro.owl imports/ro_terms_combined.txt if [ $(IMP) = true ]; then $(ROBOT) query -i $< --update ../sparql/preprocess-module.ru \\ extract -T imports/ro_terms_combined.txt --force true --individuals exclude --method BOT \\ query --update ../sparql/inject-subset-declaration.ru --update ../sparql/postprocess-module.ru \\ annotate --ontology-iri $(ONTBASE)/$@ $(ANNOTATE_ONTOLOGY_VERSION) --output $@.tmp.owl && mv $@.tmp.owl $@; fi Now feel free to change this goal to do whatever you wish it to do! It probably makes some sense (albeit not being a strict necessity), to leave most of the goal instead and replace only: extract -T imports/ro_terms_combined.txt --force true --individuals exclude --method BOT \\ to another ROBOT pipeline.","title":"Customise an import"},{"location":"odk-workflows/RepoManagement/#add-a-component","text":"A component is an import which belongs to your ontology, e.g. is managed by you and your team. Open src/ontology/cl-odk.yaml If you dont have it yet, add a new top level section components Under the components section, add a new section called products . This is where all your components are specified Under the products section, add a new component, e.g. - filename: mycomp.owl Example components: products: - filename: mycomp.owl When running sh run.sh make update_repo , a new file src/ontology/components/mycomp.owl will be created which you can edit as you see fit. Typical ways to edit: Using a ROBOT template to generate the component (see below) Manually curating the component separately with Prot\u00e9g\u00e9 or any other editor Providing a components/mycomp.owl: make target in src/ontology/cl.Makefile and provide a custom command to generate the component WARNING : Note that the custom rule to generate the component MUST NOT depend on any other ODK-generated file such as seed files and the like (see issue ). Providing an additional attribute for the component in src/ontology/cl-odk.yaml , source , to specify that this component should simply be downloaded from somewhere on the web.","title":"Add a component"},{"location":"odk-workflows/RepoManagement/#adding-a-new-component-based-on-a-robot-template","text":"Since ODK 1.3.2, it is possible to simply link a ROBOT template to a component without having to specify any of the import logic. In order to add a new component that is connected to one or more template files, follow these steps: Open src/ontology/cl-odk.yaml . Make sure that use_templates: TRUE is set in the global project options. You should also make sure that use_context: TRUE is set in case you are using prefixes in your templates that are not known to robot , such as OMOP: , CPONT: and more. All non-standard prefixes you are using should be added to config/context.json . Add another component to the products section. To activate this component to be template-driven, simply say: use_template: TRUE . This will create an empty template for you in the templates directory, which will automatically be processed when recreating the component (e.g. run.bat make recreate-mycomp ). If you want to use more than one component, use the templates field to add as many template names as you wish. ODK will look for them in the src/templates directory. Advanced: If you want to provide additional processing options, you can use the template_options field. This should be a string with option from robot template . One typical example for additional options you may want to provide is --add-prefixes config/context.json to ensure the prefix map of your context is provided to robot , see above. Example : components: products: - filename: mycomp.owl use_template: TRUE template_options: --add-prefixes config/context.json templates: - template1.tsv - template2.tsv Note : if your mirror is particularly large and complex, read this ODK recommendation .","title":"Adding a new component based on a ROBOT template"},{"location":"odk-workflows/RepositoryFileStructure/","text":"Repository structure \u00b6 The main kinds of files in the repository: Release files Imports Components Release files \u00b6 Release file are the file that are considered part of the official ontology release and to be used by the community. A detailed description of the release artefacts can be found here . Imports \u00b6 Imports are subsets of external ontologies that contain terms and axioms you would like to re-use in your ontology. These are considered \"external\", like dependencies in software development, and are not included in your \"base\" product, which is the release artefact which contains only those axioms that you personally maintain. These are the current imports in CL Import URL Type pr https://raw.githubusercontent.com/obophenotype/pro_obo_slim/master/pr_slim.owl None go http://purl.obolibrary.org/obo/go.owl None uberon http://purl.obolibrary.org/obo/uberon.owl None ro http://purl.obolibrary.org/obo/ro.owl None pato http://purl.obolibrary.org/obo/pato.owl None ncbitaxon http://purl.obolibrary.org/obo/ncbitaxon/subsets/taxslim.owl None omo http://purl.obolibrary.org/obo/omo.owl mirror Components \u00b6 Components, in contrast to imports, are considered full members of the ontology. This means that any axiom in a component is also included in the ontology base - which means it is considered native to the ontology. While this sounds complicated, consider this: conceptually, no component should be part of more than one ontology. If that seems to be the case, we are most likely talking about an import. Components are often not needed for ontologies, but there are some use cases: There is an automated process that generates and re-generates a part of the ontology A part of the ontology is managed in ROBOT templates The expressivity of the component is higher than the format of the edit file. For example, people still choose to manage their ontology in OBO format (they should not) missing out on a lot of owl features. They may choose to manage logic that is beyond OBO in a specific OWL component. These are the components in CL Filename URL hra_subset.owl None mappings.owl None blood_and_immune_upper_slim.owl None eye_upper_slim.owl None general_cell_types_upper_slim.owl None","title":"Your ODK Repository Overview"},{"location":"odk-workflows/RepositoryFileStructure/#repository-structure","text":"The main kinds of files in the repository: Release files Imports Components","title":"Repository structure"},{"location":"odk-workflows/RepositoryFileStructure/#release-files","text":"Release file are the file that are considered part of the official ontology release and to be used by the community. A detailed description of the release artefacts can be found here .","title":"Release files"},{"location":"odk-workflows/RepositoryFileStructure/#imports","text":"Imports are subsets of external ontologies that contain terms and axioms you would like to re-use in your ontology. These are considered \"external\", like dependencies in software development, and are not included in your \"base\" product, which is the release artefact which contains only those axioms that you personally maintain. These are the current imports in CL Import URL Type pr https://raw.githubusercontent.com/obophenotype/pro_obo_slim/master/pr_slim.owl None go http://purl.obolibrary.org/obo/go.owl None uberon http://purl.obolibrary.org/obo/uberon.owl None ro http://purl.obolibrary.org/obo/ro.owl None pato http://purl.obolibrary.org/obo/pato.owl None ncbitaxon http://purl.obolibrary.org/obo/ncbitaxon/subsets/taxslim.owl None omo http://purl.obolibrary.org/obo/omo.owl mirror","title":"Imports"},{"location":"odk-workflows/RepositoryFileStructure/#components","text":"Components, in contrast to imports, are considered full members of the ontology. This means that any axiom in a component is also included in the ontology base - which means it is considered native to the ontology. While this sounds complicated, consider this: conceptually, no component should be part of more than one ontology. If that seems to be the case, we are most likely talking about an import. Components are often not needed for ontologies, but there are some use cases: There is an automated process that generates and re-generates a part of the ontology A part of the ontology is managed in ROBOT templates The expressivity of the component is higher than the format of the edit file. For example, people still choose to manage their ontology in OBO format (they should not) missing out on a lot of owl features. They may choose to manage logic that is beyond OBO in a specific OWL component. These are the components in CL Filename URL hra_subset.owl None mappings.owl None blood_and_immune_upper_slim.owl None eye_upper_slim.owl None general_cell_types_upper_slim.owl None","title":"Components"},{"location":"odk-workflows/SettingUpDockerForODK/","text":"Setting up your Docker environment for ODK use \u00b6 One of the most frequent problems with running the ODK for the first time is failure because of lack of memory. This can look like a Java OutOfMemory exception, but more often than not it will appear as something like an Error 137 . There are two places you need to consider to set your memory: Your src/ontology/run.sh (or run.bat) file. You can set the memory in there by adding robot_java_args: '-Xmx8G' to your src/ontology/cl-odk.yaml file, see for example here . Set your docker memory. By default, it should be about 10-20% more than your robot_java_args variable. You can manage your memory settings by right-clicking on the docker whale in your system bar-->Preferences-->Resources-->Advanced, see picture below.","title":"Setting up Docker for ODK"},{"location":"odk-workflows/SettingUpDockerForODK/#setting-up-your-docker-environment-for-odk-use","text":"One of the most frequent problems with running the ODK for the first time is failure because of lack of memory. This can look like a Java OutOfMemory exception, but more often than not it will appear as something like an Error 137 . There are two places you need to consider to set your memory: Your src/ontology/run.sh (or run.bat) file. You can set the memory in there by adding robot_java_args: '-Xmx8G' to your src/ontology/cl-odk.yaml file, see for example here . Set your docker memory. By default, it should be about 10-20% more than your robot_java_args variable. You can manage your memory settings by right-clicking on the docker whale in your system bar-->Preferences-->Resources-->Advanced, see picture below.","title":"Setting up your Docker environment for ODK use"},{"location":"odk-workflows/UpdateImports/","text":"Update Imports Workflow \u00b6 This page discusses how to update the contents of your imports, like adding or removing terms. If you are looking to customise imports, like changing the module type, see here . Importing a new term \u00b6 Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title \"Using the Base Module approach\". Importing a new term is split into two sub-phases: Declaring the terms to be imported Refreshing imports dynamically Declaring terms to be imported \u00b6 There are three ways to declare terms that are to be imported from an external ontology. Choose the appropriate one for your particular scenario (all three can be used in parallel if need be): Prot\u00e9g\u00e9-based declaration Using term files Using the custom import template Prot\u00e9g\u00e9-based declaration \u00b6 This workflow is to be avoided, but may be appropriate if the editor does not have access to the ODK docker container . This approach also applies to ontologies that use base module import approach. Open your ontology (edit file) in Prot\u00e9g\u00e9 (5.5+). Select 'owl:Thing' Add a new class as usual. Paste the full iri in the 'Name:' field, for example, http://purl.obolibrary.org/obo/CHEBI_50906. Click 'OK' Now you can use this term for example to construct logical definitions. The next time the imports are refreshed (see how to refresh here ), the metadata (labels, definitions, etc.) for this term are imported from the respective external source ontology and becomes visible in your ontology. Using term files \u00b6 Every import has, by default a term file associated with it, which can be found in the imports directory. For example, if you have a GO import in src/ontology/go_import.owl , you will also have an associated term file src/ontology/go_terms.txt . You can add terms in there simply as a list: GO:0008150 GO:0008151 Now you can run the refresh imports workflow ) and the two terms will be imported. Using the custom import template \u00b6 This workflow is appropriate if: You prefer to manage all your imported terms in a single file (rather than multiple files like in the \"Using term files\" workflow above). You wish to augment your imported ontologies with additional information. This requires a cautionary discussion. To enable this workflow, you add the following to your ODK config file ( src/ontology/cl-odk.yaml ), and update the repository : use_custom_import_module: TRUE Now you can manage your imported terms directly in the custom external terms template, which is located at src/templates/external_import.owl . Note that this file is a ROBOT template , and can, in principle, be extended to include any axioms you like. Before extending the template, however, read the following carefully. The main purpose of the custom import template is to enable the management off all terms to be imported in a centralised place. To enable that, you do not have to do anything other than maintaining the template. So if you, say currently import APOLLO_SV:00000480 , and you wish to import APOLLO_SV:00000532 , you simply add a row like this: ID Entity Type ID TYPE APOLLO_SV:00000480 owl:Class APOLLO_SV:00000532 owl:Class When the imports are refreshed see imports refresh workflow , the term(s) will simply be imported from the configured ontologies. Now, if you wish to extend the Makefile (which is beyond these instructions) and add, say, synonyms to the imported terms, you can do that, but you need to (a) preserve the ID and ENTITY columns and (b) ensure that the ROBOT template is valid otherwise, see here . WARNING . Note that doing this is a widespread antipattern (see related issue ). You should not change the axioms of terms that do not belong into your ontology unless necessary - such changes should always be pushed into the ontology where they belong. However, since people are doing it, whether the OBO Foundry likes it or not, at least using the custom imports module as described here localises the changes to a single simple template and ensures that none of the annotations added this way are merged into the base file . Refresh imports \u00b6 If you want to refresh the import yourself (this may be necessary to pass the travis tests), and you have the ODK installed, you can do the following (using go as an example): First, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). cd src/ontology Then, you regenerate the import that will now include any new terms you have added. Note: You must have docker installed . sh run.sh make PAT=false imports/go_import.owl -B Since ODK 1.2.27, it is also possible to simply run the following, which is the same as the above: sh run.sh make refresh-go Note that in case you changed the defaults, you need to add IMP=true and/or MIR=true to the command below: sh run.sh make IMP=true MIR=true PAT=false imports/go_import.owl -B If you wish to skip refreshing the mirror, i.e. skip downloading the latest version of the source ontology for your import (e.g. go.owl for your go import) you can set MIR=false instead, which will do the exact same thing as the above, but is easier to remember: sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B Using the Base Module approach \u00b6 Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files. The idea is to only import axioms from ontologies that actually belong to it . A base file is a subset of the ontology that only contains those axioms that nominally belong there. In other words, the base file does not contain any axioms that belong to another ontology. An example would be this: Imagine this being the full Uberon ontology: Axiom 1: BFO:123 SubClassOf BFO:124 Axiom 1: UBERON:123 SubClassOf BFO:123 Axiom 1: UBERON:124 SubClassOf UBERON 123 The base file is the set of all axioms that are about UBERON terms: Axiom 1: UBERON:123 SubClassOf BFO:123 Axiom 1: UBERON:124 SubClassOf UBERON 123 I.e. Axiom 1: BFO:123 SubClassOf BFO:124 Gets removed. The base file pipeline is a bit more complex than the normal pipelines, because of the logical interactions between the imported ontologies. This is solved by _first merging all mirrors into one huge file and then extracting one mega module from it. Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies. When we use the base pipelines, we 1) First obtain the base (usually by simply downloading it, but there is also an option now to create it with ROBOT) 2) We merge all base files into one big pile 3) Then we extract a single module imports/merged_import.owl The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml. To check if your ontology uses this method, check src/ontology/cl-odk.yaml to see if use_base_merging: TRUE is declared under import_group If your ontology uses Base Module approach, please use the following steps: First, add the term to be imported to the term file associated with it (see above \"Using term files\" section if this is not clear to you) Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). cd src/ontology Then refresh imports by running sh run.sh make imports/merged_import.owl Note: if your mirrors are updated, you can run sh run.sh make no-mirror-refresh-merged This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and one of the local devs should pick this up and run the import for you. Lastly, restart Prot\u00e9g\u00e9, and the term should be imported in ready to be used.","title":"Imports management"},{"location":"odk-workflows/UpdateImports/#update-imports-workflow","text":"This page discusses how to update the contents of your imports, like adding or removing terms. If you are looking to customise imports, like changing the module type, see here .","title":"Update Imports Workflow"},{"location":"odk-workflows/UpdateImports/#importing-a-new-term","text":"Note: some ontologies now use a merged-import system to manage dynamic imports, for these please follow instructions in the section title \"Using the Base Module approach\". Importing a new term is split into two sub-phases: Declaring the terms to be imported Refreshing imports dynamically","title":"Importing a new term"},{"location":"odk-workflows/UpdateImports/#declaring-terms-to-be-imported","text":"There are three ways to declare terms that are to be imported from an external ontology. Choose the appropriate one for your particular scenario (all three can be used in parallel if need be): Prot\u00e9g\u00e9-based declaration Using term files Using the custom import template","title":"Declaring terms to be imported"},{"location":"odk-workflows/UpdateImports/#protege-based-declaration","text":"This workflow is to be avoided, but may be appropriate if the editor does not have access to the ODK docker container . This approach also applies to ontologies that use base module import approach. Open your ontology (edit file) in Prot\u00e9g\u00e9 (5.5+). Select 'owl:Thing' Add a new class as usual. Paste the full iri in the 'Name:' field, for example, http://purl.obolibrary.org/obo/CHEBI_50906. Click 'OK' Now you can use this term for example to construct logical definitions. The next time the imports are refreshed (see how to refresh here ), the metadata (labels, definitions, etc.) for this term are imported from the respective external source ontology and becomes visible in your ontology.","title":"Prot\u00e9g\u00e9-based declaration"},{"location":"odk-workflows/UpdateImports/#using-term-files","text":"Every import has, by default a term file associated with it, which can be found in the imports directory. For example, if you have a GO import in src/ontology/go_import.owl , you will also have an associated term file src/ontology/go_terms.txt . You can add terms in there simply as a list: GO:0008150 GO:0008151 Now you can run the refresh imports workflow ) and the two terms will be imported.","title":"Using term files"},{"location":"odk-workflows/UpdateImports/#using-the-custom-import-template","text":"This workflow is appropriate if: You prefer to manage all your imported terms in a single file (rather than multiple files like in the \"Using term files\" workflow above). You wish to augment your imported ontologies with additional information. This requires a cautionary discussion. To enable this workflow, you add the following to your ODK config file ( src/ontology/cl-odk.yaml ), and update the repository : use_custom_import_module: TRUE Now you can manage your imported terms directly in the custom external terms template, which is located at src/templates/external_import.owl . Note that this file is a ROBOT template , and can, in principle, be extended to include any axioms you like. Before extending the template, however, read the following carefully. The main purpose of the custom import template is to enable the management off all terms to be imported in a centralised place. To enable that, you do not have to do anything other than maintaining the template. So if you, say currently import APOLLO_SV:00000480 , and you wish to import APOLLO_SV:00000532 , you simply add a row like this: ID Entity Type ID TYPE APOLLO_SV:00000480 owl:Class APOLLO_SV:00000532 owl:Class When the imports are refreshed see imports refresh workflow , the term(s) will simply be imported from the configured ontologies. Now, if you wish to extend the Makefile (which is beyond these instructions) and add, say, synonyms to the imported terms, you can do that, but you need to (a) preserve the ID and ENTITY columns and (b) ensure that the ROBOT template is valid otherwise, see here . WARNING . Note that doing this is a widespread antipattern (see related issue ). You should not change the axioms of terms that do not belong into your ontology unless necessary - such changes should always be pushed into the ontology where they belong. However, since people are doing it, whether the OBO Foundry likes it or not, at least using the custom imports module as described here localises the changes to a single simple template and ensures that none of the annotations added this way are merged into the base file .","title":"Using the custom import template"},{"location":"odk-workflows/UpdateImports/#refresh-imports","text":"If you want to refresh the import yourself (this may be necessary to pass the travis tests), and you have the ODK installed, you can do the following (using go as an example): First, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). cd src/ontology Then, you regenerate the import that will now include any new terms you have added. Note: You must have docker installed . sh run.sh make PAT=false imports/go_import.owl -B Since ODK 1.2.27, it is also possible to simply run the following, which is the same as the above: sh run.sh make refresh-go Note that in case you changed the defaults, you need to add IMP=true and/or MIR=true to the command below: sh run.sh make IMP=true MIR=true PAT=false imports/go_import.owl -B If you wish to skip refreshing the mirror, i.e. skip downloading the latest version of the source ontology for your import (e.g. go.owl for your go import) you can set MIR=false instead, which will do the exact same thing as the above, but is easier to remember: sh run.sh make IMP=true MIR=false PAT=false imports/go_import.owl -B","title":"Refresh imports"},{"location":"odk-workflows/UpdateImports/#using-the-base-module-approach","text":"Since ODK 1.2.31, we support an entirely new approach to generate modules: Using base files. The idea is to only import axioms from ontologies that actually belong to it . A base file is a subset of the ontology that only contains those axioms that nominally belong there. In other words, the base file does not contain any axioms that belong to another ontology. An example would be this: Imagine this being the full Uberon ontology: Axiom 1: BFO:123 SubClassOf BFO:124 Axiom 1: UBERON:123 SubClassOf BFO:123 Axiom 1: UBERON:124 SubClassOf UBERON 123 The base file is the set of all axioms that are about UBERON terms: Axiom 1: UBERON:123 SubClassOf BFO:123 Axiom 1: UBERON:124 SubClassOf UBERON 123 I.e. Axiom 1: BFO:123 SubClassOf BFO:124 Gets removed. The base file pipeline is a bit more complex than the normal pipelines, because of the logical interactions between the imported ontologies. This is solved by _first merging all mirrors into one huge file and then extracting one mega module from it. Example: Let's say we are importing terms from Uberon, GO and RO in our ontologies. When we use the base pipelines, we 1) First obtain the base (usually by simply downloading it, but there is also an option now to create it with ROBOT) 2) We merge all base files into one big pile 3) Then we extract a single module imports/merged_import.owl The first implementation of this pipeline is PATO, see https://github.com/pato-ontology/pato/blob/master/src/ontology/pato-odk.yaml. To check if your ontology uses this method, check src/ontology/cl-odk.yaml to see if use_base_merging: TRUE is declared under import_group If your ontology uses Base Module approach, please use the following steps: First, add the term to be imported to the term file associated with it (see above \"Using term files\" section if this is not clear to you) Next, you navigate in your terminal to the ontology directory (underneath src in your hpo root directory). cd src/ontology Then refresh imports by running sh run.sh make imports/merged_import.owl Note: if your mirrors are updated, you can run sh run.sh make no-mirror-refresh-merged This requires quite a bit of memory on your local machine, so if you encounter an error, it might be a lack of memory on your computer. A solution would be to create a ticket in an issue tracker requesting for the term to be imported, and one of the local devs should pick this up and run the import for you. Lastly, restart Prot\u00e9g\u00e9, and the term should be imported in ready to be used.","title":"Using the Base Module approach"},{"location":"odk-workflows/components/","text":"Adding components to an ODK repo \u00b6 For details on what components are, please see component section of repository file structure document . To add custom components to an ODK repo, please follow the following steps: 1) Locate your odk yaml file and open it with your favourite text editor (src/ontology/cl-odk.yaml) 2) Search if there is already a component section to the yaml file, if not add it accordingly, adding the name of your component: components: products: - filename: your-component-name.owl 3) Add the component to your catalog file (src/ontology/catalog-v001.xml) 4) Add the component to the edit file (src/ontology/cl-edit.obo) for .obo formats: import: http://purl.obolibrary.org/obo/cl/components/your-component-name.owl for .owl formats: Import() 5) Refresh your repo by running sh run.sh make update_repo - this should create a new file in src/ontology/components. 6) In your custom makefile (src/ontology/cl.Makefile) add a goal for your custom make file. In this example, the goal is a ROBOT template. $(COMPONENTSDIR)/your-component-name.owl: $(SRC) ../templates/your-component-template.tsv $(ROBOT) template --template ../templates/your-component-template.tsv \\ annotate --ontology-iri $(ONTBASE)/$@ --output $(COMPONENTSDIR)/your-component-name.owl (If using a ROBOT template, do not forget to add your template tsv in src/templates/) 7) Make the file by running sh run.sh make components/your-component-name.owl","title":"Adding components to an ODK repo"},{"location":"odk-workflows/components/#adding-components-to-an-odk-repo","text":"For details on what components are, please see component section of repository file structure document . To add custom components to an ODK repo, please follow the following steps: 1) Locate your odk yaml file and open it with your favourite text editor (src/ontology/cl-odk.yaml) 2) Search if there is already a component section to the yaml file, if not add it accordingly, adding the name of your component: components: products: - filename: your-component-name.owl 3) Add the component to your catalog file (src/ontology/catalog-v001.xml) 4) Add the component to the edit file (src/ontology/cl-edit.obo) for .obo formats: import: http://purl.obolibrary.org/obo/cl/components/your-component-name.owl for .owl formats: Import() 5) Refresh your repo by running sh run.sh make update_repo - this should create a new file in src/ontology/components. 6) In your custom makefile (src/ontology/cl.Makefile) add a goal for your custom make file. In this example, the goal is a ROBOT template. $(COMPONENTSDIR)/your-component-name.owl: $(SRC) ../templates/your-component-template.tsv $(ROBOT) template --template ../templates/your-component-template.tsv \\ annotate --ontology-iri $(ONTBASE)/$@ --output $(COMPONENTSDIR)/your-component-name.owl (If using a ROBOT template, do not forget to add your template tsv in src/templates/) 7) Make the file by running sh run.sh make components/your-component-name.owl","title":"Adding components to an ODK repo"},{"location":"patterns/cellBearerOfQuality/","text":"cellBearerOfQuality \u00b6 http://purl.obolibrary.org/obo/cl/cellBearerOfQuality Description \u00b6 A cell that has a specific quality, such as binucleate. Contributors \u00b6 https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165 Name \u00b6 { quality } { cell } Annotations \u00b6 exact_synonym : { quality }d { cell } Definition \u00b6 Any { cell } that is { quality } Equivalent to \u00b6 { cell } and ( bearer of some { quality }) Data preview \u00b6 defined_class defined_class_label cell cell_label quality quality_label CL:0001061 abnormal cell CL:0000000 cell PATO:0000460 abnormal CL:0000225 anucleate cell CL:0000003 native cell PATO:0001405 anucleate CL:0000227 binucleate cell CL:0000003 native cell PATO:0001406 binucleate CL:0000103 bipolar neuron CL:0000099 interneuron PATO:0070006 bipolar morphology CL:4023077 bitufted neuron CL:0000099 interneuron PATO:0070012 bitufted cell morphology See full table here","title":"Cell, bearer of quality"},{"location":"patterns/cellBearerOfQuality/#cellbearerofquality","text":"http://purl.obolibrary.org/obo/cl/cellBearerOfQuality","title":"cellBearerOfQuality"},{"location":"patterns/cellBearerOfQuality/#description","text":"A cell that has a specific quality, such as binucleate.","title":"Description"},{"location":"patterns/cellBearerOfQuality/#contributors","text":"https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165","title":"Contributors"},{"location":"patterns/cellBearerOfQuality/#name","text":"{ quality } { cell }","title":"Name"},{"location":"patterns/cellBearerOfQuality/#annotations","text":"exact_synonym : { quality }d { cell }","title":"Annotations"},{"location":"patterns/cellBearerOfQuality/#definition","text":"Any { cell } that is { quality }","title":"Definition"},{"location":"patterns/cellBearerOfQuality/#equivalent-to","text":"{ cell } and ( bearer of some { quality })","title":"Equivalent to"},{"location":"patterns/cellBearerOfQuality/#data-preview","text":"defined_class defined_class_label cell cell_label quality quality_label CL:0001061 abnormal cell CL:0000000 cell PATO:0000460 abnormal CL:0000225 anucleate cell CL:0000003 native cell PATO:0001405 anucleate CL:0000227 binucleate cell CL:0000003 native cell PATO:0001406 binucleate CL:0000103 bipolar neuron CL:0000099 interneuron PATO:0070006 bipolar morphology CL:4023077 bitufted neuron CL:0000099 interneuron PATO:0070012 bitufted cell morphology See full table here","title":"Data preview"},{"location":"patterns/cellCapableOfBiologicalProcess/","text":"cellCapableOfBiologicalProcess \u00b6 http://purl.obolibrary.org/obo/cl/cellCapableOfBiologicalProcess Description \u00b6 Any cell that is involved in/capable of a particular biological process, such as acid secretion. Name \u00b6 { biological_process } { cell } Definition \u00b6 A { cell } that is capable of { biological_process }. Equivalent to \u00b6 { cell } and ( capable of some { biological_process }) Data preview \u00b6 defined_class defined_class_label biological_process biological_process_label cell cell_label CL:0000236 B cell GO:0019724 B cell mediated immunity CL:0000945 lymphocyte of B lineage CL:0000492 CD4-positive helper T cell GO:0001816 cytokine production CL:0000624 CD4-positive, alpha-beta T cell CL:0000795 CD8-positive, alpha-beta regulatory T cell GO:0050777 negative regulation of immune response CL:0000625 CD8-positive, alpha-beta T cell CL:0011005 GABAergic interneuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000099 interneuron CL:0000617 GABAergic neuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000540 neuron See full table here","title":"Cell, capable of Biological Process"},{"location":"patterns/cellCapableOfBiologicalProcess/#cellcapableofbiologicalprocess","text":"http://purl.obolibrary.org/obo/cl/cellCapableOfBiologicalProcess","title":"cellCapableOfBiologicalProcess"},{"location":"patterns/cellCapableOfBiologicalProcess/#description","text":"Any cell that is involved in/capable of a particular biological process, such as acid secretion.","title":"Description"},{"location":"patterns/cellCapableOfBiologicalProcess/#name","text":"{ biological_process } { cell }","title":"Name"},{"location":"patterns/cellCapableOfBiologicalProcess/#definition","text":"A { cell } that is capable of { biological_process }.","title":"Definition"},{"location":"patterns/cellCapableOfBiologicalProcess/#equivalent-to","text":"{ cell } and ( capable of some { biological_process })","title":"Equivalent to"},{"location":"patterns/cellCapableOfBiologicalProcess/#data-preview","text":"defined_class defined_class_label biological_process biological_process_label cell cell_label CL:0000236 B cell GO:0019724 B cell mediated immunity CL:0000945 lymphocyte of B lineage CL:0000492 CD4-positive helper T cell GO:0001816 cytokine production CL:0000624 CD4-positive, alpha-beta T cell CL:0000795 CD8-positive, alpha-beta regulatory T cell GO:0050777 negative regulation of immune response CL:0000625 CD8-positive, alpha-beta T cell CL:0011005 GABAergic interneuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000099 interneuron CL:0000617 GABAergic neuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000540 neuron See full table here","title":"Data preview"},{"location":"patterns/cellHasPlasmaMembranePartX/","text":"cellHasPlasmaMembranePartX \u00b6 http://purl.obolibrary.org/obo/cl/cellHasPlasmaMembranePartX Description \u00b6 A cell type that is characterized by a plasma membrane part, such as a cilium or receptor. Note - that this is only good for cells defined by a single plasma membrane receptor. Contributors \u00b6 https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165 Name \u00b6 { cell } { plasma_membrane } Annotations \u00b6 exact_synonym : { cell }-positive { plasma_membrane } Definition \u00b6 A { cell } that has a { plasma_membrane }. Equivalent to \u00b6 { cell } and ( has plasma membrane part some { plasma_membrane })","title":"Cell, has plasme membrane"},{"location":"patterns/cellHasPlasmaMembranePartX/#cellhasplasmamembranepartx","text":"http://purl.obolibrary.org/obo/cl/cellHasPlasmaMembranePartX","title":"cellHasPlasmaMembranePartX"},{"location":"patterns/cellHasPlasmaMembranePartX/#description","text":"A cell type that is characterized by a plasma membrane part, such as a cilium or receptor. Note - that this is only good for cells defined by a single plasma membrane receptor.","title":"Description"},{"location":"patterns/cellHasPlasmaMembranePartX/#contributors","text":"https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165","title":"Contributors"},{"location":"patterns/cellHasPlasmaMembranePartX/#name","text":"{ cell } { plasma_membrane }","title":"Name"},{"location":"patterns/cellHasPlasmaMembranePartX/#annotations","text":"exact_synonym : { cell }-positive { plasma_membrane }","title":"Annotations"},{"location":"patterns/cellHasPlasmaMembranePartX/#definition","text":"A { cell } that has a { plasma_membrane }.","title":"Definition"},{"location":"patterns/cellHasPlasmaMembranePartX/#equivalent-to","text":"{ cell } and ( has plasma membrane part some { plasma_membrane })","title":"Equivalent to"},{"location":"patterns/cellPartOfAnatomicalEntity/","text":"cellPartOfAnatomicalEntity \u00b6 http://purl.obolibrary.org/obo/cl/cellPartOfAnatomicalEntity Description \u00b6 A cell that is part of an anatomical entity. Contributors \u00b6 https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165 Name \u00b6 { cell } { anatomical_entity } Definition \u00b6 Any { cell } that is part of a { anatomical_entity }. Equivalent to \u00b6 { cell } and ( part of some { anatomical_entity }) Data preview \u00b6 defined_class defined_class_label anatomical_entity anatomical_entity_label cell cell_label CL:0009032 B cell of appendix UBERON:0001154 vermiform appendix CL:0000236 B cell CL:0009045 B cell of medullary sinus of lymph node UBERON:0009744 lymph node medullary sinus CL:0000236 B cell CL:0010007 His-Purkinje system cell UBERON:0004146 His-Purkinje system CL:0000003 native cell CL:0002680 PP cell of intestine UBERON:0000160 intestine CL:0000696 PP cell CL:0009015 Peyer's patch follicular dendritic cell UBERON:0001211 Peyer's patch CL:0000442 follicular dendritic cell See full table here","title":"Cell, part of anatomical entity"},{"location":"patterns/cellPartOfAnatomicalEntity/#cellpartofanatomicalentity","text":"http://purl.obolibrary.org/obo/cl/cellPartOfAnatomicalEntity","title":"cellPartOfAnatomicalEntity"},{"location":"patterns/cellPartOfAnatomicalEntity/#description","text":"A cell that is part of an anatomical entity.","title":"Description"},{"location":"patterns/cellPartOfAnatomicalEntity/#contributors","text":"https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165","title":"Contributors"},{"location":"patterns/cellPartOfAnatomicalEntity/#name","text":"{ cell } { anatomical_entity }","title":"Name"},{"location":"patterns/cellPartOfAnatomicalEntity/#definition","text":"Any { cell } that is part of a { anatomical_entity }.","title":"Definition"},{"location":"patterns/cellPartOfAnatomicalEntity/#equivalent-to","text":"{ cell } and ( part of some { anatomical_entity })","title":"Equivalent to"},{"location":"patterns/cellPartOfAnatomicalEntity/#data-preview","text":"defined_class defined_class_label anatomical_entity anatomical_entity_label cell cell_label CL:0009032 B cell of appendix UBERON:0001154 vermiform appendix CL:0000236 B cell CL:0009045 B cell of medullary sinus of lymph node UBERON:0009744 lymph node medullary sinus CL:0000236 B cell CL:0010007 His-Purkinje system cell UBERON:0004146 His-Purkinje system CL:0000003 native cell CL:0002680 PP cell of intestine UBERON:0000160 intestine CL:0000696 PP cell CL:0009015 Peyer's patch follicular dendritic cell UBERON:0001211 Peyer's patch CL:0000442 follicular dendritic cell See full table here","title":"Data preview"},{"location":"patterns/overview/","text":"Pattern directory \u00b6 This is a listing of all the patterns hosted as part of this directory Patterns in \u00b6 Cell bearer of quality \u00b6 A cell that has a specific quality, such as binucleate. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellBearerOfQuality Name cellBearerOfQuality Classes CL:0000000, PATO:0000001, Variables cell (CL:0000000), quality (PATO:0000001), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples mondo Data preview: \u00b6 defined:class defined:class:label cell cell:label quality quality:label CL:0001061 abnormal cell CL:0000000 cell PATO:0000460 abnormal CL:0000225 anucleate cell CL:0000003 native cell PATO:0001405 anucleate CL:0000227 binucleate cell CL:0000003 native cell PATO:0001406 binucleate CL:0000103 bipolar neuron CL:0000099 interneuron PATO:0070006 bipolar morphology CL:4023077 bitufted neuron CL:0000099 interneuron PATO:0070012 bitufted cell morphology See full table here Cell capable of biological process \u00b6 Any cell that is involved in/capable of a particular biological process, such as acid secretion. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellCapableOfBiologicalProcess Name cellCapableOfBiologicalProcess Classes CL:0000000, GO:0008150, Variables cell (CL:0000000), biological_process (GO:0008150), Contributors Examples mondo Data preview: \u00b6 defined:class defined:class:label biological:process biological:process:label cell cell:label CL:0000236 B cell GO:0019724 B cell mediated immunity CL:0000945 lymphocyte of B lineage CL:0000492 CD4-positive helper T cell GO:0001816 cytokine production CL:0000624 CD4-positive, alpha-beta T cell CL:0000795 CD8-positive, alpha-beta regulatory T cell GO:0050777 negative regulation of immune response CL:0000625 CD8-positive, alpha-beta T cell CL:0011005 GABAergic interneuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000099 interneuron CL:0000617 GABAergic neuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000540 neuron See full table here Cell has plasma membrane part x \u00b6 A cell type that is characterized by a plasma membrane part, such as a cilium or receptor. Note - that this is only good for cells defined by a single plasma membrane receptor. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellHasPlasmaMembranePartX Name cellHasPlasmaMembranePartX Classes CL:0000000, CL:0000003, GO:0005886, Variables cell (CL:0000003), plasma_membrane (GO:0005886), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples Cell part of anatomical entity \u00b6 A cell that is part of an anatomical entity. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellPartOfAnatomicalEntity Name cellPartOfAnatomicalEntity Classes CL:0000000, UBERON:0001062, Variables cell (CL:0000000), anatomical_entity (UBERON:0001062), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples mondo Data preview: \u00b6 defined:class defined:class:label anatomical:entity anatomical:entity:label cell cell:label CL:0009032 B cell of appendix UBERON:0001154 vermiform appendix CL:0000236 B cell CL:0009045 B cell of medullary sinus of lymph node UBERON:0009744 lymph node medullary sinus CL:0000236 B cell CL:0010007 His-Purkinje system cell UBERON:0004146 His-Purkinje system CL:0000003 native cell CL:0002680 PP cell of intestine UBERON:0000160 intestine CL:0000696 PP cell CL:0009015 Peyer's patch follicular dendritic cell UBERON:0001211 Peyer's patch CL:0000442 follicular dendritic cell See full table here Taxon specific \u00b6 A cell that is restricted to a specific taxon, such as CL:0001200 'lymphocyte of B lineage, CD19-positive' are only in mammals. Note - this is not to be used for any cell that is restricted to a taxon, this is for taxon-specific subclasses of existing cell types. This should hardly ever be used. Attribute Info IRI http://purl.obolibrary.org/obo/cl/taxonSpecific Name taxonSpecific Classes CL:0000000, NCBITaxon:1, Variables cell (CL:0000000), taxon (NCBITaxon:1), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples","title":"Design Patterns Overview"},{"location":"patterns/overview/#pattern-directory","text":"This is a listing of all the patterns hosted as part of this directory","title":"Pattern directory"},{"location":"patterns/overview/#patterns-in","text":"","title":"Patterns in"},{"location":"patterns/overview/#cell-bearer-of-quality","text":"A cell that has a specific quality, such as binucleate. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellBearerOfQuality Name cellBearerOfQuality Classes CL:0000000, PATO:0000001, Variables cell (CL:0000000), quality (PATO:0000001), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples mondo","title":"Cell bearer of quality"},{"location":"patterns/overview/#data-preview","text":"defined:class defined:class:label cell cell:label quality quality:label CL:0001061 abnormal cell CL:0000000 cell PATO:0000460 abnormal CL:0000225 anucleate cell CL:0000003 native cell PATO:0001405 anucleate CL:0000227 binucleate cell CL:0000003 native cell PATO:0001406 binucleate CL:0000103 bipolar neuron CL:0000099 interneuron PATO:0070006 bipolar morphology CL:4023077 bitufted neuron CL:0000099 interneuron PATO:0070012 bitufted cell morphology See full table here","title":"Data preview:"},{"location":"patterns/overview/#cell-capable-of-biological-process","text":"Any cell that is involved in/capable of a particular biological process, such as acid secretion. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellCapableOfBiologicalProcess Name cellCapableOfBiologicalProcess Classes CL:0000000, GO:0008150, Variables cell (CL:0000000), biological_process (GO:0008150), Contributors Examples mondo","title":"Cell capable of biological process"},{"location":"patterns/overview/#data-preview_1","text":"defined:class defined:class:label biological:process biological:process:label cell cell:label CL:0000236 B cell GO:0019724 B cell mediated immunity CL:0000945 lymphocyte of B lineage CL:0000492 CD4-positive helper T cell GO:0001816 cytokine production CL:0000624 CD4-positive, alpha-beta T cell CL:0000795 CD8-positive, alpha-beta regulatory T cell GO:0050777 negative regulation of immune response CL:0000625 CD8-positive, alpha-beta T cell CL:0011005 GABAergic interneuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000099 interneuron CL:0000617 GABAergic neuron GO:0061534 gamma-aminobutyric acid secretion, neurotransmission CL:0000540 neuron See full table here","title":"Data preview:"},{"location":"patterns/overview/#cell-has-plasma-membrane-part-x","text":"A cell type that is characterized by a plasma membrane part, such as a cilium or receptor. Note - that this is only good for cells defined by a single plasma membrane receptor. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellHasPlasmaMembranePartX Name cellHasPlasmaMembranePartX Classes CL:0000000, CL:0000003, GO:0005886, Variables cell (CL:0000003), plasma_membrane (GO:0005886), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples","title":"Cell has plasma membrane part x"},{"location":"patterns/overview/#cell-part-of-anatomical-entity","text":"A cell that is part of an anatomical entity. Attribute Info IRI http://purl.obolibrary.org/obo/cl/cellPartOfAnatomicalEntity Name cellPartOfAnatomicalEntity Classes CL:0000000, UBERON:0001062, Variables cell (CL:0000000), anatomical_entity (UBERON:0001062), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples mondo","title":"Cell part of anatomical entity"},{"location":"patterns/overview/#data-preview_2","text":"defined:class defined:class:label anatomical:entity anatomical:entity:label cell cell:label CL:0009032 B cell of appendix UBERON:0001154 vermiform appendix CL:0000236 B cell CL:0009045 B cell of medullary sinus of lymph node UBERON:0009744 lymph node medullary sinus CL:0000236 B cell CL:0010007 His-Purkinje system cell UBERON:0004146 His-Purkinje system CL:0000003 native cell CL:0002680 PP cell of intestine UBERON:0000160 intestine CL:0000696 PP cell CL:0009015 Peyer's patch follicular dendritic cell UBERON:0001211 Peyer's patch CL:0000442 follicular dendritic cell See full table here","title":"Data preview:"},{"location":"patterns/overview/#taxon-specific","text":"A cell that is restricted to a specific taxon, such as CL:0001200 'lymphocyte of B lineage, CD19-positive' are only in mammals. Note - this is not to be used for any cell that is restricted to a taxon, this is for taxon-specific subclasses of existing cell types. This should hardly ever be used. Attribute Info IRI http://purl.obolibrary.org/obo/cl/taxonSpecific Name taxonSpecific Classes CL:0000000, NCBITaxon:1, Variables cell (CL:0000000), taxon (NCBITaxon:1), Contributors 0000-0001-5208-3432 , 0000-0002-6601-2165 , Examples","title":"Taxon specific"},{"location":"patterns/taxonSpecific/","text":"taxonSpecific \u00b6 http://purl.obolibrary.org/obo/cl/taxonSpecific Description \u00b6 A cell that is restricted to a specific taxon, such as CL:0001200 'lymphocyte of B lineage, CD19-positive' are only in mammals. Note - this is not to be used for any cell that is restricted to a taxon, this is for taxon-specific subclasses of existing cell types. This should hardly ever be used. Contributors \u00b6 https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165 Name \u00b6 { cell }, { taxon } Definition \u00b6 Any { cell } that is part of a { taxon }. Equivalent to \u00b6 { cell } and ( in taxon some { taxon })","title":"Cell, taxon specific"},{"location":"patterns/taxonSpecific/#taxonspecific","text":"http://purl.obolibrary.org/obo/cl/taxonSpecific","title":"taxonSpecific"},{"location":"patterns/taxonSpecific/#description","text":"A cell that is restricted to a specific taxon, such as CL:0001200 'lymphocyte of B lineage, CD19-positive' are only in mammals. Note - this is not to be used for any cell that is restricted to a taxon, this is for taxon-specific subclasses of existing cell types. This should hardly ever be used.","title":"Description"},{"location":"patterns/taxonSpecific/#contributors","text":"https://orcid.org/0000-0001-5208-3432 https://orcid.org/0000-0002-6601-2165","title":"Contributors"},{"location":"patterns/taxonSpecific/#name","text":"{ cell }, { taxon }","title":"Name"},{"location":"patterns/taxonSpecific/#definition","text":"Any { cell } that is part of a { taxon }.","title":"Definition"},{"location":"patterns/taxonSpecific/#equivalent-to","text":"{ cell } and ( in taxon some { taxon })","title":"Equivalent to"}]} \ No newline at end of file diff --git a/search/worker.js b/search/worker.js new file mode 100644 index 000000000..8628dbce9 --- /dev/null +++ b/search/worker.js @@ -0,0 +1,133 @@ +var base_path = 'function' === typeof importScripts ? '.' : '/search/'; +var allowSearch = false; +var index; +var documents = {}; +var lang = ['en']; +var data; + +function getScript(script, callback) { + console.log('Loading script: ' + script); + $.getScript(base_path + script).done(function () { + callback(); + }).fail(function (jqxhr, settings, exception) { + console.log('Error: ' + exception); + }); +} + +function getScriptsInOrder(scripts, callback) { + if (scripts.length === 0) { + callback(); + return; + } + getScript(scripts[0], function() { + getScriptsInOrder(scripts.slice(1), callback); + }); +} + +function loadScripts(urls, callback) { + if( 'function' === typeof importScripts ) { + importScripts.apply(null, urls); + callback(); + } else { + getScriptsInOrder(urls, callback); + } +} + +function onJSONLoaded () { + data = JSON.parse(this.responseText); + var scriptsToLoad = ['lunr.js']; + if (data.config && data.config.lang && data.config.lang.length) { + lang = data.config.lang; + } + if (lang.length > 1 || lang[0] !== "en") { + scriptsToLoad.push('lunr.stemmer.support.js'); + if (lang.length > 1) { + scriptsToLoad.push('lunr.multi.js'); + } + if (lang.includes("ja") || lang.includes("jp")) { + scriptsToLoad.push('tinyseg.js'); + } + for (var i=0; i < lang.length; i++) { + if (lang[i] != 'en') { + scriptsToLoad.push(['lunr', lang[i], 'js'].join('.')); + } + } + } + loadScripts(scriptsToLoad, onScriptsLoaded); +} + +function onScriptsLoaded () { + console.log('All search scripts loaded, building Lunr index...'); + if (data.config && data.config.separator && data.config.separator.length) { + lunr.tokenizer.separator = new RegExp(data.config.separator); + } + + if (data.index) { + index = lunr.Index.load(data.index); + data.docs.forEach(function (doc) { + documents[doc.location] = doc; + }); + console.log('Lunr pre-built index loaded, search ready'); + } else { + index = lunr(function () { + if (lang.length === 1 && lang[0] !== "en" && lunr[lang[0]]) { + this.use(lunr[lang[0]]); + } else if (lang.length > 1) { + this.use(lunr.multiLanguage.apply(null, lang)); // spread operator not supported in all browsers: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_operator#Browser_compatibility + } + this.field('title'); + this.field('text'); + this.ref('location'); + + for (var i=0; i < data.docs.length; i++) { + var doc = data.docs[i]; + this.add(doc); + documents[doc.location] = doc; + } + }); + console.log('Lunr index built, search ready'); + } + allowSearch = true; + postMessage({config: data.config}); + postMessage({allowSearch: allowSearch}); +} + +function init () { + var oReq = new XMLHttpRequest(); + oReq.addEventListener("load", onJSONLoaded); + var index_path = base_path + '/search_index.json'; + if( 'function' === typeof importScripts ){ + index_path = 'search_index.json'; + } + oReq.open("GET", index_path); + oReq.send(); +} + +function search (query) { + if (!allowSearch) { + console.error('Assets for search still loading'); + return; + } + + var resultDocuments = []; + var results = index.search(query); + for (var i=0; i < results.length; i++){ + var result = results[i]; + doc = documents[result.ref]; + doc.summary = doc.text.substring(0, 200); + resultDocuments.push(doc); + } + return resultDocuments; +} + +if( 'function' === typeof importScripts ) { + onmessage = function (e) { + if (e.data.init) { + init(); + } else if (e.data.query) { + postMessage({ results: search(e.data.query) }); + } else { + console.error("Worker - Unrecognized message: " + e); + } + }; +} diff --git a/sitemap.xml b/sitemap.xml new file mode 100644 index 000000000..38b1bdd5f --- /dev/null +++ b/sitemap.xml @@ -0,0 +1,173 @@ + + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + + None + 2023-11-02 + daily + + \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz new file mode 100644 index 000000000..24d5a2f12 Binary files /dev/null and b/sitemap.xml.gz differ diff --git a/taxon-restrictions/index.html b/taxon-restrictions/index.html new file mode 100644 index 000000000..cca36c4cf --- /dev/null +++ b/taxon-restrictions/index.html @@ -0,0 +1,786 @@ + + + + + + + + + + + + + + + + + + + + + + Taxon constraints - CL Documentation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ +
+ + + + +
+ + +
+ +
+ + + + + + +
+
+ + + +
+
+
+ + + + +
+
+
+ + + +
+
+
+ + + +
+
+
+ + + +
+
+ + + + + +

The cell ontology (CL) follows certain conventions regarding taxon constraints. +Please review the following pages before adding taxon constraints to CL terms.

+

Explanation of taxon constraints can be found here: +https://oboacademy.github.io/obook/explanation/taxon-constraints-explainer/

+

A how-to guide can be found here: +https://oboacademy.github.io/obook/howto/add-taxon-restrictions/

+ + + + + + +
+
+ + +
+ +
+ + + +
+
+
+
+ + + + + + + + + + + \ No newline at end of file