Merge branch 'develop' of https://github.com/hermes-hmc/workflow into…

… 137-remove-click

.gitignore

@@ -18,7 +18,8 @@ docs/build/
 
 .idea/
 .venv/
+.vscode/
 dist/
 
 # HERMES workflow specifics
-.hermes
+.hermes/

docs/adr/0002-use-a-common-data-model.md

@@ -14,11 +14,14 @@ SPDX-License-Identifier: CC-BY-SA-4.0
 
 We need a data model that's
 
-- extensible, to take up metaedata that cannot yet be included in CodeMeta.json
+- extensible, to take up metadata that cannot yet be included in CodeMeta.json
 - compatible with RO-Crate
 
 to exchange data between modules.
 
+The chosen option determines the serialization of the data model, too.
+See also [ADR 11](./0011-record-provenance-of-metadata.md) about provenance records in the data model.
+
 ## Considered Options
 
 * CodeMeta + schema.org via RO-Crate
@@ -32,6 +35,7 @@ Chosen option: "CodeMeta + schema-based, extended JSON-LD for internal data mode
 
 * Compatibility with RO-Crate
 * Compatibility with CodeMeta
+* Compatibility with custom metadata to be deposited along standardized metadata
 
 ### Negative Consequences
 

docs/adr/0003-define-interfaces-for-inter-module-data-exchange.md

@@ -4,16 +4,18 @@ SPDX-FileCopyrightText: 2022 German Aerospace Center (DLR), Forschungszentrum J
 SPDX-License-Identifier: CC-BY-SA-4.0
 -->
 
-# Define interfaces for inter-module data exchange
+# <strike>Define interfaces for inter-module data exchange</strike>
 
-* Status: proposed
-* Date: 2022-03-07
+* Status: superseded
+* Date: 2023-11-15
 
 ## Context and Problem Statement
 
 This depends on the data model (ADR 0002).
 Do we have to expose different parts of the data model structure at different points in the workflow?
 
+Superseded: decisions in [ADR 2](./0002-use-a-common-data-model.md) (use JSON-LD) and [ADR 11](./0011-record-provenance-of-metadata.md)  (create a unified data model of metadata and provenance) will result in a context [DAO](https://en.wikipedia.org/wiki/Data_access_object) from beginning till end of a run.
+
 ## Decision Drivers
 
 * Different data need to be available at different points

docs/adr/0011-record-provenance-of-metadata.md

@@ -5,25 +5,44 @@ SPDX-License-Identifier: CC-BY-SA-4.0
 -->
 # Record provenance of metadata
 
-* Status: proposed
-* Date: 2022-05-30
+* Status: accepted
+* Deciders: sdruskat, jkelling, led02, poikilotherm, skernchen
+* Date: 2023-11-15
+
+Technical story: https://github.com/hermes-hmc/hermes/pull/40
 
 ## Context and Problem Statement
 
-To enable tracability of the metadata, and resolution based on metadata sources in case of duplicates, etc., we need to record the provenance of metadata values. To do this, we need to specify a way to do this.
+To enable traceability of the metadata, and resolution based on metadata sources in case of duplicates, etc., we need to record the provenance of metadata values.
+To do this, we need to specify a way to do this.
 
 ## Considered Options
 
 * Internal comment field
 * Dedicated metadata field
 * Use PROV standard
+* Separate internal provenance model
+* Create wrapped JSON-LD entities and add our metadata ([json-ld/json-ld.org#744](https://github.com/json-ld/json-ld.org/issues/744))
+* Create non-standard JSON-LD extension with custom [keywords]
 
 ## Decision Outcome
 
-Chosen option: "", because comes out best.
+Chosen option: "Create non-standard JSON-LD extension with custom [keywords]", because comes out best.
 
 ## Pros and Cons of the Options
 
+### Positive Consequences
+
+* We have a unified data model and keep both provenance and actual data at the sample place
+* Unifying the data model requires less complex handling in the implementation, as there is no need to recombine data
+* It's not possible to loose the provenance information, as it is added to any attribute stored in the model
+
+### Negative Consequences
+
+* We loose the ability to reuse the serialized internal data model as output files (CodeMeta) without further processing
+* We have to document these added keywords very well for plugin developers
+* We need to define a ontology for this (internal) provenance metadata
+
 ### Internal comment field
 
 This would be a comment attached to single metadata fields to record provenance.
@@ -46,3 +65,37 @@ This attaches provenance information following PROV-O to metadata fields
 * Good, because Standardized for provenance information
 * Good, because Not much extra documentation needed
 * Bad, because More circumvent way to describe relatively constricted cases (probably only use a few entities and `prov:wasInformedBy` or similar)
+
+### Separate internal metadata about metadata model
+
+This would create a valid JSON-LD file serializing our internal data model and a auxiliary file with the provencance data
+
+* Good, because keeping things separate enables direct reuse and validation of the data model file
+* Good, because serialization of the provenance data is free form and simple to do
+* Bad, because we need to re-combine provenance and metadata
+* Bad, because we have more files in the output which might confuse people
+* Bad, because not easy to debug when recombination fails
+
+### Create wrapped JSON-LD entities and add our metadata
+
+This would work around the limitation of RDF and JSON-LD that [value objects](https://www.w3.org/TR/json-ld/#value-objects) are non-extensible
+
+* Good, because standard compliant, still validates using standard validators
+* Bad, because very noisy in the output files
+* Bad, because still needs back references to the object when using `@id` in graph objects
+* Bad, because would require our own ontology and repeating any field ever needed (when keeping the original fields and not using a graph object)
+* Bad, because would require our own objects to keep the type and value separated, requiring reparsing when writing output files
+
+### Create non-standard JSON-LD extension with custom keywords
+
+This would work around the limitation of RDF and JSON-LD that [value objects](https://www.w3.org/TR/json-ld/#value-objects) are non-extensible
+
+* Good, because easy to implement in our custom handling of the graph as Python dictionaries
+* Good, because not very noisy
+* Good, because keywords are the JSON-LD way to provide metadata already
+* Good, because very light extension and not touching definitions from other ontologies
+* Good, because we can still make use of an ontology for the metadata objects to provide an open/closed principle compliant structure
+* Bad, because not standard compliant
+* Bad, because needs filtering when writing output files
+
+[keywords]: https://www.w3.org/TR/json-ld/#keywords

docs/adr/0012-overall-data-model-design.md

@@ -3,10 +3,11 @@ SPDX-FileCopyrightText: 2022 German Aerospace Center (DLR), Forschungszentrum J
 
 SPDX-License-Identifier: CC-BY-SA-4.0
 -->
-# Overall data model design
+# <strike>Overall data model design</strike>
 
-* Status: proposed
+* Status: superseded
 * Date: 2022-07-06
+* Deciders: led02, poikilotherm, sdruskat, skernchen
 
 ## Context and Problem Statement
 
@@ -16,6 +17,8 @@ It is also curcial to add information about the source of the data.
 In contrast in the deposit state only curated, well defined, and unambiguous data should be stored.
 The source for single attributes is not required anymore.
 
+Superseded: we no longer need to serialize additional information like provenance (see ADR 11) at the different stages in separate files.
+
 ## Considered Options
 
 * One common model for all stages

docs/source/_templates/funding.html

@@ -0,0 +1,13 @@
+<!--
+SPDX-FileCopyrightText: 2024 Forschungszentrum Jülich
+SPDX-License-Identifier: CC0-1.0
+SPDX-FileContributor: Oliver Bertuch
+-->
+<div class="funding">
+  Funded by the <i>Initiative and Networking Fund</i> of the
+  <a href='https://www.helmholtz.de/en/about-us/structure-and-governance/initiating-and-networking/' target='_blank'>
+    Helmholtz Association
+  </a>
+  in the framework of the
+  <a href='https://helmholtz-metadaten.de' targe='_blank'>Helmholtz Metadata Collaboration</a>
+</div>

docs/source/conf.py

@@ -27,11 +27,13 @@
 # -- Project information -----------------------------------------------------
 
 project = 'HERMES Workflow'
-copyright = '2022, HERMES project'
-author = 'Oliver Bertuch, Stephan Druskat, Guido Juckeland, Jeffrey Kelling, Oliver Knodel, Michael Meinel, Tobias Schlauch'
+copyright = '2024, HERMES project'
+author = 'Oliver Bertuch, Stephan Druskat, Guido Juckeland, Jeffrey Kelling, ' + \
+         'Oliver Knodel, Michael Meinel, Tobias Schlauch, Sophie Kernchen'
+
 
 # The full version, including alpha/beta/rc tags
-release = '2022-07-01'
+release = '2024-01-11'
 
 
 # -- General configuration ---------------------------------------------------
@@ -47,6 +49,7 @@
     'sphinx.ext.ifconfig',
     'sphinx.ext.githubpages',
     'sphinx.ext.autodoc',
+    'sphinx.ext.napoleon',
     'sphinx-favicon',
     'sphinxcontrib.contentui',
     'sphinxcontrib.images',
@@ -55,7 +58,6 @@
     "sphinxext.opengraph",
     'myst_parser',
     'autoapi.extension',
-    'sphinx_click',
     'sphinxcontrib.mermaid',
     'sphinx_togglebutton',
 ]
@@ -100,11 +102,11 @@
 html_static_path = ['_static']
 
 html_logo = '_static/img/hermes-visual-blue.svg'
-html_title = 'Workflow Docs'
+html_title = 'HERMES Documentation'
 
 # Custom sidebar templates, maps document names to template names.
 html_sidebars = {
-    "**": ["sidebar-logo.html", "sbt-sidebar-nav.html"]
+    "**": ["navbar-logo.html", "sbt-sidebar-nav.html", "funding.html"]
 }
 
 # Enable and customize the permanent headerlinks with a nice icon (chain symbol from FontAwesome)
@@ -113,23 +115,23 @@
 
 html_theme_options = {
     "home_page_in_toc": True,
-    "extra_navbar": "<div>Funded by the <i>Initiative and Networking Fund</i> of the <a \
-                     href='https://www.helmholtz.de/en/about-us/structure-and-governance/initiating-and-networking/' \
-                    target='_blank'>Helmholtz Association</a> in the framework of the \
-                    <a href='https://helmholtz-metadaten.de' targe='_blank'>Helmholtz Metadata \
-                    Collaboration</a></div>",
-    "repository_url": "https://github.com/hermes-hmc/workflow",
+    "repository_url": "https://github.com/hermes-hmc/hermes",
     "use_repository_button": True,
+    "navigation_with_keys": False,
 }
 
 html_css_files = [
     'custom.css',
 ]
 
+html_context = {
+    "default_mode": "light",
+}
+
 # -- Options for OpenGraph Tags ----------------------------------------------
 
 ogp_site_url = "https://docs.software-metadata.pub/"
-ogp_image = "https://docs.software-metadata.pub/_static/img/opengraph-workflow.png"
+ogp_image = "https://docs.software-metadata.pub/en/latest/_static/img/opengraph-workflow.png"
 ogp_image_alt = "The HERMES key visual on a blue background with pipelines and the Workflow subproject title"
 ogp_description_length = 200
 ogp_type = "website"

hermes.toml

@@ -3,17 +3,17 @@
 # SPDX-License-Identifier: CC0-1.0
 
 [harvest]
-from = [ "cff", "git" ]
+sources = [ "cff", "git" ]
 
 [harvest.cff]
-validate = false
+enable_validation = false
 
 [deposit]
 target = "invenio_rdm"
 
 [deposit.invenio_rdm]
 site_url = "https://sandbox.zenodo.org"
-communities = ["zenodo"]
+communities = []
 access_right = "open"
 
 [deposit.invenio_rdm.api_paths]