Skip to content

Commit

Permalink
[CI] Publish Documentation for cd70de4
Browse files Browse the repository at this point in the history
  • Loading branch information
@NPJ-OP-LUX
NPJ-OP-LUX committed Feb 23, 2024
1 parent ce477e8 commit b7882a7
Show file tree
Hide file tree
Showing 18 changed files with 5,657 additions and 5,609 deletions.
18 changes: 12 additions & 6 deletions SWS/mapping_suite/mapping-suite-structure.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,9 @@ <h1 class="page">Mapping suite package structure</h1>
</table>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_suite_package_description_for_semantic_engineers"><a class="anchor" href="#_mapping_suite_package_description_for_semantic_engineers"></a>Mapping suite package description for Semantic Engineers</h3>
<div class="sect1">
<h2 id="_mapping_suite_package_description_for_semantic_engineers"><a class="anchor" href="#_mapping_suite_package_description_for_semantic_engineers"></a>Mapping suite package description for Semantic Engineers</h2>
<div class="sectionbody">
<div class="paragraph">
<p>When Semantic Engineers start working on a new mapping suite, they first need to set up a package folder structure similar to the one described below. One package per form number is the accepted way to organise packages.</p>
</div>
Expand Down Expand Up @@ -385,8 +386,10 @@ <h3 id="_mapping_suite_package_description_for_semantic_engineers"><a class="anc
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_suite_package_description_for_software_engineers"><a class="anchor" href="#_mapping_suite_package_description_for_software_engineers"></a>Mapping suite package description for Software Engineers</h3>
</div>
<div class="sect1">
<h2 id="_mapping_suite_package_description_for_software_engineers"><a class="anchor" href="#_mapping_suite_package_description_for_software_engineers"></a>Mapping suite package description for Software Engineers</h2>
<div class="sectionbody">
<div class="paragraph">
<p>A package provided by semantic engineers (SE) is enriched with additional artefacts that are generated automatically using the package expanding tools which take as input the artefacts provided by the SE. Here are some examples of additional artefacts generated:</p>
</div>
Expand Down Expand Up @@ -453,8 +456,10 @@ <h3 id="_mapping_suite_package_description_for_software_engineers"><a class="anc
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_suite_package_description_for_semantic_engineers_after_execution"><a class="anchor" href="#_mapping_suite_package_description_for_semantic_engineers_after_execution"></a>Mapping suite package description for Semantic Engineers after execution</h3>
</div>
<div class="sect1">
<h2 id="_mapping_suite_package_description_for_semantic_engineers_after_execution"><a class="anchor" href="#_mapping_suite_package_description_for_semantic_engineers_after_execution"></a>Mapping suite package description for Semantic Engineers after execution</h2>
<div class="sectionbody">
<div class="paragraph">
<p>After the execution of a mapping, the mapping package will be further enriched, containing additional files as a result of running the mapping suite on the test data included.</p>
</div>
Expand Down Expand Up @@ -515,6 +520,7 @@ <h3 id="_mapping_suite_package_description_for_semantic_engineers_after_executio
<p><a href="https://github.com/OP-TED/ted-rdf-docs/issues/new/choose">Click here to let us know</a></p>
</div>
</div>
</div>
</article>
</div>
</main>
Expand Down
22 changes: 14 additions & 8 deletions SWS/mapping_suite/reusing.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -238,13 +238,14 @@ <h3 class="title"><a href="../index.html">TED Semantic Web Service</a></h3>
</aside>
<article class="doc">
<h1 class="page">Reusing Mapping suite packages</h1>
<div class="sect2">
<h3 id="_reusing_mapping_suite_packages"><a class="anchor" href="#_reusing_mapping_suite_packages"></a>Reusing Mapping suite packages</h3>
<div class="sect1">
<h2 id="_reusing_mapping_suite_packages"><a class="anchor" href="#_reusing_mapping_suite_packages"></a>Reusing Mapping suite packages</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The structure of the package evolves during through the different phases of the mapping development process. Three phases of the mapping development are described in this section.</p>
</div>
<div class="sect3">
<h4 id="_initial_phase"><a class="anchor" href="#_initial_phase"></a>Initial Phase</h4>
<div class="sect2">
<h3 id="_initial_phase"><a class="anchor" href="#_initial_phase"></a>Initial Phase</h3>
<div class="paragraph">
<p><strong>Audience</strong>: Semantic Engineers
In the first, initial, phase, when the Semantic Engineers start working on a new mapping suite, they should set up a package folder structure similar to the one described below.</p>
Expand Down Expand Up @@ -291,8 +292,10 @@ <h4 id="_initial_phase"><a class="anchor" href="#_initial_phase"></a>Initial Pha
</div>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_suite_package_description_for_the_software_engineers"><a class="anchor" href="#_mapping_suite_package_description_for_the_software_engineers"></a>Mapping suite package description for the Software Engineers</h3>
</div>
<div class="sect1">
<h2 id="_mapping_suite_package_description_for_the_software_engineers"><a class="anchor" href="#_mapping_suite_package_description_for_the_software_engineers"></a>Mapping suite package description for the Software Engineers</h2>
<div class="sectionbody">
<div class="paragraph">
<p>A package provided by the semantic engineers (SE) is enriched with additional artefacts that are generated automatically using the package expanding tools which take as input the artefacts provided by the SE. Here are some examples of these additional artefacts that are being generated:</p>
</div>
Expand Down Expand Up @@ -350,8 +353,10 @@ <h3 id="_mapping_suite_package_description_for_the_software_engineers"><a class=
</div>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_suite_package_description_for_the_semantic_engineers_after_the_expansion"><a class="anchor" href="#_mapping_suite_package_description_for_the_semantic_engineers_after_the_expansion"></a>Mapping suite package description for the Semantic Engineers after the expansion</h3>
</div>
<div class="sect1">
<h2 id="_mapping_suite_package_description_for_the_semantic_engineers_after_the_expansion"><a class="anchor" href="#_mapping_suite_package_description_for_the_semantic_engineers_after_the_expansion"></a>Mapping suite package description for the Semantic Engineers after the expansion</h2>
<div class="sectionbody">
<div class="paragraph">
<p>After the “execution” of a mapping, the mapping package will be further enriched, and will contain additional files, as a result of running the mapping suite on the included test data.</p>
</div>
Expand Down Expand Up @@ -414,6 +419,7 @@ <h3 id="_mapping_suite_package_description_for_the_semantic_engineers_after_the_
<p><a href="https://github.com/OP-TED/ted-rdf-docs/issues/new/choose">Click here to let us know</a></p>
</div>
</div>
</div>
</article>
</div>
</main>
Expand Down
66 changes: 44 additions & 22 deletions SWS/mapping_suite/toolchain.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -252,28 +252,35 @@ <h1 class="page">The Toolchain used in the Mapping Development Process</h1>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_resources_injector_resources_injector"><a class="anchor" href="#_resources_injector_resources_injector"></a>Resources Injector (resources_injector)</h3>
<div class="sect1">
<h2 id="_resources_injector_resources_injector"><a class="anchor" href="#_resources_injector_resources_injector"></a>Resources Injector (resources_injector)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This CLI injects the requested resources listed on the "Resources" spreadsheet of the Conceptual Mappings into the MappingSuite. Each Form has a resources list that represents the controlled value that are needed in the mapping process.</p>
</div>
</div>
<div class="sect2">
<h3 id="_rml_modules_injector_rml_modules_injector"><a class="anchor" href="#_rml_modules_injector_rml_modules_injector"></a>RML Modules Injector (rml_modules_injector)</h3>
</div>
<div class="sect1">
<h2 id="_rml_modules_injector_rml_modules_injector"><a class="anchor" href="#_rml_modules_injector_rml_modules_injector"></a>RML Modules Injector (rml_modules_injector)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This CLI injects the technical mappings modules
into the mapping suite. Each form has a module list that is needed in order to run the mapping_runner.
The modules names are listed on the "RML_Modules" spreadsheet of the Conceptual Mappings.</p>
</div>
</div>
<div class="sect2">
<h3 id="_sparql_test_generator_sparql_generator"><a class="anchor" href="#_sparql_test_generator_sparql_generator"></a>SPARQL Test Generator (sparql_generator)</h3>
</div>
<div class="sect1">
<h2 id="_sparql_test_generator_sparql_generator"><a class="anchor" href="#_sparql_test_generator_sparql_generator"></a>SPARQL Test Generator (sparql_generator)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This CLI generates a set of SPARQL queries from the conceptual mapping that is then executed by the <code>sparql_runner</code> CLI (described in the next section). Each generated query can be used to test if the related conceptual mapping is correctly generating RDF data or not.</p>
</div>
</div>
<div class="sect2">
<h3 id="_sparql_queries_runner_sparql_runner"><a class="anchor" href="#_sparql_queries_runner_sparql_runner"></a>SPARQL Queries Runner (sparql_runner)</h3>
</div>
<div class="sect1">
<h2 id="_sparql_queries_runner_sparql_runner"><a class="anchor" href="#_sparql_queries_runner_sparql_runner"></a>SPARQL Queries Runner (sparql_runner)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This CLI executes all the sparql queries generated by sparql_generator into a separate RDF result file. The result file is a report per RDF file that contains the queries and their associated success indicators (Valid, Invalid, Unverifiable, Warning or Error).</p>
</div>
Expand Down Expand Up @@ -331,44 +338,58 @@ <h3 id="_sparql_queries_runner_sparql_runner"><a class="anchor" href="#_sparql_q
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_test_runner_mapping_runner"><a class="anchor" href="#_mapping_test_runner_mapping_runner"></a>Mapping Test Runner (mapping_runner)</h3>
</div>
<div class="sect1">
<h2 id="_mapping_test_runner_mapping_runner"><a class="anchor" href="#_mapping_test_runner_mapping_runner"></a>Mapping Test Runner (mapping_runner)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This CLI applies the mapping on a certain test notice file, a batch of notice files (organised in a folder), or on all available test notices, and generates output files representing the corresponding RDF graph for each notice (see <a href="https://github.com/OP-TED/ted-rdf-mapping/tree/main/mappings/package_F03/output">RDF output examples</a>).</p>
</div>
</div>
<div class="sect2">
<h3 id="_metadata_generator_metadata_generator"><a class="anchor" href="#_metadata_generator_metadata_generator"></a>Metadata generator (metadata_generator)</h3>
</div>
<div class="sect1">
<h2 id="_metadata_generator_metadata_generator"><a class="anchor" href="#_metadata_generator_metadata_generator"></a>Metadata generator (metadata_generator)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This tool extracts the relevant metadata from the conceptual mapping file (by default: <code>conceptual_mappings.xlsx</code>) and stores it in a JSON file (by default: <code>metadata.json</code>). This metadata file will be used by various other processes (both CLIs and DAGs), mainly to inform them about the applicability of this mapping to various notices.</p>
</div>
</div>
<div class="sect2">
<h3 id="_yarrrml_to_rml_converter_yarrrml2rml_converter"><a class="anchor" href="#_yarrrml_to_rml_converter_yarrrml2rml_converter"></a>YARRRML to RML Converter (yarrrml2rml_converter)</h3>
</div>
<div class="sect1">
<h2 id="_yarrrml_to_rml_converter_yarrrml2rml_converter"><a class="anchor" href="#_yarrrml_to_rml_converter_yarrrml2rml_converter"></a>YARRRML to RML Converter (yarrrml2rml_converter)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This command line tool allows the conversion of a mapping expressed in the more user-friendly YARRRML syntax to RML. This is a very useful tool, especially at initial phases of the mapping development, or for newcomers, as it is easier and faster to write YARRRML rules than RML rules. This tool is not used anymore in the current development process as, due to technical reasons, the mappings are developed directly in RML, not in YARRRML.</p>
</div>
</div>
<div class="sect2">
<h3 id="_xpath_coverage_runner_xpath_coverage_runner"><a class="anchor" href="#_xpath_coverage_runner_xpath_coverage_runner"></a>XPATH Coverage Runner (xpath_coverage_runner)</h3>
</div>
<div class="sect1">
<h2 id="_xpath_coverage_runner_xpath_coverage_runner"><a class="anchor" href="#_xpath_coverage_runner_xpath_coverage_runner"></a>XPATH Coverage Runner (xpath_coverage_runner)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Generates reports describing XPATH coverage of the notices.</p>
</div>
</div>
<div class="sect2">
<h3 id="_shacl_validation_runner_shacl_runner"><a class="anchor" href="#_shacl_validation_runner_shacl_runner"></a>SHACL Validation Runner (shacl_runner)</h3>
</div>
<div class="sect1">
<h2 id="_shacl_validation_runner_shacl_runner"><a class="anchor" href="#_shacl_validation_runner_shacl_runner"></a>SHACL Validation Runner (shacl_runner)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Generates SHACL Validation Reports for RDF files.</p>
</div>
</div>
<div class="sect2">
<h3 id="_mapping_suite_processor_mapping_suite_processor"><a class="anchor" href="#_mapping_suite_processor_mapping_suite_processor"></a>Mapping Suite Processor (mapping_suite_processor)</h3>
</div>
<div class="sect1">
<h2 id="_mapping_suite_processor_mapping_suite_processor"><a class="anchor" href="#_mapping_suite_processor_mapping_suite_processor"></a>Mapping Suite Processor (mapping_suite_processor)</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This CLI runs all the necessary CLIs mentioned above in a logical order, to fully process a mapping suite, starting with the generation of metadata and finishing with running the mapping on all the (specified) tests data, generating all the possible associated validation artefacts. It can be run for a certain package, (set of) notice(s), or groups of commands.</p>
</div>
</div>
<div class="sect2">
<h3 id="_other_relevant_tools_and_libraries"><a class="anchor" href="#_other_relevant_tools_and_libraries"></a>Other relevant tools and libraries</h3>
</div>
<div class="sect1">
<h2 id="_other_relevant_tools_and_libraries"><a class="anchor" href="#_other_relevant_tools_and_libraries"></a>Other relevant tools and libraries</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Other relevant tools that are used in the mapping process that worth mentioning, are:</p>
</div>
Expand All @@ -389,6 +410,7 @@ <h3 id="_other_relevant_tools_and_libraries"><a class="anchor" href="#_other_rel
<p><a href="https://github.com/OP-TED/ted-rdf-docs/issues/new/choose">Click here to let us know</a></p>
</div>
</div>
</div>
</article>
</div>
</main>
Expand Down
42 changes: 28 additions & 14 deletions SWS/mapping_suite/versioning.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -245,8 +245,9 @@ <h1 class="page">Mapping Suite Versioning Rules</h1>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_semantic_versioning_in_a_nutshell"><a class="anchor" href="#_semantic_versioning_in_a_nutshell"></a>Semantic versioning in a nutshell</h3>
<div class="sect1">
<h2 id="_semantic_versioning_in_a_nutshell"><a class="anchor" href="#_semantic_versioning_in_a_nutshell"></a>Semantic versioning in a nutshell</h2>
<div class="sectionbody">
<div class="paragraph">
<p>“Dependency hell” plagues software management and impacts models, architecture and documentation. As a project expands, the complexity of changes and dependencies increase, complicating the release of new work packages. Version lock and version promiscuity impede progress, making it difficult to move projects forward safely and efficiently.</p>
</div>
Expand Down Expand Up @@ -298,14 +299,18 @@ <h3 id="_semantic_versioning_in_a_nutshell"><a class="anchor" href="#_semantic_v
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_backwards_compatibility"><a class="anchor" href="#_backwards_compatibility"></a>Backwards compatibility</h3>
</div>
<div class="sect1">
<h2 id="_backwards_compatibility"><a class="anchor" href="#_backwards_compatibility"></a>Backwards compatibility</h2>
<div class="sectionbody">
<div class="paragraph">
<p>A new version of the mapping suite is considered to be backwards compatible if it can be read directly by the same software that was able to read the previous version without requiring any modifications in its code.</p>
</div>
</div>
<div class="sect2">
<h3 id="_major_version_increment"><a class="anchor" href="#_major_version_increment"></a>Major version increment</h3>
</div>
<div class="sect1">
<h2 id="_major_version_increment"><a class="anchor" href="#_major_version_increment"></a>Major version increment</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
Expand All @@ -327,8 +332,10 @@ <h3 id="_major_version_increment"><a class="anchor" href="#_major_version_increm
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_minor_version_increment"><a class="anchor" href="#_minor_version_increment"></a>Minor version increment</h3>
</div>
<div class="sect1">
<h2 id="_minor_version_increment"><a class="anchor" href="#_minor_version_increment"></a>Minor version increment</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
Expand Down Expand Up @@ -369,8 +376,10 @@ <h3 id="_minor_version_increment"><a class="anchor" href="#_minor_version_increm
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_patch_version_increment"><a class="anchor" href="#_patch_version_increment"></a>Patch version increment</h3>
</div>
<div class="sect1">
<h2 id="_patch_version_increment"><a class="anchor" href="#_patch_version_increment"></a>Patch version increment</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
Expand All @@ -395,8 +404,10 @@ <h3 id="_patch_version_increment"><a class="anchor" href="#_patch_version_increm
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_release_labelling"><a class="anchor" href="#_release_labelling"></a>Release labelling</h3>
</div>
<div class="sect1">
<h2 id="_release_labelling"><a class="anchor" href="#_release_labelling"></a>Release labelling</h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
Expand All @@ -418,8 +429,10 @@ <h3 id="_release_labelling"><a class="anchor" href="#_release_labelling"></a>Rel
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_conclusion"><a class="anchor" href="#_conclusion"></a>Conclusion</h3>
</div>
<div class="sect1">
<h2 id="_conclusion"><a class="anchor" href="#_conclusion"></a>Conclusion</h2>
<div class="sectionbody">
<div class="paragraph">
<p>By adhering to the versioning rules outlined in this section, developers and maintainers can effectively manage versioning for mapping suites, ensuring compatibility and smooth transitions between different versions. These rules provide clear guidelines for when to increment major, minor and patch versions, considering the potential impact on SPARQL queries on the transformation output. Following these rules will help maintain consistency and compatibility across various systems that rely on the mapping suites.</p>
</div>
Expand All @@ -431,6 +444,7 @@ <h3 id="_conclusion"><a class="anchor" href="#_conclusion"></a>Conclusion</h3>
<p><a href="https://github.com/OP-TED/ted-rdf-docs/issues/new/choose">Click here to let us know</a></p>
</div>
</div>
</div>
</article>
</div>
</main>
Expand Down
Loading

0 comments on commit b7882a7

Please sign in to comment.