From c922f322781f7be34e0a31404ba67e144b43e223 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robin=20Keskis=C3=A4rkk=C3=A4?= Date: Thu, 22 Aug 2024 15:33:42 +0200 Subject: [PATCH 1/4] usage intructions --- README.md | 121 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) diff --git a/README.md b/README.md index bcd038eff..bf0e0b83f 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,127 @@ # HeFQUIN HeFQUIN is a query federation engine for heterogeneous federations of graph data sources (e.g, federated knowledge graphs) that is currently under development by [the Semantic Web research group at Linköping University](https://www.ida.liu.se/research/semanticweb/). +# Quick Start + +HeFQUIN can be run as a web service or using a Command Line Interface (CLI) tool. + +## Running HeFQUIN as a web service +The web service can be run using docker, an embedded server, or an existing servlet container (e.g., Tomcat or Jetty). + +### Run using Docker +Download or clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build and run the latest version of the engine using: +```bash +$ docker-compose build +$ docker-compose up +``` + +The optional build argument (`TAG`) can used to build the image from a specific release: +```yml + hefquin: + build: + context: . + args: + TAG: + ports: + - "8080:8080" +``` + +The engine can be configured by mounting custom configuration files when starting up the container: +```yml + hefquin: + build: . + ports: + - "8080:8080" + volumes: + - ./example/config.properties:/usr/local/tomcat/webapps/ROOT/config.properties + - ./example/ExampleEngineConf.ttl:/usr/local/tomcat/webapps/ROOT/ExampleEngineConf.ttl + - ./example/ExampleFedConf.ttl:/usr/local/tomcat/webapps/ROOT/ExampleFedConf.ttl +``` + +where the `config.properties` file has the following structure: +```bash +ENGINE_CONF_FILE=ExampleEngineConf.ttl # the engine configuration file +FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file +``` + +By default the service exposes a SPARQL endpoint at `http://localhost:8080/sparql`. + +### Run using embedded server + +__Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: +```bash +$ mvn clean package +``` +To use a specific release, checkout the tag before building: +```bash +$ git checkout +$ mvn clean package +``` + +__Option 2:__ Download a jar file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). + +Run the server from a terminal using: +```bash +java -cp target/HeFQUIN-x.y.z-SNAPSHOT.jar se.liu.ida.hefquin.service.HeFQUINServer +``` + +The engine can be configured by modifying the `config.properties` file in the working directory. The `config.properties` file has the following structure: +```bash +ENGINE_CONF_FILE=ExampleEngineConf.ttl # the engine configuration file +FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file +``` + +### Run using an existing servlet container + +__Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: +```bash +$ mvn clean package +``` +To use a specific release, checkout the tag before building: +```bash +$ git checkout +$ mvn clean package +``` + +__Option 2:__ Download a jar file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). + +Deploy `target/HeFQUIN-x.y.z-SNAPSHOT.war` in your serlet container. + +The engine can be configured by modifying the `config.properties` file in the working directory. The `config.properties` file has the following structure: +```bash +ENGINE_CONF_FILE=ExampleEngineConf.ttl # the engine configuration file +FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file +``` +> __NOTE__: The servlet will need to be restarted for any changes in the engine configuration to take effect. + + +## Run HeFQUIN using the CLI Tool + +__Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: +```bash +$ mvn clean package +``` +To use a specific release, checkout the tag before building: +```bash +$ git checkout +$ mvn clean package +``` + +__Option 2:__ Download a jar file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). + +Example usage: +```bash +java -cp target/HeFQUIN-0.0.4-SNAPSHOT.jar se.liu.ida.hefquin.cli.RunQueryWithoutSrcSel \ + --query ExampleQuery.rq \ + --federationDescription ExampleFederation.ttl \ + --confDescr ExampleEngineConf.ttl +``` + +For a full list of the available CLI options use: +```bash +java -cp target/HeFQUIN-x.y.z-SNAPSHOT.jar se.liu.ida.hefquin.cli.RunQueryWithoutSrcSel --help +``` + ### Features of HeFQUIN * Support for all features of SPARQL 1.1 (where basic graph patterns, group graph patterns (AND), union graph patterns, optional patterns, and filters are supported natively within the HeFQUIN engine, and the other features of SPARQL are supported through integration of the HeFQUIN engine into Apache Jena) * So far, support for SPARQL endpoints, TPF, and brTPF From be715c311478918339a20bdb7744b19dc5fe62a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robin=20Keskis=C3=A4rkk=C3=A4?= Date: Thu, 22 Aug 2024 15:56:57 +0200 Subject: [PATCH 2/4] collapsible sections, minor fixes --- README.md | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index bf0e0b83f..46e9e5d31 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,12 @@ HeFQUIN is a query federation engine for heterogeneous federations of graph data HeFQUIN can be run as a web service or using a Command Line Interface (CLI) tool. -## Running HeFQUIN as a web service +## Running as a web service The web service can be run using docker, an embedded server, or an existing servlet container (e.g., Tomcat or Jetty). +
+ Run using Docker + ### Run using Docker Download or clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build and run the latest version of the engine using: ```bash @@ -45,8 +48,12 @@ FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file ``` By default the service exposes a SPARQL endpoint at `http://localhost:8080/sparql`. +
-### Run using embedded server +
+ Run jar + +### Run jar __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -70,8 +77,12 @@ The engine can be configured by modifying the `config.properties` file in the wo ENGINE_CONF_FILE=ExampleEngineConf.ttl # the engine configuration file FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file ``` +
-### Run using an existing servlet container +
+ Run in an existing servlet container + +### Run in an existing servlet container __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -93,9 +104,9 @@ ENGINE_CONF_FILE=ExampleEngineConf.ttl # the engine configuration file FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file ``` > __NOTE__: The servlet will need to be restarted for any changes in the engine configuration to take effect. - - -## Run HeFQUIN using the CLI Tool +
+ +## Run using the CLI tool __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash From 2e2b5ec0284ca7bc3295738211d829c28e830859 Mon Sep 17 00:00:00 2001 From: Olaf Hartig Date: Fri, 23 Aug 2024 11:22:43 +0200 Subject: [PATCH 3/4] Changes order of the sections in README.md --- README.md | 55 +++++++++++++++++++++++++------------------------------ 1 file changed, 25 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 46e9e5d31..1b506cbf1 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,34 @@ # HeFQUIN HeFQUIN is a query federation engine for heterogeneous federations of graph data sources (e.g, federated knowledge graphs) that is currently under development by [the Semantic Web research group at Linköping University](https://www.ida.liu.se/research/semanticweb/). -# Quick Start +### Features of HeFQUIN +* Support for all features of SPARQL 1.1 (where basic graph patterns, group graph patterns (AND), union graph patterns, optional patterns, and filters are supported natively within the HeFQUIN engine, and the other features of SPARQL are supported through integration of the HeFQUIN engine into Apache Jena) +* So far, support for SPARQL endpoints, TPF, and brTPF + * [work on openCypher Property Graphs ongoing](https://github.com/LiUSemWeb/HeFQUIN/tree/main/src/main/java/se/liu/ida/hefquin/engine/wrappers/lpgwrapper) + * [work on GraphQL APIs ongoing](https://github.com/LiUSemWeb/HeFQUIN/tree/main/src/main/java/se/liu/ida/hefquin/engine/wrappers/graphqlwrapper) +* Initial support for vocabulary mappings +* [Heuristics-based logical query optimizer](https://github.com/LiUSemWeb/HeFQUIN/wiki/Heuristics-Based-Logical-Query-Optimizer) +* Several different [cost-based physical optimizers](https://github.com/LiUSemWeb/HeFQUIN/wiki/Cost-Based-Physical-Query-Optimizers) (greedy, dynamic programming, simulated annealing, randomized iterative improvement) +* Relevant [physical operators](https://github.com/LiUSemWeb/HeFQUIN/wiki/Physical-Operators); e.g., hash join, symmetric hash join (SHJ), request-based nested-loops join (NLJ), several variations of bind joins (brTPF-based, UNION-based, FILTER-based, VALUES-based) +* Two execution models (push-based and pull-based) +* Features for getting an understanding of the internals of the engine + * printing of logical and physical plans + * programmatic access to execution statistics on the level of individual operators and data structures, as well as printing of these statistics from the CLI +* 380+ unit tests + +### Current Limitations +* HeFQUIN does not yet have a source selection component. All subpatterns of the queries given to HeFQUIN need to be wrapped in SERVICE clauses. + +## Quick Start HeFQUIN can be run as a web service or using a Command Line Interface (CLI) tool. -## Running as a web service +### Running as a web service The web service can be run using docker, an embedded server, or an existing servlet container (e.g., Tomcat or Jetty).
- Run using Docker + Run using Docker -### Run using Docker Download or clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build and run the latest version of the engine using: ```bash $ docker-compose build @@ -51,9 +68,7 @@ By default the service exposes a SPARQL endpoint at `http://localhost:8080/sparq
- Run jar - -### Run jar + Run jar __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -80,9 +95,7 @@ FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file
- Run in an existing servlet container - -### Run in an existing servlet container + Run in an existing servlet container __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -106,7 +119,7 @@ FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file > __NOTE__: The servlet will need to be restarted for any changes in the engine configuration to take effect.
-## Run using the CLI tool +### Run using the CLI tool __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -133,25 +146,7 @@ For a full list of the available CLI options use: java -cp target/HeFQUIN-x.y.z-SNAPSHOT.jar se.liu.ida.hefquin.cli.RunQueryWithoutSrcSel --help ``` -### Features of HeFQUIN -* Support for all features of SPARQL 1.1 (where basic graph patterns, group graph patterns (AND), union graph patterns, optional patterns, and filters are supported natively within the HeFQUIN engine, and the other features of SPARQL are supported through integration of the HeFQUIN engine into Apache Jena) -* So far, support for SPARQL endpoints, TPF, and brTPF - * [work on openCypher Property Graphs ongoing](https://github.com/LiUSemWeb/HeFQUIN/tree/main/src/main/java/se/liu/ida/hefquin/engine/wrappers/lpgwrapper) - * [work on GraphQL APIs ongoing](https://github.com/LiUSemWeb/HeFQUIN/tree/main/src/main/java/se/liu/ida/hefquin/engine/wrappers/graphqlwrapper) -* Initial support for vocabulary mappings -* [Heuristics-based logical query optimizer](https://github.com/LiUSemWeb/HeFQUIN/wiki/Heuristics-Based-Logical-Query-Optimizer) -* Several different [cost-based physical optimizers](https://github.com/LiUSemWeb/HeFQUIN/wiki/Cost-Based-Physical-Query-Optimizers) (greedy, dynamic programming, simulated annealing, randomized iterative improvement) -* Relevant [physical operators](https://github.com/LiUSemWeb/HeFQUIN/wiki/Physical-Operators); e.g., hash join, symmetric hash join (SHJ), request-based nested-loops join (NLJ), several variations of bind joins (brTPF-based, UNION-based, FILTER-based, VALUES-based) -* Two execution models (push-based and pull-based) -* Features for getting an understanding of the internals of the engine - * printing of logical and physical plans - * programmatic access to execution statistics on the level of individual operators and data structures, as well as printing of these statistics from the CLI -* 380+ unit tests - -### Current Limitations -* HeFQUIN does not yet have a source selection component. All subpatterns of the queries given to HeFQUIN need to be wrapped in SERVICE clauses. - -### Publications related to HeFQUIN +## Publications related to HeFQUIN * Sijin Cheng and Olaf Hartig: **[FedQPL: A Language for Logical Query Plans over Heterogeneous Federations of RDF Data Sources](https://olafhartig.de/files/ChengHartig_FedQPL_iiWAS2020_Extended.pdf)**. In _Proceedings of the 22nd International Conference on Information Integration and Web-based Applications & Services (iiWAS)_, 2020. * Sijin Cheng and Olaf Hartig: **[Source Selection for SPARQL Endpoints: Fit for Heterogeneous Federations of RDF Data Sources?](https://olafhartig.de/files/ChengHartig_QuWeDa2022.pdf)**. In _Proceedings of the 6th Workshop on Storing, Querying and Benchmarking Knowledge Graphs (QuWeDa)_, 2022. * Sijin Cheng and Olaf Hartig: **[A Cost Model to Optimize Queries over Heterogeneous Federations of RDF Data Sources](https://olafhartig.de/files/ChengHartig_CostModel_DMKG2023.pdf)**. In _Proceedings of the 1st International Workshop on Data Management for Knowledge Graphs (DMKG)_, 2023. From f8bbef4e7611e51d20e617bedb15559f37ee6b2c Mon Sep 17 00:00:00 2001 From: Olaf Hartig Date: Fri, 23 Aug 2024 11:34:17 +0200 Subject: [PATCH 4/4] Edits in the README.md --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 1b506cbf1..a5d980108 100644 --- a/README.md +++ b/README.md @@ -21,10 +21,10 @@ HeFQUIN is a query federation engine for heterogeneous federations of graph data ## Quick Start -HeFQUIN can be run as a web service or using a Command Line Interface (CLI) tool. +HeFQUIN can be used as a Web service or via a Command Line Interface (CLI) tool. -### Running as a web service -The web service can be run using docker, an embedded server, or an existing servlet container (e.g., Tomcat or Jetty). +### Running as a Web service +The Web service can be run using Docker, via an embedded server, or in a separate servlet container such as Tomcat or Jetty.
Run using Docker @@ -68,7 +68,7 @@ By default the service exposes a SPARQL endpoint at `http://localhost:8080/sparq
- Run jar + Run the server embedded in the JAR file __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -80,9 +80,9 @@ $ git checkout $ mvn clean package ``` -__Option 2:__ Download a jar file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). +__Option 2:__ Download a JAR file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). -Run the server from a terminal using: +Start the server in a terminal by running the following command: ```bash java -cp target/HeFQUIN-x.y.z-SNAPSHOT.jar se.liu.ida.hefquin.service.HeFQUINServer ``` @@ -95,7 +95,7 @@ FED_CONF_FILE=ExampleFedConf.ttl # the federation configuration file
- Run in an existing servlet container + Run in a separate servlet container __Option 1:__ Clone the project from the HeFQUIN repository and navigate to the project root in a terminal. Build the latest version of the engine using: ```bash @@ -107,7 +107,7 @@ $ git checkout $ mvn clean package ``` -__Option 2:__ Download a jar file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). +__Option 2:__ Download a JAR file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). Deploy `target/HeFQUIN-x.y.z-SNAPSHOT.war` in your serlet container. @@ -131,7 +131,7 @@ $ git checkout $ mvn clean package ``` -__Option 2:__ Download a jar file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). +__Option 2:__ Download a JAR file from the set of available releases (see [Releases](https://github.com/LiUSemWeb/HeFQUIN/releases)). Example usage: ```bash @@ -141,7 +141,7 @@ java -cp target/HeFQUIN-0.0.4-SNAPSHOT.jar se.liu.ida.hefquin.cli.RunQueryWithou --confDescr ExampleEngineConf.ttl ``` -For a full list of the available CLI options use: +For a full list of the parameters with which the CLI tool can be run, use the following command: ```bash java -cp target/HeFQUIN-x.y.z-SNAPSHOT.jar se.liu.ida.hefquin.cli.RunQueryWithoutSrcSel --help ```