diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.md index 2f8c8d7..f7620ea 100644 --- a/docs/getting-started/configuration.md +++ b/docs/getting-started/configuration.md @@ -10,13 +10,13 @@ This document describes the configuration file parameters for MantisTable UI. Th 1. **Copy the Example File** - First, make a copy of the `example.env` file. This can be done using the following command in a terminal: + First, make a copy of the `.env.example.prod` file. This can be done using the following command in a terminal: ```sh - cp example.env .env + cp .env.example.prod .env ``` - The `example.env` is located inside the main folder: + The `.env.example.prod` is located inside the main folder: ```bash mantistable-ui @@ -28,7 +28,8 @@ This document describes the configuration file parameters for MantisTable UI. Th ├── components.json ├── docker-compose.yml ├── drizzle.config.ts - ├── example.env + ├── .env.example.dev + ├── .env.example.prod . ``` @@ -37,40 +38,45 @@ This document describes the configuration file parameters for MantisTable UI. Th Open the newly created `.env` file in a text editor of your choice. Modify the parameters as needed to fit your environment. Below is an example of how the file might look after customization: ```env - DATABASE_URL="postgresql://postgres:yourpassword@yourhost:5432/yourdatabase" - POSTGRESQL_PASS="your_postgres_password" - POSTGRES_HOST="your_postgres_host" - POSTGRES_PORT="your_postgres_port" - POSTGRES_DB="your_postgres_db" - POSTGRES_USER="your_postgres_user" - - MONGO_INITDB_ROOT_USERNAME="your_mongo_username" - MONGO_INITDB_ROOT_PASSWORD="your_mongo_password" - - STI_HOST="http://your.sti.host:port" - - PLUGINS_PORT="your_plugins_port" + # PostgreSQL + POSTGRESQL_PASS="..." + POSTGRES_HOST="..." + POSTGRES_PORT="..." + POSTGRES_DB="..." + POSTGRES_USER="..." + + # MongoDB + MONGO_INITDB_ROOT_USERNAME="..." + MONGO_INITDB_ROOT_PASSWORD="..." + + # STI + STI_HOST_UPLOAD_AND_PROCESS="..." + STI_HOST_DOWNLOAD_ANNOTATIONS="..." + + # Plugins + PLUGINS_HOST="..." + NEXT_PUBLIC_PLUGINS_HOST="..." + PLUGINS_PORT="..." + + # Mantis + MANTIS_OUTSIDEPORT="..." + + # Mantistablex Variables + GPT_KEY="..." + GPT_ENDPOINT="..." ``` ## Configuration Parameters ### PostgreSQL -- **`POSTGRESQL_URL`** - - ```plaintext - DATABASE_URL="postgresql://postgres:password@localhost:port/mantistableui" - ``` - - Specifies the URL for connecting to the PostgreSQL database. - - **`POSTGRESQL_PASS`** ```plaintext POSTGRESQL_PASS="password!" ``` - The password for the PostgreSQL database user. This should match the password specified in the `DATABASE_URL`. + The password for the PostgreSQL database. - **`POSTGRES_HOST`** @@ -78,15 +84,15 @@ This document describes the configuration file parameters for MantisTable UI. Th POSTGRES_HOST="localhost" ``` - The hostname or IP address of the PostgreSQL server. + The hostname or IP address of the PostgreSQL server. The service is available also inside the Docker Network, so it's possible to specify `postgres`, which is the name of the service. -- **`POSTGRES_POR`** +- **`POSTGRES_PORT`** ```plaintext POSTGRES_PORT="5432" ``` - The port on which the PostgreSQL server is running. + The port on which the PostgreSQL server is running. `5432` is the default one. - **`POSTGRES_DB`** @@ -124,13 +130,21 @@ This document describes the configuration file parameters for MantisTable UI. Th ### External STI Services -- **`STI_HOST`** +- **`STI_HOST_UPLOAD_AND_PROCESS`** + + ```plaintext + STI_HOST_UPLOAD_AND_PROCESS="http://sti_upload_and_process:5000" + ``` + + The address of the STI endpoint available to upload and process a single table. Replace this with the appropriate URL for your setup. + + - **`STI_HOST_DOWNLOAD_ANNOTATIONS`** ```plaintext - STI_HOST="http://local:5042" + STI_HOST_DOWNLOAD_ANNOTATIONS="http://sti_download:5000" ``` - Specifies the hostname, URL or IP address for an STI service. Replace this with the appropriate URL for your setup. + The address of the STI endpoint available to download the annotated table. Replace this with the appropriate URL for your setup. :::warning The external Semantic Table Interpretation service must expose APIs as indicated in [External STI Approach](/docs/sti/external-sti-approach) page. @@ -138,34 +152,91 @@ This document describes the configuration file parameters for MantisTable UI. Th ### Plugins +- **`PLUGINS_HOST`** + + ```plaintext + PLUGINS_HOST="http://api:5000" + ``` + + The address of the plugin service. The address specified here is internal in the docker network, such as `http://api:5000`, which specify the container name. + +- **`NEXT_PUBLIC_PLUGINS_HOST`** + + ```plaintext + NEXT_PUBLIC_PLUGINS_HOST="http://localhost:5001" + ``` + + The address of the plugin service. The address specified here is external to the docker network, such as `http://localhost:5001`, because it is used in the frontend module. + - **`PLUGINS_PORT`** ```plaintext - PLUGINS_PORT="port" + PLUGINS_PORT="5001" ``` The port on which the plugins service (Doker container) will run. -## Configuration File Example +### MantisTable -Below is a complete example of a configuration file with all parameters: +- **`MANTIS_OUTSIDEPORT`** -```plaintext -DATABASE_URL="postgresql://postgres:password!@localhost:5432/mantistableui" -POSTGRESQL_PASS="password!" -POSTGRES_HOST="localhost" -POSTGRES_PORT="5432" -POSTGRES_DB="mantistableui" -POSTGRES_USER="postgres" + ```plaintext + MANTIS_OUTSIDEPORT="3000" + ``` + + The port on which the frontend platform service (Doker container) will run. + +### Mantistablex + +- **`GPT_KEY`** + + ```plaintext + GPT_KEY="..." + ``` + + The private key from GPT APIs used to execute the mantistablex plugin. + +- **`GPT_ENDPOINT`** + + ```plaintext + GPT_ENDPOINT="..." + ``` -MONGO_INITDB_ROOT_USERNAME="root" -MONGO_INITDB_ROOT_PASSWORD="password!" + The endpoint from GPT APIs used to execute the mantistablex plugin. -STI_HOST="http://unimib.it:5042" +## Configuration File Example -PLUGINS_PORT="5001" +Below is a complete example of a configuration file with all parameters: + +```plaintext + # PostgreSQL + POSTGRESQL_PASS="table_ui2024!" + POSTGRES_HOST="postgres" + POSTGRES_PORT="5432" + POSTGRES_DB="postgrestableui" + POSTGRES_USER="postgres" + + # MongoDB + MONGO_INITDB_ROOT_USERNAME="root" + MONGO_INITDB_ROOT_PASSWORD="export2024!" + + # STI + STI_HOST_UPLOAD_AND_PROCESS="https://selbat.datai.disco.unimib.it/dataset/createWithArray?token=selBat_demo_2023" + STI_HOST_DOWNLOAD_ANNOTATIONS="http://vm.chronos.disco.unimib.it/dataset/${datasetName}/table/${table_id}?token=alligator_demo_2023" + + # Plugins + PLUGINS_HOST="http://api:5000" + NEXT_PUBLIC_PLUGINS_HOST="http://localhost:5001" + PLUGINS_PORT="5001" + + # Mantis + MANTIS_OUTSIDEPORT="3000" + + # Mantistablex Variables + GPT_KEY="..." + GPT_ENDPOINT="..." ``` ## Conclusion -The configuration file for MantisTable UI is **crucial** for ensuring proper connections to databases and external services. By correctly setting these parameters, you ensure the smooth operation and integration of MantisTable UI within your environment. \ No newline at end of file +The configuration file for MantisTable UI is **crucial** for ensuring proper connections to databases and external services. By correctly setting these parameters, you ensure the smooth operation and integration of MantisTable UI within your environment. diff --git a/docs/plug-in/add-on.md b/docs/plug-in/add-on.md index 305f390..425b302 100644 --- a/docs/plug-in/add-on.md +++ b/docs/plug-in/add-on.md @@ -3,3 +3,95 @@ sidebar_position: 2 --- # Add-on + +Add-on plugins enable MantisTable UI to perform arbitrary tasks on the annotated table. An example of `add-on` plugin is the lexicalization of the annotated table (`mantistablex`). + +### General Guidelines + +1. **Isolation:** Each plugin should operate independently and not interfere with other plugins or the core functionality of MantisTable UI. +2. **Configuration:** Plugins should allow configuration through environment variables or configuration files. +3. **Documentation:** Provide clear documentation for each plugin, including usage instructions, dependencies, and configuration options. + +### Export Plugin Example + +**Structure:** + +``` +/addon-plugin-example + mantistablex.py + config.json + input.json + inputData.json + input.html + output.html + README.md +``` + +**export.py:** + +```python +### Read Annotated Table +table_input_path = f"{os.path.dirname(os.path.realpath(__file__))}/input.json" +file_input = open(table_input_path, encoding="utf-8") +original_table = json.load(file_input) +# Read Input Data for the Plugin +input_data_path = f"{os.path.dirname(os.path.realpath(__file__))}/inputData.json" +file_input_data = open(input_data_path, encoding="utf-8") +input_data: dict = json.load(file_input_data) + +# ADD-ON CODE + +# Write output on output.html +output_file_path = f"{os.path.dirname(os.path.realpath(__file__))}/output.html" +``` + +**config.json:** + +```json +{ + "type": "addon", + "name": "mantistablex", + "description": "MantisTableX plugin converts structured data into natural language descriptions. Given an input table with annotations specifying the relationships and significance of its data, the tool generates coherent textual summaries or narratives that convey the table's information in readable and contextually appropriate sentences.", + "entry_file": "mantistablex.py", + "output": "output.html" +} +``` + +The entry file is specified by the `entry_file` field in the `config.json` file and it represents the entry point of the plugin, because it is the file that is executed to apply the plugin. Once the plugin is selected, the annotated table (defined in the output of the STI sytem ~ [External STI Approach](/docs/sti/external-sti-approach)) is injected in the `input.json` file. In this case, there is also the possibility to define an html form to enable the user to send input data to the add-on plugin for its execution. To add input data, it's important to define the form in the `input.html` file using a `
` as root element. Then, the `entry_file` is applied using the `subprocess` module in python and, finally, the output is read in the file defined by the `config.json` using the `output` field (usually defined as `output.html`) in the root folder. For this reason it is very important to use the following structure to identify the correct file path in the plugin: + +```python +table_input_path = f"{os.path.dirname(os.path.realpath(__file__))}/input.json" # input table +input_data_path = f"{os.path.dirname(os.path.realpath(__file__))}/inputData.json" # input data +output_file_path = f"{os.path.dirname(os.path.realpath(__file__))}/output.html" # output html +``` + +The plugin is executed on an isolated environment, so you can't install your own libraries but some are already installed: + +``` +tqdm +rdflib +pandas +numpy +requests +psycopg2 +pymongo +python-multipart +``` + +Add-on plugins are very powerful because the definition of the UI to display the data entry form and the plugin's output is left to the plugin developer, who will define the UI using HTML Markup language and inline CSS to style the content. + +## Using Plugins in MantisTable UI + +1. **Install the Plugin:** + + - Place the plugin files in the appropriate directory as specified in the configuration file. + - Ensure all dependencies are installed and properly configured. + +2. **Configure the Plugin:** + + - Update the `config.json` file with the necessary configuration parameters. + +3. **Access Plugin Functionality:** + - Export plugins can be accessed through the export menu in the UI. + - Add-on plugins are typically accessed through semantic processing options. + - Transformation plugins can be applied through the data transformation tools in the UI. diff --git a/docs/plug-in/export.md b/docs/plug-in/export.md index a41174a..4cc573f 100644 --- a/docs/plug-in/export.md +++ b/docs/plug-in/export.md @@ -18,45 +18,68 @@ Export plugins enable MantisTable UI to export data and annotation in user-defin ``` /export-plugin-example - /src - index.js - /config - config.json - /docs - README.md + export.py + config.json + input.json + output/ + kg.json + README.md ``` -**index.js:** +**export.py:** -```javascript -module.exports = { - export: (tableData) => { - // Implement custom export functionality here - return customFormattedData; - } -}; +```python +file_path = f"{os.path.dirname(os.path.realpath(__file__))}/input.json" +file_open = open(file_path, encoding="utf-8") +data = json.load(file_open) +# EXPORT CODE +export_file_path = f"{os.path.dirname(os.path.realpath(__file__))}/output/kg.json" ``` **config.json:** ```json { - "format": "custom-format", - "description": "Export plugin for custom format" + "type": "export", + "output": "kg.json", + "entry_file": "export.py", + "name": "json", + "description": "The 'json' Export Plugin allows users to export annotated tables from Wikidata into JSON triples. This plugin processes tables annotated with Wikidata properties and values, converting them into a structured JSON format that represents each cell's data as subject-predicate-object triples. It simplifies data extraction and integration for developers and researchers working with linked data and semantic web applications" } ``` +The entry file is specified by the `entry_file` field in the `config.json` file and it represents the entry point of the plugin, because it is the file that is executed to apply the plugin. Once the plugin is selected, the annotated table (defined in the output of the STI sytem ~ [External STI Approach](/docs/sti/external-sti-approach)) is injected in the `input.json` file. Then, the `entry_file` is applied using the `subprocess` module in python and, finally, the output is read in the file defined by the `config.json` using the `output` field in the `output` directory. For this reason it is very important to use the following structure to identify the correct file path in the plugin: + +```python +file_path = f"{os.path.dirname(os.path.realpath(__file__))}/input.json" # input +export_file_path = f"{os.path.dirname(os.path.realpath(__file__))}/output/kg.json" # output +``` + +The plugin is executed on an isolated environment, so you can't install your own libraries but some are already installed: + +``` +tqdm +rdflib +pandas +numpy +requests +psycopg2 +pymongo +python-multipart +``` + ## Using Plugins in MantisTable UI 1. **Install the Plugin:** + - Place the plugin files in the appropriate directory as specified in the configuration file. - Ensure all dependencies are installed and properly configured. 2. **Configure the Plugin:** - - Update the `.env` file with the necessary configuration parameters. - - Restart MantisTable UI to load the new plugins. + + - Update the `config.json` file with the necessary configuration parameters. 3. **Access Plugin Functionality:** - Export plugins can be accessed through the export menu in the UI. - Add-on plugins are typically accessed through semantic processing options. - - Transformation plugins can be applied through the data transformation tools in the UI. \ No newline at end of file + - Transformation plugins can be applied through the data transformation tools in the UI. diff --git a/docs/sti/external-sti-approach.md b/docs/sti/external-sti-approach.md index 44284d5..fe4f185 100644 --- a/docs/sti/external-sti-approach.md +++ b/docs/sti/external-sti-approach.md @@ -4,7 +4,7 @@ sidebar_position: 1 # External STI Approach -MantisTable UI can interacts with an API to integrate an external STI approach. This document describes the API endpoints, their methods, and how to configure MantisTable UI to use these endpoints. +MantisTable UI can interacts with an API to integrate an external STI approach. This document describes the API endpoints, their methods, and how to configure MantisTable UI to use these endpoints. :::warning In order to use your own STI approach you need to implement an API by following the instructions on this page. @@ -21,16 +21,16 @@ The API must consists of two main endpoints: The URL of the API endpoint and the names of the methods are configurable through the MantisTable UI configuration file (`.env`). The relevant configuration parameters are: -- `API_BASE_URL`: the base URL of the API. -- `SEMANTIC_INTERPRETATION_METHOD`: the method name for performing semantic table interpretation. -- `GET_ANNOTATIONS_METHOD`: the method name for retrieving table annotations. +- `STI_HOST_UPLOAD_AND_PROCESS`: the endpoint name for performing semantic table interpretation. +- `STI_HOST_DOWNLOAD_ANNOTATIONS`: the endpoint name for retrieving table annotations. ### Example Configuration ```js -API_BASE_URL="http://api.mantistable.com" -SEMANTIC_INTERPRETATION_METHOD="/semantic-interpretation" -GET_ANNOTATIONS_METHOD="/get-annotations" +SEMANTIC_INTERPRETATION_METHOD = + "https://selbat.datai.disco.unimib.it/dataset/createWithArray?token=selBat_demo_2023"; +GET_ANNOTATIONS_METHOD = + "http://vm.chronos.disco.unimib.it/dataset/${datasetName}/table/${table_id}?token=alligator_demo_2023"; ``` :::warning @@ -51,40 +51,82 @@ To modify the configuration file, refer to the page [Configuration](/docs/gettin **Request Body:** -```js +The data format that must be used to send the table to the STI service is: + +```typescript { - "table": [ - ["Column1", "Column2", "Column3"], - ["Value1", "Value2", "Value3"], - ... - ] + datasetName: string; + tableName: string; + header: string[]; + rows: { + idRow: number; + data: string[]; + }[]; + kgReference: string; } ``` -**Response:** +An example of table formatted for the annotation process is: ```js { - "status": "success", - "message": "Table interpreted successfully", - "data": { - "table_id": "unique_table_identifier" + "datasetName": "mantisTables", + "tableName": "table", + "header": ["col0", "col1"], + "rows": [ + { + "idRow": 1, + "data": ["garraf", "3"] + }, + { + "idRow": 2, + "data": ["beroun", ""] + }, + { + "idRow": 3, + "data": ["prat de llobregat", "2"] + }, + { + "idRow": 4, + "data": ["abisko turiststation", "1"] + } + ], + "kgReference": "wikidata" } +``` + +**Response:** + +```js +{ + "status": "Ok", + "tables": [ + { + "id": "66cdd62f961248695b90fc6b", + "datasetName": "mantisTables", + "tableName": "table" + } + ] } ``` **Example Request:** ```sh -curl -X POST http://api.mantistable.com/semantic-interpretation \ --H "Content-Type: application/json" \ --d '{ - "table": [ - ["Name", "Age", "Occupation"], - ["John Doe", "29", "Engineer"], - ["Jane Smith", "34", "Artist"] - ] -}' +curl -X 'POST' \ + 'https://selbat.datai.disco.unimib.it/dataset/createWithArray?token=selBat_demo_2023' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '[ + { + "datasetName": "mantisTables", + "tableName": "table", + "header": [ + "col0", + "col1" + ], + "rows": [ + ... ``` **Example Response:** @@ -94,7 +136,14 @@ curl -X POST http://api.mantistable.com/semantic-interpretation \ "status": "success", "message": "Table interpreted successfully", "data": { - "table_id": "1234567890abcdef" + "status": "Ok", + "tables": [ + { + "id": "66cdd62f961248695b90fc6b", + "datasetName": "mantisTables", + "tableName": "table" + } + ] } } ``` @@ -111,24 +160,699 @@ curl -X POST http://api.mantistable.com/semantic-interpretation \ **Request Parameters:** +- `datasetName`: the name of the dataset to which the table belongs. - `table_id`: The unique identifier of the table. **Response:** +The data format that must be used to receive the annotated table from the STI service is: + +```typescript +{ + data: { + datasetName: string; + tableName: string; + header: string[]; + rows: { + idRow: number; + data: string[]; + }[], + semanticAnnotations: { + cea: { + idColumn: number; + idRow: number; + entities: { + id: string; + name: string; + description: string; + match: boolean; + score: number; + features: { + id: string; + value: number; + }[]; + types: { + id: string; + name: string; + }[]; + }[]; + }[] + cta: { + idColumn: number; + types: { + id: string; + name: string; + score: number; + }[]; + }[] + cpa: { + idSourceColumn: string; + idTargetColumn: string; + predicates: { + id: string; + name: string; + score: number; + }[]; + }[] + }, + metadata: { + column: { + idColumn: number; + tag: string; + }[]; + }, + status: "TODO" | "DOING" | "DONE"; + nrows: number; +}; +} +``` + +An example of annotated table returned from the STI process is: + ```js { "status": "success", "data": { - "table": [ - ["Column1", "Column2", "Column3"], - ["Value1", "Value2", "Value3"], - ... - ], - "annotations": [ - {"column": "Column1", "type": "Name", "confidence": 0.95}, - {"column": "Column2", "type": "Age", "confidence": 0.98}, - ... - ] + "data": { + "datasetName": "mantisTables", + "tableName": "1", + "header": ["col0", "col1"], + "rows": [ + { + "idRow": 1, + "data": ["garraf", "3"] + }, + { + "idRow": 2, + "data": ["beroun", ""] + }, + { + "idRow": 3, + "data": ["prat de llobregat", "2"] + }, + { + "idRow": 4, + "data": ["abisko turiststation", "1"] + } + ], + "semanticAnnotations": { + "cea": [ + { + "idColumn": 0, + "idRow": 1, + "entities": [ + { + "id": "Q2204492", + "name": "garraf", + "types": [ + { + "id": "Q55488", + "name": "railway station" + } + ], + "description": "railway station in sitges, spain", + "match": true, + "score": 0.979, + "features": [ + { + "id": "delta", + "value": 0.721 + }, + { + "id": "omega", + "value": 0.824 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + }, + { + "id": "Q456084", + "name": "garraf", + "types": [ + { + "id": "Q40080", + "name": "beach" + }, + { + "id": "Q3744088", + "name": "tourism region" + }, + { + "id": "Q93352", + "name": "coast" + } + ], + "description": "tourism region in spain", + "match": false, + "score": 0.258, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + }, + { + "id": "Q5523785", + "name": "garraf", + "types": [ + { + "id": "Q3055118", + "name": "single entity of population" + } + ], + "description": "human settlement in sitges, garraf, penedès, spain", + "match": false, + "score": 0.14, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 1, + "entities": [] + }, + { + "idColumn": 0, + "idRow": 2, + "entities": [ + { + "id": "Q9169473", + "name": "beroun", + "types": [ + { + "id": "Q4167410", + "name": "Wikimedia disambiguation page" + } + ], + "description": "wikimedia disambiguation page", + "match": true, + "score": 0.98, + "features": [ + { + "id": "delta", + "value": 0.061 + }, + { + "id": "omega", + "value": 0.429 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q47497586", + "name": "beroun", + "types": [ + { + "id": "Q13433827", + "name": "encyclopedia article" + } + ], + "description": "encyclopedia article", + "match": false, + "score": 0.919, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q23903289", + "name": "beroun", + "types": [ + { + "id": "Q13433827", + "name": "encyclopedia article" + } + ], + "description": "encyclopedia article", + "match": false, + "score": 0.903, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 2, + "entities": [] + }, + { + "idColumn": 0, + "idRow": 3, + "entities": [ + { + "id": "Q1894527", + "name": "prat de llobregat", + "types": [ + { + "id": "Q55488", + "name": "railway station" + }, + { + "id": "Q55491", + "name": "underground railway station" + }, + { + "id": "Q22808403", + "name": "underground station" + } + ], + "description": "railway station in el prat de llobregat, catalonia, spain", + "match": true, + "score": 0.979, + "features": [ + { + "id": "delta", + "value": 0.06 + }, + { + "id": "omega", + "value": 0.428 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + }, + { + "id": "Q15619", + "name": "prat de llobregat", + "types": [ + { + "id": "Q33146843", + "name": "municipality of Catalonia" + } + ], + "description": "municipality in baix llobregat, catalonia, spain", + "match": false, + "score": 0.919, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.07 + } + ] + }, + { + "id": "Q23932050", + "name": "prat de llobregat", + "types": [ + { + "id": "Q3305213", + "name": "painting" + } + ], + "description": "painting by francesc gimeno", + "match": false, + "score": 0.643, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 3, + "entities": [] + }, + { + "idColumn": 0, + "idRow": 4, + "entities": [ + { + "id": "Q4412158", + "name": "abisko turiststation", + "types": [ + { + "id": "Q55488", + "name": "railway station" + } + ], + "description": "railway station in norrbotten county, sweden", + "match": true, + "score": 0.978, + "features": [ + { + "id": "delta", + "value": 0.058 + }, + { + "id": "omega", + "value": 0.426 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q66737990", + "name": "abisko turiststation", + "types": [ + { + "id": "Q1406318", + "name": "holiday village" + }, + { + "id": "Q27686", + "name": "hotel" + } + ], + "description": "tourist station in kiruna, sweden", + "match": false, + "score": 0.92, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q10542058", + "name": "jävre turiststation", + "types": [ + { + "id": "Q1497375", + "name": "architectural ensemble" + } + ], + "description": "none", + "match": false, + "score": 0.004, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 0.7 + }, + { + "id": "jaccard_distance", + "value": 0.5 + }, + { + "id": "popularity", + "value": 0 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 4, + "entities": [] + } + ], + "cpa": [ + { + "idSourceColumn": 0, + "idTargetColumn": 1, + "predicates": [ + { + "id": "P5595", + "name": "number of platform faces", + "score": 0.75 + }, + { + "id": "P1103", + "name": "number of platform tracks", + "score": 0.525 + }, + { + "id": "P2044", + "name": "elevation above sea level", + "score": 0.138 + }, + { + "id": "P1538", + "name": "number of households", + "score": 0.018 + }, + { + "id": "P2046", + "name": "area", + "score": 0.015 + } + ] + } + ], + "cta": [ + { + "idColumn": 0, + "types": [ + { + "id": "Q55488", + "name": "railway station", + "score": 0.75 + }, + { + "id": "Q3055118", + "name": "single entity of population", + "score": 0.25 + }, + { + "id": "Q40080", + "name": "beach", + "score": 0.25 + }, + { + "id": "Q3744088", + "name": "tourism region", + "score": 0.25 + }, + { + "id": "Q93352", + "name": "coast", + "score": 0.25 + }, + { + "id": "Q4167410", + "name": "Wikimedia disambiguation page", + "score": 0.25 + }, + { + "id": "Q13433827", + "name": "encyclopedia article", + "score": 0.25 + }, + { + "id": "Q55491", + "name": "underground railway station", + "score": 0.25 + }, + { + "id": "Q22808403", + "name": "underground station", + "score": 0.25 + }, + { + "id": "Q33146843", + "name": "municipality of Catalonia", + "score": 0.25 + } + ] + } + ] + }, + "metadata": { + "column": [ + { + "idColumn": 0, + "tag": "SUBJ" + }, + { + "idColumn": 1, + "tag": "LIT" + } + ] + }, + "status": "DONE" + }, + "pagination": { + "currentPage": 1, + "perPage": 10, + "totalPages": 1, + "totalItems": 4 + } } } ``` @@ -136,8 +860,9 @@ curl -X POST http://api.mantistable.com/semantic-interpretation \ **Example Request:** ```sh -curl -X GET "http://api.mantistable.com/get-annotations?table_id=1234567890abcdef" \ --H "Content-Type: application/json" +curl -X 'GET' \ + 'https://selbat.datai.disco.unimib.it/dataset/mantisTables/table/table?page=1&per_page=10&token=selBat_demo_2023' \ + -H 'accept: application/json' ``` **Example Response:** @@ -146,20 +871,630 @@ curl -X GET "http://api.mantistable.com/get-annotations?table_id=1234567890abcde { "status": "success", "data": { - "table": [ - ["Name", "Age", "Occupation"], - ["John Doe", "29", "Engineer"], - ["Jane Smith", "34", "Artist"] - ], - "annotations": [ - {"column": "Name", "type": "Person Name", "confidence": 0.95}, - {"column": "Age", "type": "Age", "confidence": 0.98}, - {"column": "Occupation", "type": "Job Title", "confidence": 0.92} - ] + "data": { + "datasetName": "mantisTables", + "tableName": "1", + "header": ["col0", "col1"], + "rows": [ + { + "idRow": 1, + "data": ["garraf", "3"] + }, + { + "idRow": 2, + "data": ["beroun", ""] + }, + { + "idRow": 3, + "data": ["prat de llobregat", "2"] + }, + { + "idRow": 4, + "data": ["abisko turiststation", "1"] + } + ], + "semanticAnnotations": { + "cea": [ + { + "idColumn": 0, + "idRow": 1, + "entities": [ + { + "id": "Q2204492", + "name": "garraf", + "types": [ + { + "id": "Q55488", + "name": "railway station" + } + ], + "description": "railway station in sitges, spain", + "match": true, + "score": 0.979, + "features": [ + { + "id": "delta", + "value": 0.721 + }, + { + "id": "omega", + "value": 0.824 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + }, + { + "id": "Q456084", + "name": "garraf", + "types": [ + { + "id": "Q40080", + "name": "beach" + }, + { + "id": "Q3744088", + "name": "tourism region" + }, + { + "id": "Q93352", + "name": "coast" + } + ], + "description": "tourism region in spain", + "match": false, + "score": 0.258, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + }, + { + "id": "Q5523785", + "name": "garraf", + "types": [ + { + "id": "Q3055118", + "name": "single entity of population" + } + ], + "description": "human settlement in sitges, garraf, penedès, spain", + "match": false, + "score": 0.14, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 1, + "entities": [] + }, + { + "idColumn": 0, + "idRow": 2, + "entities": [ + { + "id": "Q9169473", + "name": "beroun", + "types": [ + { + "id": "Q4167410", + "name": "Wikimedia disambiguation page" + } + ], + "description": "wikimedia disambiguation page", + "match": true, + "score": 0.98, + "features": [ + { + "id": "delta", + "value": 0.061 + }, + { + "id": "omega", + "value": 0.429 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q47497586", + "name": "beroun", + "types": [ + { + "id": "Q13433827", + "name": "encyclopedia article" + } + ], + "description": "encyclopedia article", + "match": false, + "score": 0.919, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q23903289", + "name": "beroun", + "types": [ + { + "id": "Q13433827", + "name": "encyclopedia article" + } + ], + "description": "encyclopedia article", + "match": false, + "score": 0.903, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 2, + "entities": [] + }, + { + "idColumn": 0, + "idRow": 3, + "entities": [ + { + "id": "Q1894527", + "name": "prat de llobregat", + "types": [ + { + "id": "Q55488", + "name": "railway station" + }, + { + "id": "Q55491", + "name": "underground railway station" + }, + { + "id": "Q22808403", + "name": "underground station" + } + ], + "description": "railway station in el prat de llobregat, catalonia, spain", + "match": true, + "score": 0.979, + "features": [ + { + "id": "delta", + "value": 0.06 + }, + { + "id": "omega", + "value": 0.428 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.01 + } + ] + }, + { + "id": "Q15619", + "name": "prat de llobregat", + "types": [ + { + "id": "Q33146843", + "name": "municipality of Catalonia" + } + ], + "description": "municipality in baix llobregat, catalonia, spain", + "match": false, + "score": 0.919, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0.07 + } + ] + }, + { + "id": "Q23932050", + "name": "prat de llobregat", + "types": [ + { + "id": "Q3305213", + "name": "painting" + } + ], + "description": "painting by francesc gimeno", + "match": false, + "score": 0.643, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 3, + "entities": [] + }, + { + "idColumn": 0, + "idRow": 4, + "entities": [ + { + "id": "Q4412158", + "name": "abisko turiststation", + "types": [ + { + "id": "Q55488", + "name": "railway station" + } + ], + "description": "railway station in norrbotten county, sweden", + "match": true, + "score": 0.978, + "features": [ + { + "id": "delta", + "value": 0.058 + }, + { + "id": "omega", + "value": 0.426 + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q66737990", + "name": "abisko turiststation", + "types": [ + { + "id": "Q1406318", + "name": "holiday village" + }, + { + "id": "Q27686", + "name": "hotel" + } + ], + "description": "tourist station in kiruna, sweden", + "match": false, + "score": 0.92, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 1 + }, + { + "id": "jaccard_distance", + "value": 1 + }, + { + "id": "popularity", + "value": 0 + } + ] + }, + { + "id": "Q10542058", + "name": "jävre turiststation", + "types": [ + { + "id": "Q1497375", + "name": "architectural ensemble" + } + ], + "description": "none", + "match": false, + "score": 0.004, + "features": [ + { + "id": "delta", + "value": null + }, + { + "id": "omega", + "value": null + }, + { + "id": "levenshtein_distance", + "value": 0.7 + }, + { + "id": "jaccard_distance", + "value": 0.5 + }, + { + "id": "popularity", + "value": 0 + } + ] + } + ] + }, + { + "idColumn": 1, + "idRow": 4, + "entities": [] + } + ], + "cpa": [ + { + "idSourceColumn": 0, + "idTargetColumn": 1, + "predicates": [ + { + "id": "P5595", + "name": "number of platform faces", + "score": 0.75 + }, + { + "id": "P1103", + "name": "number of platform tracks", + "score": 0.525 + }, + { + "id": "P2044", + "name": "elevation above sea level", + "score": 0.138 + }, + { + "id": "P1538", + "name": "number of households", + "score": 0.018 + }, + { + "id": "P2046", + "name": "area", + "score": 0.015 + } + ] + } + ], + "cta": [ + { + "idColumn": 0, + "types": [ + { + "id": "Q55488", + "name": "railway station", + "score": 0.75 + }, + { + "id": "Q3055118", + "name": "single entity of population", + "score": 0.25 + }, + { + "id": "Q40080", + "name": "beach", + "score": 0.25 + }, + { + "id": "Q3744088", + "name": "tourism region", + "score": 0.25 + }, + { + "id": "Q93352", + "name": "coast", + "score": 0.25 + }, + { + "id": "Q4167410", + "name": "Wikimedia disambiguation page", + "score": 0.25 + }, + { + "id": "Q13433827", + "name": "encyclopedia article", + "score": 0.25 + }, + { + "id": "Q55491", + "name": "underground railway station", + "score": 0.25 + }, + { + "id": "Q22808403", + "name": "underground station", + "score": 0.25 + }, + { + "id": "Q33146843", + "name": "municipality of Catalonia", + "score": 0.25 + } + ] + } + ] + }, + "metadata": { + "column": [ + { + "idColumn": 0, + "tag": "SUBJ" + }, + { + "idColumn": 1, + "tag": "LIT" + } + ] + }, + "status": "DONE" + }, + "pagination": { + "currentPage": 1, + "perPage": 10, + "totalPages": 1, + "totalItems": 4 + } } } ``` ## Conclusion -The API provides essential functionalities for semantic table interpretation and retrieval of annotations, which are crucial for the MantisTable UI. Proper configuration of the API endpoints ensures seamless interaction between the MantisTable UI and the external API. \ No newline at end of file +The API provides essential functionalities for semantic table interpretation and retrieval of annotations, which are crucial for the MantisTable UI. Proper configuration of the API endpoints ensures seamless interaction between the MantisTable UI and the external API.