diff --git a/README.md b/README.md index 44528ac02e..2033d2bf1d 100644 --- a/README.md +++ b/README.md @@ -210,3 +210,4 @@ YourKit supports open source projects with its full-featured Java Profiler. YourKit, LLC is the creator of YourKit Java Profiler and YourKit .NET Profiler, innovative and intelligent tools for profiling Java and .NET applications. + diff --git a/docs/src/paradox/00-release-notes/next.md b/docs/src/paradox/00-release-notes/next.md index 9bc0b65d9a..ec493158c8 100644 --- a/docs/src/paradox/00-release-notes/next.md +++ b/docs/src/paradox/00-release-notes/next.md @@ -24,7 +24,17 @@ Also, please change the **HINT** to the appropriate level: - MAJOR: Separate the `knora-admin` ontology from `knora-api` (@github[#1263](#1263)). Existing repositories must be updated; see `upgrade/1263-knora-admin` for instructions. - + - FEATURE: Add support for searching for specific list values in Gravsearch for both the simple and complex schema (@github[#1314](#1314)). - REFACTOR: List value responses contain the list node's label in the simple schema only (@github[#1321](#1321)) + +- MAJOR: Change API v1 file uploads to work like API v2 (@github[#1233](#1233)). To enable + Knora and Sipi to work without sharing a filesystem, the procedure + for uploading files in API v1 has changed; see + @ref:[Adding Resources with Image Files](../03-apis/api-v1/adding-resources.md#adding-resources-with-image-files) + and @ref:[Bulk Import with Image Files](../03-apis/api-v1/adding-resources.md#bulk-import-with-image-files). + Also, for consistency with API v2 responses, the `temporaryBaseIIIFUrl` returned by the Sipi + `/upload` route no longer includes the image filename; see + @ref:[Upload Files to Sipi](../03-apis/api-v2/editing-values.md#upload-files-to-sipi). + \ No newline at end of file diff --git a/docs/src/paradox/03-apis/api-v1/adding-resources.md b/docs/src/paradox/03-apis/api-v1/adding-resources.md index 5140ff6d96..68fd095946 100644 --- a/docs/src/paradox/03-apis/api-v1/adding-resources.md +++ b/docs/src/paradox/03-apis/api-v1/adding-resources.md @@ -49,67 +49,53 @@ The request header's content type has to be set to `application/json`. ## Adding Resources with Image Files -Certain resource classes can have attached image files. There are two ways to -attach a file to a resource: Either by submitting directly the binaries of the file in a -an HTTP Multipart request, or by indicating the location of the file. The two cases are referred to -as non-GUI case and GUI case (see @ref:[Sipi and Knora](../../07-sipi/sipi-and-knora.md)). - -### Including the binaries (non-GUI case) - -In order to include the binaries, a HTTP Multipart request has to be -sent. One part contains the JSON (same format as described for -[Adding Resources Without Images Files](#adding-resources-without-a-digital-representation)) -and has to be named `json`. The other part contains the file's name, its binaries, and its mime type -and has to be named `file`. The following example illustrates how to -make this type of request using Python 3: - -```python -#!/usr/bin/env python3 - -import requests, json - -# a Python dictionary that will be turned into a JSON object -resourceParams = { - 'restype_id': 'http://www.knora.org/ontology/test#testType', - 'properties': { - 'http://www.knora.org/ontology/test#testtext': [ - {'richtext_value': {'utf8str': "test"}} - ], - 'http://www.knora.org/ontology/test#testnumber': [ - {'int_value': 1} - ] - }, - 'label': "test resource", - 'project_id': 'http://rdfh.ch/projects/testproject' -} - -# the name of the file to be submitted -filename = "myimage.jpg" +The first step is to upload an image file to Sipi, using a +`multipart/form-data` request, where `sipihost` represents the host and +port on which Sipi is running: -# a tuple containing the file's name, its binaries and its mimetype -file = {'file': (filename, open(filename, 'rb'), "image/jpeg")} # use name "file" +``` +HTTP POST to http://sipihost/upload?token=TOKEN +``` -# do a POST request providing both the JSON and the binaries -r = requests.post("http://host/v1/resources", - data={'json': json.dumps(resourceParams)}, # use name "json" - files=file, - auth=('user', 'password')) +The `TOKEN` is the `sid` returned by Knora in response to the +client's login request (see @ref:[Authentication](authentication.md)). +The request must contain a body part providing the file as well as a parameter +`filename`, providing the file's original filename, which both Knora and Sipi will +store; these filenames can be descriptive and need not be unique. + +Sipi will then convert the uploaded image file to JPEG 2000 format and store +it in a temporary location. If this is successful, it will return a JSON +response that looks something like this: + +```json +{ + "uploadedFiles": [{ + "originalFilename": "manuscript-1234-page-1.tiff", + "internalFilename": "3UIsXH9bP0j-BV0D4sN51Xz.jp2", + "temporaryBaseIIIFUrl": "http://sipihost/tmp" + }] +} ``` -Please note that the file has to be read in binary mode (by default it -would be read in text mode). +This provides: -### Indicating the location of a file (GUI case) +- the `originalFilename`, which we submitted when uploading the file +- the unique `internalFilename` that Sipi has randomly generated for the file +- the `temporaryBaseIIIFUrl`, which we can use to construct a IIIF URL for + previewing the file -This request works similarly to -[Adding Resources Without Image Files](#adding-resources-without-a-digital-representation). The JSON format is described -in the TypeScript interface `createResourceWithRepresentationRequest` in -module `createResourceFormats`. The request header's content type has to -set to `application/json`. +The client may now wish to get a thumbnail of the uploaded image, to allow +the user to confirm that the correct files have been uploaded. This can be done +by adding the filename and IIIF parameters to `temporaryBaseIIIFUrl`. For example, to get +a JPG thumbnail image whose width and height are at most 128 pixels wide, you would request +`http://sipihost/tmp/3UIsXH9bP0j-BV0D4sN51Xz.jp2/full/!128,128/0/default.jpg`. -In addition to [Adding Resources Without Image Files](#adding-resources-without-a-digital-representation), the -(temporary) name of the file, its original name, and mime type have to -be provided (see @ref:[GUI Case](../../07-sipi/sipi-and-knora.md#gui-case)). +The request to Knora works similarly to +[Adding Resources Without Image Files](#adding-resources-without-image-files), +with the addition of `file`, whose value is the `internalFilename` that Sipi returned. +See the TypeScript interface `createResourceWithRepresentationRequest` in +module `createResourceFormats` for details. The request header's content type must be +set to `application/json`. ## Response to a Resource Creation @@ -124,7 +110,7 @@ The JSON format of the response is described in the TypeScript interface ## Changing a Resource's Label A resource's label can be changed by making a PUT request to the path -segments `resources/label`. The resource's Iri has to be provided in the +segments `resources/label`. The resource's IRI has to be provided in the URL (as its last segment). The new label has to submitted as JSON in the HTTP request's body. @@ -152,18 +138,24 @@ Only system or project administrators may use the bulk import. The procedure for using this feature is as follows (see the @ref:[example below](#bulk-import-example)). -1. Make an HTTP GET request to Knora to @ref:[get XML schemas](#1-get-xml-schemas) describing - the XML to be provided for the import. -2. @ref:[Generate an XML import document](#2-generate-xml-import-document) representing the - data to be imported, following the Knora import schemas that were generated in step 1. - You will probably want to write a script to do this. Knora is not involved in this step. - If you are also importing image files, this XML document needs to - @ref:[contain the filesystem paths](#bulk-import-with-image-files) of those files. -3. @ref:[Validate your XML import document](#3-validate-xml-import-document), using an XML schema validator such as - [Apache Xerces](http://xerces.apache.org) or [Saxon](http://www.saxonica.com), or an - XML development environment such as [Oxygen](https://www.oxygenxml.com). This will - help ensure that the data you submit to Knora is correct. Knora is not involved in this step. -4. @ref:[Submit the XML import document to Knora](#4-submit-xml-import-document-to-knora). +1. Make an HTTP GET request to Knora to @ref:[get XML schemas](#1-get-xml-schemas) describing + the XML to be provided for the import. + +2. If you are importing image files, @ref:[upload files to Sipi](#2-upload-files-to-sipi). + +3. @ref:[Generate an XML import document](#3-generate-xml-import-document) representing the + data to be imported, following the Knora import schemas that were generated in step 1. + You will probably want to write a script to do this. Knora is not involved in this step. + If you are also importing image files, this XML document needs to + @ref:[contain the filenames](#bulk-import-with-image-files) that Sipi returned + for the files you uploaded in step 2. + +4. @ref:[Validate your XML import document](#4-validate-xml-import-document), using an XML schema validator such as + [Apache Xerces](http://xerces.apache.org) or [Saxon](http://www.saxonica.com), or an + XML development environment such as [Oxygen](https://www.oxygenxml.com). This will + help ensure that the data you submit to Knora is correct. Knora is not involved in this step. + +5. @ref:[Submit the XML import document to Knora](#5-submit-xml-import-document-to-knora). In this procedure, the person responsible for generating the XML import data need not be familiar with RDF or with the ontologies involved. @@ -208,7 +200,12 @@ containing three files: - `knoraXmlImport.xsd`: The standard Knora XML import schema, used by all XML imports. -#### 2. Generate XML Import Document +#### 2. Upload Files to Sipi + +See @ref:[Upload Files to Sipi](../api-v2/editing-values.md#upload-files-to-sipi) in +the Knora API v2 documentation. + +#### 3. Generate XML Import Document We now convert our existing data to XML, probably by writing a custom script. The resulting XML import document could look like this: @@ -341,7 +338,7 @@ This illustrates several aspects of XML imports: - A text value can have a `lang` attribute, whose value is an ISO 639-1 code specifying the language of the text. -#### 3. Validate XML Import Document +#### 4. Validate XML Import Document You can use an XML schema validator such as [Apache Xerces](http://xerces.apache.org) or [Saxon](http://saxon.sourceforge.net/), or an XML development environment @@ -354,7 +351,7 @@ For example, using Saxon: java -cp ./saxon9ee.jar com.saxonica.Validate -xsd:p0801-biblio.xsd -s:data.xml ``` -#### 4. Submit XML Import Document to Knora +#### 5. Submit XML Import Document to Knora To create these resources in Knora, make an HTTP post request with the XML import document as the request body. The URL must specify the (URL-encoded) IRI of the project in which @@ -411,8 +408,8 @@ contains the IRI of the target resource. To attach an image file to a resource, we must provide the element `knoraXmlImport:file` before the property elements. In this -element, we must give the absolute filesystem path to the file that -should be attached to the resource, along with its MIME type: +element, we must provide a `filename` attribute, containing the `internalFilename` +that Sipi returned for the file in @ref:[2. Upload Files to Sipi](#2-upload-files-to-sipi). ```xml @@ -427,7 +424,7 @@ should be attached to the resource, along with its MIME type: a page with an image - + Chlaus 1a @@ -438,6 +435,5 @@ should be attached to the resource, along with its MIME type: ``` -During the processing of the bulk import, Knora will -communicate the location of file to Sipi, which will convert it to JPEG 2000 -for storage. +During the processing of the bulk import, Knora will ask Sipi for the rest of the +file's metadata, and store that metadata in a file value attached to the resource. diff --git a/docs/src/paradox/03-apis/api-v1/changing-values.md b/docs/src/paradox/03-apis/api-v1/changing-values.md index efc3ac70d6..beaca1e156 100644 --- a/docs/src/paradox/03-apis/api-v1/changing-values.md +++ b/docs/src/paradox/03-apis/api-v1/changing-values.md @@ -56,52 +56,21 @@ has to be used in order to create a new value (all these TypeScript interfaces a ## Modifying a File Value -In order to exchange a file value (digital representation of a -resource), the path segment `filevalue` has to be used. The IRI of the -resource whose file value is to be exchanged has to be appended: +To change a file value, the client first uploads the new file to +Sipi, following the procedure described in +@ref:[Adding Resources with Image Files](adding-resources.md#adding-resources-with-image-files). + +Then the client sends a request to Knora, using this following route: ``` HTTP PUT to http://host/filevalue/resourceIRI ``` -Please note that the resource IRI has to be URL encoded. - -There are two ways to change a file of a resource: Either by submitting -directly the binaries of the file in a HTTP Multipart request or by -indicating the location of the file. The two cases are referred to as -non-GUI case and GUI case (TODO: add a link to "Sipi and Knora"). - -### Including the binaries (non-GUI case) - -Here, a HTTP MULTIPART request has to be made simply providing the -binaries (without JSON): - -```python -#!/usr/bin/env python3 - -import requests, json, urllib - -# the name of the file to be submitted -filename = 'myimage.tif' - -# a tuple containing the file's name, its binaries and its mimetype -files = {'file': (filename, open(filename, 'rb'), "image/tiff")} - -resIri = urllib.parse.quote_plus('http://rdfh.ch/xy') - -r = requests.put("http://host/filevalue/" + resIri, - files=files) -``` - -Please note that the file has to be read in binary mode (by default it -would be read in text mode). - -### Indicating the location of a file (GUI case) - -Here, simply the location of the new file has to be submitted as JSON. -The JSON format is described in the TypeScript interface -`changeFileValueRequest` in module `changeValueFormats`. The request -header's content type has to set to `application/json`. +Here, `resourceIRI` is the URL-encoded IRI of the resource whose file value is +to be changed. The body of the request is a JSON object described in the TypeScript +interface `changeFileValueRequest` in module `changeValueFormats`, and contains +`file`, whose value is the `internalFilename` that Sipi returned. The request header's +content type must be set to `application/json`. ## Response on Value Change diff --git a/docs/src/paradox/03-apis/api-v2/editing-values.md b/docs/src/paradox/03-apis/api-v2/editing-values.md index 7c574c2da9..d4e47902f9 100644 --- a/docs/src/paradox/03-apis/api-v2/editing-values.md +++ b/docs/src/paradox/03-apis/api-v2/editing-values.md @@ -260,11 +260,11 @@ Sipi then returns a JSON response that looks something like this: "uploadedFiles": [{ "originalFilename": "manuscript-1234-page-1.tiff", "internalFilename": "3UIsXH9bP0j-BV0D4sN51Xz.jp2", - "temporaryBaseIIIFUrl": "http://sipihost/tmp/3UIsXH9bP0j-BV0D4sN51Xz.jp2" + "temporaryBaseIIIFUrl": "http://sipihost/tmp" }, { "originalFilename": "manuscript-1234-page-2.tiff", "internalFilename": "2RvJgguglpe-B45EOk0Gx8H.jp2", - "temporaryBaseIIIFUrl": "http://sipihost/tmp/2RvJgguglpe-B45EOk0Gx8H.jp2" + "temporaryBaseIIIFUrl": "http://sipihost/tmp" }] } ``` diff --git a/docs/src/paradox/05-internals/design/api-v2/sipi.md b/docs/src/paradox/05-internals/design/api-v2/sipi.md index c8bc5ca8c4..896e83c741 100644 --- a/docs/src/paradox/05-internals/design/api-v2/sipi.md +++ b/docs/src/paradox/05-internals/design/api-v2/sipi.md @@ -43,7 +43,7 @@ are described below. The `upload.lua` script is available at Sipi's `upload` route. It processes one or more file uploads submitted to Sipi. It converts uploaded images to JPEG 2000 format, and stores them in Sipi's `tmp` directory. The usage of this script is described in -@ref:[Creating File Values](../../../03-apis/api-v2/editing-values.md#creating-file-values). +@ref:[Upload Files to Sipi](../../../03-apis/api-v2/editing-values.md#upload-files-to-sipi). Each time `upload.lua` processes a request, it also deletes old temporary files from `tmp` and (recursively) from any subdirectories. The maximum allowed age of @@ -96,7 +96,7 @@ If it encounters an error, it returns `SipiException`. to create or change a file value. The request includes Sipi's internal filename. 3. During parsing of this JSON-LD request, a `StillImageFileValueContentV2` is constructed to represent the file value. During the construction of this - object, a `GetImageMetadataRequestV2` is sent to `SipiConnector`, which + object, a `GetFileMetadataRequestV2` is sent to `SipiConnector`, which uses Sipi's built-in `knora.json` route to get the rest of the file's metadata. 4. A responder (`ResourcesResponderV2` or `ValuesResponderV2`) validates diff --git a/docs/src/paradox/07-sipi/setup-sipi-for-knora.md b/docs/src/paradox/07-sipi/setup-sipi-for-knora.md index 05171dbe37..546b14258b 100644 --- a/docs/src/paradox/07-sipi/setup-sipi-for-knora.md +++ b/docs/src/paradox/07-sipi/setup-sipi-for-knora.md @@ -45,11 +45,10 @@ Whenever a file is requested from Sipi (e.g. a browser trying to dereference an image link served by Knora), a preflight function is called. This function is defined in `sipi.init-knora.lua` present in the Sipi root directory. It takes three parameters: `prefix`, `identifier` -(the name of the requested file), and `cookie`. File links created by -Knora use the prefix `knora`, e.g. -`http://localhost:1024/knora/incunabula_0000000002.jp2/full/2613,3505/0/default.jpg`. +(the name of the requested file), and `cookie`. The prefix is the shortcode +of the project that the resource containing the file value belongs to. -Given these information, Sipi asks Knora about the current's users +Given this information, Sipi asks Knora about the current's users permissions on the given file. The cookie contains the current user's Knora session id, so Knora can match Sipi's request with a given user profile and determine the permissions this user has on the file. If the @@ -60,8 +59,8 @@ refuses to serve the file. However, all of this behaviour is defined in the preflight function in Sipi and not controlled by Knora. Knora only provides the permission code. -See @ref:[Sharing the Session ID with Sipi](sipi-and-knora.md#sharing-the-session-id-with-sipi) for more -information about sharing the session id. +See @ref:[Authentication of Users with Sipi](sipi-and-knora.md#authentication-of-users-with-sipi) for more +information about sharing the session ID. ## Using Sipi in Test Mode @@ -76,24 +75,11 @@ $ docker run --rm -it --add-host webapihost:$DOCKERHOST -v $PWD/config:/sipi/con ``` Then always the same test file will be served which is included in Sipi. In test mode, Sipi will -not aks Knora about the user's permission on the requested file. - -## Using Sipi in production behind a proxy - -For SIPI to work with Salsah1 (non-angular) GUI, we need to define an additional set of -environment variables if we want to run SIPI behind a proxy: - -- `SIPI_EXTERNAL_PROTOCOL=https` -- `SIPI_EXTERNAL_HOSTNAME=iiif.example.org` -- `SIPI_EXTERNAL_PORT=443` - -These variables are only used by `make_thumbnail.lua`: - -@@snip[make_thumbnail.lua](../../../../sipi/scripts/make_thumbnail.lua) { #snip_marker } +not ask Knora about the user's permission on the requested file. ## Additional Sipi Environment Variables -Additionaly, these environment variables can be used to further configure sipi: +Additionally, these environment variables can be used to further configure sipi: - `SIPI_WEBAPI_HOSTNAME=localhost`: overrides `knora_path` in Sipi's config - `SIPI_WEBAPI_PORT=3333`: overrides `knora_port` in Sipi's config diff --git a/docs/src/paradox/07-sipi/sipi-and-knora.md b/docs/src/paradox/07-sipi/sipi-and-knora.md index 003fec61e3..01f765e0dd 100644 --- a/docs/src/paradox/07-sipi/sipi-and-knora.md +++ b/docs/src/paradox/07-sipi/sipi-and-knora.md @@ -19,8 +19,6 @@ License along with Knora. If not, see . # Interaction Between Sipi and Knora -TODO: reorganise this to make clear that it describes Knora API v1. - ## General Remarks Knora and Sipi (Simple Image Presentation Interface) are two @@ -37,224 +35,31 @@ for the client to request them from Sipi, but the whole handling of files (storing, naming, organization of the internal directory structure, format conversions, and serving) is taken care of by Sipi. -## Adding Files to Knora: Using the GUI or directly the API - -To create a resource with a digital representation attached to, either -the browser-based GUI (SALSAH) can be used or this can be done by -*directly* addressing the API. (Of course, also the GUI uses the API. -But the user does not need to know about it.) The same applies for -changing an existing digital representation for a resource. Subsequently, the first -case will be called the *GUI case* and the second the *non-GUI case*. - -### GUI Case - -In this case, the user may choose a file to upload using his -web-browser. The file is directly sent to Sipi (route: -`create_thumbnail`) to calculate a thumbnail hosted by Sipi which then -gets displayed to the user in the browser. Sipi copies the original file -into a temporary directory and keeps it there (for later processing in -another request). In its answer (JSON), Sipi returns: - -- `preview_path`: the path to the thumbnail (accessible to a - web-browser) -- `filename`: the name of the temporarily stored original file - (managed by Sipi) -- `original_mimetype`: mime type of the original file -- `original_filename`: the original name of the file submitted by - the client - -Once the user finally wants to attach the file to a resource, the -request is sent to Knora's API providing all the required parameters to -create the resource along with additional information about the file to -be attached. **However, the file itself is not submitted to the Knora -Api, but its filename returned by Sipi (from the `create_thumbnail` -response).** - -#### Create a new Resource with a Digital Representation - -The POST request is handled in `ResourcesRouteV1.scala` and parsed to a -`CreateResourceApiRequestV1`. Information about the file is sent -separately from the other resource parameters (properties) under the -name `file`: - -- `originalFilename`: original name of the file (returned by Sipi - when creating the thumbnail) -- `originalMimeType`: original mime type of the file (returned by - Sipi when creating the thumbnail) -- `filename`: name of the temporarily stored original file (returned - by Sipi when creating the thumbnail) - -In the route, a `SipiResponderConversionFileRequestV1` is created -representing the information about the file to be attached to the new -resource. Along with the other parameters, it is sent to the resources -responder. - -See @ref:[Further Handling of the GUI and the non GUI-case in the Resources Responder](#further-handling-of-the-gui-and-the-non-gui-case-in-the-resources-responder) for -details of how the resources responder then handles the request. +## Adding Files to Knora -#### Change the Digital Representation of a Resource - -The request is taken care of in `ValuesRouteV1.scala`. The PUT request -is handled in path `v1/filevalue/{resIri}` which receives the resource -Iri as a part of the URL: *The submitted file will update the existing -file values of the given resource.* - -The file parameters are submitted as json and are parsed into a -`ChangeFileValueApiRequestV1`. To represent the conversion request for -the Sipi responder, a `SipiResponderConversionFileRequestV1` is created. -A `ChangeFileValueRequestV1` containing the resource Iri and the message -for Sipi is then created and sent to the values responder. - -See @ref:[Further Handling of the GUI and the non GUI-case in the Values Responder](#further-handling-of-the-gui-and-the-non-gui-case-in-the-values-responder) -for details of how the values responder then handles the request. - -### Non-GUI case - -In this case, the API receives an HTTP multipart request containing the -binary data. - -#### Create a new Resource with a Digital Representation - -The request is handled in `ResourcesRouteV1.scala`. The multipart POST -request consists of two named body parts: `json` containing the resource -parameters (properties) and `file` containing the binary data as well as -the file name and its mime type. Using Python's [request -module](http://docs.python-requests.org/en/master/user/quickstart/#post-a-multipart-encoded-file), -a request could look like this: - -```python -import requests, json - -params = {...} // resource parameters -files = {'file': (filename, open(path + filename, 'rb'), mimetype)} // filename, binary data, and mime type - -r = requests.post(knora_url + '/resources', - data={'json': json.dumps(params)}, - files=files, - headers=None) -``` +A file is first uploaded to Sipi, then its metadata is submitted to +Knora. The implementation of this procedure is described in +@ref:[Knora and Sipi](../05-internals/design/api-v2/sipi.md). Instructions +for the client are given in +@ref:[Creating File Values](../03-apis/api-v2/editing-values.md#creating-file-values) +(for Knora API v2) and in +@ref:[Adding Resources with Image Files](../03-apis/api-v1/adding-resources.md#adding-resources-with-image-files) +(for API v1). -The binary data is saved to a temporary location by Knora. The route -then creates a `SipiResponderConversionPathRequestV1` representing the -information about the file (i.e. the temporary path to the file) to be -attached to the new resource. Along with the other parameters, it is -sent to the resources responder. - -See @ref:[Further Handling of the GUI and the non GUI-case in the Resources Responder](#further-handling-of-the-gui-and-the-non-gui-case-in-the-resources-responder) for -details of how the resources responder then handles the request. - -#### Change the Digital Representation of a Resource - -The request is taken care of in `ValuesRouteV1.scala`. The multipart PUT -request is handled in path `v1/filevalue/{resIri}` which receives the -resource Iri as a part of the URL: *The submitted file will update the -existing file values of the given resource.* - -For the request, no json parameters are required. So its body just -consists of the binary data -(see @ref:[Create a new Resource with a Digital Representation](#create-a-new-resource-with-a-digital-representation)). -The values route stores the submitted -binaries as a temporary file and creates a -`SipiResponderConversionPathRequestV1`. A `ChangeFileValueRequestV1` -containing the resource Iri and the message for Sipi is then created and -sent to the values responder. - -See @ref:[Further Handling of the GUI and the non GUI-case in the Values Responder](#further-handling-of-the-gui-and-the-non-gui-case-in-the-values-responder) for details -of how the values responder then handles the request. - -### Further Handling of the GUI and the Non-GUI case in the Resources Responder - -Once a `SipiResponderConversionFileRequestV1` (GUI case) or a -`SipiResponderConversionPathRequestV1` (non-GUI case) has been created -and passed to the resources responder, the GUI and the non-GUI case can -be handled in a very similar way. This is why they are both -implementations of the trait `SipiResponderConversionRequestV1`. - -The resource responder calls the ontology responder to check if all -required properties were submitted for the given resource type. Also it -is checked if the given resource type may have a digital representation. -The resources responder then sends a message to Sipi responder that does -a request to the Sipi server. Depending on the type of the message -(`SipiResponderConversionFileRequestV1` or -`SipiResponderConversionPathRequestV1`), a different Sipi route is -called. In the first case (GUI case), the file is already managed by -Sipi and only the filename has to be indicated. In the latter case, Sipi -is told about the location where Knora has saved the binary data to. - -To make this handling easy for Knora, both messages have their own -implementation for creating the parameters for Sipi (declared in the -trait as `toFormData`). If Knora deals with a -`SipiResponderConversionPathRequestV1`, it has to delete the temporary -file after it has been processed by SIPI. Here, we assume that we deal -with an image. - -For both cases, Sipi returns the same answer containing the following -information: - - - `file_type`: the type of the file that has been handled by Sipi - (image | video | audio | text | binary) - - `mimetype_full` and `mimetype_thumb`: mime types of the full image - representation and the thumbnail - - `original_mimetype`: the mime type of the original file - - `original_filename`: the name of the original file - - `nx_full`, `ny_full`, `nx_thumb`, and `ny_thumb`: the x and y - dimensions of both the full image and the thumbnail - - `filename_full` and `filename_full`: the names of the full image - and the thumbnail (needed to request the images from Sipi) - -The `file_type` is important because representations for resources are -restricted to media types: image, audio, video or a generic binary file. -If a resource type requires an image representations (subclass of -`StillImageRepresentation`), the `file_type` has to be an image. -Otherwise, the ontology's restrictions would be violated. Because of -this requirement, there is a construct `fileType2FileValueProperty` -mapping file types to file value properties. Also all the possible file -types are defined in enumeration. - -Depending on the given file type, Sipi responder can create the apt -message (here: `StillImageFileValueV1`) to save the data to the -triplestore. - -### Further Handling of the GUI and the non-GUI case in the Values Responder - -In the values responder, `ChangeFileValueRequestV1` is passed to the -method `changeFileValueV1`. Unlike ordinary value change requests, the -Iris of the value objects to be updated are not known yet. Because of -this, all the existing file values of the given resource Iri have to be -queried first. Also their quality levels are queried because in case of -a `StillImageFileValue`, we have to deal with a file value for the -thumbnail and another one for the full quality representation. When -these two file values are being updated, the quality levels have to be -considered for the sake of consistency (otherwise a full quality value's -`knora-base:previous-value` may point to a thumbnail file value). - -With the file values being returned, we actually know about the current -Iris of the value objects. Now the Sipi responder is called to handle -the file conversion request (see @ref:[Further Handling of the GUI and the non GUI-case in the Resources Responder](#further-handling-of-the-gui-and-the-non-gui-case-in-the-resources-responder)). -After that, it is checked that the `file_type` returned by Sipi responder -corresponds to the property type of the existing file values. For -example, if the `file_type` is an image, the property pointing to the -current file values must be a `hasStillImageFileValue`. Otherwise, the -user submitted a non image file that has to be rejected. - -Depending on the `file_type`, messages of type `ChangeValueRequestV1` -can be created. For each existing file value, such a message is -instantiated containing the current value Iri and the new value to be -created (returned by the sipi responder). These messages are passed to -`changeValueV1` because with the described handling done in -`changeFileValueV1`, the file values can be changed like any other value -type. +## Retrieving Files from Sipi -In case of success, a `ChangeFileValueResponseV1` is sent back to the -client, containing a list of the single `ChangeValueResponseV1`. +### File URLs in API v2 -## Retrieving Files from Sipi +In Knora API v2, image file URLs are provided in [IIIF](https://iiif.io/) format. In the simple +@ref:[ontology schema](../03-apis/api-v2/introduction.md#api-schema), a file value is simply +a IIIF URL that can be used to retrieve the file from Sipi. In the complex schema, +it is a `StillImageFileValue` with additional properties that the client can use to construct +different IIIF URLs, e.g. at different resolutions. See the `knora-api` ontology for details. -### URL creation +### File URLs in API v1 -Binary representions of Knora locations are served by Sipi. For each -file value, Knora creates several locations representing different -quality levels: +In API v1, for each file value, Knora creates several Sipi URLs for accessing the file at different +resolutions: ``` "resinfo": { @@ -296,31 +101,15 @@ quality levels: ``` Each of these paths has to be handled by the browser by making a call to -Sipi, obtaining the binary representation in the desired quality. To -deal with different image quality levels, Sipi implements the [IIIF -standard](http://iiif.io/api/image/2.0/). The different quality level -paths are created by Knora in `ValueUtilV1`. - -Whenever Sipi serves a binary representation of a Knora file value -(indicated by using the prefix `knora` in the path), it has to make a -request to Knora's Sipi responder to get the user's permissions on the -requested file. Sipi's request to Knora contains a cookie with the Knora -session id the user has obtained when logging in to Knora: As a response -to a successful login, Knora returns the user's session id and this id -is automatically sent to Sipi by the browser, setting a second cookie -for the communication with Sipi. The reason the Knora session id is set -in two cookies, is the fact that cookies can not be shared among -different domains. Since Knora and Sipi are likely to be running under -different domains, this solution offers the necessary flexibility. - +Sipi, obtaining the binary representation in the desired quality. -## Authentication of users with Sipi +## Authentication of Users with Sipi Whenever a file is requested, Sipi asks Knora about the current user's permissions on the given file. This is achieved by sharing the Knora session cookie with Sipi. When the user logs in to Knora using his browser (using either `V1` or `V2` authentication route), a session cookie containing a JWT token representing -the user is stored in the user's client. This session cookie is then read by Sipi and used to query for +the user is stored in the user's client. This session cookie is then read by Sipi and used to ask Knora for the user's image permissions. For the session cookie to be sent to Sipi, both the Knora API and Sipi endpoints need to -be under the same domain, e.g., `api.example.com` and `iiif.example.com`. \ No newline at end of file +be under the same domain, e.g., `api.example.com` and `iiif.example.com`. diff --git a/salsah1/src/public/index.html b/salsah1/src/public/index.html index 9d729a23c8..3332494525 100644 --- a/salsah1/src/public/index.html +++ b/salsah1/src/public/index.html @@ -212,7 +212,9 @@ // When retrieving a file from Sipi (e.g. an IIIF URL), Sipi can send the session id to Knora with the request, // identifying the user that is making the request to Knora. SALSAH.userprofile = data.userProfile; + window.sessionStorage.setItem('userprofile', JSON.stringify(SALSAH.userprofile)); + window.sessionStorage.setItem('token', data.sid); $('#dologin').simpledialog('loginbox', 'close'); @@ -254,6 +256,8 @@ SALSAH.ApiDelete('session', function(data) { if (data.status == ApiErrors.OK) { window.sessionStorage.removeItem('userprofile'); + window.sessionStorage.removeItem('token'); + SALSAH.userprofile = null; $('#dologut').simpledialog('logoutbox', 'close'); diff --git a/salsah1/src/public/js/jquery.location.js b/salsah1/src/public/js/jquery.location.js index 370a798a67..d9a1633bb5 100644 --- a/salsah1/src/public/js/jquery.location.js +++ b/salsah1/src/public/js/jquery.location.js @@ -91,7 +91,7 @@ $.ajax({ type:'POST', - url: SIPI_URL + "/make_thumbnail", + url: SIPI_URL + "/upload?token=" + window.sessionStorage.getItem('token'), data: fd, cache: false, contentType: false, @@ -104,8 +104,11 @@ $this.find('.progressNumber').remove(); $this.find('.uploadButton').remove(); + var uploadedFile = data["uploadedFiles"][0]; + var preview_path = uploadedFile["temporaryBaseIIIFUrl"] + "/" + uploadedFile["internalFilename"] + "/full/!128,128/0/default.jpg"; + $this.append($('
').addClass('thumbNail') - .append($('', {src: data.preview_path, style: "image-orientation: from-image"})) + .append($('', {src: preview_path, style: "image-orientation: from-image"})) .append($('
')) .append(data.original_filename) ); @@ -113,7 +116,7 @@ //$this.append($('').attr({'type': 'hidden'}).addClass('fileData').val(event.target.responseText)); - localdata.sipi_response = data; + localdata.sipi_response = uploadedFile; }, error: function(jqXHR, textStatus, errorThrown) { if (errorThrown !== undefined && jqXHR !== undefined && jqXHR.responseJSON !== undefined) { diff --git a/salsah1/src/public/js/jquery.propedit.js b/salsah1/src/public/js/jquery.propedit.js index fb751e1b59..46a030deec 100644 --- a/salsah1/src/public/js/jquery.propedit.js +++ b/salsah1/src/public/js/jquery.propedit.js @@ -1348,11 +1348,7 @@ } else { data = { - file: { - originalFilename: sipi_response["original_filename"], - originalMimeType: sipi_response["original_mimetype"], - filename: sipi_response["filename"] - } + file: sipi_response["internalFilename"] }; SALSAH.ApiPut('filevalue/' + encodeURIComponent(res_id), data, function(data) { if (data.status == ApiErrors.OK) { diff --git a/salsah1/src/public/js/jquery.resadd.js b/salsah1/src/public/js/jquery.resadd.js index 3763bf3413..48f0cfeaa9 100644 --- a/salsah1/src/public/js/jquery.resadd.js +++ b/salsah1/src/public/js/jquery.resadd.js @@ -1051,11 +1051,7 @@ ele = form.find('[name="' + propname + '"]'); var sipi_response = ele.location('value'); - file = { - originalFilename: sipi_response["original_filename"], - originalMimeType: sipi_response["original_mimetype"], - filename: sipi_response["filename"] - }; + file = sipi_response["internalFilename"]; break; diff --git a/salsah1/src/typescript_interfaces/basicMessageComponents.ts b/salsah1/src/typescript_interfaces/basicMessageComponents.ts index 564d8c22a4..ff5f8a286d 100644 --- a/salsah1/src/typescript_interfaces/basicMessageComponents.ts +++ b/salsah1/src/typescript_interfaces/basicMessageComponents.ts @@ -404,31 +404,14 @@ export module basicMessageComponents { } /** - * Describes a file value (for GUI-case) + * Describes a file value. */ export interface createOrChangeFileValueRequest { /** - * Describes a file value (for GUI-case) + * The internal filename returned by Sipi. */ - file: { - - /** - * The file's original name - */ - originalFilename: string; - - /** - * The original mime type of the file - */ - originalMimeType: string; - - /** - * The file's temporary name - */ - filename: string; - - } + file: string; } /** diff --git a/salsah1/src/typescript_interfaces/sampleRequests/sampleChangeValues.ts b/salsah1/src/typescript_interfaces/sampleRequests/sampleChangeValues.ts index 16cb76fdf0..91dd881972 100644 --- a/salsah1/src/typescript_interfaces/sampleRequests/sampleChangeValues.ts +++ b/salsah1/src/typescript_interfaces/sampleRequests/sampleChangeValues.ts @@ -24,11 +24,7 @@ let changeIntervalValue: changeValueFormats.changeIntervalValueRequest = {"inter let changeIntervalValueResponse: changeValueFormats.changeValueResponse = {"id":"http://rdfh.ch/a-thing/values/G58MBZ5ES7yxmKX2l5QTPg","status":0,"comment":null,"rights":8,"value":{"timeval1":0,"timeval2":36000}}; let changeFileValueRequest: changeValueFormats.changeFileValueRequest = { - 'file': { - 'originalFilename' : "myfile.jpg", - 'originalMimeType' : "image/jpeg", - 'filename' : "tmpname.jpg" - } + "file": "3UIsXH9bP0j-BV0D4sN51Xz.jp2" }; let changeFileValueResponse: changeValueFormats.changeFileValueResponse = {"locations":[{"duration":0,"nx":128,"path":"http://localhost:1024/knora/5XTEI1z10A2-D8ojQHrMiUz.jpg/full/max/0/default.jpg","ny":72,"fps":0,"format_name":"JPEG","origname":"2016-06-26+12.26.45.jpg","protocol":"file"},{"duration":0,"nx":3264,"path":"http://localhost:1024/knora/5XTEI1z10A2-D8ojQHrMiUz.jpx/full/3264,1836/0/default.jpg","ny":1836,"fps":0,"format_name":"JPEG2000","origname":"2016-06-26+12.26.45.jpg","protocol":"file"}],"status":0}; diff --git a/salsah1/src/typescript_interfaces/sampleRequests/sampleCreateResources.ts b/salsah1/src/typescript_interfaces/sampleRequests/sampleCreateResources.ts index 48a079bc82..3bb9f8c809 100644 --- a/salsah1/src/typescript_interfaces/sampleRequests/sampleCreateResources.ts +++ b/salsah1/src/typescript_interfaces/sampleRequests/sampleCreateResources.ts @@ -50,11 +50,7 @@ let thingWithFile: createResourceFormats.createResourceWithRepresentationRequest "http://www.knora.org/ontology/0001/anything#hasListItem": [{"hlist_value":"http://rdfh.ch/anything/treeList10"}], "http://www.knora.org/ontology/0001/anything#hasInterval": [{"interval_value": [1000000000000000.0000000000000001, 1000000000000000.0000000000000002]}] }, - "file": { - 'originalFilename' : "myfile.jpg", - 'originalMimeType' : "image/jpeg", - 'filename' : "tmp.jpg" - } + "file": "3UIsXH9bP0j-BV0D4sN51Xz.jp2" }; let createResourceResponse: createResourceFormats.createResourceResponse = {"res_id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw","results":{"http://www.knora.org/ontology/0001/anything#hasDecimal":[{"value":{"dateval1":null,"ival":null,"dateprecision1":null,"textval":{"string":"3.3"},"person_id":{"string":"http://rdfh.ch/users/91e19f1e01"},"property_id":{"string":"http://www.knora.org/ontology/0001/anything#hasDecimal"},"calendar":null,"timeval2":null,"dval":{"decimal":3.3},"dateval2":null,"order":{"integer":1},"resource_id":{"string":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw"},"timeval1":null,"dateprecision2":null},"id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw/values/1WQOgjJWS86laX0BYKoyGw"}],"http://www.knora.org/ontology/0001/anything#hasColor":[{"value":{"dateval1":null,"ival":null,"dateprecision1":null,"textval":{"string":"#ff3333"},"person_id":{"string":"http://rdfh.ch/users/91e19f1e01"},"property_id":{"string":"http://www.knora.org/ontology/0001/anything#hasColor"},"calendar":null,"timeval2":null,"dval":null,"dateval2":null,"order":{"integer":1},"resource_id":{"string":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw"},"timeval1":null,"dateprecision2":null},"id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw/values/x5EBVPVHTReZsz-UCnZR0g"}],"http://www.knora.org/ontology/0001/anything#hasInteger":[{"value":{"dateval1":null,"ival":{"integer":1},"dateprecision1":null,"textval":{"string":"1"},"person_id":{"string":"http://rdfh.ch/users/91e19f1e01"},"property_id":{"string":"http://www.knora.org/ontology/0001/anything#hasInteger"},"calendar":null,"timeval2":null,"dval":null,"dateval2":null,"order":{"integer":1},"resource_id":{"string":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw"},"timeval1":null,"dateprecision2":null},"id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw/values/6mbnuiGeSIWaB_sXw3-cJA"}],"http://www.knora.org/ontology/0001/anything#hasInterval":[{"value":{"dateval1":null,"ival":null,"dateprecision1":null,"textval":{"string":"IntervalValueV1(0,0)"},"person_id":{"string":"http://rdfh.ch/users/91e19f1e01"},"property_id":{"string":"http://www.knora.org/ontology/0001/anything#hasInterval"},"calendar":null,"timeval2":null,"dval":null,"dateval2":null,"order":{"integer":1},"resource_id":{"string":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw"},"timeval1":null,"dateprecision2":null},"id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw/values/VRlXZ-taQoKonY4y6_Y9wQ"}],"http://www.knora.org/ontology/0001/anything#hasDate":[{"value":{"dateval1":{"string":"2016-07-14"},"ival":null,"dateprecision1":{"string":"DAY"},"textval":{"string":"2016-07-14"},"person_id":{"string":"http://rdfh.ch/users/91e19f1e01"},"property_id":{"string":"http://www.knora.org/ontology/0001/anything#hasDate"},"calendar":{"string":"GREGORIAN"},"timeval2":null,"dval":null,"dateval2":{"string":"2016-07-14 CE"},"order":{"integer":1},"resource_id":{"string":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw"},"timeval1":null,"dateprecision2":{"string":"DAY"}},"id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw/values/M15JNh0rRjGvYrL7G257EQ"}],"http://www.knora.org/ontology/0001/anything#hasListItem":[{"value":{"dateval1":null,"ival":null,"dateprecision1":null,"textval":{"string":"http://rdfh.ch/anything/treeList01"},"person_id":{"string":"http://rdfh.ch/users/91e19f1e01"},"property_id":{"string":"http://www.knora.org/ontology/0001/anything#hasListItem"},"calendar":null,"timeval2":null,"dval":null,"dateval2":null,"order":{"integer":1},"resource_id":{"string":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw"},"timeval1":null,"dateprecision2":null},"id":"http://rdfh.ch/Bc7yXd3ETJ6SjNuq86eBdw/values/FuZ9sVvzQgywFhkf5SuE8Q"}]},"status":0} diff --git a/sipi/config/sipi.knora-docker-config.lua b/sipi/config/sipi.knora-docker-config.lua index 702f0c776e..88e2e8c3cc 100644 --- a/sipi/config/sipi.knora-docker-config.lua +++ b/sipi/config/sipi.knora-docker-config.lua @@ -167,21 +167,6 @@ fileserver = { -- Custom routes. Each route is an URL path associated with a Lua script. -- routes = { - { - method = 'POST', - route = '/make_thumbnail', - script = 'make_thumbnail.lua' - }, - { - method = 'POST', - route = '/convert_from_path', - script = 'convert_from_path.lua' - }, - { - method = 'POST', - route = '/convert_from_file', - script = 'convert_from_file.lua' - }, { method = 'POST', route = '/upload', @@ -217,4 +202,3 @@ routes = { } } - diff --git a/sipi/config/sipi.knora-docker-no-auth-config.lua b/sipi/config/sipi.knora-docker-no-auth-config.lua index d627b604d2..126d917eaa 100644 --- a/sipi/config/sipi.knora-docker-no-auth-config.lua +++ b/sipi/config/sipi.knora-docker-no-auth-config.lua @@ -46,7 +46,7 @@ sipi = { -- max_post_size = '30M', - -- + -- -- indicates the path to the root of the image directory. Depending on the settings of the variable -- "prefix_as_path" the images are search at // (prefix_as_path = TRUE) -- or / (prefix_as_path = FALSE). Please note that "prefix" and "imageid" are @@ -179,21 +179,6 @@ fileserver = { -- Custom routes. Each route is URL path associated with a Lua script. -- routes = { - { - method = 'POST', - route = '/make_thumbnail', - script = 'make_thumbnail.lua' - }, - { - method = 'POST', - route = '/convert_from_path', - script = 'convert_from_path.lua' - }, - { - method = 'POST', - route = '/convert_from_file', - script = 'convert_from_file.lua' - }, --{ -- method = 'POST', -- route = '/Knora_login', @@ -235,4 +220,4 @@ routes = { script = 'delete_temp_file.lua' } -} +} \ No newline at end of file diff --git a/sipi/config/sipi.knora-docker-test-config.lua b/sipi/config/sipi.knora-docker-test-config.lua index da0425998e..d1ca13574d 100644 --- a/sipi/config/sipi.knora-docker-test-config.lua +++ b/sipi/config/sipi.knora-docker-test-config.lua @@ -46,7 +46,7 @@ sipi = { -- max_post_size = '30M', - -- + -- -- indicates the path to the root of the image directory. Depending on the settings of the variable -- "prefix_as_path" the images are search at // (prefix_as_path = TRUE) -- or / (prefix_as_path = FALSE). Please note that "prefix" and "imageid" are @@ -178,21 +178,6 @@ fileserver = { -- Custom routes. Each route is URL path associated with a Lua script. -- routes = { - { - method = 'POST', - route = '/make_thumbnail', - script = 'make_thumbnail.lua' - }, - { - method = 'POST', - route = '/convert_from_path', - script = 'convert_from_path.lua' - }, - { - method = 'POST', - route = '/convert_from_file', - script = 'convert_from_file.lua' - }, { method = 'POST', route = '/admin_upload', @@ -229,4 +214,4 @@ routes = { script = 'test_knora_session_cookie.lua' } -} +} \ No newline at end of file diff --git a/sipi/config/sipi.knora-local-config.lua b/sipi/config/sipi.knora-local-config.lua index b35db97d02..7b7c0a9e13 100644 --- a/sipi/config/sipi.knora-local-config.lua +++ b/sipi/config/sipi.knora-local-config.lua @@ -162,21 +162,6 @@ fileserver = { -- executes the given script defined below -- routes = { - { - method = 'POST', - route = '/make_thumbnail', - script = 'make_thumbnail.lua' - }, - { - method = 'POST', - route = '/convert_from_path', - script = 'convert_from_path.lua' - }, - { - method = 'POST', - route = '/convert_from_file', - script = 'convert_from_file.lua' - }, { method = 'POST', route = '/upload', @@ -207,4 +192,4 @@ routes = { route = '/test_knora_session_cookie', script = 'test_knora_session_cookie.lua' } -} +} \ No newline at end of file diff --git a/sipi/docker-compose.yml b/sipi/docker-compose.yml index 4b2b914d29..85a6d4c692 100644 --- a/sipi/docker-compose.yml +++ b/sipi/docker-compose.yml @@ -2,46 +2,46 @@ version: '3' services: - # sipi using default (production-like) configuration with additional routes for testing - sipi: - image: ${SIPI_IMAGE} - container_name: sipi - ports: - - "1024:1024" - volumes: - - $PWD/config:/sipi/config - - $PWD/scripts:/sipi/scripts - - /tmp:/tmp - - $HOME:$HOME - - $PWD/images:/sipi/images - - $PWD/server:/sipi/server - extra_hosts: - - api:${DOCKERHOST} - restart: unless-stopped - environment: - - SIPI_EXTERNAL_PROTOCOL=http - - SIPI_EXTERNAL_HOSTNAME=0.0.0.0 - - SIPI_EXTERNAL_PORT=1024 - command: --config=/sipi/config/sipi.knora-docker-config.lua + # sipi using default (production-like) configuration with additional routes for testing + sipi: + image: ${SIPI_IMAGE} + container_name: sipi + ports: + - "1024:1024" + volumes: + - $PWD/config:/sipi/config + - $PWD/scripts:/sipi/scripts + - /tmp:/tmp + - $HOME:$HOME + - $PWD/images:/sipi/images + - $PWD/server:/sipi/server + extra_hosts: + - api:${DOCKERHOST} + restart: unless-stopped + environment: + - SIPI_EXTERNAL_PROTOCOL=http + - SIPI_EXTERNAL_HOSTNAME=0.0.0.0 + - SIPI_EXTERNAL_PORT=1024 + command: --config=/sipi/config/sipi.knora-docker-config.lua - # sipi using configuration which disables authentication - sipi-no-auth: - image: ${SIPI_IMAGE} - container_name: sipi-no-auth - ports: - - "1024:1024" - volumes: - - $PWD/config:/sipi/config - - $PWD/scripts:/sipi/scripts - - /tmp:/tmp - - $HOME:$HOME - - $PWD/images:/sipi/images - - $PWD/server:/sipi/server - extra_hosts: - - api:${DOCKERHOST} - restart: unless-stopped - environment: - - SIPI_EXTERNAL_PROTOCOL=http - - SIPI_EXTERNAL_HOSTNAME=0.0.0.0 - - SIPI_EXTERNAL_PORT=1024 - command: --config=/sipi/config/sipi.knora-docker-no-auth-config.lua + # sipi using configuration which disables authentication + sipi-no-auth: + image: ${SIPI_IMAGE} + container_name: sipi-no-auth + ports: + - "1024:1024" + volumes: + - $PWD/config:/sipi/config + - $PWD/scripts:/sipi/scripts + - /tmp:/tmp + - $HOME:$HOME + - $PWD/images:/sipi/images + - $PWD/server:/sipi/server + extra_hosts: + - api:${DOCKERHOST} + restart: unless-stopped + environment: + - SIPI_EXTERNAL_PROTOCOL=http + - SIPI_EXTERNAL_HOSTNAME=0.0.0.0 + - SIPI_EXTERNAL_PORT=1024 + command: --config=/sipi/config/sipi.knora-docker-no-auth-config.lua \ No newline at end of file diff --git a/sipi/scripts/convert_from_file.lua b/sipi/scripts/convert_from_file.lua deleted file mode 100644 index ca83434d5b..0000000000 --- a/sipi/scripts/convert_from_file.lua +++ /dev/null @@ -1,208 +0,0 @@ --- Copyright © 2015-2019 the contributors (see Contributors.md). --- --- This file is part of Knora. --- --- Knora is free software: you can redistribute it and/or modify --- it under the terms of the GNU Affero General Public License as published --- by the Free Software Foundation, either version 3 of the License, or --- (at your option) any later version. --- --- Knora is distributed in the hope that it will be useful, --- but WITHOUT ANY WARRANTY; without even the implied warranty of --- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the --- GNU Affero General Public License for more details. --- --- You should have received a copy of the GNU Affero General Public --- License along with Knora. If not, see . - --- Knora GUI-case: Sipi has already saved the file that is supposed to be converted --- the file was saved to: config.imgroot .. '/tmp/' (route make_thumbnail) - -require "send_response" - -local success, errmsg = server.setBuffer() -if not success then - send_error(500, "server.setBuffer() failed: " .. errmsg) - return -end - -if server.post == nil then - send_error(400, PARAMETERS_INCORRECT .. " (post)") - return -end - --- --- check if the project directory is available, otherwise create it. --- - -local prefix = server.post['prefix'] - -if prefix == nil then - send_error(400, PARAMETERS_INCORRECT .. " (prefix)") - return -end - -local projectDir = config.imgroot .. '/' .. prefix .. '/' - -local exists -success, exists = server.fs.exists(projectDir) - -if not success then - send_error(500, "server.fs.exists() failed: " .. exists) - return -end - -if not exists then - local error_msg - success, error_msg = server.fs.mkdir(projectDir, 511) - - if not success then - send_error(500, "server.fs.mkdir() failed: " .. error_msg) - return - end -end - -local originalFilename = server.post['originalFilename'] -local originalMimeType = server.post['originalMimeType'] -local filename = server.post['filename'] - --- check if all the expected params are set -if originalFilename == nil then - send_error(400, PARAMETERS_INCORRECT .. " (originalFilename)") - return -end - -if originalMimeType == nil then - send_error(400, PARAMETERS_INCORRECT .. " (originalMimeType)") - return -end - -if filename == nil then - send_error(400, PARAMETERS_INCORRECT .. " (filename)") - return -end - --- file with name given in param "filename" has been saved by make_thumbnail.lua beforehand -local tmpDir = config.imgroot .. '/tmp/' - -local hashed_filename -success, hashed_filename = helper.filename_hash(filename) - -if not success then - send_error(500, "helper.filename_hash() failed: " .. hashed_filename) - return -end - -local sourcePath = tmpDir .. hashed_filename - --- check if source is readable -local readable -success, readable = server.fs.is_readable(sourcePath) -if not success then - send_error(500, "server.fs.is_readable() failed: " .. readable) - return -end - -if not readable then - send_error(400, FILE_NOT_READABLE .. sourcePath) - return -end - --- all params are set - -local baseName -success, baseName = server.uuid62() -if not success then - send_error(500, "server.uuid62() failed: " .. baseName) - return -end - --- --- create full quality image (jp2) --- -local fullImg -success, fullImg = SipiImage.new(sourcePath) -if not success then - send_error(500, "SipiImage.new() failed: " .. fullImg) - return -end - -local submitted_mimetype -success, submitted_mimetype = server.parse_mimetype(originalMimeType) - -if not success then - send_error(400, "Couldn't parse mimetype: " .. originalMimeType) - return -end - -local check -success, check = fullImg:mimetype_consistency(submitted_mimetype.mimetype, originalFilename) - -if not success then - send_error(500, "convert_from_file.lua: fullImg:mimetype_consistency() failed: " .. check) - return -end - --- if check returns false, the user's input is invalid -if not check then - send_error(400, MIMETYPES_INCONSISTENCY) - return -end - -local fullImgName = baseName .. ".jp2" - --- --- create new full quality image file path with sublevels: --- -local newFilePath -success, newFilePath = helper.filename_hash(fullImgName); -if not success then - send_error(500, "helper.filename_hash() failed: " .. newFilePath) - return -end - -local fullDims -success, fullDims = fullImg:dims() -if not success then - send_error(500, "fullImg:dims() failed: " .. fullDims) - return -end - -fullImg:write(projectDir .. newFilePath) - --- create thumbnail (jpg) -local thumbImg -success, thumbImg = SipiImage.new(sourcePath, { size = config.thumb_size }) -if not success then - send_error(500, "SipiImage.new() failed: " .. thumbImg) - return -end - -local thumbDims -success, thumbDims = thumbImg:dims() -if not success then - send_error(500, "thumbImg:dims() failed: " .. thumbDims) - return -end - --- --- delete tmp file --- -success, errmsg = server.fs.unlink(sourcePath) -if not success then - send_error(500, "server.fs.unlink() failed: " .. errmsg) - return -end - -result = { - status = 0, - mimetype_full = "image/jp2", - filename_full = fullImgName, - nx_full = fullDims.nx, - ny_full = fullDims.ny, - original_mimetype = originalMimeType, - original_filename = originalFilename, - file_type = "image" -} - -send_success(result) diff --git a/sipi/scripts/convert_from_path.lua b/sipi/scripts/convert_from_path.lua deleted file mode 100644 index ca9c2bb065..0000000000 --- a/sipi/scripts/convert_from_path.lua +++ /dev/null @@ -1,248 +0,0 @@ --- Copyright © 2015-2019 the contributors (see Contributors.md). --- --- This file is part of Knora. --- --- Knora is free software: you can redistribute it and/or modify --- it under the terms of the GNU Affero General Public License as published --- by the Free Software Foundation, either version 3 of the License, or --- (at your option) any later version. --- --- Knora is distributed in the hope that it will be useful, --- but WITHOUT ANY WARRANTY; without even the implied warranty of --- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the --- GNU Affero General Public License for more details. --- --- You should have received a copy of the GNU Affero General Public --- License along with Knora. If not, see . - --- handles the Knora non GUI-case: Knora uploaded a file to sourcePath - -require "send_response" -require "file_info" - -local success, errmsg = server.setBuffer() -if not success then - send_error(500, "server.setBuffer() failed: " .. errmsg) - return -end - -if server.post == nil then - send_error(400, PARAMETERS_INCORRECT) - return -end - -local originalFilename = server.post['originalFilename'] -local originalMimeType = server.post['originalMimeType'] -local sourcePath = server.post['source'] -local prefix = server.post['prefix'] - --- check if all the expected params are set -if originalFilename == nil or originalMimeType == nil or sourcePath == nil or prefix == nil then - send_error(400, PARAMETERS_INCORRECT) - return -end - --- all params are set - --- check if source is readable - -local readable -success, readable = server.fs.is_readable(sourcePath) -if not success then - send_error(500, "server.fs.is_readable() failed: " .. readable) - return -end - -if not readable then - send_error(400, FILE_NOT_READABLE .. sourcePath) - return -end - --- check for the mimetype of the file -local mime_info -success, mime_info = server.file_mimetype(sourcePath) - -if not success then - send_error(500, "server.file_mimetype() failed: " .. mime_info) - return -end - -local mime_type = mime_info["mimetype"] - --- check that the submitted mimetype is the same as the real mimetype of the file - -local submitted_mimetype -success, submitted_mimetype = server.parse_mimetype(originalMimeType) - -if not success then - send_error(400, "Couldn't parse mimetype: " .. originalMimeType) - return -end - -if (mime_type ~= submitted_mimetype.mimetype) then - send_error(400, MIMETYPES_INCONSISTENCY) - return -end - --- handle the file depending on its media type (image, text file) -local file_info = get_file_info(originalFilename, mime_type) - --- in case of an unsupported mimetype, the function returns false -if not file_info then - send_error(400, "Mimetype '" .. mime_type .. "' is not supported") - return -end - -local media_type = file_info["media_type"] - --- depending on the media type, decide what to do -if media_type == IMAGE then - - -- it is an image - - -- - -- check if project directory is available, if not, create it - -- - - local projectDir = config.imgroot .. '/' .. prefix .. '/' - - local exists - success, exists = server.fs.exists(projectDir) - if not success then - send_error(500, "server.fs.exists() failed: " .. exists) - return - end - - if not exists then - success, errmsg = server.fs.mkdir(projectDir, 511) - if not success then - send_error(500, "server.fs.mkdir() failed: " .. errmsg) - return - end - end - - local baseName - success, baseName = server.uuid62() - if not success then - send_error(500, "server.uuid62() failed: " .. baseName) - return - end - - - -- - -- create full quality image (jp2) - -- - local fullImg - success, fullImg = SipiImage.new(sourcePath) - if not success then - send_error(500, "SipiImage.new() failed: " .. fullImg) - return - end - - local check - success, check = fullImg:mimetype_consistency(submitted_mimetype.mimetype, originalFilename) - - if not success then - send_error(500, "convert_from_path.lua: fullImg:mimetype_consistency() failed: " .. check) - return - end - - -- if check returns false, the user's input is invalid - if not check then - send_error(400, MIMETYPES_INCONSISTENCY) - return - end - - local fullDims - success, fullDims = fullImg:dims() - if not success then - send_error(500, "fullImg:dims() failed: " .. fullDims) - return - end - - local fullImgName = baseName .. ".jp2" - - -- - -- create new full quality image file path with sublevels: - -- - local newFilePath - success, newFilePath = helper.filename_hash(fullImgName) - if not success then - send_error(500, "helper.filename_hash: " .. newFilePath) - return - end - - success, errmsg = fullImg:write(projectDir .. newFilePath) - if not success then - send_error(500, "fullImg:write() failed: " .. errmsg) - return - end - - result = { - mimetype_full = "image/jp2", - filename_full = fullImgName, - nx_full = fullDims.nx, - ny_full = fullDims.ny, - original_mimetype = originalMimeType, - original_filename = originalFilename, - file_type = IMAGE - } - - send_success(result) - -elseif media_type == TEXT then - - -- it's a text file - - -- - -- check if project directory is available, if not, create it - -- - local projectFileDir = config.imgroot .. '/' .. prefix .. '/' - local exists - success, exists = server.fs.exists(projectFileDir) - if not success then - send_error(500, "server.fs.exists() failed: " .. exists) - return - end - - if not exists then - success, errmsg = server.fs.mkdir(projectFileDir, 511) - if not success then - send_error(500, "server.fs.mkdir() failed: " .. errmsg) - return - end - end - - local baseName - success, baseName = server.uuid62() - if not success then - send_error(500, "server.uuid62() failed: " .. baseName) - return - end - - local filename = baseName .. "." .. file_info["extension"] - local filePath = projectFileDir .. filename - - local result - success, result = server.fs.copyFile(sourcePath, filePath) - if not success then - send_error(500, "server.fs.copyFile() failed: " .. result) - return - end - - server.log("Copied " .. sourcePath .. " to " .. filePath, server.loglevel.LOG_DEBUG) - - result = { - mimetype = submitted_mimetype.mimetype, - charset = submitted_mimetype.charset, - file_type = TEXT, - filename = filename, - original_mimetype = originalMimeType, - original_filename = originalFilename - } - - send_success(result) - -else - send_error(400, "Unsupported mimetype: " .. mime_type) -end diff --git a/sipi/scripts/get_knora_session.lua b/sipi/scripts/get_knora_session.lua index e93d0c69e1..7982297ab6 100644 --- a/sipi/scripts/get_knora_session.lua +++ b/sipi/scripts/get_knora_session.lua @@ -37,8 +37,7 @@ function get_session_id(cookie) -- space is also treated as a separator -- returns nil if it cannot find the session id (pattern does not match) server.log("extracted cookie: " .. cookie, server.loglevel.LOG_DEBUG) - print("extracted cookie: " .. cookie) - session_id = string.match(cookie, "KnoraAuthentication=([^%s;]+)") + local session_id = string.match(cookie, "KnoraAuthentication=([^%s;]+)") return session_id diff --git a/sipi/scripts/jwt.lua b/sipi/scripts/jwt.lua index bf172d437d..cc80a3c1a5 100644 --- a/sipi/scripts/jwt.lua +++ b/sipi/scripts/jwt.lua @@ -25,7 +25,6 @@ function get_knora_token() local token = get_token() if token == nil then - send_error(401, "Not a Knora token") return nil end @@ -57,15 +56,15 @@ function get_token() local expiration_date = token["exp"] if expiration_date == nil then - send_error(401, "Token has no expiry date") - return nil + send_error(401, "Token has no expiry date") + return nil end local systime = server.systime() if (expiration_date <= systime) then - send_error(401, "Expired token") - return nil + send_error(401, "Expired token") + return nil end local audience = token["aud"] @@ -76,4 +75,4 @@ function get_token() end return token -end +end \ No newline at end of file diff --git a/sipi/scripts/make_thumbnail.lua b/sipi/scripts/make_thumbnail.lua deleted file mode 100644 index 7bdecdd566..0000000000 --- a/sipi/scripts/make_thumbnail.lua +++ /dev/null @@ -1,188 +0,0 @@ --- Copyright © 2015-2019 the contributors (see Contributors.md). --- --- This file is part of Knora. --- --- Knora is free software: you can redistribute it and/or modify --- it under the terms of the GNU Affero General Public License as published --- by the Free Software Foundation, either version 3 of the License, or --- (at your option) any later version. --- --- Knora is distributed in the hope that it will be useful, --- but WITHOUT ANY WARRANTY; without even the implied warranty of --- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the --- GNU Affero General Public License for more details. --- --- You should have received a copy of the GNU Affero General Public --- License along with Knora. If not, see . - --- Knora GUI-case: create a thumbnail - -require "send_response" -require "util" - -success, errormsg = server.setBuffer() -if not success then - return -1 -end - --- --- check if temporary directory is available, if not, create it. --- -local tmpDir = config.imgroot .. '/tmp/' - -local success, exists = server.fs.exists(tmpDir) -if not success then - send_error(500, "server.fs.exists() failed: " .. exists) - return -end - -if not exists then - local result - success, result = server.fs.mkdir(tmpDir, 511) - if not success then - send_error(500, "server.fs.mkdir() failed: " .. result) - return - end -end - --- --- check if thumbs directory is available, if not, create it. --- -local thumbsDir = config.imgroot .. '/thumbs/' - -success, exists = server.fs.exists(thumbsDir) -if not success then - send_error(500, "server.fs.exists() failed: " .. exists) - return -end -if not exists then - local result - success, result = server.fs.mkdir(thumbsDir, 511) - if not success then - send_error(500, "server.fs.mkdir() failed: " .. result) - return - end -end - --- --- check if something was uploaded --- -if server.uploads == nil then - send_error(400, "no image uploaded") - return -end - -for imgindex, imgparam in pairs(server.uploads) do - - -- - -- copy the uploaded file (from config.tmpdir) to tmpDir so we have access to it in later requests - -- - - -- create tmp name - local tmpName - success, tmpName = server.uuid62() - if not success then - send_error(500, "server.uuid62() failed: " .. tmpName) - return - end - - local hashed_tmpName - success, hashed_tmpName = helper.filename_hash(tmpName) - - if not success then - send_error(500, "helper.filename_hash() failed: " .. hashed_tmpName) - return - end - - local tmpPath = tmpDir .. hashed_tmpName - - local result - success, result = server.copyTmpfile(imgindex, tmpPath) - if not success then - send_error(500, "server.copyTmpfile() failed: " .. result) - return - end - - - -- - -- create a thumnail sized SipiImage - -- - local thumbImg - success, thumbImg = SipiImage.new(tmpPath, {size = config.thumb_size}) - if not success then - send_error(500, "SipiImage.new() failed: " .. thumbImg) - return - end - - local filename = imgparam["origname"] - local submitted_mimetype - success, submitted_mimetype = server.parse_mimetype(imgparam["mimetype"]) - - if not success then - send_error(400, "Couldn't parse mimetype: " .. imgparam["mimetype"]) - return - end - - local check - success, check = thumbImg:mimetype_consistency(submitted_mimetype.mimetype, filename) - if not success then - send_error(500, "make_thumbnail.lua: thumbImg:mimetype_consistency() failed: " .. check) - return - end - - -- - -- if check returns false, the user's input is invalid - -- - - if not check then - send_error(400, MIMETYPES_INCONSISTENCY) - return -1 - end - - -- - -- get the dimensions - -- - local dims - success, dims = thumbImg:dims() - if not success then - send_error(500, "thumbImg:dims() failed: " .. dims) - return - end - - - -- - -- write the thumbnail file - -- - local thumbName = tmpName .. ".jpg" - local thumbPath = thumbsDir .. thumbName - - server.log("thumbnail path: " .. thumbPath, server.loglevel.LOG_DEBUG) - - success, result = thumbImg:write(thumbPath) - if not success then - send_error(500, "thumbImg:write() failed: " .. result) - return - end - - -- #snip_marker - server.log("make_thumbnail - external_protocol: " .. get_external_protocol(), server.loglevel.LOG_DEBUG) - - server.log("make_thumbnail - external_hostname: " .. get_external_hostname(), server.loglevel.LOG_DEBUG) - - server.log("make_thumbnail - external_port: " .. get_external_port(), server.loglevel.LOG_DEBUG) - - answer = { - nx_thumb = dims.nx, - ny_thumb = dims.ny, - mimetype_thumb = 'image/jpeg', - preview_path = get_external_protocol() .. "://" .. get_external_hostname() .. ":" .. get_external_port() .."/thumbs/" .. thumbName .. "/full/max/0/default.jpg", - filename = tmpName, -- make this a IIIF URL - original_mimetype = submitted_mimetype.mimetype, - original_filename = filename, - file_type = 'IMAGE' - } - -- #snip_marker - -end - -send_success(answer) diff --git a/sipi/scripts/upload.lua b/sipi/scripts/upload.lua index cde54b95f7..3a7c7debd5 100644 --- a/sipi/scripts/upload.lua +++ b/sipi/scripts/upload.lua @@ -39,7 +39,7 @@ end local token = get_knora_token() if token == nil then - return + return end -- Check that the temp folder is created diff --git a/webapi/src/it/scala/org/knora/webapi/ITKnoraLiveSpec.scala b/webapi/src/it/scala/org/knora/webapi/ITKnoraLiveSpec.scala index 26f126b40b..fd8c973586 100644 --- a/webapi/src/it/scala/org/knora/webapi/ITKnoraLiveSpec.scala +++ b/webapi/src/it/scala/org/knora/webapi/ITKnoraLiveSpec.scala @@ -19,9 +19,13 @@ package org.knora.webapi +import java.io.File + import akka.actor.{ActorRef, ActorSystem, Props} +import akka.event.LoggingAdapter import akka.http.scaladsl.Http import akka.http.scaladsl.client.RequestBuilding +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import akka.http.scaladsl.model._ import akka.stream.Materializer import com.typesafe.config.{Config, ConfigFactory} @@ -34,9 +38,9 @@ import org.knora.webapi.util.{StartupUtils, StringFormatter} import org.scalatest.matchers.should.Matchers import org.scalatest.wordspec.AnyWordSpecLike import org.scalatest.{BeforeAndAfterAll, Suite} -import spray.json.{JsObject, _} +import spray.json._ -import scala.concurrent.duration.{Duration, _} +import scala.concurrent.duration._ import scala.concurrent.{Await, ExecutionContext} import scala.languageFeature.postfixOps @@ -68,7 +72,7 @@ class ITKnoraLiveSpec(_system: ActorSystem) extends Core with StartupUtils with /* Needs to be initialized before any responders */ StringFormatter.initForTest() - val log = akka.event.Logging(system, this.getClass) + val log: LoggingAdapter = akka.event.Logging(system, this.getClass) lazy val appActor: ActorRef = system.actorOf(Props(new ApplicationActor with LiveManagers), name = APPLICATION_MANAGER_ACTOR_NAME) @@ -145,4 +149,90 @@ class ITKnoraLiveSpec(_system: ActorSystem) extends Core with StartupUtils with val responseBodyStr = getResponseString(request) JsonLDUtil.parseJsonLD(responseBodyStr) } + + /** + * Represents a file to be uploaded to Sipi. + * + * @param path the path of the file. + * @param mimeType the MIME type of the file. + */ + protected case class FileToUpload(path: String, mimeType: ContentType) + + /** + * Represents an image file to be uploaded to Sipi. + * + * @param fileToUpload the file to be uploaded. + * @param width the image's width in pixels. + * @param height the image's height in pixels. + */ + protected case class InputFile(fileToUpload: FileToUpload, width: Int, height: Int) + + /** + * Represents the information that Sipi returns about each file that has been uploaded. + * + * @param originalFilename the original filename that was submitted to Sipi. + * @param internalFilename Sipi's internal filename for the stored temporary file. + * @param temporaryUrl the URL at which the temporary file can be accessed. + * @param fileType `image`, `text`, or `document`. + */ + protected case class SipiUploadResponseEntry(originalFilename: String, internalFilename: String, temporaryUrl: String, fileType: String) + + /** + * Represents Sipi's response to a file upload request. + * + * @param uploadedFiles the information about each file that was uploaded. + */ + protected case class SipiUploadResponse(uploadedFiles: Seq[SipiUploadResponseEntry]) + + object SipiUploadResponseJsonProtocol extends SprayJsonSupport with DefaultJsonProtocol { + implicit val sipiUploadResponseEntryFormat: RootJsonFormat[SipiUploadResponseEntry] = jsonFormat4(SipiUploadResponseEntry) + implicit val sipiUploadResponseFormat: RootJsonFormat[SipiUploadResponse] = jsonFormat1(SipiUploadResponse) + } + + import SipiUploadResponseJsonProtocol._ + + /** + * Uploads a file to Sipi and returns the information in Sipi's response. + * + * @param loginToken the login token to be included in the request to Sipi. + * @param filesToUpload the files to be uploaded. + * @return a [[SipiUploadResponse]] representing Sipi's response. + */ + protected def uploadToSipi(loginToken: String, filesToUpload: Seq[FileToUpload]): SipiUploadResponse = { + // Make a multipart/form-data request containing the files. + + val formDataParts: Seq[Multipart.FormData.BodyPart] = filesToUpload.map { + fileToUpload => + val fileToSend = new File(fileToUpload.path) + assert(fileToSend.exists(), s"File ${fileToUpload.path} does not exist") + + Multipart.FormData.BodyPart( + "file", + HttpEntity.fromPath(fileToUpload.mimeType, fileToSend.toPath), + Map("filename" -> fileToSend.getName) + ) + } + + val sipiFormData = Multipart.FormData(formDataParts: _*) + + // Send Sipi the file in a POST request. + val sipiRequest = Post(s"$baseInternalSipiUrl/upload?token=$loginToken", sipiFormData) + + val sipiUploadResponseJson: JsObject = getResponseJson(sipiRequest) + // println(sipiUploadResponseJson.prettyPrint) + val sipiUploadResponse: SipiUploadResponse = sipiUploadResponseJson.convertTo[SipiUploadResponse] + + // Request the temporary file from Sipi. + for (responseEntry <- sipiUploadResponse.uploadedFiles) { + val sipiGetTmpFileRequest: HttpRequest = if (responseEntry.fileType == "image") { + Get(responseEntry.temporaryUrl + "/full/full/0/default.jpg") + } else { + Get(responseEntry.temporaryUrl) + } + + checkResponseOK(sipiGetTmpFileRequest) + } + + sipiUploadResponse + } } diff --git a/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiIntegrationV1ITSpec.scala b/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiIntegrationV1ITSpec.scala index 6cad44677e..b9dffc186c 100644 --- a/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiIntegrationV1ITSpec.scala +++ b/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiIntegrationV1ITSpec.scala @@ -22,11 +22,13 @@ package org.knora.webapi.e2e.v1 import java.io.{File, FileInputStream, FileOutputStream} import java.net.URLEncoder +import akka.http.scaladsl.model._ import akka.http.scaladsl.model.headers._ -import akka.http.scaladsl.model.{HttpEntity, _} +import akka.http.scaladsl.unmarshalling.Unmarshal import com.typesafe.config.{Config, ConfigFactory} import org.knora.webapi._ import org.knora.webapi.messages.store.triplestoremessages.{RdfDataObject, TriplestoreJsonProtocol} +import org.knora.webapi.messages.v2.routing.authenticationmessages.{AuthenticationV2JsonProtocol, LoginResponse} import org.knora.webapi.util.{FileUtil, MutableTestIri} import org.xmlunit.builder.{DiffBuilder, Input} import org.xmlunit.diff.Diff @@ -50,21 +52,20 @@ object KnoraSipiIntegrationV1ITSpec { * End-to-End (E2E) test specification for testing Knora-Sipi integration. Sipi must be running with the config file * `sipi.knora-docker-config.lua`. */ -class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV1ITSpec.config) with TriplestoreJsonProtocol { +class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV1ITSpec.config) with AuthenticationV2JsonProtocol with TriplestoreJsonProtocol { override lazy val rdfDataObjects: List[RdfDataObject] = List( RdfDataObject(path = "_test_data/all_data/incunabula-data.ttl", name = "http://www.knora.org/data/0803/incunabula"), RdfDataObject(path = "_test_data/all_data/anything-data.ttl", name = "http://www.knora.org/data/0001/anything") ) - private val username = "root@example.com" + private val userEmail = "root@example.com" private val password = "test" private val pathToChlaus = "_test_data/test_route/images/Chlaus.jpg" private val pathToMarbles = "_test_data/test_route/images/marbles.tif" private val pathToMarblesWithWrongExtension = "_test_data/test_route/images/marbles_with_wrong_extension.jpg" private val pathToXSLTransformation = "_test_data/test_route/texts/letterToHtml.xsl" private val pathToMappingWithXSLT = "_test_data/test_route/texts/mappingForLetterWithXSLTransformation.xml" - private val firstPageIri = new MutableTestIri private val secondPageIri = new MutableTestIri private val pathToBEOLBodyXSLTransformation = "_test_data/test_route/texts/beol/standoffToTEI.xsl" @@ -132,7 +133,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV case _ => false } - case _ => throw InvalidApiJsonException("bulk import response should have memeber 'createdResources'") + case _ => throw InvalidApiJsonException("bulk import response should have member 'createdResources'") } if (resIriOption.nonEmpty) { @@ -153,175 +154,37 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV } "Knora and Sipi" should { + var loginToken: String = "" - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 - "create an 'incunabula:page' with binary data" ignore { + "log in as a Knora user" in { + /* Correct username and correct password */ - // JSON describing the resource to be created. - val paramsPageWithBinaries = + val params = s""" |{ - | "restype_id": "http://www.knora.org/ontology/0803/incunabula#page", - | "label": "test", - | "project_id": "http://rdfh.ch/projects/0803", - | "properties": { - | "http://www.knora.org/ontology/0803/incunabula#pagenum": [ - | { - | "richtext_value": { - | "utf8str": "test_page" - | } - | } - | ], - | "http://www.knora.org/ontology/0803/incunabula#origname": [ - | { - | "richtext_value": { - | "utf8str": "test" - | } - | } - | ], - | "http://www.knora.org/ontology/0803/incunabula#partOf": [ - | { - | "link_value": "http://rdfh.ch/0803/5e77e98d2603" - | } - | ], - | "http://www.knora.org/ontology/0803/incunabula#seqnum": [ - | { - | "int_value": 999 - | } - | ] - | } + | "email": "$userEmail", + | "password": "$password" |} - """.stripMargin - - // The image to be uploaded. - val fileToSend = new File(pathToChlaus) - assert(fileToSend.exists(), s"File $pathToChlaus does not exist") - - // A multipart/form-data request containing the image and the JSON. - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "json", - HttpEntity(ContentTypes.`application/json`, paramsPageWithBinaries) - ), - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - // Send the multipart/form-data request to the Knora API server. - val knoraPostRequest = Post(baseApiUrl + "/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(username, password)) - val knoraPostResponseJson = getResponseJson(knoraPostRequest) - - // Get the IRI of the newly created resource. - val resourceIri: String = knoraPostResponseJson.fields("res_id").asInstanceOf[JsString].value - firstPageIri.set(resourceIri) - - // Request the resource from the Knora API server. - val knoraRequestNewResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(resourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(username, password)) - val knoraNewResourceJson = getResponseJson(knoraRequestNewResource) - - // Get the URL of the image that was uploaded. - val iiifUrl = knoraNewResourceJson.fields.get("resinfo") match { - case Some(resinfo: JsObject) => - resinfo.fields.get("locdata") match { - case Some(locdata: JsObject) => - locdata.fields.get("path") match { - case Some(JsString(path)) => path - case None => throw InvalidApiJsonException("no 'path' given") - case _ => throw InvalidApiJsonException("'path' could not pe parsed correctly") - } - case None => throw InvalidApiJsonException("no 'locdata' given") - - case _ => throw InvalidApiJsonException("'locdata' could not pe parsed correctly") - } - - case None => throw InvalidApiJsonException("no 'resinfo' given") - - case _ => throw InvalidApiJsonException("'resinfo' could not pe parsed correctly") - } - - // Request the image from Sipi. - val sipiGetRequest = Get(iiifUrl) ~> addCredentials(BasicHttpCredentials(username, password)) - checkResponseOK(sipiGetRequest) - } - - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 - "change an 'incunabula:page' with binary data" ignore { - // The image to be uploaded. - val fileToSend = new File(pathToMarbles) - assert(fileToSend.exists(), s"File $pathToMarbles does not exist") - - // A multipart/form-data request containing the image. - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/tiff`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - // Send the image in a PUT request to the Knora API server. - val knoraPutRequest = Put(baseApiUrl + "/v1/filevalue/" + URLEncoder.encode(firstPageIri.get, "UTF-8"), formData) ~> addCredentials(BasicHttpCredentials(username, password)) - checkResponseOK(knoraPutRequest) - } - - "reject an 'incunabula:page' with binary data if the file extension is incorrect" ignore { // Ignored because of issue #1531. - // The image to be uploaded. - val fileToSend = new File(pathToMarblesWithWrongExtension) - assert(fileToSend.exists(), s"File $pathToMarblesWithWrongExtension does not exist") - - // A multipart/form-data request containing the image. - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/tiff`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) + """.stripMargin - // Send the image in a PUT request to the Knora API server. - val knoraPutRequest = Put(baseApiUrl + "/v1/filevalue/" + URLEncoder.encode(firstPageIri.get, "UTF-8"), formData) ~> addCredentials(BasicHttpCredentials(username, password)) + val request = Post(baseApiUrl + s"/v2/authentication", HttpEntity(ContentTypes.`application/json`, params)) + val response: HttpResponse = singleAwaitingRequest(request) + assert(response.status == StatusCodes.OK) - val exception = intercept[AssertionException] { - checkResponseOK(knoraPutRequest) - } + val lr: LoginResponse = Await.result(Unmarshal(response.entity).to[LoginResponse], 1.seconds) + loginToken = lr.token - assert(exception.getMessage.contains("MIME type and/or file extension are inconsistent")) + loginToken.nonEmpty should be(true) } "create an 'incunabula:page' with parameters" in { - // The image to be uploaded. - val fileToSend = new File(pathToChlaus) - assert(fileToSend.exists(), s"File $pathToChlaus does not exist") - - // A multipart/form-data request containing the image. - val sipiFormData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) + // Upload the image to Sipi. + val sipiUploadResponse: SipiUploadResponse = uploadToSipi( + loginToken = loginToken, + filesToUpload = Seq(FileToUpload(path = pathToChlaus, mimeType = MediaTypes.`image/tiff`)) ) - // Send a POST request to Sipi, asking it to make a thumbnail of the image. - val sipiRequest = Post(baseInternalSipiUrl + "/make_thumbnail", sipiFormData) ~> addCredentials(BasicHttpCredentials(username, password)) - val sipiResponseJson = getResponseJson(sipiRequest) - - // Request the thumbnail from Sipi. - val jsonFields = sipiResponseJson.fields - val previewUrl = jsonFields("preview_path").asInstanceOf[JsString].value - val sipiGetRequest = Get(previewUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl)) ~> addCredentials(BasicHttpCredentials(username, password)) - checkResponseOK(sipiGetRequest) - - val fileParams = JsObject( - Map( - "originalFilename" -> jsonFields("original_filename"), - "originalMimeType" -> jsonFields("original_mimetype"), - "filename" -> jsonFields("filename") - ) - ) + val uploadedFile: SipiUploadResponseEntry = sipiUploadResponse.uploadedFiles.head val knoraParams = s""" @@ -339,14 +202,14 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV | ], | "http://www.knora.org/ontology/0803/incunabula#seqnum": [{"int_value": 99999999}] | }, - | "file": ${fileParams.compactPrint}, + | "file": "${uploadedFile.internalFilename}", | "label": "test page", | "project_id": "http://rdfh.ch/projects/0803" |} """.stripMargin // Send the JSON in a POST request to the Knora API server. - val knoraPostRequest = Post(baseApiUrl + "/v1/resources", HttpEntity(ContentTypes.`application/json`, knoraParams)) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPostRequest = Post(baseApiUrl + "/v1/resources", HttpEntity(ContentTypes.`application/json`, knoraParams)) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val knoraPostResponseJson = getResponseJson(knoraPostRequest) // Get the IRI of the newly created resource. @@ -354,49 +217,28 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV secondPageIri.set(resourceIri) // Request the resource from the Knora API server. - val knoraRequestNewResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(resourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraRequestNewResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(resourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(userEmail, password)) checkResponseOK(knoraRequestNewResource) } "change an 'incunabula:page' with parameters" in { - // The image to be uploaded. - val fileToSend = new File(pathToMarbles) - assert(fileToSend.exists(), s"File $pathToMarbles does not exist") - - // A multipart/form-data request containing the image. - val sipiFormData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/tiff`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) + // Upload the image to Sipi. + val sipiUploadResponse: SipiUploadResponse = uploadToSipi( + loginToken = loginToken, + filesToUpload = Seq(FileToUpload(path = pathToMarbles, mimeType = MediaTypes.`image/tiff`)) ) - // Send a POST request to Sipi, asking it to make a thumbnail of the image. - val sipiRequest = Post(baseInternalSipiUrl + "/make_thumbnail", sipiFormData) ~> addCredentials(BasicHttpCredentials(username, password)) - val sipiResponseJson = getResponseJson(sipiRequest) - - // Request the thumbnail from Sipi. - val jsonFields = sipiResponseJson.fields - val previewUrl = jsonFields("preview_path").asInstanceOf[JsString].value - val sipiGetRequest = Get(previewUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl)) ~> addCredentials(BasicHttpCredentials(username, password)) - checkResponseOK(sipiGetRequest) + val uploadedFile: SipiUploadResponseEntry = sipiUploadResponse.uploadedFiles.head // JSON describing the new image to Knora. val knoraParams = JsObject( Map( - "file" -> JsObject( - Map( - "originalFilename" -> jsonFields("original_filename"), - "originalMimeType" -> jsonFields("original_mimetype"), - "filename" -> jsonFields("filename") - ) - ) + "file" -> JsString(s"${uploadedFile.internalFilename}") ) ) // Send the JSON in a PUT request to the Knora API server. - val knoraPutRequest = Put(baseApiUrl + "/v1/filevalue/" + URLEncoder.encode(secondPageIri.get, "UTF-8"), HttpEntity(ContentTypes.`application/json`, knoraParams.compactPrint)) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPutRequest = Put(baseApiUrl + "/v1/filevalue/" + URLEncoder.encode(secondPageIri.get, "UTF-8"), HttpEntity(ContentTypes.`application/json`, knoraParams.compactPrint)) ~> addCredentials(BasicHttpCredentials(userEmail, password)) checkResponseOK(knoraPutRequest) } @@ -438,12 +280,11 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // Send the JSON in a POST request to the Knora API server. - val knoraPostRequest = Post(baseApiUrl + "/v1/resources", HttpEntity(ContentTypes.`application/json`, knoraParams.compactPrint)) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPostRequest = Post(baseApiUrl + "/v1/resources", HttpEntity(ContentTypes.`application/json`, knoraParams.compactPrint)) ~> addCredentials(BasicHttpCredentials(userEmail, password)) checkResponseOK(knoraPostRequest) } - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 - "create an 'p0803-incunabula:book' and an 'p0803-incunabula:page' with file parameters via XML import" ignore { + "create an 'p0803-incunabula:book' and an 'p0803-incunabula:page' with file parameters via XML import" in { val fileToUpload = new File(pathToChlaus) // To be able to run packaged tests inside Docker, we need to copy @@ -459,6 +300,14 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV val absoluteFilePath = dest.getAbsolutePath + // Upload the image to Sipi. + val sipiUploadResponse: SipiUploadResponse = uploadToSipi( + loginToken = loginToken, + filesToUpload = Seq(FileToUpload(path = absoluteFilePath, mimeType = MediaTypes.`image/tiff`)) + ) + + val uploadedFile: SipiUploadResponseEntry = sipiUploadResponse.uploadedFiles.head + val knoraParams = s""" | | | a page with an image - | + | | Chlaus | 1a | @@ -485,7 +334,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV val projectIri = URLEncoder.encode("http://rdfh.ch/projects/0803", "UTF-8") // Send the JSON in a POST request to the Knora API server. - val knoraPostRequest = Post(baseApiUrl + s"/v1/resources/xmlimport/$projectIri", HttpEntity(ContentType(MediaTypes.`application/xml`, HttpCharsets.`UTF-8`), knoraParams)) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPostRequest = Post(baseApiUrl + s"/v1/resources/xmlimport/$projectIri", HttpEntity(ContentType(MediaTypes.`application/xml`, HttpCharsets.`UTF-8`), knoraParams)) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val knoraPostResponseJson: JsObject = getResponseJson(knoraPostRequest) val createdResources = knoraPostResponseJson.fields("createdResources").asInstanceOf[JsArray].elements @@ -495,11 +344,11 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV val pageResourceIri = createdResources(1).asJsObject.fields("resourceIri").asInstanceOf[JsString].value // Request the book resource from the Knora API server. - val knoraRequestNewBookResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(bookResourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraRequestNewBookResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(bookResourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(userEmail, password)) checkResponseOK(knoraRequestNewBookResource) // Request the page resource from the Knora API server. - val knoraRequestNewPageResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(pageResourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraRequestNewPageResource = Get(baseApiUrl + "/v1/resources/" + URLEncoder.encode(pageResourceIri, "UTF-8")) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val pageJson: JsObject = getResponseJson(knoraRequestNewPageResource) val locdata = pageJson.fields("resinfo").asJsObject.fields("locdata").asJsObject val origname = locdata.fields("origname").asInstanceOf[JsString].value @@ -507,13 +356,14 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV assert(origname == dest.getName) // Request the file from Sipi. - val sipiGetRequest = Get(imageUrl) ~> addCredentials(BasicHttpCredentials(username, password)) + val sipiGetRequest = Get(imageUrl) ~> addCredentials(BasicHttpCredentials(userEmail, password)) checkResponseOK(sipiGetRequest) } - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 "create a TextRepresentation of type XSLTransformation and refer to it in a mapping" ignore { + // TODO: fix this when we can upload non-image files to Sipi (PR #1206). + // create an XSL transformation val knoraParams = JsObject( Map( @@ -540,7 +390,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // Send the JSON in a POST request to the Knora API server. - val knoraPostRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPostRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val responseJson: JsObject = getResponseJson(knoraPostRequest) @@ -579,13 +429,14 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // send mapping xml to route - val knoraPostRequest2 = Post(baseApiUrl + "/v1/mapping", formDataMapping) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPostRequest2 = Post(baseApiUrl + "/v1/mapping", formDataMapping) ~> addCredentials(BasicHttpCredentials(userEmail, password)) checkResponseOK(knoraPostRequest2) } - "create a sample BEOL letter" in { + "create a sample BEOL letter" ignore { + // TODO: fix this when we can upload non-image files to Sipi (PR #1206). val mapping = FileUtil.readTextFile(new File(pathToBEOLLetterMapping)) @@ -612,15 +463,15 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // send mapping xml to route - val knoraPostRequest = Post(baseApiUrl + "/v1/mapping", formDataMapping) ~> addCredentials(BasicHttpCredentials(username, password)) + val knoraPostRequest = Post(baseApiUrl + "/v1/mapping", formDataMapping) ~> addCredentials(BasicHttpCredentials(userEmail, password)) - val _: JsValue = getResponseJson(knoraPostRequest) + getResponseJson(knoraPostRequest) // create a letter via bulk import val bulkXML = FileUtil.readTextFile(new File(pathToBEOLBulkXML)) - val bulkRequest = Post(baseApiUrl + "/v1/resources/xmlimport/" + URLEncoder.encode("http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF", "UTF-8"), HttpEntity(ContentType(MediaTypes.`application/xml`, HttpCharsets.`UTF-8`), bulkXML)) ~> addCredentials(BasicHttpCredentials(username, password)) + val bulkRequest = Post(baseApiUrl + "/v1/resources/xmlimport/" + URLEncoder.encode("http://rdfh.ch/projects/yTerZGyxjZVqFMNNKXCDPF", "UTF-8"), HttpEntity(ContentType(MediaTypes.`application/xml`, HttpCharsets.`UTF-8`), bulkXML)) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val bulkResponse: JsObject = getResponseJson(bulkRequest) @@ -628,8 +479,8 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV } - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 "create a mapping for standoff conversion to TEI referring to an XSLT and also create a Gravsearch template and an XSLT for transforming TEI header data" ignore { + // TODO: fix this when we can upload non-image files to Sipi (PR #1206). // create an XSL transformation val standoffXSLTParams = JsObject( @@ -657,7 +508,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // Send the JSON in a POST request to the Knora API server. - val bodyXSLTRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(username, password)) + val bodyXSLTRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val bodyXSLTJson: JsObject = getResponseJson(bodyXSLTRequest) @@ -696,7 +547,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // send mapping xml to route - val mappingRequest = Post(baseApiUrl + "/v1/mapping", formDataMapping) ~> addCredentials(BasicHttpCredentials(username, password)) + val mappingRequest = Post(baseApiUrl + "/v1/mapping", formDataMapping) ~> addCredentials(BasicHttpCredentials(userEmail, password)) getResponseJson(mappingRequest) @@ -726,7 +577,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // Send the JSON in a POST request to the Knora API server. - val gravsearchTemplateRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formDataGravsearch) ~> addCredentials(BasicHttpCredentials(username, password)) + val gravsearchTemplateRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formDataGravsearch) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val gravsearchTemplateJSON: JsObject = getResponseJson(gravsearchTemplateRequest) @@ -763,7 +614,7 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) // Send the JSON in a POST request to the Knora API server. - val headerXSLTRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formDataHeader) ~> addCredentials(BasicHttpCredentials(username, password)) + val headerXSLTRequest: HttpRequest = Post(baseApiUrl + "/v1/resources", formDataHeader) ~> addCredentials(BasicHttpCredentials(userEmail, password)) val headerXSLTJson = getResponseJson(headerXSLTRequest) @@ -839,8 +690,9 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV } - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 "provide a helpful error message if an XSLT file is not found" ignore { + // TODO: re-enable this with the preceding test, when we can upload non-image files to Sipi (PR #1206). + val missingHeaderXSLTIri = "http://rdfh.ch/0801/608NfPLCRpeYnkXKABC5mg" val letterTEIRequest: HttpRequest = Get(baseApiUrl + "/v2/tei/" + URLEncoder.encode(letterIri.get, "UTF-8") + @@ -858,4 +710,4 @@ class KnoraSipiIntegrationV1ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV assert(responseBodyStr.contains("Sipi responded with HTTP status code 404")) } } -} +} \ No newline at end of file diff --git a/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiScriptsV1ITSpec.scala b/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiScriptsV1ITSpec.scala index cf09b6bc54..622ddae4a2 100644 --- a/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiScriptsV1ITSpec.scala +++ b/webapi/src/it/scala/org/knora/webapi/e2e/v1/KnoraSipiScriptsV1ITSpec.scala @@ -19,15 +19,10 @@ package org.knora.webapi.e2e.v1 -import java.io.{File, FileInputStream, FileOutputStream} - -import akka.http.scaladsl.model.headers._ -import akka.http.scaladsl.model.{HttpEntity, _} +import akka.event.LoggingAdapter import com.typesafe.config.{Config, ConfigFactory} import org.knora.webapi.ITKnoraFakeSpec import org.knora.webapi.messages.store.triplestoremessages.TriplestoreJsonProtocol -import org.knora.webapi.util.{FileUtil, MutableTestIri} -import spray.json._ object KnoraSipiScriptsV1ITSpec { @@ -45,14 +40,7 @@ object KnoraSipiScriptsV1ITSpec { */ class KnoraSipiScriptsV1ITSpec extends ITKnoraFakeSpec(KnoraSipiScriptsV1ITSpec.config) with TriplestoreJsonProtocol { - implicit override val log = akka.event.Logging(system, this.getClass) - - private val username = "root@example.com" - private val password = "test" - private val pathToChlaus = "_test_data/test_route/images/Chlaus.jpg" - private val pathToMarbles = "_test_data/test_route/images/marbles.tif" - private val firstPageIri = new MutableTestIri - private val secondPageIri = new MutableTestIri + implicit override val log: LoggingAdapter = akka.event.Logging(system, this.getClass) "Calling Knora Sipi Scripts" should { @@ -70,165 +58,5 @@ class KnoraSipiScriptsV1ITSpec extends ITKnoraFakeSpec(KnoraSipiScriptsV1ITSpec. val request = Get(baseInternalSipiUrl + "/test_knora_session_cookie" ) getResponseString(request) } - - "successfully call make_thumbnail.lua sipi script" in { - - // The image to be uploaded. - val fileToSend = new File(pathToChlaus) - assert(fileToSend.exists(), s"File $pathToChlaus does not exist") - - // A multipart/form-data request containing the image. - val sipiFormData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - // Send a POST request to Sipi, asking it to make a thumbnail of the image. - val sipiPostRequest = Post(baseInternalSipiUrl + "/make_thumbnail", sipiFormData) ~> addCredentials(BasicHttpCredentials(username, password)) - val sipiPostResponseJson = getResponseJson(sipiPostRequest) - - /* sipiResponseJson will be something like this - { - "mimetype_thumb":"image/jpeg", - "original_mimetype":"image/jpeg", - "nx_thumb":93, - "preview_path":"http://localhost:1024/thumbs/CjwDMhlrctI-BG7gms08BJ4.jpg/full/max/0/default.jpg", - "filename":"CjwDMhlrctI-BG7gms08BJ4", - "file_type":"IMAGE", - "original_filename":"Chlaus.jpg", - "ny_thumb":128 - } - */ - - // get the preview_path - val previewPath = sipiPostResponseJson.fields("preview_path").asInstanceOf[JsString].value - - // get the filename - val filename = sipiPostResponseJson.fields("filename").asInstanceOf[JsString].value - - /** - * Send a GET request to Sipi, asking for the preview image. - * With testcontainers it is not possible to know the random port - * in advance, so that we can provide it to Sipi at startup. - * Instead we need to replace the standard port configured - * and returned by sipi to the random port known after sipi has - * already started. - */ - val sipiGetRequest01 = Get(previewPath.replace("http://0.0.0.0:1024", baseExternalSipiUrl)) - val sipiGetResponse01: HttpResponse = singleAwaitingRequest(sipiGetRequest01) - log.debug(s"sipiGetResponse01: ${sipiGetResponse01.toString}") - sipiGetResponse01.status should be(StatusCodes.OK) - - // Send a GET request to Sipi, asking for the info.json of the image - val sipiGetRequest02 = Get(baseInternalSipiUrl + "/thumbs/" + filename + ".jpg/info.json" ) - val sipiGetResponse02: HttpResponse = singleAwaitingRequest(sipiGetRequest02) - log.debug(s"sipiGetResponse02: ${sipiGetResponse02.toString}") - sipiGetResponse02.status should be(StatusCodes.OK) - } - - "successfully call convert_from_file.lua sipi script" in { - - /* This is the case where the file is already stored on the sipi server as part of make_thumbnail*/ - - // The image to be uploaded. - val fileToSend = new File(pathToChlaus) - assert(fileToSend.exists(), s"File $pathToChlaus does not exist") - - // A multipart/form-data request containing the image. - val sipiFormData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - // Send a POST request to Sipi, asking it to make a thumbnail of the image. - val sipiMakeThumbnailRequest = Post(baseInternalSipiUrl + "/make_thumbnail", sipiFormData) - val sipiMakeThumbnailResponse: HttpResponse = singleAwaitingRequest(sipiMakeThumbnailRequest) - - val sipiMakeThumbnailResponseJson = getResponseJson(sipiMakeThumbnailRequest) - val originalFilename = sipiMakeThumbnailResponseJson.fields("original_filename").asInstanceOf[JsString].value - val originalMimetype = sipiMakeThumbnailResponseJson.fields("original_mimetype").asInstanceOf[JsString].value - val filename = sipiMakeThumbnailResponseJson.fields("filename").asInstanceOf[JsString].value - - // A form-data request containing the payload for convert_from_file. - val sipiFormData02 = FormData( - Map( - "originalFilename" -> originalFilename, - "originalMimeType" -> originalMimetype, - "prefix" -> "0001", - "filename" -> filename - ) - ) - - val convertFromFileRequest = Post(baseInternalSipiUrl + "/convert_from_file", sipiFormData02) - val convertFromFileResponseJson = getResponseJson(convertFromFileRequest) - - val filenameFull = convertFromFileResponseJson.fields("filename_full").asInstanceOf[JsString].value - - // Running with KnoraFakeService which always allows access to files. - // Send a GET request to Sipi, asking for full image - // not possible as authentication is required and file needs to be known by knora to be able to authenticate the request - val sipiGetImageRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/full/max/0/default.jpg") ~> addCredentials(BasicHttpCredentials(username, password)) - checkResponseOK(sipiGetImageRequest) - - // Send a GET request to Sipi, asking for the info.json of the image - val sipiGetInfoRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/info.json" ) ~> addCredentials(BasicHttpCredentials(username, password)) - val sipiGetInfoResponseJson = getResponseJson(sipiGetInfoRequest) - log.debug("sipiGetInfoResponseJson: {}", sipiGetInfoResponseJson) - } - - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 - "successfully call convert_from_path.lua sipi script" ignore { - - // The image to be uploaded. - val fileToSend = new File(pathToChlaus) - assert(fileToSend.exists(), s"File $pathToChlaus does not exist") - - // To be able to run packaged tests inside Docker, we need to copy - // the file first to a place which is shared with sipi - val dest = FileUtil.createTempFile(settings) - new FileOutputStream(dest) - .getChannel - .transferFrom( - new FileInputStream(fileToSend).getChannel, - 0, - Long.MaxValue - ) - - // A multipart/form-data request containing the image. - val sipiFormData = FormData( - Map( - "originalFilename" -> fileToSend.getName, - "originalMimeType" -> "image/jpeg", - "prefix" -> "0001", - "source" -> dest.getAbsolutePath - ) - ) - - // Send a POST request to Sipi, asking it to make a thumbnail of the image. - val sipiConvertFromPathPostRequest = Post(baseInternalSipiUrl + "/convert_from_path", sipiFormData) - val sipiConvertFromPathPostResponseJson = getResponseJson(sipiConvertFromPathPostRequest) - - val filenameFull = sipiConvertFromPathPostResponseJson.fields("filename_full").asInstanceOf[JsString].value - - //log.debug("sipiConvertFromPathPostResponseJson: {}", sipiConvertFromPathPostResponseJson) - - // Running with KnoraFakeService which always allows access to files. - val sipiGetImageRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/full/max/0/default.jpg") ~> addCredentials(BasicHttpCredentials(username, password)) - checkResponseOK(sipiGetImageRequest) - - // Send a GET request to Sipi, asking for the info.json of the image - val sipiGetInfoRequest = Get(baseInternalSipiUrl + "/0001/" + filenameFull + "/info.json" ) ~> addCredentials(BasicHttpCredentials(username, password)) - val sipiGetInfoResponseJson = getResponseJson(sipiGetInfoRequest) - log.debug("sipiGetInfoResponseJson: {}", sipiGetInfoResponseJson) - } - } } - - diff --git a/webapi/src/it/scala/org/knora/webapi/e2e/v2/KnoraSipiIntegrationV2ITSpec.scala b/webapi/src/it/scala/org/knora/webapi/e2e/v2/KnoraSipiIntegrationV2ITSpec.scala index b028da66dd..0059aeafd3 100644 --- a/webapi/src/it/scala/org/knora/webapi/e2e/v2/KnoraSipiIntegrationV2ITSpec.scala +++ b/webapi/src/it/scala/org/knora/webapi/e2e/v2/KnoraSipiIntegrationV2ITSpec.scala @@ -1,9 +1,27 @@ +/* + * Copyright © 2015-2019 the contributors (see Contributors.md). + * + * This file is part of Knora. + * + * Knora is free software: you can redistribute it and/or modify + * it under the terms of the GNU Affero General Public License as published + * by the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Knora is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public + * License along with Knora. If not, see . + */ + package org.knora.webapi.e2e.v2 import java.io.File import java.net.URLEncoder -import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import akka.http.scaladsl.model._ import akka.http.scaladsl.model.headers.BasicHttpCredentials import akka.http.scaladsl.unmarshalling.Unmarshal @@ -14,7 +32,6 @@ import org.knora.webapi.messages.v2.routing.authenticationmessages._ import org.knora.webapi.util.IriConversions._ import org.knora.webapi.util.jsonld._ import org.knora.webapi.util.{MutableTestIri, SmartIri, StringFormatter} -import spray.json._ import scala.concurrent.Await import scala.concurrent.duration._ @@ -68,115 +85,6 @@ class KnoraSipiIntegrationV2ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV private val csvOriginalFilename = "eggs.csv" private val pathToCsv = s"_test_data/test_route/files/$csvOriginalFilename" - /** - * Represents a file to be uploaded to Sipi. - * - * @param path the path of the file. - * @param mimeType the MIME type of the file. - * - */ - case class FileToUpload(path: String, mimeType: ContentType) - - /** - * Represents an image file to be uploaded to Sipi. - * - * @param fileToUpload the file to be uploaded. - * @param width the image's width in pixels. - * @param height the image's height in pixels. - */ - case class InputFile(fileToUpload: FileToUpload, width: Int, height: Int) - - /** - * Represents the information that Sipi returns about each file that has been uploaded. - * - * @param originalFilename the original filename that was submitted to Sipi. - * @param internalFilename Sipi's internal filename for the stored temporary file. - * @param temporaryUrl the URL at which the temporary file can be accessed. - * @param fileType `image`, `text`, or `document`. - */ - case class SipiUploadResponseEntry(originalFilename: String, internalFilename: String, temporaryUrl: String, fileType: String) - - /** - * Represents Sipi's response to a file upload request. - * - * @param uploadedFiles the information about each file that was uploaded. - */ - case class SipiUploadResponse(uploadedFiles: Seq[SipiUploadResponseEntry]) - - object SipiUploadResponseJsonProtocol extends SprayJsonSupport with DefaultJsonProtocol { - implicit val sipiUploadResponseEntryFormat: RootJsonFormat[SipiUploadResponseEntry] = jsonFormat4(SipiUploadResponseEntry) - implicit val sipiUploadResponseFormat: RootJsonFormat[SipiUploadResponse] = jsonFormat1(SipiUploadResponse) - } - - import SipiUploadResponseJsonProtocol._ - - /** - * Represents the information that Knora returns about an image file value that was created. - * - * @param internalFilename the image's internal filename. - * @param iiifUrl the image's IIIF URL. - * @param width the image's width in pixels. - * @param height the image's height in pixels. - */ - case class SavedImage(internalFilename: String, iiifUrl: String, width: Int, height: Int) - - /** - * Represents the information that Knora returns about a document file value that was created. - * - * @param internalFilename the files's internal filename. - * @param url the file's URL. - * @param pageCount the document's page count. - * @param width the document's width in pixels. - * @param height the document's height in pixels. - */ - case class SavedDocument(internalFilename: String, url: String, pageCount: Int, width: Option[Int], height: Option[Int]) - - /** - * Uploads a file to Sipi and returns the information in Sipi's response. - * - * @param loginToken the login token to be included in the request to Sipi. - * @param filesToUpload the files to be uploaded. - * @return a [[SipiUploadResponse]] representing Sipi's response. - */ - private def uploadToSipi(loginToken: String, filesToUpload: Seq[FileToUpload]): SipiUploadResponse = { - // Make a multipart/form-data request containing the files. - - val formDataParts: Seq[Multipart.FormData.BodyPart] = filesToUpload.map { - fileToUpload => - val fileToSend = new File(fileToUpload.path) - assert(fileToSend.exists(), s"File ${fileToUpload.path} does not exist") - - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(fileToUpload.mimeType, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - } - - val sipiFormData = Multipart.FormData(formDataParts: _*) - - // Send Sipi the file in a POST request. - val sipiRequest = Post(s"$baseInternalSipiUrl/upload?token=$loginToken", sipiFormData) - - val sipiUploadResponseJson: JsObject = getResponseJson(sipiRequest) - // println(sipiUploadResponseJson.prettyPrint) - val sipiUploadResponse: SipiUploadResponse = sipiUploadResponseJson.convertTo[SipiUploadResponse] - // println(s"sipiUploadResponse: $sipiUploadResponse") - - // Request the temporary file from Sipi. - for (responseEntry <- sipiUploadResponse.uploadedFiles) { - val sipiGetTmpFileRequest: HttpRequest = if (responseEntry.fileType == "image") { - Get(responseEntry.temporaryUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl) + "/full/max/0/default.jpg") - } else { - Get(responseEntry.temporaryUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl)) - } - - checkResponseOK(sipiGetTmpFileRequest) - } - - sipiUploadResponse - } - /** * Given a JSON-LD document representing a resource, returns a JSON-LD array containing the values of the specified * property. @@ -220,6 +128,27 @@ class KnoraSipiIntegrationV2ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV matchingValues.head } + /** + * Represents the information that Knora returns about an image file value that was created. + * + * @param internalFilename the image's internal filename. + * @param iiifUrl the image's IIIF URL. + * @param width the image's width in pixels. + * @param height the image's height in pixels. + */ + protected case class SavedImage(internalFilename: String, iiifUrl: String, width: Int, height: Int) + + /** + * Represents the information that Knora returns about a document file value that was created. + * + * @param internalFilename the files's internal filename. + * @param url the file's URL. + * @param pageCount the document's page count. + * @param width the document's width in pixels. + * @param height the document's height in pixels. + */ + case class SavedDocument(internalFilename: String, url: String, pageCount: Int, width: Option[Int], height: Option[Int]) + /** * Given a JSON-LD object representing a Knora image file value, returns a [[SavedImage]] containing the same information. * @@ -462,7 +391,7 @@ class KnoraSipiIntegrationV2ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV ) val internalFilename = sipiUploadResponse.uploadedFiles.head.internalFilename - val temporaryBaseIIIFUrl = sipiUploadResponse.uploadedFiles.head.temporaryUrl.replace("http://0.0.0.0:1024", baseExternalSipiUrl) + val temporaryBaseIIIFUrl = sipiUploadResponse.uploadedFiles.head.temporaryUrl // JSON describing the new image to Knora. val jsonLdEntity = @@ -485,7 +414,7 @@ class KnoraSipiIntegrationV2ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV assert(knoraPostResponse.status == StatusCodes.Forbidden) // Request the temporary image from Sipi. - val sipiGetTmpFileRequest = Get(temporaryBaseIIIFUrl) + val sipiGetTmpFileRequest = Get(temporaryBaseIIIFUrl + "/" + internalFilename + "/full/full/0/default.jpg") val sipiResponse = singleAwaitingRequest(sipiGetTmpFileRequest) assert(sipiResponse.status == StatusCodes.NotFound) } @@ -676,4 +605,4 @@ class KnoraSipiIntegrationV2ITSpec extends ITKnoraLiveSpec(KnoraSipiIntegrationV assert(savedDocument.height.isEmpty) } } -} +} \ No newline at end of file diff --git a/webapi/src/it/scala/org/knora/webapi/other/v1/DrawingsGodsV1ITSpec.scala b/webapi/src/it/scala/org/knora/webapi/other/v1/DrawingsGodsV1ITSpec.scala index 57e350ea0f..c9711bf922 100644 --- a/webapi/src/it/scala/org/knora/webapi/other/v1/DrawingsGodsV1ITSpec.scala +++ b/webapi/src/it/scala/org/knora/webapi/other/v1/DrawingsGodsV1ITSpec.scala @@ -19,18 +19,23 @@ package org.knora.webapi.other.v1 -import java.io.File import java.net.URLEncoder +import akka.event.LoggingAdapter import akka.http.scaladsl.model._ import akka.http.scaladsl.model.headers.BasicHttpCredentials -import com.typesafe.config.ConfigFactory +import akka.http.scaladsl.unmarshalling.Unmarshal +import com.typesafe.config.{Config, ConfigFactory} import org.knora.webapi.messages.store.triplestoremessages.{RdfDataObject, TriplestoreJsonProtocol} +import org.knora.webapi.messages.v2.routing.authenticationmessages.{AuthenticationV2JsonProtocol, LoginResponse} import org.knora.webapi.{ITKnoraLiveSpec, InvalidApiJsonException} import spray.json._ +import scala.concurrent.Await +import scala.concurrent.duration._ + object DrawingsGodsV1ITSpec { - val config = ConfigFactory.parseString( + val config: Config = ConfigFactory.parseString( """ akka.loglevel = "DEBUG" akka.stdout-loglevel = "DEBUG" @@ -38,9 +43,9 @@ object DrawingsGodsV1ITSpec { } /** - * End-to-End (E2E) test specification for additional testing of permissions. - */ -class DrawingsGodsV1ITSpec extends ITKnoraLiveSpec(DrawingsGodsV1ITSpec.config) with TriplestoreJsonProtocol { + * End-to-End (E2E) test specification for additional testing of permissions. + */ +class DrawingsGodsV1ITSpec extends ITKnoraLiveSpec(DrawingsGodsV1ITSpec.config) with AuthenticationV2JsonProtocol with TriplestoreJsonProtocol { override lazy val rdfDataObjects: List[RdfDataObject] = List( RdfDataObject(path = "_test_data/other.v1.DrawingsGodsV1Spec/drawings-gods_admin-data.ttl", name = "http://www.knora.org/data/admin"), @@ -54,9 +59,37 @@ class DrawingsGodsV1ITSpec extends ITKnoraLiveSpec(DrawingsGodsV1ITSpec.config) val drawingsOfGodsUserEmail = "ddd1@unil.ch" val testPass = "test" val pathToChlaus = "_test_data/test_route/images/Chlaus.jpg" + var loginToken: String = "" - // TODO: fix as part of https://github.com/dasch-swiss/knora-api/pull/1233 - "be able to create a resource, only find one DOAP (with combined resource class / property), and have permission to access the image" ignore { + "log in as a Knora user" in { + /* Correct username and correct password */ + + val params = + s""" + |{ + | "email": "$drawingsOfGodsUserEmail", + | "password": "$testPass" + |} + """.stripMargin + + val request = Post(baseApiUrl + s"/v2/authentication", HttpEntity(ContentTypes.`application/json`, params)) + val response: HttpResponse = singleAwaitingRequest(request) + assert(response.status == StatusCodes.OK) + + val lr: LoginResponse = Await.result(Unmarshal(response.entity).to[LoginResponse], 1.seconds) + loginToken = lr.token + + loginToken.nonEmpty should be(true) + } + + "be able to create a resource, only find one DOAP (with combined resource class / property), and have permission to access the image" in { + // Upload the image to Sipi. + val sipiUploadResponse: SipiUploadResponse = uploadToSipi( + loginToken = loginToken, + filesToUpload = Seq(FileToUpload(path = pathToChlaus, mimeType = MediaTypes.`image/tiff`)) + ) + + val uploadedFile: SipiUploadResponseEntry = sipiUploadResponse.uploadedFiles.head val params = s""" @@ -72,30 +105,14 @@ class DrawingsGodsV1ITSpec extends ITKnoraLiveSpec(DrawingsGodsV1ITSpec.config) | "http://www.knora.org/ontology/0105/drawings-gods#hasCommentAuthor":[{"hlist_value":"http://rdfh.ch/lists/0105/drawings-gods-2016-list-CommentAuthorList-child"}], | "http://www.knora.org/ontology/0105/drawings-gods#hasCodeVerso":[{"richtext_value":{"utf8str":"dayyad"}}] | }, + | "file": "${uploadedFile.internalFilename}", | "project_id":"http://rdfh.ch/projects/0105", | "label":"dayyad" |} """.stripMargin - // The image to be uploaded. - val fileToSend = new File(pathToChlaus) - assert(fileToSend.exists(), s"File $pathToChlaus does not exist") - - // A multipart/form-data request containing the image and the JSON. - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "json", - HttpEntity(ContentTypes.`application/json`, params) - ), - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - // Send the multipart/form-data request to the Knora API server. - val knoraPostRequest = Post(baseApiUrl + "/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(drawingsOfGodsUserEmail, testPass)) + // Send the JSON in a POST request to the Knora API server. + val knoraPostRequest = Post(baseApiUrl + "/v1/resources", HttpEntity(ContentTypes.`application/json`, params)) ~> addCredentials(BasicHttpCredentials(drawingsOfGodsUserEmail, testPass)) val knoraPostResponseJson = getResponseJson(knoraPostRequest) // Get the IRI of the newly created resource. @@ -132,4 +149,4 @@ class DrawingsGodsV1ITSpec extends ITKnoraLiveSpec(DrawingsGodsV1ITSpec.config) } } -} +} \ No newline at end of file diff --git a/webapi/src/main/resources/application.conf b/webapi/src/main/resources/application.conf index 8472b532dd..431572a248 100644 --- a/webapi/src/main/resources/application.conf +++ b/webapi/src/main/resources/application.conf @@ -345,11 +345,6 @@ app { file-server-path = "server" - v1 { - path-conversion-route = "convert_from_path" - file-conversion-route = "convert_from_file" - } - v2 { file-metadata-route = "knora.json" move-file-route = "store" diff --git a/webapi/src/main/resources/knoraXmlImport.xsd b/webapi/src/main/resources/knoraXmlImport.xsd index 2c128f5ff3..d44bd9fc9d 100644 --- a/webapi/src/main/resources/knoraXmlImport.xsd +++ b/webapi/src/main/resources/knoraXmlImport.xsd @@ -31,8 +31,7 @@ - - + diff --git a/webapi/src/main/scala/org/knora/webapi/KnoraSettings.scala b/webapi/src/main/scala/org/knora/webapi/KnoraSettings.scala index 7d08440bcb..0695f38bc6 100644 --- a/webapi/src/main/scala/org/knora/webapi/KnoraSettings.scala +++ b/webapi/src/main/scala/org/knora/webapi/KnoraSettings.scala @@ -101,16 +101,8 @@ class KnoraSettingsImpl(config: Config) extends Extension { val externalSipiHost: String = config.getString("app.sipi.external-host") val externalSipiPort: Int = config.getInt("app.sipi.external-port") val externalSipiBaseUrl: String = externalSipiProtocol + "://" + externalSipiHost + (if (externalSipiPort != 80) ":" + externalSipiPort else "") - - val sipiFileServerPrefix: String = config.getString("app.sipi.file-server-path") - val externalSipiIIIFGetUrl: String = externalSipiBaseUrl - - val internalSipiImageConversionUrlV1: String = s"$internalSipiBaseUrl" - val sipiPathConversionRouteV1: String = config.getString("app.sipi.v1.path-conversion-route") - val sipiFileConversionRouteV1: String = config.getString("app.sipi.v1.file-conversion-route") - val sipiFileMetadataRouteV2: String = config.getString("app.sipi.v2.file-metadata-route") val sipiMoveFileRouteV2: String = config.getString("app.sipi.v2.move-file-route") val sipiDeleteTempFileRouteV2: String = config.getString("app.sipi.v2.delete-temp-file-route") diff --git a/webapi/src/main/scala/org/knora/webapi/messages/store/sipimessages/SipiMessages.scala b/webapi/src/main/scala/org/knora/webapi/messages/store/sipimessages/SipiMessages.scala index c559322df7..1b79bdd615 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/store/sipimessages/SipiMessages.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/store/sipimessages/SipiMessages.scala @@ -19,20 +19,13 @@ package org.knora.webapi.messages.store.sipimessages -import java.io.File - import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport -import org.knora.webapi._ +import org.knora.webapi.SipiException import org.knora.webapi.messages.RequestWithSender import org.knora.webapi.messages.admin.responder.usersmessages.UserADM import org.knora.webapi.messages.store.StoreRequest -import org.knora.webapi.messages.v1.responder.usermessages.UserProfileV1 -import org.knora.webapi.messages.v1.responder.valuemessages.FileValueV1 import spray.json._ -////////////////////////////////////////////////////////////////////////////////////////////////////////////////// -// Messages V1 - /** * An abstract trait for messages that can be sent to the [[org.knora.webapi.store.iiif.IIIFManager]] */ @@ -41,276 +34,7 @@ sealed trait IIIFRequest extends StoreRequest /** * An abstract trait for messages that can be sent to [[org.knora.webapi.store.iiif.SipiConnector]]. */ -sealed trait SipiRequestV1 extends IIIFRequest - -/** - * Abstract trait to represent a conversion request to Sipi Connector. - * - * For each type of conversion request, an implementation of `toFormData` must be provided. - * - */ -sealed trait SipiConversionRequestV1 extends SipiRequestV1 { - val originalFilename: String - val originalMimeType: String - val projectShortcode: String - val userProfile: UserProfileV1 - - /** - * Creates a Map representing the parameters to be submitted to Sipi's conversion routes. - * This method must be implemented for each type of conversion request - * because different Sipi routes are called and the parameters differ. - * - * @return a Map of key-value pairs that can be turned into form data by Sipi responder. - */ - def toFormData: Map[String, String] - - def toJsValue: JsValue -} - - -/** - * Represents a binary file that has been temporarily stored by Knora (non GUI-case). Knora route received a multipart request - * containing binary data which it saved to a temporary location, so it can be accessed by Sipi. Knora has to delete that file afterwards. - * For further details, please read the docs: Sipi -> Interaction Between Sipi and Knora. - * - * @param originalFilename the original name of the binary file. - * @param originalMimeType the MIME type of the binary file (e.g. image/tiff). - * @param source the temporary location of the source file on disk (absolute path). - * @param userProfile the user making the request. - */ -case class SipiConversionPathRequestV1(originalFilename: String, - originalMimeType: String, - projectShortcode: String, - source: File, - userProfile: UserProfileV1) extends SipiConversionRequestV1 { - - /** - * Creates the parameters needed to call the Sipi route convert_path. - * - * Required parameters: - * - originalFilename: original name of the file to be converted. - * - originalMimeType: original mime type of the file to be converted. - * - source: path to the file to be converted (file was created by Knora). - * - * @return a Map of key-value pairs that can be turned into form data by Sipi responder. - */ - def toFormData: Map[String, String] = { - Map( - "originalFilename" -> originalFilename, - "originalMimeType" -> originalMimeType, - "source" -> source.toString, - "prefix" -> projectShortcode - ) - } - - def toJsValue: JsValue = RepresentationV1JsonProtocol.SipiConversionPathRequestV1Format.write(this) -} - -/** - * Represents an binary file that has been temporarily stored by Sipi (GUI-case). Knora route received a request telling it about - * a file that is already managed by Sipi. The binary file data have already been sent to Sipi by the client (browser-based GUI). - * Knora has to tell Sipi about the name of the file to be converted. - * For further details, please read the docs: Sipi -> Interaction Between Sipi and Knora. - * - * @param originalFilename the original name of the binary file. - * @param originalMimeType the MIME type of the binary file (e.g. image/tiff). - * @param filename the name of the binary file created by SIPI. - * @param userProfile the user making the request. - */ - -case class SipiConversionFileRequestV1(originalFilename: String, - originalMimeType: String, - projectShortcode: String, - filename: String, - userProfile: UserProfileV1) extends SipiConversionRequestV1 { - - /** - * Creates the parameters needed to call the Sipi route convert_file. - * - * Required parameters: - * - originalFilename: original name of the file to be converted. - * - originalMimeType: original mime type of the file to be converted. - * - filename: name of the file to be converted (already managed by Sipi). - * - * @return a Map of key-value pairs that can be turned into form data by Sipi responder. - */ - def toFormData: Map[String, String] = { - Map( - "originalFilename" -> originalFilename, - "originalMimeType" -> originalMimeType, - "filename" -> filename, - "prefix" -> projectShortcode - ) - } - - def toJsValue: JsValue = RepresentationV1JsonProtocol.SipiConversionFileRequestV1Format.write(this) - -} - - -/** - * Represents the response received from SIPI after an image conversion request. - * - * @param nx_full x dim of the full quality representation. - * @param ny_full y dim of the full quality representation. - * @param mimetype_full mime type of the full quality representation. - * @param filename_full filename of the full quality representation. - * @param original_mimetype mime type of the original file. - * @param original_filename name of the original file. - * @param file_type type of file that has been converted (image). - */ -case class SipiImageConversionResponse(nx_full: Int, - ny_full: Int, - mimetype_full: String, - filename_full: String, - original_mimetype: String, - original_filename: String, - file_type: String) - -/** - * Represents the response received from Sipi after a text file store request. - * - * @param mimetype mime type of the text file. - * @param charset encoding of the text file. - * @param filename filename of the text file. - * @param original_mimetype original mime type of the text file (equals `mimetype`). - * @param original_filename original name of the text file. - * @param file_type type of file that has been stored (text). - */ -case class SipiTextResponse(mimetype: String, - charset: String, - filename: String, - original_mimetype: String, - original_filename: String, - file_type: String) - - -object SipiConstants { - // TODO: Shall we better use an ErrorHandlingMap here? - // map file types converted by Sipi to file value properties in Knora - val fileType2FileValueProperty: Map[FileType.Value, IRI] = Map( - FileType.TEXT -> OntologyConstants.KnoraBase.HasTextFileValue, - FileType.IMAGE -> OntologyConstants.KnoraBase.HasStillImageFileValue, - FileType.MOVIE -> OntologyConstants.KnoraBase.HasMovingImageFileValue, - FileType.AUDIO -> OntologyConstants.KnoraBase.HasAudioFileValue, - FileType.BINARY -> OntologyConstants.KnoraBase.HasDocumentFileValue - - ) - - object FileType extends Enumeration { - // the string representations correspond to Sipi's internal enum. - val IMAGE: Value = Value(0, "image") - val TEXT: Value = Value(1, "text") - val MOVIE: Value = Value(2, "movie") - val AUDIO: Value = Value(3, "audio") - val BINARY: Value = Value(4, "binary") - - val valueMap: Map[String, Value] = values.map(v => (v.toString, v)).toMap - - /** - * Given the name of a file type in this enumeration, returns the file type. If the file type is not found, throws an - * [[SipiException]]. - * - * @param filetype the name of the file type. - * @return the requested file type. - */ - def lookup(filetype: String): Value = { - valueMap.get(filetype) match { - case Some(ftype) => ftype - case None => throw SipiException(message = s"File type $filetype returned by Sipi not found in enumeration") - } - } - - } - - object StillImage { - val fullQuality = "full" - val thumbnailQuality = "thumbnail" - } - -} - -/** - * Response from [[org.knora.webapi.store.iiif.SipiConnector]] to a [[SipiConversionRequestV1]] representing a [[FileValueV1]]. - * - * @param fileValueV1 a [[FileValueV1]] - */ -case class SipiConversionResponseV1(fileValueV1: FileValueV1, file_type: SipiConstants.FileType.Value) - - -////////////////////////////////////////////////////////////////////////////////////////////////////////////////// -// JSON formatting V1 - -/** - * A spray-json protocol for generating Knora API v1 JSON providing data about representations of a resource. - */ -object RepresentationV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol with NullOptions { - - /** - * Converts between [[SipiConversionPathRequestV1]] objects and [[JsValue]] objects. - */ - implicit object SipiConversionPathRequestV1Format extends RootJsonFormat[SipiConversionPathRequestV1] { - /** - * Not implemented. - */ - def read(jsonVal: JsValue): SipiConversionPathRequestV1 = ??? - - /** - * Converts a [[SipiConversionPathRequestV1]] into [[JsValue]] for formatting as JSON. - * - * @param request the [[SipiConversionPathRequestV1]] to be converted. - * @return a [[JsValue]]. - */ - def write(request: SipiConversionPathRequestV1): JsValue = { - - val fields = Map( - "originalFilename" -> request.originalFilename.toJson, - "originalMimeType" -> request.originalMimeType.toJson, - "source" -> request.source.toString.toJson - ) - - JsObject(fields) - } - } - - /** - * Converts between [[SipiConversionFileRequestV1]] objects and [[JsValue]] objects. - */ - implicit object SipiConversionFileRequestV1Format extends RootJsonFormat[SipiConversionFileRequestV1] { - /** - * Not implemented. - */ - def read(jsonVal: JsValue) = ??? - - /** - * Converts a [[SipiConversionFileRequestV1]] into [[JsValue]] for formatting as JSON. - * - * @param request the [[SipiConversionFileRequestV1]] to be converted. - * @return a [[JsValue]]. - */ - def write(request: SipiConversionFileRequestV1): JsValue = { - - val fields = Map( - "originalFilename" -> request.originalFilename.toJson, - "originalMimeType" -> request.originalMimeType.toJson, - "filename" -> request.filename.toJson - ) - - JsObject(fields) - } - } - - implicit val sipiImageConversionResponseFormat: RootJsonFormat[SipiImageConversionResponse] = jsonFormat7(SipiImageConversionResponse) - implicit val textStoreResponseFormat: RootJsonFormat[SipiTextResponse] = jsonFormat6(SipiTextResponse) -} - -////////////////////////////////////////////////////////////////////////////////////////////////////////////////// -// Messages V2 - -/** - * An abstract trait for messages that can be sent to [[org.knora.webapi.store.iiif.SipiConnector]]. - */ -sealed trait SipiRequestV2 extends IIIFRequest { +sealed trait SipiRequest extends IIIFRequest { def requestingUser: UserADM } @@ -320,9 +44,8 @@ sealed trait SipiRequestV2 extends IIIFRequest { * @param fileUrl the URL at which Sipi can serve the file. * @param requestingUser the user making the request. */ -case class GetFileMetadataRequestV2(fileUrl: String, - requestingUser: UserADM) extends SipiRequestV2 - +case class GetFileMetadataRequest(fileUrl: String, + requestingUser: UserADM) extends SipiRequest /** * Represents a response from Sipi providing metadata about an image file. @@ -349,7 +72,7 @@ case class GetFileMetadataResponseV2(originalFilename: Option[String], } object GetFileMetadataResponseV2JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol { - implicit val getImageMetadataResponseV2Format: RootJsonFormat[GetFileMetadataResponseV2] = jsonFormat6(GetFileMetadataResponseV2) + implicit val GetFileMetadataResponseV2Format: RootJsonFormat[GetFileMetadataResponseV2] = jsonFormat6(GetFileMetadataResponseV2) } /** @@ -359,9 +82,9 @@ object GetFileMetadataResponseV2JsonProtocol extends SprayJsonSupport with Defau * @param prefix the prefix under which the file should be stored. * @param requestingUser the user making the request. */ -case class MoveTemporaryFileToPermanentStorageRequestV2(internalFilename: String, - prefix: String, - requestingUser: UserADM) extends SipiRequestV2 +case class MoveTemporaryFileToPermanentStorageRequest(internalFilename: String, + prefix: String, + requestingUser: UserADM) extends SipiRequest /** * Asks Sipi to delete a temporary file. @@ -369,8 +92,8 @@ case class MoveTemporaryFileToPermanentStorageRequestV2(internalFilename: String * @param internalFilename the name of the file. * @param requestingUser the user making the request. */ -case class DeleteTemporaryFileRequestV2(internalFilename: String, - requestingUser: UserADM) extends SipiRequestV2 +case class DeleteTemporaryFileRequest(internalFilename: String, + requestingUser: UserADM) extends SipiRequest /** @@ -381,7 +104,7 @@ case class DeleteTemporaryFileRequestV2(internalFilename: String, */ case class SipiGetTextFileRequest(fileUrl: String, requestingUser: UserADM, - senderName: String) extends SipiRequestV2 with RequestWithSender + senderName: String) extends SipiRequest with RequestWithSender /** * Represents a response for [[SipiGetTextFileRequest]]. diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala index 4c01346de4..797905d62d 100755 --- a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/resourcemessages/ResourceMessagesV1.scala @@ -24,10 +24,11 @@ import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ +import org.knora.webapi.messages.admin.responder.projectsmessages.ProjectADM import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.{SipiConversionPathRequestV1, SipiConversionRequestV1} import org.knora.webapi.messages.v1.responder.valuemessages._ import org.knora.webapi.messages.v1.responder.{KnoraRequestV1, KnoraResponseV1} +import org.knora.webapi.messages.v2.responder.UpdateResultInProject import spray.json._ import scala.collection.breakOut @@ -42,16 +43,16 @@ import scala.collection.breakOut * @param restype_id the resource type of the resource to be created. * @param label the rdfs:label of the resource. * @param properties the properties to be created as a Map of property types to property value(s). - * @param file a file to be attached to the resource (GUI-case). + * @param file the filename of a file that has been uploaded to Sipi's temporary storage. * @param project_id the IRI of the project the resources is added to. */ case class CreateResourceApiRequestV1(restype_id: IRI, label: String, properties: Map[IRI, Seq[CreateResourceValueV1]], - file: Option[CreateFileV1] = None, + file: Option[String] = None, project_id: IRI) { - def toJsValue = ResourceV1JsonProtocol.createResourceApiRequestV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.createResourceApiRequestV1Format.write(this) } @@ -62,14 +63,14 @@ case class CreateResourceApiRequestV1(restype_id: IRI, * @param label the resource's label. * @param client_id the client's unique ID for the resource. * @param properties the resource's properties. - * @param file a file on disk that should be attached to the resource. + * @param file a file in Sipi's temporary storage that should be attached to the resource. * @param creationDate the creation date that should be attached to the resource. */ case class CreateResourceFromXmlImportRequestV1(restype_id: IRI, client_id: String, label: String, properties: Map[IRI, Seq[CreateResourceValueV1]], - file: Option[ReadFileV1] = None, + file: Option[String] = None, creationDate: Option[Instant]) /** @@ -203,7 +204,7 @@ case class ResourceSearchGetRequestV1(searchString: String, resourceTypeIri: Opt * @param resourceTypeIri the type of the new resource. * @param label the rdfs:label of the resource. * @param values the properties to add: type and value(s): a Map of propertyIris to ApiValueV1. - * @param file a file that should be attached to the resource. + * @param file a file that has been uploaded to Sipi's temporary storage and should be attached to the resource. * @param projectIri the IRI of the project the resources is added to. * @param userProfile the profile of the user making the request. * @param apiRequestID the ID of the API request. @@ -211,7 +212,7 @@ case class ResourceSearchGetRequestV1(searchString: String, resourceTypeIri: Opt case class ResourceCreateRequestV1(resourceTypeIri: IRI, label: String, values: Map[IRI, Seq[CreateValueV1WithComment]], - file: Option[SipiConversionRequestV1] = None, + file: Option[StillImageFileValueV1] = None, projectIri: IRI, userProfile: UserADM, apiRequestID: UUID) extends ResourcesResponderRequestV1 @@ -224,14 +225,14 @@ case class ResourceCreateRequestV1(resourceTypeIri: IRI, * @param clientResourceID the client's ID for the resource. * @param label the rdfs:label of the resource. * @param values the properties to add: type and value(s): a Map of propertyIris to ApiValueV1. - * @param file a file on disk that should be stored by Sipi and should be attached to the resource. + * @param file a file in Sipi's temporary storage that should be attached to the resource. * @param creationDate the creation date that should be attached to the resource. */ case class OneOfMultipleResourceCreateRequestV1(resourceTypeIri: IRI, clientResourceID: String, label: String, values: Map[IRI, Seq[CreateValueV1WithComment]], - file: Option[SipiConversionPathRequestV1] = None, + file: Option[StillImageFileValueV1] = None, creationDate: Option[Instant]) /** @@ -254,9 +255,9 @@ case class MultipleResourceCreateRequestV1(resourcesToCreate: Seq[OneOfMultipleR * @param createdResources created resources * */ -case class MultipleResourceCreateResponseV1(createdResources: Seq[OneOfMultipleResourcesCreateResponseV1]) extends KnoraResponseV1 { +case class MultipleResourceCreateResponseV1(createdResources: Seq[OneOfMultipleResourcesCreateResponseV1], projectADM: ProjectADM) extends KnoraResponseV1 with UpdateResultInProject { - def toJsValue: JsValue = ResourceV1JsonProtocol.multipleResourceCreateResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.MultipleResourceCreateResponseV1Format.write(this) } @@ -304,7 +305,7 @@ case class ResourceCheckClassResponseV1(isInClass: Boolean) * @param id the IRI of the resource that was marked as deleted. */ case class ResourceDeleteResponseV1(id: IRI) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.resourceDeleteResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceDeleteResponseV1Format.write(this) } /** @@ -315,7 +316,7 @@ case class ResourceDeleteResponseV1(id: IRI) extends KnoraResponseV1 { */ case class ResourceInfoResponseV1(resource_info: Option[ResourceInfoV1] = None, rights: Option[Int] = None) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.resourceInfoResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceInfoResponseV1Format.write(this) } /** @@ -333,7 +334,7 @@ case class ResourceFullResponseV1(resinfo: Option[ResourceInfoV1] = None, props: Option[PropsV1] = None, incoming: Seq[IncomingV1] = Nil, access: String) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.resourceFullResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceFullResponseV1Format.write(this) } /** @@ -342,7 +343,7 @@ case class ResourceFullResponseV1(resinfo: Option[ResourceInfoV1] = None, * @param resource_context resources relating to this resource via `knora-base:partOf`. */ case class ResourceContextResponseV1(resource_context: ResourceContextV1) extends KnoraResponseV1 { - def toJsValue = ResourceContextV1JsonProtocol.resourceContextResponseV1Format.write(this) + def toJsValue: JsValue = ResourceContextV1JsonProtocol.resourceContextResponseV1Format.write(this) } @@ -353,7 +354,7 @@ case class ResourceContextResponseV1(resource_context: ResourceContextV1) extend */ case class ResourceRightsResponseV1(rights: Option[Int]) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.resourceRightsResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceRightsResponseV1Format.write(this) } /** @@ -364,7 +365,7 @@ case class ResourceRightsResponseV1(rights: Option[Int]) extends KnoraResponseV1 */ case class ResourceSearchResponseV1(resources: Seq[ResourceSearchResultRowV1] = Vector.empty[ResourceSearchResultRowV1]) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.resourceSearchResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceSearchResponseV1Format.write(this) } /** @@ -373,13 +374,14 @@ case class ResourceSearchResponseV1(resources: Seq[ResourceSearchResultRowV1] = * @param res_id the IRI ow the new resource. * @param results the values that have been attached to the resource. The key in the Map refers * to the property IRI and the Seq contains all instances of values of this type. + * @param projectADM the project in which the resource is to be created. */ case class ResourceCreateResponseV1(res_id: IRI, - results: Map[IRI, Seq[ResourceCreateValueResponseV1]] = Map.empty[IRI, Seq[ResourceCreateValueResponseV1]]) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.resourceCreateResponseV1Format.write(this) + results: Map[IRI, Seq[ResourceCreateValueResponseV1]] = Map.empty[IRI, Seq[ResourceCreateValueResponseV1]], + projectADM: ProjectADM) extends KnoraResponseV1 with UpdateResultInProject { + def toJsValue: JsValue = ResourceV1JsonProtocol.ResourceCreateResponseV1Format.write(this) } - /** * Requests the properties of a given resource. * @@ -396,7 +398,7 @@ case class PropertiesGetRequestV1(iri: IRI, userProfile: UserADM) extends Resour * @param properties the properties of the specified resource. */ case class PropertiesGetResponseV1(properties: PropsGetV1) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.propertiesGetResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.propertiesGetResponseV1Format.write(this) } /** @@ -417,7 +419,7 @@ case class ChangeResourceLabelRequestV1(resourceIri: IRI, label: String, userADM * @param label the resource's new label. */ case class ChangeResourceLabelResponseV1(res_id: IRI, label: String) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.changeResourceLabelResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.changeResourceLabelResponseV1Format.write(this) } /** @@ -438,7 +440,7 @@ case class GraphDataGetRequestV1(resourceIri: IRI, depth: Int, userADM: UserADM) * @param edges the edges that are visible in the graph. */ case class GraphDataGetResponseV1(nodes: Seq[GraphNodeV1], edges: Seq[GraphEdgeV1]) extends KnoraResponseV1 { - def toJsValue = ResourceV1JsonProtocol.graphDataGetResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.graphDataGetResponseV1Format.write(this) } /** @@ -461,17 +463,17 @@ object ResourceContextCodeV1 extends Enumeration { /** * Indicates that a resource has no parts and is not part of another resource. */ - val RESOURCE_CONTEXT_NONE = Value(0) + val RESOURCE_CONTEXT_NONE: Value = Value(0) /** * Indicates that a resource is part of another resource. */ - val RESOURCE_CONTEXT_IS_PARTOF = Value(1) + val RESOURCE_CONTEXT_IS_PARTOF: Value = Value(1) /** * Indicates that a resource has parts. */ - val RESOURCE_CONTEXT_IS_COMPOUND = Value(2) + val RESOURCE_CONTEXT_IS_COMPOUND: Value = Value(2) object ResourceContextCodeV1Protocol extends DefaultJsonProtocol { @@ -742,7 +744,7 @@ case class ResourceSearchResultRowV1(id: IRI, value: Seq[String], rights: Option[Int] = None) { - def toJsValue = ResourceV1JsonProtocol.resourceSearchResultV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceSearchResultV1Format.write(this) } ////////////////////////////////////////////////////////////////////////////////////////////////////////////////// @@ -814,7 +816,7 @@ object SalsahGuiConversions { * @param id the value object IRI of the value. */ case class ResourceCreateValueResponseV1(value: ResourceCreateValueObjectResponseV1, id: IRI) { - def toJsValue = ResourceV1JsonProtocol.resourceCreateValueResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceCreateValueResponseV1Format.write(this) } /** @@ -823,9 +825,9 @@ case class ResourceCreateValueResponseV1(value: ResourceCreateValueObjectRespons */ object LiteralValueType extends Enumeration { type ValueType = Value - val StringValue = Value(0, "string") - val IntegerValue = Value(1, "integer") - val DecimalValue = Value(2, "decimal") + val StringValue: Value = Value(0, "string") + val IntegerValue: Value = Value(1, "integer") + val DecimalValue: Value = Value(2, "decimal") object LiteralValueTypeV1Protocol extends DefaultJsonProtocol { @@ -873,7 +875,7 @@ case class ResourceCreateValueObjectResponseV1(textval: Map[LiteralValueType.Val order: Map[LiteralValueType.Value, Int]) { // TODO: do we need to add geonames here? - def toJsValue = ResourceV1JsonProtocol.resourceCreateValueObjectResponseV1Format.write(this) + def toJsValue: JsValue = ResourceV1JsonProtocol.resourceCreateValueObjectResponseV1Format.write(this) } /** @@ -937,7 +939,7 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol /** * Not implemented. */ - def read(jsonVal: JsValue) = ??? + def read(jsonVal: JsValue): PropsV1 = ??? /** * Converts a [[PropsV1]] into a [[JsValue]]. @@ -983,7 +985,7 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol /** * Not implemented. */ - def read(jsonVal: JsValue) = ??? + def read(jsonVal: JsValue): PropsGetV1 = ??? /** * Converts a [[PropsGetV1]] into a [[JsValue]]. @@ -1040,7 +1042,7 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol * @param jsonVal the [[JsValue]] to be converted. * @return a [[PropsGetForRegionV1]]. */ - def read(jsonVal: JsValue) = { + def read(jsonVal: JsValue): PropsGetForRegionV1 = { val jsonObj = jsonVal.asJsObject @@ -1100,7 +1102,6 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol } } - /** * Converts between [[ResourceInfoV1]] objects and [[JsValue]] objects. */ @@ -1108,7 +1109,7 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol /** * Not implemented. */ - def read(jsonVal: JsValue) = ??? + def read(jsonVal: JsValue): ResourceInfoV1 = ??? /** * Converts a [[ResourceInfoV1]] into [[JsValue]] for formatting as JSON. @@ -1145,6 +1146,40 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol } } + /** + * Converts between [[MultipleResourceCreateResponseV1]] objects and [[JsValue]] objects. + */ + implicit object MultipleResourceCreateResponseV1Format extends JsonFormat[MultipleResourceCreateResponseV1] { + /** + * Not implemented. + */ + override def read(json: JsValue): MultipleResourceCreateResponseV1 = ??? + + /** + * Converts a [[MultipleResourceCreateResponseV1]] into a [[JsValue]] for formatting as JSON. + */ + override def write(response: MultipleResourceCreateResponseV1): JsValue = { + val fields = Map( + "createdResources" -> response.createdResources.toJson + ) + + JsObject(fields) + } + } + + implicit object ResourceCreateResponseV1Format extends JsonFormat[ResourceCreateResponseV1] { + override def read(json: JsValue): ResourceCreateResponseV1 = ??? + + override def write(obj: ResourceCreateResponseV1): JsValue = { + val fields = Map( + "res_id" -> obj.res_id.toJson, + "results" -> obj.results.toJson + ) + + JsObject(fields) + } + } + implicit val createResourceValueV1Format: RootJsonFormat[CreateResourceValueV1] = jsonFormat15(CreateResourceValueV1) implicit val createResourceApiRequestV1Format: RootJsonFormat[CreateResourceApiRequestV1] = jsonFormat5(CreateResourceApiRequestV1) implicit val ChangeResourceLabelApiRequestV1Format: RootJsonFormat[ChangeResourceLabelApiRequestV1] = jsonFormat1(ChangeResourceLabelApiRequestV1) @@ -1161,8 +1196,6 @@ object ResourceV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol implicit val resourceCreateValueObjectResponseV1Format: RootJsonFormat[ResourceCreateValueObjectResponseV1] = jsonFormat14(ResourceCreateValueObjectResponseV1) implicit val resourceCreateValueResponseV1Format: RootJsonFormat[ResourceCreateValueResponseV1] = jsonFormat2(ResourceCreateValueResponseV1) implicit val oneOfMultipleResourcesCreateResponseFormat: JsonFormat[OneOfMultipleResourcesCreateResponseV1] = jsonFormat3(OneOfMultipleResourcesCreateResponseV1) - implicit val multipleResourceCreateResponseV1Format: RootJsonFormat[MultipleResourceCreateResponseV1] = jsonFormat1(MultipleResourceCreateResponseV1) - implicit val resourceCreateResponseV1Format: RootJsonFormat[ResourceCreateResponseV1] = jsonFormat2(ResourceCreateResponseV1) implicit val resourceDeleteResponseV1Format: RootJsonFormat[ResourceDeleteResponseV1] = jsonFormat1(ResourceDeleteResponseV1) implicit val changeResourceLabelResponseV1Format: RootJsonFormat[ChangeResourceLabelResponseV1] = jsonFormat2(ChangeResourceLabelResponseV1) implicit val graphNodeV1Format: JsonFormat[GraphNodeV1] = jsonFormat4(GraphNodeV1) diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala index 153626acd6..d42f13deb1 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v1/responder/valuemessages/ValueMessagesV1.scala @@ -19,17 +19,18 @@ package org.knora.webapi.messages.v1.responder.valuemessages -import java.io.File import java.time.Instant import java.util.UUID import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport import org.knora.webapi._ +import org.knora.webapi.messages.admin.responder.projectsmessages.ProjectADM import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.SipiConversionRequestV1 import org.knora.webapi.messages.v1.responder.resourcemessages.LocationV1 import org.knora.webapi.messages.v1.responder.{KnoraRequestV1, KnoraResponseV1} +import org.knora.webapi.messages.v2.responder.UpdateResultInProject import org.knora.webapi.messages.v2.responder.standoffmessages._ +import org.knora.webapi.messages.v2.responder.valuemessages.{FileValueV2, StillImageFileValueContentV2} import org.knora.webapi.util.standoff.StandoffTagUtilV2 import org.knora.webapi.util.{DateUtilV1, StringFormatter} import spray.json._ @@ -141,14 +142,6 @@ case class CreateFileV1(originalFilename: String, } -/** - * Represents a file on disk to be added to a Knora resource in the context of a bulk import. - * - * @param file the file. - * @param mimeType the file's MIME type. - */ -case class ReadFileV1(file: File, mimeType: String) - /** * Represents an API request payload that asks the Knora API server to change a value of a resource property (i.e. to * update its version history). @@ -205,11 +198,11 @@ case class ChangeValueApiRequestV1(richtext_value: Option[CreateRichtextV1] = No /** * Represents an API request payload that asks the Knora API server to change the file attached to a resource - * (i. e. to create a new version of its file values). + * (i. e. to create a new version of its file value). * - * @param file the new file to be attached to the resource (GUI-case). + * @param file the name of a file that has been uploaded to Sipi's temporary storage. */ -case class ChangeFileValueApiRequestV1(file: CreateFileV1) { +case class ChangeFileValueApiRequestV1(file: String) { def toJsValue: JsValue = ApiValueV1JsonProtocol.changeFileValueApiRequestV1Format.write(this) } @@ -484,9 +477,9 @@ case class DeleteValueResponseV1(id: IRI) extends KnoraResponseV1 { * In case of an image, two file valueshave to be changed: thumbnail and full quality. * * @param resourceIri the resource whose files value(s) should be changed. - * @param file the file to be created and added. + * @param file a file that has been uploaded to Sipi's temporary storage. */ -case class ChangeFileValueRequestV1(resourceIri: IRI, file: SipiConversionRequestV1, apiRequestID: UUID, userProfile: UserADM) extends ValuesResponderRequestV1 +case class ChangeFileValueRequestV1(resourceIri: IRI, file: StillImageFileValueV1, apiRequestID: UUID, userProfile: UserADM) extends ValuesResponderRequestV1 /** * Represents a response to a [[ChangeFileValueRequestV1]]. @@ -494,8 +487,8 @@ case class ChangeFileValueRequestV1(resourceIri: IRI, file: SipiConversionReques * * @param locations the updated file value(s). */ -case class ChangeFileValueResponseV1(locations: Vector[LocationV1]) extends KnoraResponseV1 { - def toJsValue: JsValue = ApiValueV1JsonProtocol.changeFileValueResponseV1Format.write(this) +case class ChangeFileValueResponseV1(locations: Vector[LocationV1], projectADM: ProjectADM) extends KnoraResponseV1 with UpdateResultInProject { + def toJsValue: JsValue = ApiValueV1JsonProtocol.ChangeFileValueResponseV1Format.write(this) } ////////////////////////////////////////////////////////////////////////////////////////////////////////////////// @@ -1493,6 +1486,20 @@ case class StillImageFileValueV1(internalMimeType: String, case other => throw InconsistentTriplestoreDataException(s"Cannot compare a $valueTypeIri to a ${other.valueTypeIri}") } } + + def toStillImageFileValueContentV2: StillImageFileValueContentV2 = { + StillImageFileValueContentV2( + ontologySchema = InternalSchema, + fileValue = FileValueV2( + internalFilename = internalFilename, + internalMimeType = internalMimeType, + originalFilename = Some(originalFilename), + originalMimeType = Some(internalMimeType) + ), + dimX = dimX, + dimY = dimY + ) + } } case class MovingImageFileValueV1(internalMimeType: String, @@ -1613,7 +1620,7 @@ object ApiValueV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol def write(calendarV1Value: KnoraCalendarV1.Value): JsValue = JsString(calendarV1Value.toString) } - /** + /**å * Converts between [[KnoraPrecisionV1]] objects and [[JsValue]] objects. */ implicit object KnoraPrecisionV1JsonFormat extends JsonFormat[KnoraPrecisionV1.Value] { @@ -1643,6 +1650,16 @@ object ApiValueV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol def write(valueV1: ApiValueV1): JsValue = valueV1.toJsValue } + implicit object ChangeFileValueResponseV1Format extends JsonFormat[ChangeFileValueResponseV1] { + override def read(json: JsValue): ChangeFileValueResponseV1 = ??? + + override def write(obj: ChangeFileValueResponseV1): JsValue = { + JsObject(Map( + "locations" -> obj.locations.toJson + )) + } + } + implicit val createFileV1Format: RootJsonFormat[CreateFileV1] = jsonFormat3(CreateFileV1) implicit val valueGetResponseV1Format: RootJsonFormat[ValueGetResponseV1] = jsonFormat7(ValueGetResponseV1) implicit val dateValueV1Format: JsonFormat[DateValueV1] = jsonFormat5(DateValueV1) @@ -1659,5 +1676,4 @@ object ApiValueV1JsonProtocol extends SprayJsonSupport with DefaultJsonProtocol implicit val changeValueResponseV1Format: RootJsonFormat[ChangeValueResponseV1] = jsonFormat4(ChangeValueResponseV1) implicit val deleteValueResponseV1Format: RootJsonFormat[DeleteValueResponseV1] = jsonFormat1(DeleteValueResponseV1) implicit val changeFileValueApiRequestV1Format: RootJsonFormat[ChangeFileValueApiRequestV1] = jsonFormat1(ChangeFileValueApiRequestV1) - implicit val changeFileValueResponseV1Format: RootJsonFormat[ChangeFileValueResponseV1] = jsonFormat1(ChangeFileValueResponseV1) } diff --git a/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/valuemessages/ValueMessagesV2.scala b/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/valuemessages/ValueMessagesV2.scala index f9fde2978a..535c6b75b1 100644 --- a/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/valuemessages/ValueMessagesV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/messages/v2/responder/valuemessages/ValueMessagesV2.scala @@ -30,7 +30,7 @@ import akka.util.Timeout import org.knora.webapi._ import org.knora.webapi.messages.admin.responder.projectsmessages.ProjectADM import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.{GetFileMetadataRequestV2, GetFileMetadataResponseV2} +import org.knora.webapi.messages.store.sipimessages.{GetFileMetadataRequest, GetFileMetadataResponseV2} import org.knora.webapi.messages.v2.responder._ import org.knora.webapi.messages.v2.responder.resourcemessages.ReadResourceV2 import org.knora.webapi.messages.v2.responder.standoffmessages._ @@ -2647,8 +2647,8 @@ object FileValueWithSipiMetadata { internalFilename <- Future(jsonLDObject.requireStringWithValidation(OntologyConstants.KnoraApiV2Complex.FileValueHasFilename, stringFormatter.toSparqlEncodedString)) // Ask Sipi about the rest of the file's metadata. - tempFileUrl = s"${settings.internalSipiBaseUrl}/tmp/$internalFilename" - fileMetadataResponse: GetFileMetadataResponseV2 <- (storeManager ? GetFileMetadataRequestV2(fileUrl = tempFileUrl, requestingUser = requestingUser)).mapTo[GetFileMetadataResponseV2] + tempFileUrl = stringFormatter.makeSipiTempFileUrl(settings, internalFilename) + fileMetadataResponse: GetFileMetadataResponseV2 <- (storeManager ? GetFileMetadataRequest(fileUrl = tempFileUrl, requestingUser = requestingUser)).mapTo[GetFileMetadataResponseV2] fileValue = FileValueV2( internalFilename = internalFilename, internalMimeType = fileMetadataResponse.internalMimeType, diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala index 5561037383..565845a12d 100755 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ResourcesResponderV1.scala @@ -26,17 +26,20 @@ import akka.http.scaladsl.util.FastFuture import akka.pattern._ import org.knora.webapi._ import org.knora.webapi.messages.admin.responder.permissionsmessages.{DefaultObjectAccessPermissionsStringForPropertyGetADM, DefaultObjectAccessPermissionsStringForResourceClassGetADM, DefaultObjectAccessPermissionsStringResponseADM, ResourceCreateOperation} +import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectADM, ProjectGetRequestADM, ProjectGetResponseADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages._ import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.v1.responder.ontologymessages._ import org.knora.webapi.messages.v1.responder.projectmessages._ -import org.knora.webapi.messages.v1.responder.resourcemessages.{MultipleResourceCreateResponseV1, _} +import org.knora.webapi.messages.v1.responder.resourcemessages._ import org.knora.webapi.messages.v1.responder.valuemessages._ +import org.knora.webapi.messages.v2.responder.UpdateResultInProject import org.knora.webapi.messages.v2.responder.ontologymessages.Cardinality.KnoraCardinalityInfo import org.knora.webapi.messages.v2.responder.ontologymessages.{Cardinality, OntologyMetadataGetByIriRequestV2, OntologyMetadataV2, ReadOntologyMetadataV2} +import org.knora.webapi.messages.v2.responder.valuemessages.StillImageFileValueContentV2 import org.knora.webapi.responders.Responder.handleUnexpectedMessage import org.knora.webapi.responders.v1.GroupedProps._ +import org.knora.webapi.responders.v2.ResourceUtilV2 import org.knora.webapi.responders.{IriLocker, Responder, ResponderData} import org.knora.webapi.twirl.SparqlTemplateResourceToCreate import org.knora.webapi.util.ApacheLuceneSupport.MatchStringWhileTyping @@ -45,7 +48,7 @@ import org.knora.webapi.util._ import scala.collection.immutable import scala.concurrent.Future -import scala.util.Try +import scala.util.{Failure, Success, Try} /** * Responds to requests for information about resources, and returns responses in Knora API v1 format. @@ -65,7 +68,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo case ResourceRightsGetRequestV1(resourceIri, userProfile) => getRightsResponseV1(resourceIri, userProfile) case graphDataGetRequest: GraphDataGetRequestV1 => getGraphDataResponseV1(graphDataGetRequest) case ResourceSearchGetRequestV1(searchString: String, resourceIri: Option[IRI], numberOfProps: Int, limitOfResults: Int, userProfile: UserADM) => getResourceSearchResponseV1(searchString, resourceIri, numberOfProps, limitOfResults, userProfile) - case ResourceCreateRequestV1(resourceTypeIri, label, values, convertRequest, projectIri, userProfile, apiRequestID) => createNewResource(resourceTypeIri, label, values, convertRequest, projectIri, userProfile, apiRequestID) + case ResourceCreateRequestV1(resourceTypeIri, label, values, file, projectIri, userProfile, apiRequestID) => createNewResource(resourceTypeIri, label, values, file, projectIri, userProfile, apiRequestID) case MultipleResourceCreateRequestV1(resourcesToCreate, projectIri, userProfile, apiRequestID) => createMultipleNewResources(resourcesToCreate, projectIri, userProfile, apiRequestID) case ResourceCheckClassRequestV1(resourceIri: IRI, owlClass: IRI, userProfile: UserADM) => checkResourceClass(resourceIri, owlClass, userProfile) case PropertiesGetRequestV1(resourceIri: IRI, userProfile: UserADM) => getPropertiesV1(resourceIri = resourceIri, userProfile = userProfile) @@ -1190,15 +1193,14 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo ) } } - else - { - // ?firstProp is sufficient: the client requested just one property per resource that was found - ResourceSearchResultRowV1( - id = row.rowMap("resourceIri"), - value = Vector(firstProp), - rights = permissionCode - ) - } + else { + // ?firstProp is sufficient: the client requested just one property per resource that was found + ResourceSearchResultRowV1( + id = row.rowMap("resourceIri"), + value = Vector(firstProp), + rights = permissionCode + ) + } } yield searchResultRow @@ -1217,36 +1219,42 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo * @param resourcesToCreate collection of ResourceRequests . * @param projectIri IRI of the project . * @param apiRequestID the the ID of the API request. - * @param userProfile the profile of the user making the request. + * @param requestingUser the user making the request. * @return a [[MultipleResourceCreateResponseV1]] informing the client about the new resources. */ private def createMultipleNewResources(resourcesToCreate: Seq[OneOfMultipleResourceCreateRequestV1], projectIri: IRI, - userProfile: UserADM, + requestingUser: UserADM, apiRequestID: UUID): Future[MultipleResourceCreateResponseV1] = { - val userProfileV1 = userProfile.asUserProfileV1 + // Convert all the image metadata in the request to StillImageFileValueContentV2 instances, so we + // can use ResourceUtilV2.doSipiPostUpdate after updating the triplestore. + val stillImageFileValueContentV2s: Seq[StillImageFileValueContentV2] = resourcesToCreate.flatMap { + resourceToCreate => resourceToCreate.file.map(_.toStillImageFileValueContentV2) + } - for { + val updateFuture: Future[MultipleResourceCreateResponseV1] = for { // Get user's IRI and don't allow anonymous users to create resources. userIri: IRI <- Future { - if (userProfile.isAnonymousUser) { + if (requestingUser.isAnonymousUser) { throw ForbiddenException("Anonymous users aren't allowed to create resources") } else { - userProfile.id + requestingUser.id } } // Get information about the project in which the resources will be created. projectInfoResponse <- { - responderManager ? ProjectInfoByIRIGetRequestV1( - projectIri, - Some(userProfileV1) + responderManager ? ProjectGetRequestADM( + identifier = ProjectIdentifierADM(maybeIri = Some(projectIri)), + requestingUser = requestingUser ) - }.mapTo[ProjectInfoResponseV1] + }.mapTo[ProjectGetResponseADM] + + projectADM = projectInfoResponse.project // Ensure that the project isn't the system project or the shared ontologies project. - resourceProjectIri: IRI = projectInfoResponse.project_info.id + resourceProjectIri: IRI = projectADM.id _ = if (resourceProjectIri == OntologyConstants.KnoraAdmin.SystemProject || resourceProjectIri == OntologyConstants.KnoraAdmin.DefaultSharedOntologiesProject) { throw BadRequestException(s"Resources cannot be created in project $resourceProjectIri") @@ -1254,11 +1262,11 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // Ensure that the resource class isn't from a non-shared ontology in another project. - namedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfoResponse.project_info) + namedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV2(projectADM) // Create random IRIs for resources, collect in Map[clientResourceID, IRI] clientResourceIDsToResourceIris: Map[String, IRI] = new ErrorHandlingMap( - toWrap = resourcesToCreate.map(resRequest => resRequest.clientResourceID -> stringFormatter.makeRandomResourceIri(projectInfoResponse.project_info.shortcode)).toMap, + toWrap = resourcesToCreate.map(resRequest => resRequest.clientResourceID -> stringFormatter.makeRandomResourceIri(projectADM.shortcode)).toMap, errorTemplateFun = { key => s"Resource $key is the target of a link, but was not provided in the request" }, errorFun = { errorMsg => throw BadRequestException(errorMsg) } ) @@ -1277,7 +1285,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // Ensure that none of the resource classes is from a non-shared ontology in another project. resourceClassOntologyIris: Set[SmartIri] = resourceClasses.map(_.toSmartIri.getOntologyFromEntity) - readOntologyMetadataV2: ReadOntologyMetadataV2 <- (responderManager ? OntologyMetadataGetByIriRequestV2(resourceClassOntologyIris, userProfile)).mapTo[ReadOntologyMetadataV2] + readOntologyMetadataV2: ReadOntologyMetadataV2 <- (responderManager ? OntologyMetadataGetByIriRequestV2(resourceClassOntologyIris, requestingUser)).mapTo[ReadOntologyMetadataV2] _ = for (ontologyMetadata <- readOntologyMetadataV2.ontologies) { val ontologyProjectIri: IRI = ontologyMetadata.projectIri.getOrElse(throw InconsistentTriplestoreDataException(s"Ontology ${ontologyMetadata.ontologyIri} has no project")).toString @@ -1290,7 +1298,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceClassesEntityInfoResponse: EntityInfoGetResponseV1 <- (responderManager ? EntityInfoGetRequestV1( resourceClassIris = resourceClasses, propertyIris = Set.empty[IRI], - userProfile = userProfile + userProfile = requestingUser )).mapTo[EntityInfoGetResponseV1] allPropertyIris: Set[IRI] = resourceClassesEntityInfoResponse.resourceClassInfoMap.flatMap { @@ -1301,7 +1309,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo propertyEntityInfoResponse: EntityInfoGetResponseV1 <- (responderManager ? EntityInfoGetRequestV1( resourceClassIris = Set.empty[IRI], propertyIris = allPropertyIris, - userProfile = userProfile + userProfile = requestingUser )).mapTo[EntityInfoGetResponseV1] propertyEntityInfoMapsPerResource: Map[IRI, Map[IRI, PropertyInfoV1]] = resourceClassesEntityInfoResponse.resourceClassInfoMap.map { @@ -1323,7 +1331,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo responderManager ? DefaultObjectAccessPermissionsStringForResourceClassGetADM( projectIri = projectIri, resourceClassIri = resourceClassIri, - targetUser = userProfile, + targetUser = requestingUser, requestingUser = KnoraSystemInstances.Users.SystemUser ) }.mapTo[DefaultObjectAccessPermissionsStringResponseADM] @@ -1343,7 +1351,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo projectIri = projectIri, resourceClassIri = resourceClassIri, propertyIri = propertyIri, - targetUser = userProfile, + targetUser = requestingUser, requestingUser = KnoraSystemInstances.Users.SystemUser) }.mapTo[DefaultObjectAccessPermissionsStringResponseADM] } yield (propertyIri, defaultObjectAccessPermissions.permissionLiteral) @@ -1386,9 +1394,9 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceClassInfo = resourceClassesEntityInfoResponse.resourceClassInfoMap(resourceCreateRequest.resourceTypeIri), propertyInfoMap = propertyEntityInfoMapsPerResource(resourceCreateRequest.resourceTypeIri), values = resourceCreateRequest.values, - sipiConversionRequest = resourceCreateRequest.file, + convertedFile = resourceCreateRequest.file, clientResourceIDsToResourceClasses = clientResourceIDsToResourceClasses, - userProfile = userProfile + userProfile = requestingUser ) // Convert each LinkToClientIDUpdateV1 into a LinkUpdateV1. @@ -1421,7 +1429,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo clientResourceIDsToResourceIris = clientResourceIDsToResourceIris, creationDate = creationDate, fileValues = fileValues, - userProfile = userProfile, + userProfile = requestingUser, apiRequestID = apiRequestID ) @@ -1461,7 +1469,43 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo label = resourceToCreate.label ) } - } yield MultipleResourceCreateResponseV1(responses) + } yield MultipleResourceCreateResponseV1(responses, projectADM) + + doSipiPostUpdateForResources( + updateFuture = updateFuture, + fileValueContentV2s = stillImageFileValueContentV2s, + requestingUser = requestingUser + ) + } + + /** + * Asks Sipi to to move temporary image files to permanent storage if a triplestore update was successful, + * or to delete the temporary files if the triplestore update failed. + * + * @param updateFuture the future resulting from the triplestore update. + * @param fileValueContentV2s the file values that were created, if any. + * @param requestingUser the user making the request. + * @return `updateFuture`, or a failed future (if Sipi failed to move a file to permanent storage). + */ + private def doSipiPostUpdateForResources[T <: UpdateResultInProject](updateFuture: Future[T], + fileValueContentV2s: Seq[StillImageFileValueContentV2], + requestingUser: UserADM): Future[T] = { + val resultFutures: Seq[Future[T]] = fileValueContentV2s.map { + valueContent => + ResourceUtilV2.doSipiPostUpdate( + updateFuture = updateFuture, + valueContent = valueContent, + requestingUser = requestingUser, + responderManager = responderManager, + storeManager = storeManager, + log = log + ) + } + + Future.sequence(resultFutures).transformWith { + case Success(_) => updateFuture + case Failure(e) => Future.failed(e) + } } /** @@ -1473,7 +1517,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo * @param values values to be created for resource. If `linkTargetsAlreadyExist` is true, any links must be represented as [[LinkUpdateV1]] instances. * Otherwise, they must be represented as [[LinkToClientIDUpdateV1]] instances, so that appropriate error messages can * be generated for links to missing resources. - * @param sipiConversionRequest a file (binary representation) to be attached to the resource (GUI and non GUI-case). + * @param convertedFile an already converted file to be attached to the resource. * @param clientResourceIDsToResourceClasses for each client resource ID, the IRI of the resource's class. Used only if `linkTargetsAlreadyExist` is false. * @param userProfile the profile of the user making the request. * @return a tuple (IRI, Vector[CreateValueV1WithComment]) containing the IRI of the resource and a collection of holders of [[UpdateValueV1]] and comment. @@ -1482,19 +1526,14 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceClassInfo: ClassInfoV1, propertyInfoMap: Map[IRI, PropertyInfoV1], values: Map[IRI, Seq[CreateValueV1WithComment]], - sipiConversionRequest: Option[SipiConversionRequestV1], + convertedFile: Option[StillImageFileValueV1], clientResourceIDsToResourceClasses: Map[String, IRI] = new ErrorHandlingMap[IRI, IRI]( toWrap = Map.empty[IRI, IRI], errorTemplateFun = { key => s"Resource $key is the target of a link, but was not provided in the request" }, errorFun = { errorMsg => throw BadRequestException(errorMsg) } ), userProfile: UserADM): Future[Option[(IRI, Vector[CreateValueV1WithComment])]] = { - val userProfileV1 = userProfile.asUserProfileV1 - for { - // Get ontology information about the resource class's cardinalities and about each property's knora-base:objectClassConstraint. - - // Check that each submitted value is consistent with the knora-base:objectClassConstraint of the property that is supposed to // point to it. propertyObjectClassConstraintChecks: Seq[Unit] <- Future.sequence { @@ -1567,12 +1606,9 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo } } - // maximally one file value can be handled here - _ = if (resourceClassInfo.fileValueProperties.size > 1) throw BadRequestException(s"The given resource type $resourceClassIri requires more than on file value. This is not supported for API V1") - // Check that no required values are missing. requiredProps: Set[IRI] = resourceClassInfo.knoraResourceCardinalities.filter { - case (propIri, cardinalityInfo) => cardinalityInfo.cardinality == Cardinality.MustHaveOne || cardinalityInfo.cardinality == Cardinality.MustHaveSome + case (_, cardinalityInfo) => cardinalityInfo.cardinality == Cardinality.MustHaveOne || cardinalityInfo.cardinality == Cardinality.MustHaveSome }.keySet -- resourceClassInfo.linkValueProperties -- resourceClassInfo.fileValueProperties // exclude link value and file value properties from checking submittedPropertyIris = values.keySet @@ -1583,35 +1619,23 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo } // check if a file value is required by the ontology - fileValues: Option[(IRI, Vector[CreateValueV1WithComment])] <- if (resourceClassInfo.fileValueProperties.nonEmpty) { - // call sipi responder - for { - sipiResponse: SipiConversionResponseV1 <- (storeManager ? sipiConversionRequest.getOrElse(throw OntologyConstraintException(s"No file (required) given for resource type $resourceClassIri"))).mapTo[SipiConversionResponseV1] - - // check if the file type returned by Sipi corresponds to the expected fileValue property in resourceClassInfo.fileValueProperties.head - _ = if (SipiConstants.fileType2FileValueProperty(sipiResponse.file_type) != resourceClassInfo.fileValueProperties.head) { - // TODO: remove the file from SIPI (delete request) - throw BadRequestException(s"Type of submitted file (${sipiResponse.file_type}) does not correspond to expected property type ${resourceClassInfo.fileValueProperties.head}") - } + fileValues: Option[(IRI, Vector[CreateValueV1WithComment])] = if (resourceClassInfo.fileValueProperties.nonEmpty) { + convertedFile match { + case Some(converted) => + // TODO: check if the file type returned by Sipi corresponds to the expected fileValue property in resourceClassInfo.fileValueProperties.head + Some(resourceClassInfo.fileValueProperties.head -> Vector(CreateValueV1WithComment(converted))) - // in case we deal with a SipiResponderConversionPathRequestV1 (non GUI-case), the tmp file created by resources route - // has already been deleted by the SipiResponder + case None => throw BadRequestException(s"File required but none submitted") + } - } yield Some(resourceClassInfo.fileValueProperties.head -> Vector(CreateValueV1WithComment(sipiResponse.fileValueV1))) } else { - // resource class requires no binary representation - // check if there was no file sent - // TODO: in all cases of an error, the tmp file has to be deleted - sipiConversionRequest match { - case None => Future(None) // expected behaviour - case Some(_: SipiConversionFileRequestV1) => - throw BadRequestException(s"File params (GUI-case) are given but resource class $resourceClassIri does not allow any representation") - case Some(_: SipiConversionPathRequestV1) => - throw BadRequestException(s"A binary file was provided (non GUI-case) but resource class $resourceClassIri does not have any binary representation") + if (convertedFile.nonEmpty) { + throw BadRequestException(s"File params are given but resource class $resourceClassIri does not allow any representation") + } else { + None } } } yield fileValues - } /** @@ -1685,6 +1709,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo * @param creatorIri the creator of the resources to be created. * @param createNewResourceSparql Sparql query to create the resource . * @param generateSparqlForValuesResponse Sparql statement for creation of values of resource. + * @param projectADM the project in which the resource was created. * @param userProfile the profile of the user making the request. * @return a [[ResourceCreateResponseV1]] containing information about the created resource . */ @@ -1692,6 +1717,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo creatorIri: IRI, createNewResourceSparql: String, generateSparqlForValuesResponse: GenerateSparqlToCreateMultipleValuesResponseV1, + projectADM: ProjectADM, userProfile: UserADM): Future[ResourceCreateResponseV1] = { // Verify that the resource was created. for { @@ -1727,7 +1753,11 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo }) } - apiResponse: ResourceCreateResponseV1 = ResourceCreateResponseV1(results = resourceCreateValueResponses, res_id = resourceIri) + apiResponse: ResourceCreateResponseV1 = ResourceCreateResponseV1( + results = resourceCreateValueResponses, + res_id = resourceIri, + projectADM = projectADM + ) } yield apiResponse } @@ -1735,30 +1765,37 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo /** * Does pre-update checks, creates a resource, and verifies that it was created. * - * @param resourceIri the IRI of the resource to be created. - * @param values the values to be attached to the resource. - * @param creatorIri the creator of the resource to be created. - * @param namedGraph the named graph the resource belongs to. - * @param apiRequestID the request ID used for locking the resource. + * @param resourceClassIri the IRI of the resource class. + * @param projectADM the project in which the resource should be created. + * @param label the `rdfs:label` of the resource to be created. + * @param resourceIri the IRI of the resource to be created. + * @param values the values to be attached to the resource. + * @param file a file that has been uploaded to Sipi's temporary storage and should be attached to the resource. + * @param creatorIri the creator of the resource to be created. + * @param namedGraph the named graph the resource belongs to. + * @param requestingUser the user making the request. + * @param apiRequestID the request ID used for locking the resource. * @return a [[ResourceCreateResponseV1]] containing information about the created resource. */ def createResourceAndCheck(resourceClassIri: IRI, - projectIri: IRI, + projectADM: ProjectADM, label: String, resourceIri: IRI, values: Map[IRI, Seq[CreateValueV1WithComment]], - sipiConversionRequest: Option[SipiConversionRequestV1], + file: Option[StillImageFileValueV1], creatorIri: IRI, namedGraph: IRI, - userProfile: UserADM, + requestingUser: UserADM, apiRequestID: UUID): Future[ResourceCreateResponseV1] = { - for { + val fileValueContent: Option[StillImageFileValueContentV2] = file.map(_.toStillImageFileValueContentV2) + + val updateFuture = for { // Get ontology information about the resource class and its properties. resourceClassEntityInfoResponse: EntityInfoGetResponseV1 <- (responderManager ? EntityInfoGetRequestV1( resourceClassIris = Set(resourceClassIri), propertyIris = Set.empty[IRI], - userProfile = userProfile + userProfile = requestingUser )).mapTo[EntityInfoGetResponseV1] resourceClassInfo = resourceClassEntityInfoResponse.resourceClassInfoMap(resourceClassIri) @@ -1766,7 +1803,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo propertyEntityInfoResponse: EntityInfoGetResponseV1 <- (responderManager ? EntityInfoGetRequestV1( resourceClassIris = Set.empty[IRI], propertyIris = resourceClassInfo.knoraResourceCardinalities.keySet, - userProfile = userProfile + userProfile = requestingUser )).mapTo[EntityInfoGetResponseV1] propertyInfoMap = propertyEntityInfoResponse.propertyInfoMap @@ -1775,9 +1812,9 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo defaultResourceClassAccessPermissionsResponse: DefaultObjectAccessPermissionsStringResponseADM <- { responderManager ? DefaultObjectAccessPermissionsStringForResourceClassGetADM( - projectIri = projectIri, + projectIri = projectADM.id, resourceClassIri = resourceClassIri, - targetUser = userProfile, + targetUser = requestingUser, requestingUser = KnoraSystemInstances.Users.SystemUser ) }.mapTo[DefaultObjectAccessPermissionsStringResponseADM] @@ -1789,10 +1826,10 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo for { defaultObjectAccessPermissions <- { responderManager ? DefaultObjectAccessPermissionsStringForPropertyGetADM( - projectIri = projectIri, + projectIri = projectADM.id, resourceClassIri = resourceClassIri, propertyIri = propertyIri, - targetUser = userProfile, + targetUser = requestingUser, requestingUser = KnoraSystemInstances.Users.SystemUser) }.mapTo[DefaultObjectAccessPermissionsStringResponseADM] } yield (propertyIri, defaultObjectAccessPermissions.permissionLiteral) @@ -1806,8 +1843,8 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceClassInfo = resourceClassInfo, propertyInfoMap = propertyInfoMap, values = values, - sipiConversionRequest = sipiConversionRequest, - userProfile = userProfile + convertedFile = file, + userProfile = requestingUser ) // Everything looks OK, so we can create the resource and its values. @@ -1816,7 +1853,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo creationDate: Instant = Instant.now generateSparqlForValuesResponse: GenerateSparqlToCreateMultipleValuesResponseV1 <- generateSparqlForValuesOfNewResource( - projectIri = projectIri, + projectIri = projectADM.id, resourceIri = resourceIri, resourceClassIri = resourceClassIri, defaultPropertyAccessPermissions = defaultPropertyAccessPermissions, @@ -1824,7 +1861,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo fileValues = fileValues, clientResourceIDsToResourceIris = Map.empty[String, IRI], creationDate = creationDate, - userProfile = userProfile, + userProfile = requestingUser, apiRequestID = apiRequestID ) @@ -1841,7 +1878,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo createNewResourceSparql = generateSparqlForNewResources( resourcesToCreate = resourcesToCreate, - projectIri = projectIri, + projectIri = projectADM.id, namedGraph = namedGraph, creatorIri = creatorIri ) @@ -1854,32 +1891,37 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo creatorIri = creatorIri, createNewResourceSparql = createNewResourceSparql, generateSparqlForValuesResponse = generateSparqlForValuesResponse, - userProfile = userProfile + projectADM = projectADM, + userProfile = requestingUser ) } yield apiResponse + + doSipiPostUpdateForResources( + updateFuture = updateFuture, + fileValueContentV2s = fileValueContent.toSeq, + requestingUser = requestingUser + ) } /** * Creates a new resource and attaches the given values to it. * - * @param resourceClassIri the resource type of the resource to be created. - * @param values the values to be attached to the resource. - * @param sipiConversionRequest a file (binary representation) to be attached to the resource (GUI and non GUI-case) - * @param projectIri the project the resource belongs to. - * @param userProfile the user that is creating the resource - * @param apiRequestID the ID of this API request. + * @param resourceClassIri the resource type of the resource to be created. + * @param values the values to be attached to the resource. + * @param file a file that has been uploaded to Sipi's temporary storage and should be attached to the resource. + * @param projectIri the project the resource belongs to. + * @param userProfile the user that is creating the resource + * @param apiRequestID the ID of this API request. * @return a [[ResourceCreateResponseV1]] informing the client about the new resource. */ private def createNewResource(resourceClassIri: IRI, label: String, values: Map[IRI, Seq[CreateValueV1WithComment]], - sipiConversionRequest: Option[SipiConversionRequestV1] = None, + file: Option[StillImageFileValueV1] = None, projectIri: IRI, userProfile: UserADM, apiRequestID: UUID): Future[ResourceCreateResponseV1] = { - val userProfileV1 = userProfile.asUserProfileV1 - - val resultFuture = for { + for { // Get user's IRI and don't allow anonymous users to create resources. userIri: IRI <- Future { @@ -1894,16 +1936,17 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo throw BadRequestException(s"Instances of knora-base:Resource cannot be created, only instances of subclasses") } - projectInfoResponse <- { - responderManager ? ProjectInfoByIRIGetRequestV1( - projectIri, - Some(userProfileV1) + // Get project info + projectResponse <- { + responderManager ? ProjectGetRequestADM( + identifier = ProjectIdentifierADM(maybeIri = Some(projectIri)), + requestingUser = userProfile ) - }.mapTo[ProjectInfoResponseV1] + }.mapTo[ProjectGetResponseADM] // Ensure that the project isn't the system project or the shared ontologies project. - resourceProjectIri: IRI = projectInfoResponse.project_info.id + resourceProjectIri: IRI = projectResponse.project.id _ = if (resourceProjectIri == OntologyConstants.KnoraAdmin.SystemProject || resourceProjectIri == OntologyConstants.KnoraAdmin.DefaultSharedOntologiesProject) { throw BadRequestException(s"Resources cannot be created in project $resourceProjectIri") @@ -1920,8 +1963,8 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo throw BadRequestException(s"Cannot create a resource in project $resourceProjectIri with resource class $resourceClassIri, which is defined in a non-shared ontology in another project") } - namedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfoResponse.project_info) - resourceIri: IRI = stringFormatter.makeRandomResourceIri(projectInfoResponse.project_info.shortcode) + namedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV2(projectResponse.project) + resourceIri: IRI = stringFormatter.makeRandomResourceIri(projectResponse.project.shortcode) // Check user's PermissionProfile (part of UserADM) to see if the user has the permission to // create a new resource in the given project. @@ -1934,33 +1977,18 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo resourceIri, () => createResourceAndCheck( resourceClassIri = resourceClassIri, - projectIri = resourceProjectIri, + projectADM = projectResponse.project, label = label, resourceIri = resourceIri, values = values, - sipiConversionRequest = sipiConversionRequest, + file = file, creatorIri = userIri, namedGraph = namedGraph, - userProfile = userProfile, + requestingUser = userProfile, apiRequestID = apiRequestID ) ) } yield result - - // If a temporary file was created, ensure that it's deleted, regardless of whether the request succeeded or failed. - resultFuture.andThen { - case _ => - sipiConversionRequest match { - case Some(conversionRequest) => - conversionRequest match { - case conversionPathRequest: SipiConversionPathRequestV1 => - // a tmp file has been created by the resources route (non GUI-case), delete it - FileUtil.deleteFileFromTmpLocation(conversionPathRequest.source, log) - case _ => () - } - case None => () - } - } } /** @@ -1992,7 +2020,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo // Create update sparql string sparqlUpdate = queries.sparql.v1.txt.deleteResource( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfoResponse.project_info), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfoResponse.project_info), triplestore = settings.triplestoreType, resourceIri = resourceDeleteRequest.resourceIri, maybeDeleteComment = resourceDeleteRequest.deleteComment, @@ -2095,7 +2123,7 @@ class ResourcesResponderV1(responderData: ResponderData) extends Responder(respo }.mapTo[ProjectInfoResponseV1] // get the named graph the resource is contained in by the resource's project - namedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfoResponse.project_info) + namedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfoResponse.project_info) // Make a timestamp to indicate when the resource was updated. currentTime: String = Instant.now.toString diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ValueUtilV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ValueUtilV1.scala index 67a0c0109d..b027a3f1e7 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ValueUtilV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ValueUtilV1.scala @@ -73,7 +73,7 @@ class ValueUtilV1(private val settings: KnoraSettingsImpl) { } def makeSipiImagePreviewGetUrlFromFilename(projectShortcode: String, filename: String): String = { - s"${settings.externalSipiIIIFGetUrl}/$projectShortcode/$filename/full/max/0/default.jpg" + s"${settings.externalSipiIIIFGetUrl}/$projectShortcode/$filename/full/!128,128/0/default.jpg" } /** @@ -107,6 +107,7 @@ class ValueUtilV1(private val settings: KnoraSettingsImpl) { "application/octet-stream" -> "BINARY-UNKNOWN", "image/jpeg" -> "JPEG", "image/jp2" -> "JPEG2000", + "image/jpx" -> "JPEG2000", "application/pdf" -> "PDF", "application/postscript" -> "POSTSCRIPT", "application/vnd.ms-powerpoint" -> "PPT", diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala b/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala index 7be1b5d35f..bd462d1a5b 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v1/ValuesResponderV1.scala @@ -24,8 +24,8 @@ import java.time.Instant import akka.pattern._ import org.knora.webapi._ import org.knora.webapi.messages.admin.responder.permissionsmessages.{DefaultObjectAccessPermissionsStringForPropertyGetADM, DefaultObjectAccessPermissionsStringResponseADM, PermissionADM, PermissionType} +import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectADM, ProjectGetRequestADM, ProjectGetResponseADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.{SipiConstants, SipiConversionPathRequestV1, SipiConversionRequestV1, SipiConversionResponseV1} import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.v1.responder.ontologymessages.{EntityInfoGetRequestV1, EntityInfoGetResponseV1} import org.knora.webapi.messages.v1.responder.projectmessages.{ProjectInfoByIRIGetV1, ProjectInfoV1} @@ -34,7 +34,9 @@ import org.knora.webapi.messages.v1.responder.usermessages.{UserProfileByIRIGetV import org.knora.webapi.messages.v1.responder.valuemessages._ import org.knora.webapi.messages.v2.responder.ontologymessages.Cardinality import org.knora.webapi.messages.v2.responder.standoffmessages._ +import org.knora.webapi.messages.v2.responder.valuemessages.StillImageFileValueContentV2 import org.knora.webapi.responders.Responder.handleUnexpectedMessage +import org.knora.webapi.responders.v2.ResourceUtilV2 import org.knora.webapi.responders.{IriLocker, Responder, ResponderData} import org.knora.webapi.twirl.SparqlTemplateLinkUpdate import org.knora.webapi.util.IriConversions._ @@ -214,7 +216,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde // Everything seems OK, so create the value. unverifiedValue <- createValueV1AfterChecks( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), projectIri = resourceFullResponse.resinfo.get.project_id, resourceIri = createValueRequest.resourceIri, propertyIri = createValueRequest.propertyIri, @@ -611,26 +613,18 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde */ case class CurrentFileValue(property: IRI, valueObjectIri: IRI, quality: Option[Int]) - def changeFileValue(oldFileValue: CurrentFileValue, newFileValue: FileValueV1): Future[ChangeValueResponseV1] = { - changeValueV1(ChangeValueRequestV1( - valueIri = oldFileValue.valueObjectIri, - value = newFileValue, - userProfile = changeFileValueRequest.userProfile, - apiRequestID = changeFileValueRequest.apiRequestID // re-use the same id - )) - } - /** - * Preprocesses a file value change request by calling the Sipi responder to create a new file - * and calls [[changeValueV1]] to actually change the file value in Knora. + * Changes a file value in the triplestore. * * @param changeFileValueRequest a [[ChangeFileValueRequestV1]] sent by the values route. + * @param projectADM the project in which the value is being updated. * @return a [[ChangeFileValueResponseV1]] representing all the changed file values. */ - def makeTaskFuture(changeFileValueRequest: ChangeFileValueRequestV1): Future[ChangeFileValueResponseV1] = { + def makeTaskFuture(changeFileValueRequest: ChangeFileValueRequestV1, projectADM: ProjectADM): Future[ChangeFileValueResponseV1] = { + val fileValueContent: StillImageFileValueContentV2 = changeFileValueRequest.file.toStillImageFileValueContentV2 // get the Iris of the current file value(s) - val resultFuture = for { + val triplestoreUpdateFuture = for { resourceIri <- Future(changeFileValueRequest.resourceIri) @@ -648,7 +642,6 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde // get the property Iris, file value Iris and qualities attached to the resource fileValues: Seq[CurrentFileValue] = getFileValuesResponse.results.bindings.map { row: VariableResultsRow => - CurrentFileValue( property = row.rowMap("p"), valueObjectIri = row.rowMap("fileValueIri"), @@ -659,56 +652,44 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde ) } - // the message to be sent to SipiConnector - sipiConversionRequest: SipiConversionRequestV1 = changeFileValueRequest.file + // TODO: check if the file type returned by Sipi corresponds to the already existing file value type - sipiResponse: SipiConversionResponseV1 <- (storeManager ? sipiConversionRequest).mapTo[SipiConversionResponseV1] + response: ChangeValueResponseV1 <- changeValueV1(ChangeValueRequestV1( + valueIri = fileValues.head.valueObjectIri, + value = changeFileValueRequest.file, + userProfile = changeFileValueRequest.userProfile, + apiRequestID = changeFileValueRequest.apiRequestID // re-use the same id + )) - // check if the file type returned by Sipi corresponds to the already existing file value type (e.g., hasStillImageRepresentation) - _ = if (SipiConstants.fileType2FileValueProperty(sipiResponse.file_type) != fileValues.head.property) { - // TODO: remove the file from SIPI (delete request) - throw BadRequestException(s"Type of submitted file (${sipiResponse.file_type}) does not correspond to expected property type ${fileValues.head.property}") - } - - // - // handle file types individually - // - - // create the apt case class depending on the file type returned by Sipi - changedLocation: LocationV1 <- sipiResponse.file_type match { - case SipiConstants.FileType.IMAGE => - if (fileValues.size != 1) { - throw InconsistentTriplestoreDataException(s"Expected 1 file value for $resourceIri, but ${fileValues.size} given.") - } - - val oldFileValue: CurrentFileValue = fileValues.head - val newFileValue: FileValueV1 = sipiResponse.fileValueV1 - - for { - response: ChangeValueResponseV1 <- changeFileValue(oldFileValue, newFileValue) - } yield response.value match { - case fileValueV1: FileValueV1 => valueUtilV1.fileValueV12LocationV1(fileValueV1) - case other => throw AssertionException(s"Expected Sipi to change a file value, but it changed one of these: ${other.valueTypeIri}") - } - - case otherFileType => throw NotImplementedException(s"File type $otherFileType not yet supported") + changedLocation = response.value match { + case fileValueV1: FileValueV1 => valueUtilV1.fileValueV12LocationV1(fileValueV1) + case other => throw AssertionException(s"Expected Sipi to change a file value, but it changed one of these: ${other.valueTypeIri}") } } yield ChangeFileValueResponseV1( - locations = Vector(changedLocation) + locations = Vector(changedLocation), + projectADM = projectADM ) - // If a temporary file was created, ensure that it's deleted, regardless of whether the request succeeded or failed. - resultFuture.andThen { - case _ => changeFileValueRequest.file match { - case conversionPathRequest: SipiConversionPathRequestV1 => - // a tmp file has been created by the resources route (non GUI-case), delete it - FileUtil.deleteFileFromTmpLocation(conversionPathRequest.source, log) - case _ => () - } - } + ResourceUtilV2.doSipiPostUpdate( + updateFuture = triplestoreUpdateFuture, + valueContent = fileValueContent, + requestingUser = changeFileValueRequest.userProfile, + responderManager = responderManager, + storeManager = storeManager, + log = log + ) } for { + resourceInfoResponse <- (responderManager ? ResourceInfoGetRequestV1(iri = changeFileValueRequest.resourceIri, userProfile = changeFileValueRequest.userProfile)).mapTo[ResourceInfoResponseV1] + + // Get project info + projectResponse <- { + responderManager ? ProjectGetRequestADM( + identifier = ProjectIdentifierADM(maybeIri = Some(resourceInfoResponse.resource_info.get.project_id)), + requestingUser = changeFileValueRequest.userProfile + ) + }.mapTo[ProjectGetResponseADM] // Do the preparations of a file value change while already holding an update lock on the resource. // This is necessary because in `makeTaskFuture` the current file value Iris for the given resource IRI have to been retrieved. @@ -718,7 +699,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde taskResult <- IriLocker.runWithIriLock( changeFileValueRequest.apiRequestID, changeFileValueRequest.resourceIri, - () => makeTaskFuture(changeFileValueRequest) + () => makeTaskFuture(changeFileValueRequest, projectResponse.project) ) } yield taskResult @@ -880,7 +861,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde // We'll need to create a new LinkValue. changeLinkValueV1AfterChecks(projectIri = currentValueQueryResult.projectIri, - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), resourceIri = findResourceWithValueResult.resourceIri, propertyIri = propertyIri, currentLinkValueV1 = currentLinkValueQueryResult.value, @@ -983,7 +964,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde // Generate a SPARQL update. sparqlUpdate = queries.sparql.v1.txt.changeComment( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), triplestore = settings.triplestoreType, resourceIri = findResourceWithValueResult.resourceIri, propertyIri = findResourceWithValueResult.propertyIri, @@ -1101,7 +1082,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde ) sparqlUpdate = queries.sparql.v1.txt.deleteLink( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), triplestore = settings.triplestoreType, linkSourceIri = findResourceWithValueResult.resourceIri, linkUpdate = sparqlTemplateLinkUpdate, @@ -1152,7 +1133,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde } sparqlUpdate = queries.sparql.v1.txt.deleteValue( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), triplestore = settings.triplestoreType, resourceIri = findResourceWithValueResult.resourceIri, propertyIri = findResourceWithValueResult.propertyIri, @@ -2174,7 +2155,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde // Generate a SPARQL update string. sparqlUpdate = queries.sparql.v1.txt.changeLink( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), triplestore = settings.triplestoreType, linkSourceIri = resourceIri, linkUpdateForCurrentLink = sparqlTemplateLinkUpdateForCurrentLink, @@ -2321,7 +2302,7 @@ class ValuesResponderV1(responderData: ResponderData) extends Responder(responde // Generate a SPARQL update. sparqlUpdate = queries.sparql.v1.txt.addValueVersion( - dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraph(projectInfo), + dataNamedGraph = StringFormatter.getGeneralInstance.projectDataNamedGraphV1(projectInfo), triplestore = settings.triplestoreType, resourceIri = resourceIri, propertyIri = propertyIri, diff --git a/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourceUtilV2.scala b/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourceUtilV2.scala index c072117314..e749cca0fc 100644 --- a/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourceUtilV2.scala +++ b/webapi/src/main/scala/org/knora/webapi/responders/v2/ResourceUtilV2.scala @@ -25,7 +25,7 @@ import akka.util.Timeout import com.typesafe.scalalogging.Logger import org.knora.webapi.messages.admin.responder.permissionsmessages.{DefaultObjectAccessPermissionsStringForPropertyGetADM, DefaultObjectAccessPermissionsStringResponseADM} import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.{DeleteTemporaryFileRequestV2, MoveTemporaryFileToPermanentStorageRequestV2} +import org.knora.webapi.messages.store.sipimessages.{DeleteTemporaryFileRequest, MoveTemporaryFileToPermanentStorageRequest} import org.knora.webapi.messages.store.triplestoremessages.{SparqlAskRequest, SparqlAskResponse} import org.knora.webapi.messages.v2.responder.resourcemessages.ReadResourceV2 import org.knora.webapi.messages.v2.responder.valuemessages.{FileValueContentV2, ReadValueV2, ValueContentV2} @@ -159,7 +159,7 @@ object ResourceUtilV2 { updateFuture.transformWith { case Success(updateInProject: UpdateResultInProject) => // Yes. Ask Sipi to move the file to permanent storage. - val sipiRequest = MoveTemporaryFileToPermanentStorageRequestV2( + val sipiRequest = MoveTemporaryFileToPermanentStorageRequest( internalFilename = fileValueContent.fileValue.internalFilename, prefix = updateInProject.projectADM.shortcode, requestingUser = requestingUser @@ -170,7 +170,7 @@ object ResourceUtilV2 { case Failure(_) => // The file value update failed. Ask Sipi to delete the temporary file. - val sipiRequest = DeleteTemporaryFileRequestV2( + val sipiRequest = DeleteTemporaryFileRequest( internalFilename = fileValueContent.fileValue.internalFilename, requestingUser = requestingUser ) diff --git a/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala b/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala index 6242a672a4..70e265e35b 100644 --- a/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/routing/v1/ResourcesRouteV1.scala @@ -22,26 +22,22 @@ package org.knora.webapi.routing.v1 import java.io._ import java.nio.charset.StandardCharsets -import java.nio.file.Paths import java.time.Instant import java.util.UUID -import akka.http.scaladsl.model.Multipart.BodyPart import akka.http.scaladsl.model._ import akka.http.scaladsl.model.headers._ import akka.http.scaladsl.server.Directives._ import akka.http.scaladsl.server.Route -import akka.http.scaladsl.server.directives.FileInfo import akka.http.scaladsl.util.FastFuture import akka.pattern._ -import akka.stream.scaladsl.FileIO import javax.xml.XMLConstants import javax.xml.transform.stream.StreamSource import javax.xml.validation.{Schema, SchemaFactory, Validator} import org.knora.webapi._ import org.knora.webapi.messages.admin.responder.projectsmessages.{ProjectGetRequestADM, ProjectGetResponseADM, ProjectIdentifierADM} import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.{SipiConversionFileRequestV1, SipiConversionPathRequestV1} +import org.knora.webapi.messages.store.sipimessages.{GetFileMetadataRequest, GetFileMetadataResponseV2} import org.knora.webapi.messages.v1.responder.ontologymessages._ import org.knora.webapi.messages.v1.responder.resourcemessages.ResourceV1JsonProtocol._ import org.knora.webapi.messages.v1.responder.resourcemessages._ @@ -54,11 +50,9 @@ import org.knora.webapi.util.{DateUtilV1, FileUtil, SmartIri} import org.knora.webapi.viewhandlers.ResourceHtmlView import org.w3c.dom.ls.{LSInput, LSResourceResolver} import org.xml.sax.SAXException -import spray.json._ import scala.collection.immutable -import scala.concurrent.duration._ -import scala.concurrent.{Future, Promise} +import scala.concurrent.Future import scala.util.{Failure, Success, Try} import scala.xml._ @@ -224,7 +218,7 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) } - def makeCreateResourceRequestMessage(apiRequest: CreateResourceApiRequestV1, multipartConversionRequest: Option[SipiConversionPathRequestV1] = None, userADM: UserADM): Future[ResourceCreateRequestV1] = { + def makeCreateResourceRequestMessage(apiRequest: CreateResourceApiRequestV1, userADM: UserADM): Future[ResourceCreateRequestV1] = { val projectIri = stringFormatter.validateAndEscapeIri(apiRequest.project_id, throw BadRequestException(s"Invalid project IRI: ${apiRequest.project_id}")) val resourceTypeIri = stringFormatter.validateAndEscapeIri(apiRequest.restype_id, throw BadRequestException(s"Invalid resource IRI: ${apiRequest.restype_id}")) val label = stringFormatter.toSparqlEncodedString(apiRequest.label, throw BadRequestException(s"Invalid label: '${apiRequest.label}'")) @@ -234,18 +228,26 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) projectResponse: ProjectGetResponseADM <- (responderManager ? ProjectGetRequestADM(ProjectIdentifierADM(maybeIri = Some(projectIri)), requestingUser = userADM)).mapTo[ProjectGetResponseADM] } yield projectResponse.project.shortcode - // for GUI-case: - // file has already been stored by Sipi. - // TODO: in the old SALSAH, the file params were sent as a property salsah:__location__ -> the GUI has to be adapated - paramConversionRequest: Option[SipiConversionFileRequestV1] = apiRequest.file match { - case Some(createFile: CreateFileV1) => Some(SipiConversionFileRequestV1( - originalFilename = stringFormatter.toSparqlEncodedString(createFile.originalFilename, throw BadRequestException(s"The original filename is invalid: '${createFile.originalFilename}'")), - originalMimeType = stringFormatter.toSparqlEncodedString(createFile.originalMimeType, throw BadRequestException(s"The original MIME type is invalid: '${createFile.originalMimeType}'")), - projectShortcode = projectShortcode, - filename = stringFormatter.toSparqlEncodedString(createFile.filename, throw BadRequestException(s"Invalid filename: '${createFile.filename}'")), - userProfile = userADM.asUserProfileV1 - )) - case None => None + file <- apiRequest.file match { + case Some(filename) => + // Ask Sipi about the file's metadata. + val tempFileUrl = stringFormatter.makeSipiTempFileUrl(settings, filename) + + for { + fileMetadataResponse: GetFileMetadataResponseV2 <- (storeManager ? GetFileMetadataRequest(fileUrl = tempFileUrl, requestingUser = userADM)).mapTo[GetFileMetadataResponseV2] + + // TODO: check that the file stored is an image. + } yield Some(StillImageFileValueV1( + internalFilename = filename, + internalMimeType = fileMetadataResponse.internalMimeType, + originalFilename = fileMetadataResponse.originalFilename.getOrElse(throw SipiException(s"Sipi did not return the original filename of the image")), + originalMimeType = fileMetadataResponse.originalMimeType, + projectShortcode = projectShortcode, + dimX = fileMetadataResponse.width.getOrElse(throw SipiException(s"Sipi did not return the width of the image")), + dimY = fileMetadataResponse.height.getOrElse(throw SipiException(s"Sipi did not return the height of the image")) + )) + + case None => FastFuture.successful(None) } valuesToBeCreatedWithFuture: Map[IRI, Future[Seq[CreateValueV1WithComment]]] = valuesToCreate( @@ -254,10 +256,6 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) userProfile = userADM ) - // since this function `makeCreateResourceRequestMessage` is called by the POST multipart route receiving the binaries (non GUI-case) - // and by the other POST route, either multipartConversionRequest or paramConversionRequest is set if a file should be attached to the resource, but not both. - _ = if (multipartConversionRequest.nonEmpty && paramConversionRequest.nonEmpty) throw BadRequestException("Binaries sent and file params set to route. This is illegal.") - // make the whole Map a Future valuesToBeCreated: Iterable[(IRI, Seq[CreateValueV1WithComment])] <- Future.traverse(valuesToBeCreatedWithFuture) { case (propIri: IRI, valuesFuture: Future[Seq[CreateValueV1WithComment]]) => @@ -270,11 +268,7 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) label = label, projectIri = projectIri, values = valuesToBeCreated.toMap, - file = if (multipartConversionRequest.nonEmpty) // either multipartConversionRequest or paramConversionRequest might be given, but never both - multipartConversionRequest // Non GUI-case - else if (paramConversionRequest.nonEmpty) - paramConversionRequest // GUI-case - else None, // no file given + file = file, userProfile = userADM, apiRequestID = UUID.randomUUID ) @@ -296,21 +290,34 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) values <- valuesFuture } yield propIri -> values } + + convertedFile <- resourceRequest.file match { + case Some(filename) => + // Ask Sipi about the file's metadata. + val tempFileUrl = stringFormatter.makeSipiTempFileUrl(settings, filename) + + for { + fileMetadataResponse: GetFileMetadataResponseV2 <- (storeManager ? GetFileMetadataRequest(fileUrl = tempFileUrl, requestingUser = userProfile)).mapTo[GetFileMetadataResponseV2] + + // TODO: check that the file stored is an image. + } yield Some(StillImageFileValueV1( + internalFilename = filename, + internalMimeType = fileMetadataResponse.internalMimeType, + originalFilename = fileMetadataResponse.originalFilename.getOrElse(throw SipiException(s"Sipi did not return the original filename of the image")), + originalMimeType = fileMetadataResponse.originalMimeType, + projectShortcode = projectShortcode, + dimX = fileMetadataResponse.width.getOrElse(throw SipiException(s"Sipi did not return the width of the image")), + dimY = fileMetadataResponse.height.getOrElse(throw SipiException(s"Sipi did not return the height of the image")) + )) + + case None => FastFuture.successful(None) + } } yield OneOfMultipleResourceCreateRequestV1( resourceTypeIri = resourceRequest.restype_id, clientResourceID = resourceRequest.client_id, label = stringFormatter.toSparqlEncodedString(resourceRequest.label, throw BadRequestException(s"The resource label is invalid: '${resourceRequest.label}'")), values = valuesToBeCreated.toMap, - file = resourceRequest.file.map { - fileToRead => - SipiConversionPathRequestV1( - originalFilename = stringFormatter.toSparqlEncodedString(fileToRead.file.getName, throw BadRequestException(s"The filename is invalid: '${fileToRead.file.getName}'")), - originalMimeType = stringFormatter.toSparqlEncodedString(fileToRead.mimeType, throw BadRequestException(s"The MIME type is invalid: '${fileToRead.mimeType}'")), - projectShortcode = projectShortcode, - source = fileToRead.file, - userProfile = userProfile.asUserProfileV1 - ) - }, + file = convertedFile, creationDate = resourceRequest.creationDate ) } @@ -714,21 +721,12 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) val childElementsAfterLabel = childElements.tail - // Get the resource's file metadata, if any. This represents a file that has already been stored by Sipi. + // Get the name of the resource's file, if any. This represents a file that in Sipi's temporary storage. // If provided, it must be the second child element of the resource element. - val file: Option[ReadFileV1] = childElementsAfterLabel.headOption match { + val file: Option[String] = childElementsAfterLabel.headOption match { case Some(secondChildElem) => if (secondChildElem.label == "file") { - val path = Paths.get(secondChildElem.attribute("path").get.text) - - if (!path.isAbsolute) { - throw BadRequestException(s"File path $path in resource '$clientIDForResource' is not absolute") - } - - Some(ReadFileV1( - file = path.toFile, - mimeType = secondChildElem.attribute("mimetype").get.text - )) + Some(secondChildElem.attribute("filename").get.text) } else { None } @@ -952,7 +950,7 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) log = log ) } ~ post { - // Create a new resource with he given type and possibly a file (GUI-case). + // Create a new resource with the given type and possibly a file. // The binary file is already managed by Sipi. // For further details, please read the docs: Sipi -> Interaction Between Sipi and Knora. entity(as[CreateResourceApiRequestV1]) { apiRequest => @@ -962,100 +960,6 @@ class ResourcesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) request <- makeCreateResourceRequestMessage(apiRequest = apiRequest, userADM = userProfile) } yield request - RouteUtilV1.runJsonRouteWithFuture( - requestMessageF = requestMessageFuture, - requestContext = requestContext, - settings = settings, - responderManager = responderManager, - log = log - ) - } - } ~ post { - // Create a new resource with the given type, properties, and binary data (file) (non GUI-case). - // The binary data are contained in the request and have to be temporarily stored by Knora. - // For further details, please read the docs: Sipi -> Interaction Between Sipi and Knora. - entity(as[Multipart.FormData]) { formdata: Multipart.FormData => - requestContext => - - log.debug("/v1/resources - POST - Multipart.FormData - Route") - - type Name = String - - val JSON_PART = "json" - val FILE_PART = "file" - - val receivedFile = Promise[File] - - log.debug(s"receivedFile is completed before: ${receivedFile.isCompleted}") - - // collect all parts of the multipart as it arrives into a map - val allPartsFuture: Future[Map[Name, Any]] = formdata.parts.mapAsync[(Name, Any)](1) { - case b: BodyPart if b.name == JSON_PART => - log.debug(s"inside allPartsFuture - processing $JSON_PART") - b.toStrict(2.seconds).map(strict => (b.name, strict.entity.data.utf8String.parseJson)) - - case b: BodyPart if b.name == FILE_PART => - log.debug(s"inside allPartsFuture - processing $FILE_PART") - val filename = b.filename.getOrElse(throw BadRequestException(s"Filename is not given")) - val tmpFile = FileUtil.createTempFile(settings) - val written = b.entity.dataBytes.runWith(FileIO.toPath(tmpFile.toPath)) - written.map { written => - //println(s"written result: ${written.wasSuccessful}, ${b.filename.get}, ${tmpFile.getAbsolutePath}") - receivedFile.success(tmpFile) - (b.name, FileInfo(b.name, b.filename.get, b.entity.contentType)) - } - - case b: BodyPart if b.name.isEmpty => throw BadRequestException("part of HTTP multipart request has no name") - case b: BodyPart => throw BadRequestException(s"multipart contains invalid name: ${b.name}") - }.runFold(Map.empty[Name, Any])((map, tuple) => map + tuple) - - // this file will be deleted by Knora once it is not needed anymore - // TODO: add a script that cleans files in the tmp location that have a certain age - // TODO (in case they were not deleted by Knora which should not happen -> this has also to be implemented for Sipi for the thumbnails) - // TODO: how to check if the user has sent multiple files? - - val requestMessageFuture: Future[ResourceCreateRequestV1] = for { - - userADM <- getUserADM(requestContext) - userProfile = userADM.asUserProfileV1 - - allParts <- allPartsFuture - // get the json params and turn them into a case class - apiRequest: CreateResourceApiRequestV1 = try { - allParts.getOrElse(JSON_PART, throw BadRequestException(s"MultiPart POST request was sent without required '$JSON_PART' part!")).asInstanceOf[JsValue].convertTo[CreateResourceApiRequestV1] - } catch { - case e: DeserializationException => throw BadRequestException("JSON params structure is invalid: " + e.toString) - } - - // check if the API request contains file information: this is illegal for this route - _ = if (apiRequest.file.nonEmpty) throw BadRequestException("param 'file' is set for a post multipart request. This is not allowed.") - - sourcePath <- receivedFile.future - - // get the file info containing the original filename and content type. - fileInfo = allParts.getOrElse(FILE_PART, throw BadRequestException(s"MultiPart POST request was sent without required '$FILE_PART' part!")).asInstanceOf[FileInfo] - originalFilename = fileInfo.fileName - originalMimeType = fileInfo.contentType.toString - - projectIri = stringFormatter.validateAndEscapeIri(apiRequest.project_id, throw BadRequestException(s"Invalid project IRI: ${apiRequest.project_id}")) - - projectResponse: ProjectGetResponseADM <- (responderManager ? ProjectGetRequestADM(ProjectIdentifierADM(maybeIri = Some(projectIri)), requestingUser = userADM)).mapTo[ProjectGetResponseADM] - - sipiConvertPathRequest = SipiConversionPathRequestV1( - originalFilename = stringFormatter.toSparqlEncodedString(originalFilename, throw BadRequestException(s"Original filename is invalid: '$originalFilename'")), - originalMimeType = stringFormatter.toSparqlEncodedString(originalMimeType, throw BadRequestException(s"Original MIME type is invalid: '$originalMimeType'")), - projectShortcode = projectResponse.project.shortcode, - source = sourcePath, - userProfile = userProfile - ) - - requestMessage <- makeCreateResourceRequestMessage( - apiRequest = apiRequest, - multipartConversionRequest = Some(sipiConvertPathRequest), - userADM = userADM - ) - } yield requestMessage - RouteUtilV1.runJsonRouteWithFuture( requestMessageF = requestMessageFuture, requestContext = requestContext, diff --git a/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala b/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala index 723447a376..97f3f25dc0 100644 --- a/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala +++ b/webapi/src/main/scala/org/knora/webapi/routing/v1/ValuesRouteV1.scala @@ -19,7 +19,6 @@ package org.knora.webapi.routing.v1 -import java.io.File import java.time.Instant import java.util.UUID @@ -27,24 +26,22 @@ import akka.http.scaladsl.model.Multipart import akka.http.scaladsl.model.Multipart.FormData import akka.http.scaladsl.server.Directives._ import akka.http.scaladsl.server.Route -import akka.http.scaladsl.server.directives.FileInfo import akka.http.scaladsl.util.FastFuture import akka.pattern._ -import akka.stream.scaladsl.FileIO import org.knora.webapi._ import org.knora.webapi.messages.admin.responder.usersmessages.UserADM -import org.knora.webapi.messages.store.sipimessages.{SipiConversionFileRequestV1, SipiConversionPathRequestV1} +import org.knora.webapi.messages.store.sipimessages.{GetFileMetadataRequest, GetFileMetadataResponseV2} import org.knora.webapi.messages.v1.responder.resourcemessages.{ResourceInfoGetRequestV1, ResourceInfoResponseV1} import org.knora.webapi.messages.v1.responder.valuemessages.ApiValueV1JsonProtocol._ import org.knora.webapi.messages.v1.responder.valuemessages._ import org.knora.webapi.routing.{Authenticator, KnoraRoute, KnoraRouteData, RouteUtilV1} +import org.knora.webapi.util.DateUtilV1 import org.knora.webapi.util.standoff.StandoffTagUtilV2.TextWithStandoffTagsV2 -import org.knora.webapi.util.{DateUtilV1, FileUtil} -import scala.concurrent.{Future, Promise} +import scala.concurrent.Future /** - * Provides a spray-routing function for API routes that deal with values. + * Provides an Akka routing function for API routes that deal with values. */ class ValuesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) with Authenticator { @@ -323,39 +320,28 @@ class ValuesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) wit ) } - def makeChangeFileValueRequest(resIriStr: IRI, projectShortcode: String, apiRequest: Option[ChangeFileValueApiRequestV1], multipartConversionRequest: Option[SipiConversionPathRequestV1], userADM: UserADM): ChangeFileValueRequestV1 = { - if (apiRequest.nonEmpty && multipartConversionRequest.nonEmpty) throw BadRequestException("File information is present twice, only one is allowed.") - + def makeChangeFileValueRequest(resIriStr: IRI, projectShortcode: String, apiRequest: ChangeFileValueApiRequestV1, userADM: UserADM): Future[ChangeFileValueRequestV1] = { val resourceIri = stringFormatter.validateAndEscapeIri(resIriStr, throw BadRequestException(s"Invalid resource IRI: $resIriStr")) + val tempFileUrl = stringFormatter.makeSipiTempFileUrl(settings, apiRequest.file) - if (apiRequest.nonEmpty) { - // GUI-case - val fileRequest = SipiConversionFileRequestV1( - originalFilename = stringFormatter.toSparqlEncodedString(apiRequest.get.file.originalFilename, throw BadRequestException(s"The original filename is invalid: '${apiRequest.get.file.originalFilename}'")), - originalMimeType = stringFormatter.toSparqlEncodedString(apiRequest.get.file.originalMimeType, throw BadRequestException(s"The original MIME type is invalid: '${apiRequest.get.file.originalMimeType}'")), - projectShortcode = projectShortcode, - filename = stringFormatter.toSparqlEncodedString(apiRequest.get.file.filename, throw BadRequestException(s"Invalid filename: '${apiRequest.get.file.filename}'")), - userProfile = userADM.asUserProfileV1 - ) - - ChangeFileValueRequestV1( - resourceIri = resourceIri, - file = fileRequest, - apiRequestID = UUID.randomUUID, - userProfile = userADM) - } - else if (multipartConversionRequest.nonEmpty) { - // non GUI-case - ChangeFileValueRequestV1( - resourceIri = resourceIri, - file = multipartConversionRequest.get, - apiRequestID = UUID.randomUUID, - userProfile = userADM) - } else { - // no file information was provided - throw BadRequestException("A file value change was requested but no file information was provided") - } + for { + fileMetadataResponse: GetFileMetadataResponseV2 <- (storeManager ? GetFileMetadataRequest(fileUrl = tempFileUrl, requestingUser = userADM)).mapTo[GetFileMetadataResponseV2] + // TODO: check that the file stored is an image. + } yield ChangeFileValueRequestV1( + resourceIri = resourceIri, + file = StillImageFileValueV1( + internalFilename = apiRequest.file, + internalMimeType = fileMetadataResponse.internalMimeType, + originalFilename = fileMetadataResponse.originalFilename.getOrElse(throw SipiException(s"Sipi did not return the original filename of the image")), + originalMimeType = fileMetadataResponse.originalMimeType, + projectShortcode = projectShortcode, + dimX = fileMetadataResponse.width.getOrElse(throw SipiException(s"Sipi did not return the width of the image")), + dimY = fileMetadataResponse.height.getOrElse(throw SipiException(s"Sipi did not return the height of the image")) + ), + apiRequestID = UUID.randomUUID, + userProfile = userADM + ) } // Version history request requires 3 URL path segments: resource IRI, property IRI, and current value IRI @@ -484,97 +470,22 @@ class ValuesRouteV1(routeData: KnoraRouteData) extends KnoraRoute(routeData) wit put { entity(as[ChangeFileValueApiRequestV1]) { apiRequest => requestContext => - val requestMessage = for { userADM <- getUserADM(requestContext) resourceIri = stringFormatter.validateAndEscapeIri(resIriStr, throw BadRequestException(s"Invalid resource IRI: $resIriStr")) resourceInfoResponse <- (responderManager ? ResourceInfoGetRequestV1(resourceIri, userADM)).mapTo[ResourceInfoResponseV1] projectShortcode = resourceInfoResponse.resource_info.getOrElse(throw NotFoundException(s"Resource not found: $resourceIri")).project_shortcode - } yield makeChangeFileValueRequest( - resIriStr = resIriStr, - projectShortcode = projectShortcode, - apiRequest = Some(apiRequest), - multipartConversionRequest = None, - userADM = userADM - ) - - RouteUtilV1.runJsonRouteWithFuture( - requestMessage, - requestContext, - settings, - responderManager, - log - ) - } - } ~ put { - entity(as[Multipart.FormData]) { formdata => - requestContext => - - log.debug("/v1/filevalue - PUT - Multipart.FormData - Route") - - - val FILE_PART = "file" - - type Name = String - - val receivedFile = Promise[File] - - // this file will be deleted by Knora once it is not needed anymore - // TODO: add a script that cleans files in the tmp location that have a certain age - // TODO (in case they were not deleted by Knora which should not happen -> this has also to be implemented for Sipi for the thumbnails) - // TODO: how to check if the user has sent multiple files? - - /* get the file data and save file to temporary location */ - // collect all parts of the multipart as it arrives into a map - val allPartsFuture: Future[Map[Name, Any]] = formdata.parts.mapAsync[(Name, Any)](1) { - b: FormData.BodyPart => - if (b.name == FILE_PART) { - log.debug(s"inside allPartsFuture - processing $FILE_PART") - val filename = b.filename.getOrElse(throw BadRequestException(s"Filename is not given")) - val tmpFile = FileUtil.createTempFile(settings) - val written = b.entity.dataBytes.runWith(FileIO.toPath(tmpFile.toPath)) - written.map { written => - log.debug(s"written result: ${b.filename.get}, ${tmpFile.getAbsolutePath}") - receivedFile.success(tmpFile) - (b.name, FileInfo(b.name, filename, b.entity.contentType)) - } - } else { - throw BadRequestException(s"Unexpected body part '${b.name}' in multipart request") - } - }.runFold(Map.empty[Name, Any])((map, tuple) => map + tuple) - - val requestMessageFuture = for { - userADM <- getUserADM(requestContext) - allParts <- allPartsFuture - sourcePath <- receivedFile.future - // get the file info containing the original filename and content type. - fileInfo = allParts.getOrElse(FILE_PART, throw BadRequestException(s"MultiPart POST request was sent without required '$FILE_PART' part!")).asInstanceOf[FileInfo] - originalFilename = fileInfo.fileName - originalMimeType = fileInfo.contentType.toString - - resourceIri = stringFormatter.validateAndEscapeIri(resIriStr, throw BadRequestException(s"Invalid resource IRI: $resIriStr")) - resourceInfoResponse <- (responderManager ? ResourceInfoGetRequestV1(resourceIri, userADM)).mapTo[ResourceInfoResponseV1] - projectShortcode = resourceInfoResponse.resource_info.getOrElse(throw NotFoundException(s"Resource not found: $resourceIri")).project_shortcode - - sipiConvertPathRequest = SipiConversionPathRequestV1( - originalFilename = stringFormatter.toSparqlEncodedString(originalFilename, throw BadRequestException(s"The original filename is invalid: '$originalFilename'")), - originalMimeType = stringFormatter.toSparqlEncodedString(originalMimeType, throw BadRequestException(s"The original MIME type is invalid: '$originalMimeType'")), + request <- makeChangeFileValueRequest( + resIriStr = resIriStr, projectShortcode = projectShortcode, - source = sourcePath, - userProfile = userADM.asUserProfileV1 + apiRequest = apiRequest, + userADM = userADM ) - - } yield makeChangeFileValueRequest( - resIriStr = resIriStr, - projectShortcode = projectShortcode, - apiRequest = None, - multipartConversionRequest = Some(sipiConvertPathRequest), - userADM = userADM - ) + } yield request RouteUtilV1.runJsonRouteWithFuture( - requestMessageFuture, + requestMessage, requestContext, settings, responderManager, diff --git a/webapi/src/main/scala/org/knora/webapi/store/iiif/SipiConnector.scala b/webapi/src/main/scala/org/knora/webapi/store/iiif/SipiConnector.scala index 769708c12b..dd4fa70d0c 100644 --- a/webapi/src/main/scala/org/knora/webapi/store/iiif/SipiConnector.scala +++ b/webapi/src/main/scala/org/knora/webapi/store/iiif/SipiConnector.scala @@ -31,15 +31,12 @@ import org.apache.http.message.BasicNameValuePair import org.apache.http.util.EntityUtils import org.apache.http.{Consts, HttpHost, HttpRequest, NameValuePair} import org.knora.webapi.messages.store.sipimessages.GetFileMetadataResponseV2JsonProtocol._ -import org.knora.webapi.messages.store.sipimessages.RepresentationV1JsonProtocol._ -import org.knora.webapi.messages.store.sipimessages.SipiConstants.FileType import org.knora.webapi.messages.store.sipimessages._ -import org.knora.webapi.messages.v1.responder.valuemessages.{FileValueV1, StillImageFileValueV1, TextFileValueV1} import org.knora.webapi.messages.v2.responder.SuccessResponseV2 import org.knora.webapi.routing.JWTHelper import org.knora.webapi.util.ActorUtil.{handleUnexpectedMessage, try2Message} import org.knora.webapi.util.{SipiUtil, StringFormatter} -import org.knora.webapi.{BadRequestException, KnoraDispatchers, NotImplementedException, KnoraSettings, SipiException} +import org.knora.webapi.{BadRequestException, KnoraDispatchers, KnoraSettings, SipiException} import spray.json._ import scala.concurrent.ExecutionContext @@ -70,173 +67,21 @@ class SipiConnector extends Actor with ActorLogging { private val httpClient: CloseableHttpClient = HttpClients.custom.setDefaultRequestConfig(sipiRequestConfig).build override def receive: Receive = { - case convertPathRequest: SipiConversionPathRequestV1 => try2Message(sender(), convertPathV1(convertPathRequest), log) - case convertFileRequest: SipiConversionFileRequestV1 => try2Message(sender(), convertFileV1(convertFileRequest), log) - case getFileMetadataRequestV2: GetFileMetadataRequestV2 => try2Message(sender(), getFileMetadataV2(getFileMetadataRequestV2), log) - case moveTemporaryFileToPermanentStorageRequestV2: MoveTemporaryFileToPermanentStorageRequestV2 => try2Message(sender(), moveTemporaryFileToPermanentStorageV2(moveTemporaryFileToPermanentStorageRequestV2), log) - case deleteTemporaryFileRequestV2: DeleteTemporaryFileRequestV2 => try2Message(sender(), deleteTemporaryFileV2(deleteTemporaryFileRequestV2), log) - case getTextFileRequest: SipiGetTextFileRequest => try2Message(sender(), sipiGetTextFileRequestV2(getTextFileRequest), log) + case getFileMetadataRequest: GetFileMetadataRequest => try2Message(sender(), getFileMetadata(getFileMetadataRequest), log) + case moveTemporaryFileToPermanentStorageRequest: MoveTemporaryFileToPermanentStorageRequest => try2Message(sender(), moveTemporaryFileToPermanentStorage(moveTemporaryFileToPermanentStorageRequest), log) + case deleteTemporaryFileRequest: DeleteTemporaryFileRequest => try2Message(sender(), deleteTemporaryFile(deleteTemporaryFileRequest), log) + case getTextFileRequest: SipiGetTextFileRequest => try2Message(sender(), sipiGetTextFileRequest(getTextFileRequest), log) case IIIFServiceGetStatus => try2Message(sender(), iiifGetStatus(), log) case other => handleUnexpectedMessage(sender(), other, log, this.getClass.getName) } /** - * Convert a file that has been sent to Knora (non GUI-case). - * - * @param conversionRequest the information about the file (uploaded by Knora). - * @return a [[SipiConversionResponseV1]] representing the file values to be added to the triplestore. - */ - private def convertPathV1(conversionRequest: SipiConversionPathRequestV1): Try[SipiConversionResponseV1] = { - val url = s"${settings.internalSipiImageConversionUrlV1}/${settings.sipiPathConversionRouteV1}" - - callSipiConvertRoute(url, conversionRequest) - } - - /** - * Convert a file that is already managed by Sipi (GUI-case). - * - * @param conversionRequest the information about the file (managed by Sipi). - * @return a [[SipiConversionResponseV1]] representing the file values to be added to the triplestore. - */ - private def convertFileV1(conversionRequest: SipiConversionFileRequestV1): Try[SipiConversionResponseV1] = { - val url = s"${settings.internalSipiImageConversionUrlV1}/${settings.sipiFileConversionRouteV1}" - - callSipiConvertRoute(url, conversionRequest) - } - - /** - * Makes a conversion request to Sipi and creates a [[SipiConversionResponseV1]] - * containing the file values to be added to the triplestore. - * - * @param urlPath the Sipi route to be called. - * @param conversionRequest the message holding the information to make the request. - * @return a [[SipiConversionResponseV1]]. - */ - private def callSipiConvertRoute(urlPath: String, conversionRequest: SipiConversionRequestV1): Try[SipiConversionResponseV1] = { - val context: HttpClientContext = HttpClientContext.create - - val formParams = new util.ArrayList[NameValuePair]() - - for ((key, value) <- conversionRequest.toFormData) { - formParams.add(new BasicNameValuePair(key, value)) - } - - val postEntity = new UrlEncodedFormEntity(formParams, Consts.UTF_8) - val httpPost = new HttpPost(urlPath) - httpPost.setEntity(postEntity) - - val conversionResultTry: Try[String] = Try { - var maybeResponse: Option[CloseableHttpResponse] = None - - try { - maybeResponse = Some(httpClient.execute(targetHost, httpPost, context)) - - val responseEntityStr: String = Option(maybeResponse.get.getEntity) match { - case Some(responseEntity) => EntityUtils.toString(responseEntity) - case None => "" - } - - val statusCode: Int = maybeResponse.get.getStatusLine.getStatusCode - val statusCategory: Int = statusCode / 100 - - // Was the request successful? - if (statusCategory == 2) { - // Yes. - responseEntityStr - } else { - // No. Throw an appropriate exception. - val sipiErrorMsg = SipiUtil.getSipiErrorMessage(responseEntityStr) - - if (statusCategory == 4) { - throw BadRequestException(s"Sipi responded with HTTP status code $statusCode: $sipiErrorMsg") - } else { - throw SipiException(s"Sipi responded with HTTP status code $statusCode: $sipiErrorMsg") - } - } - } finally { - maybeResponse match { - case Some(response) => response.close() - case None => () - } - } - } - - // - // handle unsuccessful requests to Sipi - // - val recoveredConversionResultTry = conversionResultTry.recoverWith { - case badRequestException: BadRequestException => throw badRequestException - case sipiException: SipiException => throw sipiException - case e: Exception => throw SipiException("Failed to connect to Sipi", e, log) - } - - for { - responseAsStr: String <- recoveredConversionResultTry - - /* get json from response body */ - responseAsJson: JsValue = JsonParser(responseAsStr) - - // get file type from Sipi response - fileType: String = responseAsJson.asJsObject.fields.getOrElse("file_type", throw SipiException(message = "Sipi did not return a file type")) match { - case JsString(ftype: String) => ftype - case other => throw SipiException(message = s"Sipi returned an invalid file type: $other") - } - - // turn fileType returned by Sipi (a string) into an enum - fileTypeEnum: FileType.Value = SipiConstants.FileType.lookup(fileType) - - // create the apt case class depending on the file type returned by Sipi - fileValueV1: FileValueV1 = fileTypeEnum match { - case SipiConstants.FileType.IMAGE => - // parse response as a [[SipiImageConversionResponse]] - val imageConversionResult = try { - responseAsJson.convertTo[SipiImageConversionResponse] - } catch { - case e: DeserializationException => throw SipiException(message = "JSON response returned by Sipi is invalid, it cannot be turned into a SipiImageConversionResponse", e = e, log = log) - } - - StillImageFileValueV1( - internalMimeType = stringFormatter.toSparqlEncodedString(imageConversionResult.mimetype_full, throw BadRequestException(s"The internal MIME type returned by Sipi is invalid: '${imageConversionResult.mimetype_full}")), - originalFilename = stringFormatter.toSparqlEncodedString(imageConversionResult.original_filename, throw BadRequestException(s"The original filename returned by Sipi is invalid: '${imageConversionResult.original_filename}")), - originalMimeType = Some(stringFormatter.toSparqlEncodedString(imageConversionResult.original_mimetype, throw BadRequestException(s"The original MIME type returned by Sipi is invalid: '${imageConversionResult.original_mimetype}"))), - projectShortcode = conversionRequest.projectShortcode, - dimX = imageConversionResult.nx_full, - dimY = imageConversionResult.ny_full, - internalFilename = stringFormatter.toSparqlEncodedString(imageConversionResult.filename_full, throw BadRequestException(s"The internal filename returned by Sipi is invalid: '${imageConversionResult.filename_full}")) - ) - - case SipiConstants.FileType.TEXT => - - // parse response as a [[SipiTextResponse]] - val textStoreResult = try { - responseAsJson.convertTo[SipiTextResponse] - } catch { - case e: DeserializationException => throw SipiException(message = "JSON response returned by Sipi is invalid, it cannot be turned into a SipiTextResponse", e = e, log = log) - } - - TextFileValueV1( - internalMimeType = stringFormatter.toSparqlEncodedString(textStoreResult.mimetype, throw BadRequestException(s"The internal MIME type returned by Sipi is invalid: '${textStoreResult.mimetype}")), - internalFilename = stringFormatter.toSparqlEncodedString(textStoreResult.filename, throw BadRequestException(s"The internal filename returned by Sipi is invalid: '${textStoreResult.filename}")), - originalFilename = stringFormatter.toSparqlEncodedString(textStoreResult.original_filename, throw BadRequestException(s"The internal filename returned by Sipi is invalid: '${textStoreResult.original_filename}")), - originalMimeType = Some(stringFormatter.toSparqlEncodedString(textStoreResult.mimetype, throw BadRequestException(s"The orignal MIME type returned by Sipi is invalid: '${textStoreResult.original_mimetype}"))), - projectShortcode = conversionRequest.projectShortcode - ) - - case unknownType => throw NotImplementedException(s"Could not handle file type $unknownType") - - // TODO: add missing file types - } - - } yield SipiConversionResponseV1(fileValueV1, file_type = fileTypeEnum) - } - - /** - * Asks Sipi for metadata about a file. - * - * @param getFileMetadataRequestV2 the request. - * @return a [[GetFileMetadataResponseV2]] containing the requested metadata. - */ - private def getFileMetadataV2(getFileMetadataRequestV2: GetFileMetadataRequestV2): Try[GetFileMetadataResponseV2] = { + * Asks Sipi for metadata about a file. + * + * @param getFileMetadataRequestV2 the request. + * @return a [[GetFileMetadataResponseV2]] containing the requested metadata. + */ + private def getFileMetadata(getFileMetadataRequestV2: GetFileMetadataRequest): Try[GetFileMetadataResponseV2] = { val knoraInfoUrl = getFileMetadataRequestV2.fileUrl + "/knora.json" val request = new HttpGet(knoraInfoUrl) @@ -247,12 +92,12 @@ class SipiConnector extends Actor with ActorLogging { } /** - * Asks Sipi to move a file from temporary storage to permanent storage. - * - * @param moveTemporaryFileToPermanentStorageRequestV2 the request. - * @return a [[SuccessResponseV2]]. - */ - private def moveTemporaryFileToPermanentStorageV2(moveTemporaryFileToPermanentStorageRequestV2: MoveTemporaryFileToPermanentStorageRequestV2): Try[SuccessResponseV2] = { + * Asks Sipi to move a file from temporary storage to permanent storage. + * + * @param moveTemporaryFileToPermanentStorageRequestV2 the request. + * @return a [[SuccessResponseV2]]. + */ + private def moveTemporaryFileToPermanentStorage(moveTemporaryFileToPermanentStorageRequestV2: MoveTemporaryFileToPermanentStorageRequest): Try[SuccessResponseV2] = { val token: String = JWTHelper.createToken( userIri = moveTemporaryFileToPermanentStorageRequestV2.requestingUser.id, secret = settings.jwtSecretKey, @@ -283,12 +128,12 @@ class SipiConnector extends Actor with ActorLogging { } /** - * Asks Sipi to delete a temporary file. - * - * @param deleteTemporaryFileRequestV2 the request. - * @return a [[SuccessResponseV2]]. - */ - private def deleteTemporaryFileV2(deleteTemporaryFileRequestV2: DeleteTemporaryFileRequestV2): Try[SuccessResponseV2] = { + * Asks Sipi to delete a temporary file. + * + * @param deleteTemporaryFileRequestV2 the request. + * @return a [[SuccessResponseV2]]. + */ + private def deleteTemporaryFile(deleteTemporaryFileRequestV2: DeleteTemporaryFileRequest): Try[SuccessResponseV2] = { val token: String = JWTHelper.createToken( userIri = deleteTemporaryFileRequestV2.requestingUser.id, secret = settings.jwtSecretKey, @@ -316,7 +161,7 @@ class SipiConnector extends Actor with ActorLogging { * * @param textFileRequest the request message. */ - private def sipiGetTextFileRequestV2(textFileRequest: SipiGetTextFileRequest): Try[SipiGetTextFileResponse] = { + private def sipiGetTextFileRequest(textFileRequest: SipiGetTextFileRequest): Try[SipiGetTextFileResponse] = { val httpRequest = new HttpGet(textFileRequest.fileUrl) val sipiResponseTry: Try[SipiGetTextFileResponse] = for { @@ -394,4 +239,4 @@ class SipiConnector extends Actor with ActorLogging { case e: Exception => throw SipiException("Failed to connect to Sipi", e, log) } } -} +} \ No newline at end of file diff --git a/webapi/src/main/scala/org/knora/webapi/util/StringFormatter.scala b/webapi/src/main/scala/org/knora/webapi/util/StringFormatter.scala index e41501c515..ac8c2e8eac 100644 --- a/webapi/src/main/scala/org/knora/webapi/util/StringFormatter.scala +++ b/webapi/src/main/scala/org/knora/webapi/util/StringFormatter.scala @@ -2376,7 +2376,7 @@ class StringFormatter private(val maybeSettings: Option[KnoraSettingsImpl] = Non * @param projectInfo the project's [[ProjectInfoV1]]. * @return the IRI of the project's data named graph. */ - def projectDataNamedGraph(projectInfo: ProjectInfoV1): IRI = { + def projectDataNamedGraphV1(projectInfo: ProjectInfoV1): IRI = { OntologyConstants.NamedGraphs.DataNamedGraphStart + "/" + projectInfo.shortcode + "/" + projectInfo.shortname } @@ -2749,6 +2749,17 @@ class StringFormatter private(val maybeSettings: Option[KnoraSettingsImpl] = Non } } + /** + * Constructs a URL for accessing a file that has been uploaded to Sipi's temporary storage. + * + * @param settings the application settings. + * @param filename the filename. + * @return a URL for accessing the file. + */ + def makeSipiTempFileUrl(settings: KnoraSettingsImpl, filename: String): String = { + s"${settings.internalSipiBaseUrl}/tmp/$filename" + } + /** * Checks whether an IRI already exists in the triplestore. * diff --git a/webapi/src/test/scala/org/knora/webapi/e2e/v1/SipiV1R2RSpec.scala b/webapi/src/test/scala/org/knora/webapi/e2e/v1/SipiV1R2RSpec.scala index bf948ef436..9e80cb4ada 100644 --- a/webapi/src/test/scala/org/knora/webapi/e2e/v1/SipiV1R2RSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/e2e/v1/SipiV1R2RSpec.scala @@ -23,18 +23,16 @@ import java.io.File import java.net.URLEncoder import java.nio.file.{Files, Paths} -import akka.actor.{Props, _} +import akka.actor._ import akka.http.scaladsl.model._ import akka.http.scaladsl.model.headers.BasicHttpCredentials -import akka.http.scaladsl.server.Route import akka.http.scaladsl.testkit.RouteTestTimeout import org.knora.webapi._ import org.knora.webapi.app.{APPLICATION_MANAGER_ACTOR_NAME, ApplicationActor} import org.knora.webapi.messages.store.triplestoremessages.RdfDataObject import org.knora.webapi.messages.v1.responder.resourcemessages.{CreateResourceApiRequestV1, CreateResourceValueV1} -import org.knora.webapi.messages.v1.responder.valuemessages.{ChangeFileValueApiRequestV1, CreateFileV1, CreateRichtextV1} +import org.knora.webapi.messages.v1.responder.valuemessages.{ChangeFileValueApiRequestV1, CreateRichtextV1} import org.knora.webapi.routing.v1.{ResourcesRouteV1, ValuesRouteV1} -import org.knora.webapi.store.iiif.SourcePath import org.knora.webapi.testing.tags.E2ETest /** @@ -53,9 +51,8 @@ class SipiV1R2RSpec extends R2RSpec { private val resourcesPath = new ResourcesRouteV1(routeData).knoraApiPath private val valuesPath = new ValuesRouteV1(routeData).knoraApiPath - implicit def default(implicit system: ActorSystem) = RouteTestTimeout(settings.defaultTimeout) + implicit def default(implicit system: ActorSystem): RouteTestTimeout = RouteTestTimeout(settings.defaultTimeout) - private val rootEmail = SharedTestDataV1.rootUser.userData.email.get private val incunabulaProjectAdminEmail = SharedTestDataV1.incunabulaProjectAdminUser.userData.email.get private val testPass = "test" @@ -111,78 +108,11 @@ class SipiV1R2RSpec extends R2RSpec { "The Resources Endpoint" should { - "create a resource with a digital representation doing a multipart request containing the binary data (non GUI-case)" in { - - val fileToSend = new File(RequestParams.pathToFile) - // check if the file exists - assert(fileToSend.exists(), s"File ${RequestParams.pathToFile} does not exist") - - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "json", - HttpEntity(ContentTypes.`application/json`, RequestParams.createResourceParams.toJsValue.compactPrint) - ), - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - RequestParams.createTmpFileDir() - - Post("/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(incunabulaProjectAdminEmail, testPass)) ~> resourcesPath ~> check { - - val tmpFile = SourcePath.getSourcePath() - - //println("response in test: " + responseAs[String]) - assert(!tmpFile.exists(), s"Tmp file $tmpFile was not deleted.") - assert(status == StatusCodes.OK, "Status code is not set to OK, Knora says:\n" + responseAs[String]) - } - } - - "try to create a resource sending binaries (multipart request) but fail because the mimetype is wrong" in { - - val fileToSend = new File(RequestParams.pathToFile) - // check if the file exists - assert(fileToSend.exists(), s"File ${RequestParams.pathToFile} does not exist") - - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "json", - HttpEntity(MediaTypes.`application/json`, RequestParams.createResourceParams.toJsValue.compactPrint) - ), - // set mimetype tiff, but jpeg is expected - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/tiff`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - RequestParams.createTmpFileDir() - - Post("/v1/resources", formData) ~> addCredentials(BasicHttpCredentials(incunabulaProjectAdminEmail, testPass)) ~> Route.seal(resourcesPath) ~> check { - - val tmpFile = SourcePath.getSourcePath() - - // this test is expected to fail - - // check that the tmp file is also deleted in case the test fails - assert(!tmpFile.exists(), s"Tmp file $tmpFile was not deleted.") - //FIXME: Check for correct status code. This would then also test if the negative case is handled correctly inside Knora. - assert(status != StatusCodes.OK, "Status code is not set to OK, Knora says:\n" + responseAs[String]) - } - } - - "create a resource with a digital representation doing a params only request without binary data (GUI-case)" in { + "create a resource with a digital representation" in { + val internalFilename = "IQUO3t1AABm-FSLC0vNvVpr.jp2" val params = RequestParams.createResourceParams.copy( - file = Some(CreateFileV1( - originalFilename = "Chlaus.jpg", - originalMimeType = "image/jpeg", - filename = "./test_server/images/Chlaus.jpg" - )) + file = Some(internalFilename) ) Post("/v1/resources", HttpEntity(MediaTypes.`application/json`, params.toJsValue.compactPrint)) ~> addCredentials(BasicHttpCredentials(incunabulaProjectAdminEmail, testPass)) ~> resourcesPath ~> check { @@ -194,76 +124,11 @@ class SipiV1R2RSpec extends R2RSpec { "The Values endpoint" should { - "change the file value of an existing page (submitting binaries)" in { - - val fileToSend = new File(RequestParams.pathToFile) - // check if the file exists - assert(fileToSend.exists(), s"File ${RequestParams.pathToFile} does not exist") - - val formData = Multipart.FormData( - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/jpeg`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - RequestParams.createTmpFileDir() - - val resIri = URLEncoder.encode("http://rdfh.ch/0803/8a0b1e75", "UTF-8") - - Put("/v1/filevalue/" + resIri, formData) ~> addCredentials(BasicHttpCredentials(incunabulaProjectAdminEmail, testPass)) ~> valuesPath ~> check { - - val tmpFile = SourcePath.getSourcePath() - - assert(!tmpFile.exists(), s"Tmp file $tmpFile was not deleted.") - assert(status == StatusCodes.OK, "Status code is not set to OK, Knora says:\n" + responseAs[String]) - } - - } - - "try to change the file value of an existing page (submitting binaries) but fail because the mimetype is wrong" in { - - val fileToSend = new File(RequestParams.pathToFile) - // check if the file exists - assert(fileToSend.exists(), s"File ${RequestParams.pathToFile} does not exist") - - val formData = Multipart.FormData( - // set mimetype tiff, but jpeg is expected - Multipart.FormData.BodyPart( - "file", - HttpEntity.fromPath(MediaTypes.`image/tiff`, fileToSend.toPath), - Map("filename" -> fileToSend.getName) - ) - ) - - RequestParams.createTmpFileDir() - - val resIri = URLEncoder.encode("http://rdfh.ch/0803/8a0b1e75", "UTF-8") - - Put("/v1/filevalue/" + resIri, formData) ~> addCredentials(BasicHttpCredentials(incunabulaProjectAdminEmail, testPass)) ~> valuesPath ~> check { - - val tmpFile = SourcePath.getSourcePath() - - // this test is expected to fail - - // check that the tmp file is also deleted in case the test fails - assert(!tmpFile.exists(), s"Tmp file $tmpFile was not deleted.") - //FIXME: Check for correct status code. This would then also test if the negative case is handled correctly inside Knora. - assert(status != StatusCodes.OK, "Status code is not set to OK, Knora says:\n" + responseAs[String]) - } - - } - - - "change the file value of an existing page (submitting params only, no binaries)" in { + "change the file value of an existing page" in { + val internalFilename ="FSLC0vNvVpr-IQUO3t1AABm.jp2" val params = ChangeFileValueApiRequestV1( - file = CreateFileV1( - originalFilename = "Chlaus.jpg", - originalMimeType = "image/jpeg", - filename = "./test_server/images/Chlaus.jpg" - ) + file = internalFilename ) val resIri = URLEncoder.encode("http://rdfh.ch/0803/8a0b1e75", "UTF-8") diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala index 576c6ac1df..e2263ebb4e 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v1/ResourcesResponderV1Spec.scala @@ -28,7 +28,6 @@ import org.knora.webapi.SharedOntologyTestDataADM._ import org.knora.webapi._ import org.knora.webapi.app.{APPLICATION_MANAGER_ACTOR_NAME, ApplicationActor} import org.knora.webapi.messages.admin.responder.permissionsmessages.{ObjectAccessPermissionADM, ObjectAccessPermissionsForResourceGetADM, PermissionADM} -import org.knora.webapi.messages.store.sipimessages.SipiConversionFileRequestV1 import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.v1.responder.resourcemessages._ import org.knora.webapi.messages.v1.responder.valuemessages._ @@ -1211,7 +1210,7 @@ class ResourcesResponderV1Spec extends CoreSpec(ResourcesResponderV1Spec.config) val origname = TextValueSimpleV1("Blatt") val seqnum = IntegerValueV1(1) - val fileValueFull = StillImageFileValueV1( + val fileValue = StillImageFileValueV1( internalMimeType = "image/jp2", internalFilename = "gaga.jpg", originalFilename = "test.jpg", @@ -1237,7 +1236,7 @@ class ResourcesResponderV1Spec extends CoreSpec(ResourcesResponderV1Spec.config) "http://www.knora.org/ontology/0803/incunabula#partOf" -> Vector(LinkV1(book)), "http://www.knora.org/ontology/0803/incunabula#origname" -> Vector(origname), "http://www.knora.org/ontology/0803/incunabula#seqnum" -> Vector(seqnum), - OntologyConstants.KnoraBase.HasStillImageFileValue -> Vector(fileValueFull) + OntologyConstants.KnoraBase.HasStillImageFileValue -> Vector(fileValue) ) responderManager ! ResourceCreateRequestV1( @@ -1245,13 +1244,7 @@ class ResourcesResponderV1Spec extends CoreSpec(ResourcesResponderV1Spec.config) label = "Test-Page", projectIri = SharedTestDataADM.INCUNABULA_PROJECT_IRI, values = valuesToBeCreated, - file = Some(SipiConversionFileRequestV1( - originalFilename = "test.jpg", - originalMimeType = "image/jpeg", - filename = "./test_server/images/Chlaus.jpg", - projectShortcode = "0803", - userProfile = SharedTestDataADM.incunabulaProjectAdminUser.asUserProfileV1 - )), + file = Some(fileValue), userProfile = SharedTestDataADM.incunabulaProjectAdminUser, apiRequestID = UUID.randomUUID ) diff --git a/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala b/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala index 87a6d581f5..87eda6c17c 100644 --- a/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala +++ b/webapi/src/test/scala/org/knora/webapi/responders/v1/ValuesResponderV1Spec.scala @@ -29,7 +29,6 @@ import org.knora.webapi.SharedOntologyTestDataADM._ import org.knora.webapi.SharedTestDataADM._ import org.knora.webapi._ import org.knora.webapi.app.{APPLICATION_MANAGER_ACTOR_NAME, ApplicationActor} -import org.knora.webapi.messages.store.sipimessages.SipiConversionFileRequestV1 import org.knora.webapi.messages.store.triplestoremessages._ import org.knora.webapi.messages.v1.responder.resourcemessages.{LocationV1, ResourceFullGetRequestV1, ResourceFullResponseV1} import org.knora.webapi.messages.v1.responder.valuemessages._ @@ -1608,17 +1607,19 @@ class ValuesResponderV1Spec extends CoreSpec(ValuesResponderV1Spec.config) with "add a new image file value to an incunabula:page" in { - val fileRequest = SipiConversionFileRequestV1( - originalFilename = "Chlaus.jpg", - originalMimeType = "image/jpeg", + val fileValue = StillImageFileValueV1( + internalMimeType = "image/jp2", + internalFilename = "gaga.jpg", + originalFilename = "test.jpg", + originalMimeType = Some("image/jpg"), projectShortcode = "0803", - filename = "./test_server/images/Chlaus.jpg", - userProfile = incunabulaUser.asUserProfileV1 + dimX = 1000, + dimY = 1000 ) val fileChangeRequest = ChangeFileValueRequestV1( resourceIri = "http://rdfh.ch/0803/8a0b1e75", - file = fileRequest, + file = fileValue, apiRequestID = UUID.randomUUID, userProfile = incunabulaUser) diff --git a/webapi/src/test/scala/org/knora/webapi/store/iiif/MockSipiConnector.scala b/webapi/src/test/scala/org/knora/webapi/store/iiif/MockSipiConnector.scala index be649dfdc2..47497dd20b 100644 --- a/webapi/src/test/scala/org/knora/webapi/store/iiif/MockSipiConnector.scala +++ b/webapi/src/test/scala/org/knora/webapi/store/iiif/MockSipiConnector.scala @@ -19,35 +19,16 @@ package org.knora.webapi.store.iiif -import java.io.File - import akka.actor.{Actor, ActorLogging, ActorSystem} import akka.http.scaladsl.util.FastFuture import org.knora.webapi.messages.store.sipimessages._ -import org.knora.webapi.messages.v1.responder.valuemessages.StillImageFileValueV1 import org.knora.webapi.messages.v2.responder.SuccessResponseV2 import org.knora.webapi.util.ActorUtil._ -import org.knora.webapi.{BadRequestException, KnoraDispatchers, KnoraSettings, SipiException} +import org.knora.webapi.{KnoraDispatchers, SipiException} -import scala.concurrent.{ExecutionContext, Future} +import scala.concurrent.ExecutionContext import scala.util.{Failure, Success, Try} -/** - * Keep track of the temporary files that was written in the route - * when submitting a multipart request - */ -object SourcePath { - private var sourcePath: File = new File("") // for init - - def setSourcePath(path: File) = { - sourcePath = path - } - - def getSourcePath() = { - sourcePath - } -} - /** * Constants for [[MockSipiConnector]]. */ @@ -69,58 +50,15 @@ class MockSipiConnector extends Actor with ActorLogging { implicit val system: ActorSystem = context.system implicit val executionContext: ExecutionContext = system.dispatchers.lookup(KnoraDispatchers.KnoraActorDispatcher) - val settings = KnoraSettings(system) - - def receive = { - case sipiResponderConversionFileRequest: SipiConversionFileRequestV1 => future2Message(sender(), imageConversionResponse(sipiResponderConversionFileRequest), log) - case sipiResponderConversionPathRequest: SipiConversionPathRequestV1 => future2Message(sender(), imageConversionResponse(sipiResponderConversionPathRequest), log) - case getFileMetadataRequestV2: GetFileMetadataRequestV2 => try2Message(sender(), getFileMetadataV2(getFileMetadataRequestV2), log) - case moveTemporaryFileToPermanentStorageRequestV2: MoveTemporaryFileToPermanentStorageRequestV2 => try2Message(sender(), moveTemporaryFileToPermanentStorageV2(moveTemporaryFileToPermanentStorageRequestV2), log) - case deleteTemporaryFileRequestV2: DeleteTemporaryFileRequestV2 => try2Message(sender(), deleteTemporaryFileV2(deleteTemporaryFileRequestV2), log) + case getFileMetadataRequest: GetFileMetadataRequest => try2Message(sender(), getFileMetadata(getFileMetadataRequest), log) + case moveTemporaryFileToPermanentStorageRequest: MoveTemporaryFileToPermanentStorageRequest => try2Message(sender(), moveTemporaryFileToPermanentStorage(moveTemporaryFileToPermanentStorageRequest), log) + case deleteTemporaryFileRequest: DeleteTemporaryFileRequest => try2Message(sender(), deleteTemporaryFile(deleteTemporaryFileRequest), log) case IIIFServiceGetStatus => future2Message(sender(), FastFuture.successful(IIIFServiceStatusOK), log) case other => handleUnexpectedMessage(sender(), other, log, this.getClass.getName) } - /** - * Imitates the Sipi server by returning a [[SipiConversionResponseV1]] representing an image conversion request. - * - * @param conversionRequest the conversion request to be handled. - * @return a [[SipiConversionResponseV1]] imitating the answer from Sipi. - */ - private def imageConversionResponse(conversionRequest: SipiConversionRequestV1): Future[SipiConversionResponseV1] = { - Future { - val originalFilename = conversionRequest.originalFilename - val originalMimeType: String = conversionRequest.originalMimeType - - // we expect original mimetype to be "image/jpeg" - if (originalMimeType != "image/jpeg") throw BadRequestException("Wrong mimetype for jpg file") - - val fileValueV1 = StillImageFileValueV1( - internalMimeType = "image/jp2", - originalFilename = originalFilename, - originalMimeType = Some(originalMimeType), - projectShortcode = conversionRequest.projectShortcode, - dimX = 800, - dimY = 800, - internalFilename = "full.jp2" - ) - - // Whenever Knora had to create a temporary file, store its path - // the calling test context can then make sure that is has actually been deleted after the test is done - // (on successful or failed conversion) - conversionRequest match { - case conversionPathRequest: SipiConversionPathRequestV1 => - // store path to tmp file - SourcePath.setSourcePath(conversionPathRequest.source) - case _ => () // params request only - } - - SipiConversionResponseV1(fileValueV1, file_type = SipiConstants.FileType.IMAGE) - } - } - - private def getFileMetadataV2(getFileMetadataRequestV2: GetFileMetadataRequestV2): Try[GetFileMetadataResponseV2] = + private def getFileMetadata(getFileMetadataRequestV2: GetFileMetadataRequest): Try[GetFileMetadataResponseV2] = Success { GetFileMetadataResponseV2( originalFilename = Some("test2.tiff"), @@ -132,7 +70,7 @@ class MockSipiConnector extends Actor with ActorLogging { ) } - private def moveTemporaryFileToPermanentStorageV2(moveTemporaryFileToPermanentStorageRequestV2: MoveTemporaryFileToPermanentStorageRequestV2): Try[SuccessResponseV2] = { + private def moveTemporaryFileToPermanentStorage(moveTemporaryFileToPermanentStorageRequestV2: MoveTemporaryFileToPermanentStorageRequest): Try[SuccessResponseV2] = { if (moveTemporaryFileToPermanentStorageRequestV2.internalFilename == MockSipiConnector.FAILURE_FILENAME) { Failure(SipiException("Sipi failed to move file to permanent storage")) } else { @@ -140,11 +78,11 @@ class MockSipiConnector extends Actor with ActorLogging { } } - private def deleteTemporaryFileV2(deleteTemporaryFileRequestV2: DeleteTemporaryFileRequestV2): Try[SuccessResponseV2] = { + private def deleteTemporaryFile(deleteTemporaryFileRequestV2: DeleteTemporaryFileRequest): Try[SuccessResponseV2] = { if (deleteTemporaryFileRequestV2.internalFilename == MockSipiConnector.FAILURE_FILENAME) { Failure(SipiException("Sipi failed to delete temporary file")) } else { Success(SuccessResponseV2("Deleted temporary file")) } } -} +} \ No newline at end of file diff --git a/webapi/src/test/scala/org/knora/webapi/util/StringFormatterSpec.scala b/webapi/src/test/scala/org/knora/webapi/util/StringFormatterSpec.scala index 0f55b9e9b3..0eb6cbd5d1 100644 --- a/webapi/src/test/scala/org/knora/webapi/util/StringFormatterSpec.scala +++ b/webapi/src/test/scala/org/knora/webapi/util/StringFormatterSpec.scala @@ -918,7 +918,7 @@ class StringFormatterSpec extends CoreSpec() { val shortcode = SharedTestDataV1.imagesProjectInfo.shortcode val shortname = SharedTestDataV1.imagesProjectInfo.shortname val expected = s"http://www.knora.org/data/$shortcode/$shortname" - val result = stringFormatter.projectDataNamedGraph(SharedTestDataV1.imagesProjectInfo) + val result = stringFormatter.projectDataNamedGraphV1(SharedTestDataV1.imagesProjectInfo) result should be(expected) // check consistency of our test data