Skip to content

Commit

Permalink
update docs with new response=raw, mode=sync and transmissionMode=val…
Browse files Browse the repository at this point in the history
…ue/reference supports (relates to #376, #377)
  • Loading branch information
fmigneault committed Mar 23, 2022
1 parent fd0e83e commit 68ba237
Show file tree
Hide file tree
Showing 2 changed files with 66 additions and 31 deletions.
85 changes: 56 additions & 29 deletions docs/source/processes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -477,9 +477,12 @@ and parametrization of various input/output combinations. Let's employ the follo
- `OGC API - Processes, Execution Mode <https://docs.ogc.org/is/18-062r2/18-062r2.html#sc_execution_mode>`_
for more general details on the execution negotiation (formerly with ``mode`` parameter) and more recently
with ``Prefer`` header.
- `OGC API - Processes, Response <https://docs.ogc.org/is/18-062r2/18-062r2.html#sc_execute_response>`_
- |ogc-exec-sync-responses|_ and |ogc-exec-async-responses|_
for a complete listing of available ``response`` formats considering all other parameters.

.. |exec-api| replace:: OpenAPI Execute
.. _exec-api: `exec-req`_

.. _proc_exec_body:

Execution Body
Expand Down Expand Up @@ -512,23 +515,11 @@ This could be added later if ``Multipart`` raw data representation is required.
Please |submit-issue|_ to request this feature if it is relevant for your use-cases.

.. fixme::
Filtering not implemented (everything always available).
Filtering of ``outputs`` not implemented (everything always available).
https://github.com/crim-ca/weaver/issues/380

.. fixme::
Transmission mode ``value`` not implemented. Only ``reference`` is supported.
https://github.com/crim-ca/weaver/issues/377

.. fixme::
The ``response`` field is only supported with the ``document`` value. This parameter is present only for
compatibility with other :term:`ADES` implementation, but does not actually affects `Weaver`'s response.

Response representation mode ``raw`` to be implemented.
https://github.com/crim-ca/weaver/issues/376

.. |exec-api| replace:: OpenAPI Execute
.. _exec-api: `exec-req`_

Other parameters presented in the above examples, namely ``mode`` and ``response`` are further detailed in
the following :ref:`proc_exec_mode` section.

.. _proc_exec_mode:

Expand Down Expand Up @@ -584,17 +575,36 @@ to wait on resources for too long), the :term:`Job` execution will automatically
The distinction between an `asynchronous` or `synchronous` response when executing a :term:`Job` can be
observed in multiple ways. The easiest is with the HTTP status code of the response, 200 being for
a :term:`Job` *entirely completed* synchronously, and 201 for a created :term:`Job` that should be
:ref:`monitored <proc_op_monitor>` asynchronously. Another method is to observe the ``"status"`` value. If the
status is ``accepted`` or ``running``, it means the operation is guaranteed to be `asynchronous`, since `synchronous`
always return a final status (``succeeded`` or ``failed``). Note that a final status is possible in both modes, so
seing one of those values does not *guarantee* it was executed `synchronously`, but the complete :term:`Job` status
can be immediately requested with the :ref:`GetStatus <proc_op_status>` request. In general, a `synchronous` response
will be much more verbose than an `asynchronous` one, since details are not yet all available. Finally, it is possible
to extract the ``Preference-Applied`` response header which will clearly indicate if the submitted ``Prefer`` header
was respected (because it could be with available worker resources) or not. In general, this means that if
the :term:`Job` submission request was not provided with ``Prefer: wait=X`` **AND** replied with the
same ``Preference-Applied`` value, it is safe to assume `Weaver` decided to queue the :term:`Job` for `asynchronous`
execution. That :term:`Job` could be executed immediately, or at a later time, according to worker availability.
:ref:`monitored <proc_op_monitor>` asynchronously. Another method is to observe the ``"status"`` value.
Effectively, a :term:`Job` that is executed `asynchronously` will return status information contents, while
a `synchronous` :term:`Job` will return the results directly, along a ``Location`` header referring to the
equivalent contents returned by :ref:`GetStatus <proc_op_status>` as in the case of `asynchronous` :term:`Job`.
It is also possible to extract the ``Preference-Applied`` response header which will clearly indicate if the
submitted ``Prefer`` header was respected (because it could be with available worker resources) or not.
In general, this means that if the :term:`Job` submission request was not provided with ``Prefer: wait=X`` **AND**
replied with the same ``Preference-Applied`` value, it is safe to assume `Weaver` decided to queue the :term:`Job`
for `asynchronous` execution. That :term:`Job` could be executed immediately, or at a later time, according to
worker availability.

It is also possible that a ``failed`` :term:`Job`, even when `synchronous`, will respond with equivalent contents
to the status location instead of results. This is because it is impossible for `Weaver` to return
the result(s) as outputs would not be generated by the incomplete :term:`Job`.

Finally, the ``response`` parameter defines how to return the results produced by the :term:`Process`.
When ``response=document``, regardless of ``mode=async`` or ``mode=sync``, and regardless of requested
outputs ``transmissionMode=value`` or ``transmissionMode=reference``, the results will be returned in
a :term:`JSON` format containing either literal values or URL references to produced files. If ``mode=async``,
this results *document* is obtained with |result-req|_ request, while ``mode=sync`` returns it directly.
When ``response=raw``, the specific contents (type and quantity), HTTP ``Link`` headers or a mix of those components
depends both on the number of available :term:`Process` outputs, which ones were requested, and how they were
requested (i.e.: ``transmissionMode``). It is also possible that further content negotiation gets involved
accordingly to the ``Accept`` header and available ``Content-Type`` of the outputs if multiple formats are supported
by the :term:`Process`. For more details regarding those combination, the official
|ogc-exec-sync-responses|_ and |ogc-exec-async-responses|_ should be employed as reference.

For any of the previous combinations, it is always possible to obtain :term:`Job` outputs, along with logs, exceptions
and other details using the :ref:`proc_op_result` endpoints.


.. _proc_exec_steps:

Expand Down Expand Up @@ -1233,10 +1243,10 @@ format is employed according to the chosen location.

.. _proc_op_result:

Obtaining output results, logs or errors
Obtaining results, outputs, logs or errors
---------------------------------------------------------------------

In the case of successful :term:`Job` execution, the outputs can be retrieved with |result-req|_ request to list
In the case of successful :term:`Job` execution, the *outputs* can be retrieved with |output-req|_ request to list
each corresponding output ``id`` with the generated file reference URL. Keep in mind that the purpose of those URLs are
only to fetch the results (not persistent storage), and could therefore be purged after some reasonable amount of time.
The format should be similar to the following example, with minor variations according to :ref:`Configuration`
Expand All @@ -1253,6 +1263,15 @@ parameters for the base :term:`WPS` output location:
]
}
For the :term:`OGC` compliant endpoint, the |result-req| request can be employed instead.
In the event of a :term:`Job` executed with ``response=document``, the contents will be very similar.
On the other hand, a :term:`Job` submitted with ``response=raw`` can produce many alternative variations according
to :term:`OGC` requirements. For this reason, the *outputs* endpoint will always provide all data and file references
in the response body as :term:`Job`, no matter the original ``response`` format. The *outputs* endpoint can also
receive additional query parameters, such as ``schema``, to return contents formatted similarly to *results*, but
enforcing a :term:`JSON` body as if ``response=document`` was specified during submission of the :term:`Process`
execution.

In situations where the :term:`Job` resulted into ``failed`` status, the |except-req|_ can be use to retrieve
the potential cause of failure, by capturing any raised exception. Below is an example of such exception details.

Expand Down Expand Up @@ -1288,6 +1307,14 @@ Note again that the more the :term:`Process` is verbose, the more tracking will
.. literalinclude:: ../../weaver/wps_restapi/examples/job_logs.json
:language: json


.. note::
All endpoints to retrieve any of the above information about a :term:`Job` can either be requested directly
(i.e.: ``/jobs/{jobID}/...``) or with equivalent :term:`Provider` and/or :term:`Process` prefixed endpoints,
if the requested :term:`Job` did refer to those :term:`Provider` and/or :term:`Process`.
A *local* :term:`Process` would have its :term:`Job` references as ``/processes/{processId}/jobs/{jobID}/...``
while a :ref:`proc_remote_provider` will use ``/provider/{providerName}/processes/{processId}/jobs/{jobID}/...``.

.. _vault:

Uploading File to the Vault
Expand Down
12 changes: 10 additions & 2 deletions docs/source/references.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@
.. _ogc-home: `ogc`_
.. |ogc-proc-api| replace:: OGC API - Processes
.. _ogc-proc-api: https://github.com/opengeospatial/ogcapi-processes
.. |ogc-exec-sync-responses| replace:: OGC API - Processes, Responses (sync)
.. _ogc-exec-sync-responses: https://docs.ogc.org/is/18-062r2/18-062r2.html#sc_execute_response
.. |ogc-exec-async-responses| replace:: OGC API - Processes, Responses (async)
.. _ogc-exec-async-responses: https://docs.ogc.org/is/18-062r2/18-062r2.html#response_7
.. |pywps| replace:: PyWPS
.. _pywps: https://github.com/geopython/pywps/
.. |pywps-status| replace:: Progress and Status Report
Expand Down Expand Up @@ -135,8 +139,12 @@
.. _status-req-name: `status-req`_
.. |status-req| replace:: ``GET {WEAVER_URL}/processes/{id}/jobs/{id}`` (GetStatus)
.. _status-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Status%2Fpaths%2F~1processes~1{process_id}~1jobs~1{job_id}%2Fget
.. |result-req| replace:: ``GET {WEAVER_URL}/processes/{id}/jobs/{id}/result`` (GetResult)
.. _result-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Results%2Fpaths%2F~1processes~1%7Bprocess_id%7D~1jobs~1%7Bjob_id%7D~1result%2Fget
.. |inputs-req| replace:: ``GET {WEAVER_URL}/jobs/{id}/inputs`` (Inputs)
.. _input-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/inputs/paths/~1jobs~1{job_id}~1inputs/get
.. |output-req| replace:: ``GET {WEAVER_URL}/jobs/{id}/outputs`` (Outputs)
.. _output-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/outputs/paths/~1jobs~1{job_id}~1outputs/get
.. |result-req| replace:: ``GET {WEAVER_URL}/jobs/{id}/results`` (Results)
.. _result-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/Results/paths/~1jobs~1{job_id}~1results/get
.. |update-token-req| replace:: Update Token
.. _update-token-req: https://pavics-weaver.readthedocs.io/en/latest/api.html#tag/UpdateToken/paths/~1processes~1{process_id}/put
.. |vault-upload-req| replace:: Vault File Upload (POST)
Expand Down

0 comments on commit 68ba237

Please sign in to comment.