Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Step-by-step input/output job snapshot, status and retrieval #239

Open
fmigneault opened this issue Mar 19, 2021 · 0 comments
Open
Assignees
Labels
feature/job Issues related to job execution, reporting and logging. project/CRIM-DEVOPS Project linked to the CRIM project DevOps/Weaver (https://crim-ca.atlassian.net/browse/GD-47). project/DACCS Related to DACCS project (https://github.com/orgs/DACCS-Climate) triage/enhancement New feature or request triage/feature New requested feature.

Comments

@fmigneault
Copy link
Collaborator

fmigneault commented Mar 19, 2021

Because of the multiple I/O format/conversion handling between WPS-REST, WPS-1/2, actual WPS XML requests and CWL package, it is hard to pinpoint the source of an error when one occurs.

Job object saved to db should provide, inputs_xml (values for WPS resquest, see #240), inputs_json (set after parsing of raw inputs and validation) and inputs_cwl (set before package execution) properties to get/set the inputs definition at any given execution step on top of the existing property for submitted inputs.
Idem for outputs.

When sending a request to /jobs/{id}/(inputs|outputs), the full details should be retrievable for each step using a query parameter to control all|cwl|wps|submit (default: submit to preserve current behaviour)

Note

Returned input values should consider masking according to #511

Adding this detail would help identify where a conversion did not occur correctly, or where an error was raised by looking at the last inputs/outputs section that had values and next that didn't have any. Another property Job.step should be added to directly running this interpretation mechanism and return a value identifying the last reached step. The step should be returned in JSON response of Job status.

relates to #221, #238, DAC-532

@fmigneault fmigneault added triage/enhancement New feature or request triage/feature New requested feature. feature/job Issues related to job execution, reporting and logging. labels Mar 19, 2021
@fmigneault fmigneault self-assigned this Mar 19, 2021
@fmigneault fmigneault added project/DACCS Related to DACCS project (https://github.com/orgs/DACCS-Climate) project/CRIM-DEVOPS Project linked to the CRIM project DevOps/Weaver (https://crim-ca.atlassian.net/browse/GD-47). labels May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/job Issues related to job execution, reporting and logging. project/CRIM-DEVOPS Project linked to the CRIM project DevOps/Weaver (https://crim-ca.atlassian.net/browse/GD-47). project/DACCS Related to DACCS project (https://github.com/orgs/DACCS-Climate) triage/enhancement New feature or request triage/feature New requested feature.
Projects
None yet
Development

No branches or pull requests

1 participant