-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Job output transform #548
base: master
Are you sure you want to change the base?
Job output transform #548
Conversation
Direct Links in job output + makefile + tests
Testing transformer engine added
Remove requirements-trfm Add to main requirements
Why not pandas
Update requirements
Ajout Multi frame Images
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a few linting issues that can be resolved automatically using the make fix[...]
targets.
Validate as well using make check[...]
targets any other flagged issues that might need some manual adjustments. I didn't comment on all of them.
I did not go in depth into the actual implementation of the transform functions.
I think there is a few tests missing to validate that the actual results from each transformation is valid (eg: pixel values from one format to another are properly rendered or expected text from one format to another is properly structured).
# noqa: E241,E501 | ||
(uri_provider_jobs, Visibility.PUBLIC, self.user_admin_id, filter_service(public_jobs)), # noqa: E241,E501 | ||
|
||
] # type: List[Tuple[str, str, Union[None, int], List[Job]]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Revert auto-formating by your IDE here and above.
weaver/formats.py
Outdated
|
||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove extra lines
weaver/wps_restapi/jobs/jobs.py
Outdated
Retrieve the possible formats of an output job. | ||
""" | ||
job = get_job(request) | ||
return get_job_possible_output_formats(job) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should do some kind of sd.JobTranformerSchema().deserialize(<body>)
check before returning the contents to ensure they match the format of weaver/wps_restapi/examples/job_transformer.json
weaver/tranform/tiff.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add missing docstrings and typings.
weaver/tranform/transform.py
Outdated
self.file_path = file_path | ||
self.cmt = current_media_type.lower() | ||
self.wmt = wanted_media_type.lower() | ||
self.output_path = self.file_path |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The output path needs to be modifiable (either from init arg, or modified after before the process
/get
calls.
More importantly, no additional files should be added under the original WPS outputs sub-directories, because their contents could be critical to the packing/detection of other apps in a workflow chain.
We need to consider the creation of some "sibling" directory next to the WPS-output-ID dir to avoid modifying the output checksums.
…sform for linting
fixes #18
fixes #100
Pour les conversions image, si GeoTiff, distinguo fait dans class Tiff