Add cleaned option #3234
Add cleaned option #3234
Conversation
aleph/views/collections_api.py
Outdated
| # This will disable (if True) values validation for all types of all entities / properties | ||
| # (will pass cleaned=True to the model.get_proxy() in the aleph/logic/processing.py) | ||
| cleaned = get_flag("cleaned", default=False) | ||
| # Flag is only available for admins: | ||
| if not request.authz.is_admin: | ||
| cleaned = False | ||
|
|
There was a problem hiding this comment.
Hey @ksotik, thanks for opening your first PR in the Aleph repo! :) 🎉
Could you add a test for the new flag?
https://github.com/alephdata/aleph/blob/develop/aleph/tests/test_collections_api.py
aleph/views/collections_api.py
Outdated
| # loading of document data: | ||
| safe = get_flag("safe", default=True) | ||
| # Flag is only available for admins: | ||
| if not request.authz.is_admin: | ||
| safe = True | ||
|
|
||
| # This will disable (if True) values validation for all types of all entities / properties | ||
| # (will pass cleaned=True to the model.get_proxy() in the aleph/logic/processing.py) | ||
| cleaned = get_flag("cleaned", default=False) |
There was a problem hiding this comment.
Slightly concerned about the fact that the two flags cleaned and safe do related things but behave in opposite ways when set to true:
cleaned=true: Assume that the data can be trusted and contains only valid values.safe=true: Assume that the data can NOT be trusted, so perform additional security measures.
Not sure though if there is a better alternative and probably not a huge problem as this is mostly relevant in edge cases. A validate flag is used in some other endpoint. It’s has a slightly different behavior though (validate=true means that Aleph will return an error instead of silently discarding invalid values).
There was a problem hiding this comment.
Oh, maybe the flag could be clean instead of cleaned (and default to true)?
clean=truemeans that the data cannot be trusted and that the data should be cleaned from invalid values.safe=truemeans that the data cannot be trusted and that file checksums should be removed.
|
@tillprochaska Can you take another look at this, ensure that this is now up to your standards. |
* Add cleaned option * Change cleaned=True to clean=False; add clean flag tests
This option allow to disable server-side values validation for all types.
Also fix "safe" option docs and default value inverse logic.