Extractor permission handling

[max-zilla]

For several projects, there is interest in controlling who can see and trigger extractors. Consider two cases:

1. Start an extractor that only I, a single user, can see or trigger
2. Start an extractor that can only be triggered on files/datasets in a certain space and is not enabled globally by default

We have gone through several iterations/ideas. Originally we discussed adding new fields to extractor_info where users could define these things, but that would be potentially cumbersome to maintain (e.g. docker images for different permission sets) and introduced weird interactions with Clowder's existing per-space extractor system and registering the same extractor later when extractor_info has changed.

As per recent discussion, the current idea is to track a new piece of information for extractors separate from extractor_info that maps extractor IDs to permissions, with API endpoints to control them. For hiding extractors from global lists of extractors, a DISABLED category could control whether it is enabled by default on new/existing spaces (is this already there?)

Let's say we introduce a new collection in Mongo called extractor_permissions. I imagine a document that could look like one of these examples:

# User Filter - only user UUID 1234 or a super-admin can trigger this extractor.
{
    extractor_id: 'some.extractor',
    users: ['1234'] 
}

# Space Filter - only resources in space UUID 5678 or 9000 can auto-trigger this extractor; manual users must be a member
{
    extractor_id:  'some.extractor',
    spaces: ['5678', '9000']
}

# Combination - user 1234 can only trigger this on stuff on those two spaces
{
    extractor_id:  'some.extractor',
    users: ['1234'],
    spaces: ['5678', '9000']
}

By separating these pieces, we allow the extractor version to be updated while maintaining permissions.

We could still support a permissions: {} object in extractor_info POST JSON that applies those settings to the extractor on startup, but we don't store it permanently with extractor_info in the database. It is only additive to the permissions list (if that makes sense?)

Things to consider:

• if a space ID is given, then that implies two restrictions: resource must be in space, user must be in space.

• When extractor_info is POSTed, permissions of the POSTing user could be applied. So if user A starts a new version of extractor that can only trigger in space M, but A is not in M, do we apply the permission and let the user lock themselves out? Or do we say no, you don't have permission to update this extractor_info?

• Superadmin and spaceadmin. Super can do everything, but how about space admins? If extractor has user UUIDs listed, it should presumably be withheld from enable by space option... but what if the user is in the space they admin and has asked to turn it on? Can all space admins see all extractors?

• if only a user UUID is provided, the user can submit any resource in any space they have access to? Meaning it needs to be enabled on any space the user is in? or does user permission bypass space check? How much control should spaceadmin have? I assume they should be able to block users from running any extractor they want on spaces they admin, but that could mean space admin sees 20 Workbench sidecar things to enable/disable. How should this work?

[max-zilla]

@robkooper suggested we introduce the concept of extractor owner/admins. Similar to superadmins or space admins, these are the people who have the final say in extractor permissions and can always see it in their lists, even if it is disabled globally and disabled in all spaces.

USE CASES

• Extractor UX owned by Max and is a User extractor (e.g. workbench sidecar) associated with him. We generally want him to be able to use it where allowed, but no one else.

• Extractor EX is owned by Max and is a regular extractor, intended for anyone who wants to use it.

• Spaces Y and N. Max is in Space Y, but not Space N.

• Max can enable both extractors on Y only. No one else will be able to submit to UX or see it on the list.

• The SpaceAdmins can disable the extractor for either space.

• The SpaceAdmins can enable EX for either space, but not UX. (right?)

• In addition to Spaces, there is a list of Users that Max can modify which control who can submit to the extractor. This is not the same as admin (they can't control enabling/disabling).

• Should we support multiple Extractor Admins? This could let multiple people submit to my Workbench sidecar extractor for better or worse...

All of this would work nicely with user group features as well.

[max-zilla]

To be clear, my approach unless there are other comments:

1. Add an ExtractorAdmins list to extractor that can have 1+ UUID and can be changed. Their permissions behave as described above (interaction with space admin)
2. When fetching extractor list, we check whether there is a permissions list that can restrict the access to specific space or user. If there is, we perform that check on the endpoints.

[robkooper]

Was chatting with Mike yesterday, what about the following:

• add a new field owner this field is either a UUID or an email this is the initial admin of the extractor

In clowder now the owner can add additional people to the extractor, as either

• user (can use the extractor anywhere in clowder)
• admin (can add extractor to a space, or give other users access to extractor)
• superadmin (can do anything)

If an extractor is added to a space, all users in that space can use that extractor.

Finally the extractor will show up in the list of extractors, and on the extractor details page it shows the owner of the extractor (not the other admins).

Finally, for the workbench, maybe we can add a second option hidden that will prevent the extractor from showing up in the extractor catalog.

[robkooper]

Thinking is that we don't need to restart the extractor if we want to add anohter user, or another space. But if the owner has left the organization, we can modify the extractor_info change the owner, and restart it to tell clowder about the new owner.

[max-zilla]

Questions from discussion with @lmarini -
If we are storing RMQ or other connection credentials in WB configuration to enable connection to Clowder, are those available to the user?

PrivateExtractor with it's own API endpoint, separate from current (space) extractors. When registered, it appends a unique ID to the end of the queue name that can have multiple users associated, but it can be instanced so multiple groups of users can have separate queues.