Dynamic Global Shortcuts

[Teddy-Kun]

I am not really that knowledgeable as to the specifics but correct me if I am wrong. Currently the way the portal works, is that an application has to pre-register exactly what shortcuts it needs.
This works for most applications, but as outlined in this clip, the current way the global shortcuts portal works, does not fulfill the needs of all applications. In the specific use-case of OBS (and probably some other applications as well) there is a need for dynamically changing shortcuts; enable a custom filter here, change to another scene there, etc. These are unpredictable as they are based on user actions inside the application.

Under the current system OBS could request slots for "Dynamic Action" 1-10 and then remap those internally, however that is limiting, cumbersome and unintuitive for the end-user. As such I suggest at least a discussion around adding Dynamic Global Shortcuts in addition to the current system.

I have thought of 2 different approaches each with pros and cons to enabling the usage of those dynamic global shortcuts. This is mostly based on my experience with the implementation in KDE, as that is, afaik, currently the only desktop environment with support for global shortcuts.

1. All compositors could add exceptions for applications that should receive all keystrokes similar to KDEs settings for X11 applications.

Pros:

• very simple to implement

Cons:

• Security: I think I don't need to explain this, but giving any application access to read all your keystrokes could, even just on accident, enable reading passwords out as you type them. We abandoned the X11 ways for a reason and I don't think we should go back.
• Unintuitive: While this would be exactly how it used to work before, this could create confusion as to why some applications work like before, but have to ask for it, and why some other have a different menu for all of that.
  Additionally this wouldn't require a portal, all desktops would just need to agree that this is a good idea, which I doubt will happen, thus we come.

2. Just let all applications create new shortcuts and destroy old ones as they want.

Pros:

• Intuitive: Works similar to the current global shortcut system.
• Security: All shortcuts are still managed by the compositor and applications wouldn't know what keys are pressed.

Cons:

• Some badly written applications may start to treat all shortcuts as dynamic.
• While a malicious actor couldn't read out keybindings this way , they could technically create infinite create and/or destroy events to possible crash the compositor and/or make the shortcuts UI unusable. I don't think this is likely or even that bad and also should be easily detectable

I think this would be the better approach, with a few more limitations. How I would imagine the process going is: an application requests X dynamic shortcuts. The user would get a prompt allowing or denying that, or allowing all dynamic shortcuts for this "session". Those shortcuts would survive closing and reopening the application for consistency reasons. They should not be removed, unless the application in question is either uninstalled, request the removal or the user removes them manually.

Other suggestions, ideas and/or thoughts are of course welcome.

Small note: I am not affiliated with anyone, I just want my application to work.

[Mikenux]

Hello @Teddy-Kun!

Here is a linked issue: #1101.

Option 2 is obviously the way to go.

• they could technically create infinite create and/or destroy event

What do you mean here? Trying to bind an "infinite" number of shortcuts or using BindShortcuts multiple times?

Conflicts

As noted in the linked issue, conflict handling seems necessary, especially if letting the app bind shortcuts and allowing users to use the shortcuts directly is wanted. Note that users should know how to get a list of shortcuts they can use.

Apps should avoid conflicts with system shortcuts. However, how to handle conflicts with another app also using dynamic global shortcuts? Do you expect your workflow to be interrupted (i.e. displaying a window asking which app shortcut to use) or is this something to avoid? (for example, it wouldn't be nice to be interrupted while recording)

[Teddy-Kun]

Hello @Mikenux,
sorry for the late answer. First of all thank you for linking that issue, I tried to search if something like that does exist, but I must have missed it. I haven't read it yet, only skimmed through it a bit.

What do you mean here? Trying to bind an "infinite" number of shortcuts or using BindShortcuts multiple times?

Yes one could theoretically call BindShortcuts multiple times and/or without limit. This wouldn't even have to be a malicious and could just be bad code. I think that is something that should be detected and blocked in some way by the portal.

Do you expect your workflow to be interrupted...

I don't think in the specific case of OBS that would be an issue. Usually such shortcuts would be created when creating a scene/source/filter etc. which is something that is normally not done when the user is streaming or recording. Still I also think that that is an application specific thing and its maybe worth to just not allow pre-declaring combinations for global shortcuts from the application side, because of conflicts. Or the application has to not declare them when it detects it would disrupt the workflow.

I think a generic "this application is trying to set a global shortcut, do you want to allow it?" prompt can be easily avoided and if desired the compositor could just give the desktop notification, that application X has added a shortcut, instead. That may just be the best decision as it informs the user.

All in all I think it would be very nice to get another example of an application that would like to use dynamic shortcuts, as basing it all around OBS seems like its a bit limiting.

[Mikenux]

Yes one could theoretically call BindShortcuts multiple times and/or without limit. This wouldn't even have to be a malicious and could just be bad code. I think that is something that should be detected and blocked in some way by the portal.

At least for registering shortcuts, allow to send one list of shortcuts after having pressed one (that is already registered).

For conflicts, even if there is problem with system shortcuts, choosing which action to trigger is still an option; the thing would be just that app should avoid that.

Shortcuts can certainly be silently registered without key combination (as requested in the linked issue). But maybe this issue is linked to KDE as they seems to show shortcuts of an app within their system settings? (while in GNOME, calling for a list from the app would be probably preferred)

[Mikenux]

Note that a notification is also disruptive if the user has chosen to display them.

Maybe the following:

• The app silently registers their shortcuts, with or without key combinations. On the system side, nothing is presented to the user.
• If the user triggers a key combination registered by the app, present a window listing all conflicts by highlighting the shortcut to display the list of shortcuts. In this window, have the option to not use keyboard shortcuts. The user would be interrupted once, but not again after that if wanted.
• Questions: Present the window only when a key combination conflicts with another shortcut or even if it does not (to inform and see if there are conflicting shortcuts)? Present in the window all the shortcuts to inform (i.e. not only those in conflict)?

[jadahl]

I think more consideration to a different approach is needed here; letting clients arbitrarily bind shortcuts at their own leisure is trivial to abuse, doesn't handle potential conflicts, so doesn't sound like a good solution. In the end, the user should be in control. Routing all key events to a set of applications via the portal to me is out of the question.

[Mikenux]

This appears to be to allow the user to access the system settings to assign keyboard shortcuts whenever they want (the shortcuts are inactive), based on the current application context. However, this seems to be for KDE, I'm not sure how you want to make it available in GNOME.

Is it possible to allow an app to register all the sets of shortcuts it has at once?

[Mikenux]

(forget that for now as I haven't fully understood the OBS case yet)

[Mikenux]

I guess we need a signal from the portal to know when shortcuts can be registered. For example, after sending the Activated signal. However, this assumes that users always use keyboard shortcuts, which is not the case.

The only thing I see (at least for now) is registering stable shortcuts. For this, a logical system could be used. For example, to switch to a specific scene: [shortcut to switch scene] + number/letter.

Thoughts? Is this a direction to take?