Add Key, Text and Paste action support for X11 and Mac OS using pynput #72
Conversation
Dragonfly's typeable and keyboard related files have been heavily modified to do this: - The *dragonfly/actions/keyboard.py* file is now a sub package instead. - The Windows Keyboard class has been moved to a new file: *dragonfly/actions/keyboard/_win32.py*. - KeySymbols classes for supported platforms have been defined that map common key names to platform-specific key symbols, e.g. "LSUPER" -> "win32con.VK_LWIN". - "from dragonfly import Keyboard, Typeable" (or equivalent) will now import classes for the current platform.
Python 2.7 raises errors in 'except' blocks as expected, but Python 3 doesn't. This commit changes Key to check if the typeable exists using 'get()' instead of catching a KeyError.
This is possible now that the Key and Text actions are working. The system clipboard is used through the pyperclip Clipboard class on Mac OS and X11. This commit also fixes a bug where None can be passed to pyperclip.copy() and raise an error.
drmfinlay
added
Enhancement
The "xev" program is used to capture key presses, so it must be installed. This script is not a unit test file.
|
I've added a test script for X11 using the xev program. It will need to be installed for the script to work. It should print two lines from xev per key press: one down event and one up event. For example: I've added a delay so that the output can be observed as the keys are pressed. The script will press keys like insert, numlock, caps lock, etc twice to attempt to maintain state as it was before the script was started. Errors can occur if keys are not included in the current XKB keymap (this is why the F13-24 keys didn't work for me) or if keys have been remapped (e.g. the compose key has been set to right alt). |
|
Hi @Danesprite, I'm not sure how you want me to test the script, but on a Linux (Linux Mint 19 Cinnamon based on Ubuntu 18.04) machine I cloned & installed dictation-toolbox/dragonfly.git, then ran the script with both Python 2.7.15rc1 and 3.6.7, then clicked on the xev window, then pressed a few keys. I got errors in both Python 2 and 3 but they are quite different, so I'm showing both here. Python 2:When I click on the xev window and press keys, it prints the 2 lines for each key, but after roughly 10 seconds it crashes with this error message while leaving the xev window open in the background: Python3:On startup it shows an error message but continues running anyway: Then it shows some more error messages but still continues: Then it seems to correctly press the many keyboard keys. I can confirm that it changes my console to fullscreen (by hitting F11) and opens my guake terminal (by hitting F12) just before it prints the alphabet: But I notice that with Python 3, even if I keep pressing keys in the xev window the whole time it's running, none of my pressed keys are shown in the console. (Even though it did work in Python 2) |
|
Hi Shervin. Thanks for testing this! :-) The Python 2 error you got indicates the action-pynput branch version isn't being used. Try running The first Python 3 error didn't occur for me because I tested with Python 3.5. I think the keys weren't showing in the console because the output from xev wasn't being printed after that error. I'll fix the error so that it works with The second Python 3 error means that key code 65452 (0xffac) cannot be typed for some reason. This is the "KP_Separator" or "npsep" number pad key which typically types either "," or ".". The key's behaviour makes it a bit of a special case and I guess that pynput has some trouble simulating it. I don't think this is a big problem though because that key isn't something you would normally simulate. The error also doesn't occur for me on Debian 9, but does on Lubuntu, so it appears to be system-dependent. Sorry about the fullscreen and guake terminal issues. I'll change the script to press F11 twice, but it would be difficult to make it work nicely in every environment. |
|
Oh you're right, I had installed the master branch on my system for Python2! After uninstalling it then running with Python 2 from the root folder, I get all the expected behaviour (ie: alphabet & Function keys pressed in the Key test and alphabet pressed in the Text test). It does show these errors, even though it does work: (But seems to be working fine in Python 2 for all the other keys). |
- Press a few more keys twice: F11, Super_L and Super_R. - Decode lines from 'xev' before using them. - Remove the "npsep" key because it sometimes doesn't work.
|
Good to hear it's all working apart from the separator key. I've fixed the I've just removed the separator key from the tests. The "KP_Decimal" / "npdec" key should be good enough. One benefit of using xdo/xdotool instead is that it can type keys that aren't in the current key map. |
|
Yep I can confirm that both Python 2.7.15rc1 and 3.6.7 are running perfectly now :-) |
|
Thanks again for the testing! I'll merge this soon. :-) |
- Make each mocked action refer to a class instead. - Replace mock_dyn_str_action with a mocked Mouse class, as that action is the only DynStrActionBase sub-class left that isn't implemented for other platforms.
Re: #8, #66.
I've implemented the
Key,TextandPasteactions for Mac OS and Linux/X11 using pynput. Dragonfly's documentation has been updated to mention Mac OS and X11 support forKeyandText. Dragonfly'sMouseaction can also be implemented with pynput.Windows keyboard support is unaffected by these changes; dragonfly's own win32 keyboard implementation is still used.
The dragonfly/action/keyboard.py file is now a sub-package that imports different
Keyboard,KeySymbolsandTypeableclasses depending on the platform. The virtual key codes in typeables.py have been moved into separate files for win32 and pynput. This design allows supporting other platforms in the future (e.g. Wayland) or using different libraries for current platforms. A warning message will be printed if the current platform isn't supported.As there are quite a few changes in this PR, it would be great if others could look over it and/or test it to make sure I haven't broken anything.
X11
pynput doesn't currently include virtual key codes for numpad and media keys on X11. I've added them in manually using key symbols defined in a few Xorg header files on my system. The media keys appear to be vendor-specific and might not work with all X11 server implementations, but should work on Debian-based systems like Ubuntu. The F13-24 keys don't work properly for some reason, but I don't think this is a big problem. All the other keys seem to work correctly from my testing.
The "XDG_SESSION_TYPE" environment variable must be set to "x11" for X11 keyboard support to work. This variable is typically set automatically, but if it isn't set for some reason, it can be set manually in the ~/.profile file or elsewhere.
Mac OS
Some keys don't work properly on Mac OS. Some have been removed/rebound by Apple, e.g. insert, pause, and scroll/num lock. Some just don't get typed properly. For example ":" will be typed as "a". There is a pynput issue on that which I might try to fix myself, as I suspect I know what the problem is.
Lots of warning messages about unsupported keys will be printed when dragonfly is imported on a mac. This is normal.
ValueErrorswill also be raised if unsupported keys are typed.