-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rework input_events
API and expose KeyCharacterMap
bindings
#102
Conversation
6a986fe
to
57e506a
Compare
Cc: @lucasmerlin it could be good if you'd also be able to take a look at this, considering the overlap with the recent GameTextInput changes. |
Although this is a fairly big change I'm really hoping to land this relatively quickly ideally so we can land corresponding support in Winit 0.29 🤞 (currently Winit just has a hacky mapping from raw key codes to characters which is embarrassingly limited). |
- Lets us build with cargo ndk 3+ - Lets us remove suppression for false-negative clippy warning about unsafe blocks in unsafe functions - Should unblock CI for #102 - 1.68.0 notably also builds the standard library with a newer r25 NDK toolchain which avoid the need for awkward libgcc workarounds, so it's anyway a desirable baseline for Android projects.
ef335b7
to
62f3158
Compare
ae546f6
to
bb0f1af
Compare
I had a look over the changes and everything seems good 👍 I am not a keyboard input expert though. |
bb0f1af
to
cb85782
Compare
a2fe446
to
d0f10a0
Compare
Cool, thanks for taking a look @lucasmerlin I've just pushed some documentation updates and also renamed I'm currently planning to merge once it passes CI |
cb85782
to
26b450d
Compare
With the way events are delivered via an `InputQueue` with `NativeActivity` there is no direct access to the underlying KeyEvent and MotionEvent Java objects and no `ndk` API that supports the equivalent of `KeyEvent.getUnicodeChar()` What `getUnicodeChar` does under the hood though is to do lookups into a `KeyCharacterMap` for the corresponding `InputDevice` based on the event's `key_code` and `meta_state` - which are things we can do via some JNI bindings for `KeyCharacterMap`. Although it's still awkward to expose an API like `key_event.get_unicode_char()` we can instead provide an API that lets you look up a `KeyCharacterMap` for any `device_id` and applications can then use that for character mapping. This approach is also more general than the `getUnicodeChar` utility since it exposes other useful state, such as being able to check what kind of keyboard input events are coming from (such as a full physical keyboard vs a virtual / 'predictive' keyboard) For consistency this exposes the same API through the game-activity backend, even though the game-activity backend is technically able to support unicode lookups via `getUnicodeChar` (since it has access to the Java `KeyEvent` object). This highlighted a need to be able to use other `AndroidApp` APIs while processing input, which wasn't possible with the `.input_events()` API design because the `AndroidApp` held a lock over the backend while iterating events. This changes `input_events()` to `input_events_iter()` which now returns a form of lending iterator and instead of taking a callback that gets called repeatedly by `input_events()` a similar callback is now passed to `iter.next(callback)`. The API isn't as ergonomic as I would have liked, considering that lending iterators aren't a standard feature for Rust yet but also since we still want to have the handling for each individual event go via a callback that can report whether an event was "handled". I think the slightly awkward ergonomics are acceptable though considering that the API will generally be used as an implementation detail within middleware frameworks like Winit. Since this is the first example where we're creating non-trivial Java bindings for an Android SDK API this adds some JNI utilities and establishes a pattern for how we can implement a class binding. It's an implementation detail but with how I wrote the binding I tried to keep in mind the possibility of creating a procmacro later that would generate some of the JNI boilerplate involved.
26b450d
to
af331e3
Compare
(Note: I've marked as draft while I still need to update the docs more)
With the way events are delivered via an
InputQueue
withNativeActivity
there is no direct access to the underlying KeyEvent and MotionEvent Java objects and nondk
API that supports the equivalent ofKeyEvent.getUnicodeChar()
What
getUnicodeChar
does under the hood though is lookup into aKeyCharacterMap
for the correspondingInputDevice
based on the event'skey_code
andmeta_state
- which we can do via some JNI bindings forKeyCharacterMap
.Although it's still awkward to expose an API like
key_event.get_unicode_char()
we can instead provide an API that lets you look up aKeyCharacterMap
for anydevice_id
and applications can then use that for character mapping.This approach is also more general than the
getUnicodeChar
utility since it exposes other useful state, such as being able to check what kind of keyboard input events are coming from (such as a full physical keyboard vs a virtual / 'predictive' keyboard)For consistency this exposes the same API through the game-activity backend, even though the game-activity backend is technically able to support unicode lookups via
getUnicodeChar
(since it has access to the JavaKeyEvent
object).This highlighted a need to be able to use other
AndroidApp
APIs while processing input, which wasn't possible with the.input_events()
API design because theAndroidApp
held a lock over the backend while iterating events.This changes
input_events()
toinput_events_iter()
which now returns a form of lending iterator and instead of taking a callback that gets called repeatedly byinput_events()
a similar callback is now passed toiter.next(callback)
.Code that iterates events now looks something like:
Code to handle unicode character mapping, including handling dead key handling would look something like:
The API isn't as ergonomic as I would have liked, considering that lending iterators aren't a standard feature for Rust yet but also since we still want to have the handling for each individual event go via a callback that can report whether an event was "handled". I think the slightly awkward ergonomics are acceptable though considering that the API will generally be used as an implementation detail within middleware frameworks like Winit.
Since this is the first example where we're creating non-trivial Java bindings for an Android SDK API this adds some JNI utilities and establishes a pattern for how we can implement a class binding.
There is now a public Error and Result type that can convey JNI errors (but without exposing any
jni-rs
types in the public API) as well as failures to get an input iterator (which could happen if an app attempted to get more than one iterator at the same time).It's an implementation detail but with how I wrote the binding I tried to keep in mind the possibility of creating a procmacro later that would generate some of the JNI boilerplate involved.