Skip to content

Commit

Permalink
Rework input_events API and expose KeyCharacterMap bindings
Browse files Browse the repository at this point in the history
With the way events are delivered via an `InputQueue` with
`NativeActivity` there is no direct access to the underlying KeyEvent
and MotionEvent Java objects and no `ndk` API that supports the
equivalent of `KeyEvent.getUnicodeChar()`

What `getUnicodeChar` does under the hood though is to do lookups into a
`KeyCharacterMap` for the corresponding `InputDevice` based on the
event's `key_code` and `meta_state` - which are things we can do via
some JNI bindings for `KeyCharacterMap`.

Although it's still awkward to expose an API like
`key_event.get_unicode_char()` we can instead provide an API that
lets you look up a `KeyCharacterMap` for any `device_id` and
applications can then use that for character mapping.

This approach is also more general than the `getUnicodeChar` utility
since it exposes other useful state, such as being able to check what
kind of keyboard input events are coming from (such as a full physical
keyboard vs a virtual / 'predictive' keyboard)

For consistency this exposes the same API through the game-activity
backend, even though the game-activity backend is technically able to
support unicode lookups via `getUnicodeChar` (since it has access to the
Java `KeyEvent` object).

This highlighted a need to be able to use other `AndroidApp` APIs while
processing input, which wasn't possible with the `.input_events()` API
design because the `AndroidApp` held a lock over the backend while
iterating events.

This changes `input_events()` to `input_events_iter()` which now returns
a form of lending iterator and instead of taking a callback that gets
called repeatedly by `input_events()` a similar callback is now passed
to `iter.next(callback)`.

The API isn't as ergonomic as I would have liked, considering that
lending iterators aren't a standard feature for Rust yet but also since
we still want to have the handling for each individual event go via a
callback that can report whether an event was "handled". I think the
slightly awkward ergonomics are acceptable though considering that
the API will generally be used as an implementation detail within
middleware frameworks like Winit.

Since this is the first example where we're creating non-trivial Java
bindings for an Android SDK API this adds some JNI utilities and
establishes a pattern for how we can implement a class binding.

It's an implementation detail but with how I wrote the binding I tried
to keep in mind the possibility of creating a procmacro later that would
generate some of the JNI boilerplate involved.
  • Loading branch information
rib committed Aug 3, 2023
1 parent ed2dc53 commit 6a986fe
Show file tree
Hide file tree
Showing 12 changed files with 829 additions and 211 deletions.
88 changes: 88 additions & 0 deletions android-activity/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,96 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- Added `KeyEvent::meta_state()` for being able to query the state of meta keys, needed for character mapping
- Added `KeyCharacterMap` JNI bindings to the corresponding Android SDK API
- Added `AndroidApp::device_key_character_map()` for being able to get a `KeyCharacterMap` for a given `device_id` for unicode character mapping

Code that handling character mapping, including dead key combining looks something like:

```rust
let mut combining_accent = None;
// Snip


let combined_key_char = if let Ok(map) = app.device_key_character_map(device_id) {
match map.get(key_event.key_code(), key_event.meta_state()) {
Ok(KeyMapChar::Unicode(unicode)) => {
let combined_unicode = if let Some(accent) = combining_accent {
match map.get_dead_key(accent, unicode) {
Ok(Some(key)) => {
info!("KeyEvent: Combined '{unicode}' with accent '{accent}' to give '{key}'");
Some(key)
}
Ok(None) => None,
Err(err) => {
log::error!("KeyEvent: Failed to combine 'dead key' accent '{accent}' with '{unicode}': {err:?}");
None
}
}
} else {
info!("KeyEvent: Pressed '{unicode}'");
Some(unicode)
};
combining_accent = None;
combined_unicode.map(|unicode| KeyMapChar::Unicode(unicode))
}
Ok(KeyMapChar::CombiningAccent(accent)) => {
info!("KeyEvent: Pressed 'dead key' combining accent '{accent}'");
combining_accent = Some(accent);
Some(KeyMapChar::CombiningAccent(accent))
}
Ok(KeyMapChar::None) => {
info!("KeyEvent: Pressed non-unicode key");
combining_accent = None;
None
}
Err(err) => {
log::error!("KeyEvent: Failed to get key map character: {err:?}");
combining_accent = None;
None
}
}
} else {
None
};
```

### Changed
- GameActivity updated to 2.0.2 (requires the corresponding 2.0.2 `.aar` release from Google) ([#88](https://github.com/rust-mobile/android-activity/pull/88))
- `AndroidApp::input_events()` is replaced by `AndroidApp::input_events_iter()` and event iteration now looks something like:

```rust
match app.input_events_iter() {
Ok(mut iter) => {
loop {
let read_input = iter.next(|event| {
let handled = match event {
InputEvent::KeyEvent(key_event) => {
// Snip
}
InputEvent::MotionEvent(motion_event) => {
// Snip
}
event => {
// Snip
}
};

handled
});

if !read_input {
break;
}
}
}
Err(err) => {
log::error!("Failed to get input events iterator: {err:?}");
}
}
```

## [0.4.3] - 2022-07-30
### Fixed
Expand Down
2 changes: 2 additions & 0 deletions android-activity/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,15 @@ native-activity = []
log = "0.4"
jni-sys = "0.3"
cesu8 = "1"
jni = "0.21"
ndk = "0.7"
ndk-sys = "0.4"
ndk-context = "0.1"
android-properties = "0.2"
num_enum = "0.6"
bitflags = "2.0"
libc = "0.2"
thiserror = "1"

[build-dependencies]
cc = { version = "1.0", features = ["parallel"] }
Expand Down
24 changes: 18 additions & 6 deletions android-activity/LICENSE
Original file line number Diff line number Diff line change
@@ -1,12 +1,24 @@
The third-party glue code, under the native-activity-csrc/ and game-activity-csrc/ directories
is covered by the Apache 2.0 license only:
# License

Apache License, Version 2.0 (docs/LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
## GameActivity

The third-party glue code, under the game-activity-csrc/ directory is covered by
the Apache 2.0 license only:

Apache License, Version 2.0 (docs/LICENSE-APACHE or <http://www.apache.org/licenses/LICENSE-2.0>)

## SDK Documentation

Documentation for APIs that are direct bindings of Android platform APIs are covered
by the Apache 2.0 license only:

Apache License, Version 2.0 (docs/LICENSE-APACHE or <http://www.apache.org/licenses/LICENSE-2.0>)

## android-activity

All other code is dual-licensed under either

* MIT License (docs/LICENSE-MIT or http://opensource.org/licenses/MIT)
* Apache License, Version 2.0 (docs/LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT License (docs/LICENSE-MIT or <http://opensource.org/licenses/MIT>)
- Apache License, Version 2.0 (docs/LICENSE-APACHE or <http://www.apache.org/licenses/LICENSE-2.0>)

at your option.
at your option.
58 changes: 58 additions & 0 deletions android-activity/src/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
use thiserror::Error;

#[derive(Error, Debug)]
pub enum AppError {
#[error("Operation only supported from the android_main() thread: {0}")]
NonMainThread(String),

#[error("Java VM or JNI error, including Java exceptions")]
JavaError(String),

#[error("Input unavailable")]
InputUnavailable,
}

pub type Result<T> = std::result::Result<T, AppError>;

// XXX: we don't want to expose jni-rs in the public API
// so we have an internal error type that we can generally
// use in the backends and then we can strip the error
// in the frontend of the API.
//
// This way we avoid exposing a public trait implementation for
// `From<jni::errors::Error>`
#[derive(Error, Debug)]
pub(crate) enum InternalAppError {
#[error("A JNI error")]
JniError(jni::errors::JniError),
#[error("A Java Exception was thrown via a JNI method call")]
JniException(String),
#[error("A Java VM error")]
JvmError(jni::errors::Error),
#[error("Input unavailable")]
InputUnavailable,
}

pub(crate) type InternalResult<T> = std::result::Result<T, InternalAppError>;

impl From<jni::errors::Error> for InternalAppError {
fn from(value: jni::errors::Error) -> Self {
InternalAppError::JvmError(value)
}
}
impl From<jni::errors::JniError> for InternalAppError {
fn from(value: jni::errors::JniError) -> Self {
InternalAppError::JniError(value)
}
}

impl From<InternalAppError> for AppError {
fn from(value: InternalAppError) -> Self {
match value {
InternalAppError::JniError(err) => AppError::JavaError(err.to_string()),
InternalAppError::JniException(msg) => AppError::JavaError(msg),
InternalAppError::JvmError(err) => AppError::JavaError(err.to_string()),
InternalAppError::InputUnavailable => AppError::InputUnavailable,
}
}
}
2 changes: 1 addition & 1 deletion android-activity/src/game_activity/input.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
use num_enum::{IntoPrimitive, TryFromPrimitive};
use std::convert::TryInto;

use crate::game_activity::ffi::{GameActivityKeyEvent, GameActivityMotionEvent};
use crate::activity_impl::ffi::{GameActivityKeyEvent, GameActivityMotionEvent};
use crate::input::{Class, Source};

// Note: try to keep this wrapper API compatible with the AInputEvent API if possible
Expand Down
Loading

0 comments on commit 6a986fe

Please sign in to comment.