API: Gestures

[bryphe]

Pulling out the gestures piece from #227 to track, since I think it's a bit further out then a specific fix for #227

From @jordwalke:

It would be nice if revery left room for gesture handlers to be built - a click is just a special case of a more complicated gesture system. but even with clicks, you often want very gesture-like cancellation. For example, if you mouse down, and then someone scrolls on the trackpad, the scrolling gesture should be able to prevent the click gesture from being recognized.

And @wokalski:

cc @rauanmayemir we talked about it yesterday. It'd be interesting to have the same API surface (we could have an empty .mli as a dep to enforce it automatically.)

Definitely on board! Would be great to have a common 'gesture' infra for all the projects to share. I'm curious though if Brisk would be using the native gesture handlers, as there are NSGestureRecognizer/UIGestureRecognizer.

The gesture working isn't high on my priority list (at the moment), since it is not on the critical path for Oni 2 - but would ❤️ help if someone is interested in experimenting. It's very important long-term!

Although cross-compilation is not available, we could prototype APIs by adding touch event handlers, and compiling to a WebGL and JS target (and hosting in Cordova) - so at least we could play with the gesture handling and the behavior of the APIs.

[jordwalke]

More notes for posterity:

Even the standard click interaction with a mouse is a pretty involved gesture that is just a special case of multitouch tap.

There is a hit testing region, that makes some gesture recognizer enter into a "possible" state, and then another rectangle which might be different than the initial hit test rectangle that defines when the gesture recognize if the touch/mouse is released. Dragging in and out of this (often even larger) rectangle makes the click either recognized or not when the mouse button is released. Usually you get streaming feedback during this "simple" click interaction anytime the mouse moves and you highlight or unhighlight the button to convey that a release of the mouse will trigger a "recognized" click or not.

Other gestures like scrolling the mouse wheel / ping zoom would be another gesture that "cancels" the click when the mouse button is still held down but not released. If a modal dialog pops up, that would trigger a cancelation of the click, and people building "simple click" handlers should be able to specify how the UI responds to that.

For a simple clicky button you might do:

• mouse down: Animate into a depressed state.
• mouse out and then back in, do the animation in forwards/reverse.
• when back in the rect, if a modal popup is triggered by some other system, the gesture (click) should be canceled and you'd respond to that by reversing the depressed animation. If no modal pops up and you release the mouse button, a special "click" animation is triggered which bounces the button back into its resting state.

This is definitely important for mobile apps but I also think it's great for other UIs and games etc where you don't use stock widgets and so exaggerated feedback helps users quickly learn the non-standard non-OS UI.

[bryphe]

Thanks for the feedback, @jordwalke ! And for sharing out the work you've done already on gestures: reasonml/reason-react@ac13a43

Looking forward to integrating it!