UI - ScrollView: Momentum scrolling + Bounce

[bryphe]

Momentum scrolling is a natural interaction for scrolls - it'd be great to have this instead of the 'instant' scroll behavior today in #195 . The momentum scrolling gives more context during the interaction and more control - making it easier for you to scroll to the right element.

A physically based scroll model

• When using the mousewheel, apply a 'force' to the scroll thumb. Force = mass * acceleration, and we can simplify and just assume a mass of 1 for the thumb, which would be Force = acceleration. Each frame, we'd increase the velocity of the thumb based on the acceleration * t. In addition, we'd adjust the actual position of the thumb based on velocity (position = position + velocity * t). This would be similiar for other gestures, like a scroll gesture on touch.

• When clicking on the track, use an appropriate force. This would be similiar to the mousewheel, but would need to be calibrated to use more 'force'.

• Model 'friction' - without any friction, as soon as you scroll, the cursor would just fly to the end! So we'd need to model friction. Friction is a constant resistive force - this could be simplified by subtracting the acceleration by a constant factor * t every frame. This could be modeled in more detail by static friction and kinetic friction, but I'm not sure if that actually is useful for the UX.

• Model 'springs' at the boundaries - Without modelling any sort of spring at the boundaries, the scroll would be smooth until the end, and then it would just instantly stop. iOS implements a nice 'bounce' effect - this can be modeled physically by using the spring equation F=-kx, where k is a spring constant (can play with different values), and x is the distance passed the boundary. In effect, as you go further out the boundary, the force applied increases - this gives a nice bounce effect. Different values of k give different types of bounce (the stiffness of the spring). We'd have to calculate the force due to spring acceleration with this, and apply it to our formula too.

Implementation

To manage momentum scrolling, we'd need to store some state for each scrollbar:

• The current position (this is already tracked)
• The current velocity
• The current acceleration

We'd also need to track the time between each frame (t or delta time, to apply to the equations).

While the physics 'simulation' is in effect, we'd want to render every frame at (60fps). This could be implemented by setting a state value (like the last frame time) every time we want a new frame - this will trigger a re-render because the reconciler will mark the tree as dirty. We should do this whenever a force is applied, or whenever the velocity is greater than a certain threshold. Once the scrollbar has 'settled' and is no longer in motion, we should no longer re-render.

Each time we render, we'd want to calculate a new position:

position = position + velocity * t

And a new velocity:

velocity = velocity + acceleration * t

And the real interesting stuff is the acceleration:

acceleration = acceleration - frictionAcceleration * t + springAcceleration * t;

We could use an option(Time.t) to track the last frametime. If it is None, there is no animation in progress - if it is Some(t), we know there is an animation and we should use it to calculate new state. Whenever a force is applied (via mousewheel, clicking on a track, etc), we should set Some(t). Once the velocity settles below a certain threshold, we should set it back to None.

This would be a fun experiment in applying animations in Revery, but a lot of moving parts!

Other Challenges

We currently use the Slider control as the scrollbar - would it make sense to factor it out to a separate control? I'm not sure the momentum aspect makes sense for a Slider, so it seems logical to factor that out to a separate ScrollBar component.

Platform-Specific Concerns

This isn't a platform-native feature - but with this physical model, we could potential tweak the constants (the friction applied, the force applied, the spring stiffness) to work well on different platforms. For example - potentially on Windows, the 'spring' at the boundaries of the scroll could be much stiffer than on OSX/iOS.

[jchavarri]

Popmotion has one library called "Pure" that has a functional design and has made a lot of good decisions to shape the API so that is usable in different environments (desktop, iPad, VR...), like separating velocity from device frame rate (might sound obvious, but many libraries don't do that).

It recently added Inertia to the API:

• https://popmotion.io/blog/20190123-introducing-inertia-animation-for-popmotion-pure/
• https://popmotion.io/api/inertia/

Maybe this could serve as inspiration, it seems many of the data types and functions could be ported to Reason idioms pretty nicely. Idk if these interpolation primitives should leave in Revery in the long-term? 🤔 but we could definitely start experimenting "in-repo" 😄

@bryphe Do you think taking the interesting parts from Popmotion to allow for inertia, and putting them under a single module in Revery would be a step in the good direction?

Edit: I stumbled upon these two articles that include great explanations on how iOS scrolling works:

• JavaScript Kinetic Scrolling: Part 2
• Flick list with its momentum scrolling and deceleration

There is an implementation here.

[jchavarri]

Hey @bryphe! I was investigating this and realized a couple of things:

• The wheel events already have some momentum, at least in my Mac. Should we just reuse those values? Or is the idea to reimplement it?
• I noticed that the pixel values for wheel events in the browser provide values way larger than the desktop build. Not sure if this is something related to jsoo, the browser, or st else. The different is almost 10x as fast (20-25 pixels max per event in desktop, up to 300 px in browser)

[bryphe]

Popmotion has one library called "Pure" that has a functional design and has made a lot of good decisions to shape the API so that is usable in different environments (desktop, iPad, VR...), like separating velocity from device frame rate (might sound obvious, but many libraries don't do that).

Neat! I didn't know about this library. I do like the functional API they provide (cc @wokalski - we were talking about animation today), and the inertia/springs/momentum would be a natural fit for this work on scrolling.

Idk if these interpolation primitives should leave in Revery in the long-term? 🤔 but we could definitely start experimenting "in-repo"

@bryphe Do you think taking the interesting parts from Popmotion to allow for inertia, and putting them under a single module in Revery would be a step in the good direction?

Experimenting in-repo sounds fine; and then we could extract them out to a separate library as it makes sense 👍 Starting with a module and porting over these primitives sounds like a great way to experiment. Would be fun to have an example app that shows a similar set of animation/physics primitives!

The only concern I had is I couldn't find license info for Popmotion Pure - do you happen to know where we can find that? (If it's not compatible with MIT, we might not be able to do a direct port).

Nice find on those links, too! Some interesting observations that I missed on my cursory investigation, like:

Using a camera and capturing few dozens scrolling movement of various iOS applications, it came to me that all the scrolling will stop after the same amount of time, regardless the size of the list or the speed of the flick. How fast you flick (which determines the initial velocity of the scrolling) only determines where the list would stop and not when.

That accompanying example code certainly be useful (handles cases I didn't think about above - like snapping to a list item).

• The wheel events already have some momentum, at least in my Mac. Should we just reuse those values? Or is the idea to reimplement it?

Interesting, was curious about this. I think if the platform supports it, we should use the platform implementation. But Windows definitely doesn't add momentum by default. And I don't think the platform would support the 'bounce' at the edges, since it wouldn't know the extents of our 'virtual' scrollview - so we'd have to reimplement that piece, at the very least.

An option would be to use a conditional flag that we pass per-platform whether we should 'add' momentum, or just use the values as-is.

• I noticed that the pixel values for wheel events in the browser provide values way larger than the desktop build. Not sure if this is something related to jsoo, the browser, or st else. The different is almost 10x as fast (20-25 pixels max per event in desktop, up to 300 px in browser)

Ouch, yes, that's quite a difference! It's possible there is some differences in scaling going on - we may also need some per-platform normalization. Taking a look at a few different references:

• MDN documentation on wheel
• Win32 GLFW scroll implementation: https://github.com/glfw/glfw/blob/7f4dde15e453093c7a12efa0b5d79514a58b5243/src/win32_window.c
• Cocoa GLFW scroll implementation: https://github.com/glfw/glfw/blob/530b37f4db964466f327cb567499da234e3bce1e/src/cocoa_window.m
• X11 GLFW scroll implementation: https://github.com/glfw/glfw/blob/78e6a0063d27ed44c2c4805606309744f6fb29fc/src/x11_window.c

I realized our current implementation of the 'scroll' JS stub doesn't take into account WheelEvent.deltaMode (the delta values can be pixel/line/page values). But if anything (like values are pages instead of pixels) - this would make it slower, not faster.

I'll double-check and see how the values reported on the JS / native side relate on Windows. I'll try using the same mouse on my Mac Mini also, and see how the values compare for each 'notch'.

[jchavarri]

The only concern I had is I couldn't find license info for Popmotion Pure - do you happen to know where we can find that?

It seems to be MIT (packages/popmotion/ is where "Pure" is located): https://github.com/Popmotion/popmotion/blob/7b399836fb1bd58bd4362095f94dfb023ffae5d6/packages/popmotion/LICENSE.md

An option would be to use a conditional flag that we pass per-platform whether we should 'add' momentum, or just use the values as-is.

Unfortunately I don't have any Windows machines at hand, so maybe I could start working on the "bounce" / "rubber band scrolling" first, as that part is clearly needed for both platforms?

I realized our current implementation of the 'scroll' JS stub doesn't take into account WheelEvent.deltaMode

I checked deltaMode in my Mac, with Firefox, and it's 0 (which seems to be value of pixels). 🤔

[jchavarri]

I just found this golden piece of data:

I'm not sure. On Mac we can do it because "wheel" events that were generated while the finger was touching the touchpad have a flag that lets us differentiate them from "wheel" events that are generated automatically for the "momentum" phase after the finger has left the touchpad, and because we receive special marker events when the fingers start and stop touching the touchpad.

I don't know if there is a way to get these pieces of information from wheel events on Windows. At some level in the system this surely must be known, because I've seen Edge do rubberband overscrolling successfully. But I don't know if we have access to this information in Firefox.

[bryphe]

Unfortunately I don't have any Windows machines at hand, so maybe I could start working on the "bounce" / "rubber band scrolling" first, as that part is clearly needed for both platforms?

Sounds good to me!

Regarding normalization:

I just ran the tests on bryphe/testing/scroll-wheel-delta Windows / OSX and on browsers on each platform, using a mouse scrollwheel, and these are the results I found:

Platform	Native	Chrome	IE	Safari
Windows	-1	-100 (wheelDelta: -120)	-160 (wheelDelta: -120)	n/a
OSX	-0.1	-4 (wheelDelta: -120)	n/a	-4 (wheelDelta: -12)

For our wheel event handler, we're currently using the deltaY and deltaX values. However, I also examined the wheelDelta value - which seems to be more consistent across browsers + platforms, except for Safari. It seems like we could use 120 as the 'base value' and then multiply it by some pixel value, for the browsers (we'd need to also account for Safari - multiply by 10 in that case).

I also stumbled across this post too which had some ideas around normalizing scrollwheel values in the browser: https://stackoverflow.com/questions/5527601/normalizing-mousewheel-speed-across-browsers.

That code normalizes to a pixel-level scroll, which I think would be reasonable for us to adopt as well. We could have normalize the scroll values to -1, 1 with the following:

• Native Windows: use as-is
• Native OSX: multiply by 10.
• Browsers: Use wheelDelta / 120. (add a multiplier of 10 for Safari)

Then, in Revery, we could multiply the -1, 1 by some pixel amount - I guess we already do this in our ScrollView here:

revery/src/UI_Components/ScrollView.re

Line 104 in 5600aee

actualScrollTop - int_of_float(wheelEvent.deltaY *. 25.);

On Mac we can do it because "wheel" events that were generated while the finger was touching the touchpad have a flag that lets us differentiate them from "wheel" events that are generated automatically for the "momentum" phase after the finger has left the touchpad, and because we receive special marker events when the fingers start and stop touching the touchpad.

Nice find! It seems like the NSEvent has a momentumPhase property that we can use to see if the scroll is due to momentum. Seems like GLFW doesn't expose this, though: https://github.com/glfw/glfw/blob/530b37f4db964466f327cb567499da234e3bce1e/src/cocoa_window.m

If it would be helpful for us, we could track an issue on GLFW or create a fork to add this as part of the context it sends about scroll/wheel events.

And we might need to look at Direct Manipulation or the direct touch APIs as we start implementing touch/gesture support. I also am not sure if it would build in momentum into the calculations, although it seems like it does add some inertia for certain transitions (we'd also have to see if it would work in the context of GLFW/OpenGL - it might need to manage the rendering for it to work). We'd also have the option for Windows of just using the direct pointer APIs, without Direct Manipulation, w/o momentum.

[jchavarri]

Woah amazing research!! 😮 Seems there's a lot of room to normalize scrolling across platforms.

That code normalizes to a pixel-level scroll, which I think would be reasonable for us to adopt as well. We could have normalize the scroll values to -1, 1 with the following

It seems that in the first answer edited solution they are giving up on normalizing altogether, no? I'm referring to:

var handleScroll = function(evt){
  if (!evt) evt = event;
  var direction = (evt.detail<0 || evt.wheelDelta>0) ? 1 : -1;
  // Use the value as you will
};
someEl.addEventListener('DOMMouseScroll',handleScroll,false); // for Firefox
someEl.addEventListener('mousewheel',    handleScroll,false); // for everyone else

It seems Firefox doesn't even have wheelDelta in the latest versions... I wonder if ignoring the values of wheelDelta and using direction exclusively would have a large impact on how the scrolling is perceived from users.

There's also one answer in that same question that is pointing to this code from Facebook's fixed-data-table which looks really well documented, and covering a lot of edge cases. They don't seem to cover Safari's edge case though? 🤔

Then, in Revery, we could multiply the -1, 1 by some pixel amount

The normalization would happen upstream in reason-glfw, right? So we can do all the browser-dependent checks for example in the js stub?

It seems like the NSEvent has a momentumPhase property that we can use to see if the scroll is due to momentum. Seems like GLFW doesn't expose this, though. If it would be helpful for us, we could track an issue on GLFW or create a fork to add this as part of the context it sends about scroll/wheel events.

Nice! I'm not sure if we'll need this yet. The only reason I can think of is because we want to use our own "inertial stream" of events instead of the native one, but it's probably better to leverage what the native platform provides.

[bryphe]

Woah amazing research!! 😮 Seems there's a lot of room to normalize scrolling across platforms.

Thanks, right back at you @jchavarri ! 😄 Definitely, I think we can get reasonably consistent behavior across platforms.

There's also one answer in that same question that is pointing to this code from Facebook's fixed-data-table which looks really well documented, and covering a lot of edge cases. They don't seem to cover Safari's edge case though? 🤔

Ah yes... That facebook code seemed to me like the best answer in that post. And that is interesting... they mention Safari's edge case in the comments (and it seems like older versions of Safari actually exhibit similiar behavior as Chrome/IE!) but indeed, it doesn't seem to be accounted for in the code - perhaps that's reason enough for us to leave it out, too.

The normalization would happen upstream in reason-glfw, right? So we can do all the browser-dependent checks for example in the js stub?

Exactly what I was thinking... If we skip out on Safari - then it seems like it would just be a matter of switching to wheelDelta and dividing by 120 in the reason-glfw stub. We could also incorporate some more of the edge cases from that fixed-data-table snippet. But this fix would make reason-glfw more consistent across browser platforms, and then revery would benefit from that.

One thing I wasn't sure of, though, is that it seems native OSX might need a 'normalization factor' of 10, at least based on the table above - I saw scroll values of 0.1/-0.1 on OSX, but the same gesture was 1/-1 on Windows (and I was thinking on the reason-glfw side - it would be helpful to normalize them to 1/-1 range, and then on revery, we could apply a pixel multiplier - the 25px in ScrollView to that).

It might be that this is specific to mouse scrollwheel and trackpad gestures are reported differently, though - so perhaps adding that multiplier might make trackpad gestures too fast. 🤔