Improve Event Delegation in qMRMLThreeDView and qMRMLSliceView

[jcfr]

Is your feature request related to a problem? Please describe.

The current event delegation mechanism in Slicer is based on Slicer-specific interactor styles, namely vtkMRMLThreeDViewInteractorStyle and vtkMRMLSliceViewInteractorStyle, which are derived from vtkMRMLViewInteractorStyle and, ultimately, vtkInteractorStyle3D.

These styles handle interaction events by adding observers for events like EnterEvent, LeaveEvent, and MouseMoveEvent, and subsequently call functions like vtkMRMLViewInteractorStyle::OnEnter(), vtkMRMLViewInteractorStyle::OnLeave(), or vtkMRMLViewInteractorStyle::OnMouseMove().

In each of these On<EventName>() functions, delegation to displayable managers and their associated widgets is performed, and if a given event isn't intercepted, the default behavior for <EventName> implemented in vtkInteractorStyle3D is executed by calling this->Superclass::On<EventName>(). The snippet below illustrates this:

if (!this->DelegateInteractionEventToDisplayableManagers(vtkCommand::<EventName>Event))
  {
  return;
  }
this->Superclass::On<EventName>();

However, a limitation arises when default event handling implemented in specific interactor styles (e.g., vtkOpenXRInteractorStyle or vtkOpenVRInteractorStyle) needs to be executed when not delegated.

Describe the solution you'd like

Refactor Interactor Styles

Refactor vtkMRMLThreeDViewInteractorStyle and vtkMRMLSliceViewInteractorStyle to derive from vtkObject and implement event delegation to MRML displayable managers and associated widgets by observing the interactor style. Depending on the observed event, default interactor style behavior would be executed using one of the following approaches:

1. Setting Abort Flag: For events such as Move3D, Button3D, Menu3D, and others where the vtkInteractorStyle class checks if the event was aborted, we will set an abort flag.
2. Explicit Call of On<EventName>(): For other events, we would explicitly call the corresponding On<EventName>() function. To support this, the existing method vtkMRMLAbstractThreeDViewDisplayableManager::PassThroughInteractorStyleEvent will be used and updated as needed.

The interactor style will be associated with its corresponding displayable manager group, allowing access to the MRML scene and registered displayable managers.

Renaming Interactor Styles

To faciliate review of changes introduced in the first phase of the refactoring, the rename the classes will happen in in a second phase. This will help avoid future confusion.

More specifically, we will ename the classes vtkMRMLThreeDViewInteractorStyle to vtkMRMLThreeDViewInteractorObserver and vtkMRMLSliceViewInteractorStyle to vtkMRMLSliceViewInteractorObserver.

Access to CameraNode from vtkMRMLThreeDViewInteractorStyle

Access to the CameraNode will be updated in the following places:

1. In qMRMLThreeDView, the code can be refactored to retrieve the camera node from the vtkMRMLCameraDisplayableManager by calling qMRMLThreeDView::displayableManagerByClassName().
2. In vtkMRMLThreeDViewInteractorStyle, the camera node will be accessed through the displayable manager group following the refactoring described above.

Default Interactor Styles

• For 3D views, the default interactor style will be vtkInteractorStyleTrackballCamera.
• For 2D views, the default interactor style will be vtkInteractorStyleUser.

Whenever possible, Slicer-specific behavior will be added either to vtkMRMLThreeDViewInteractorStyle (renamed vtkMRMLThreeDViewInteractorObserver in Phase 2) or to displayable managers to ensure that switching the interactor style does not affect critical functionality.

Describe alternatives you've considered

An alternative would be duplicating the code currently implemented in VTK-specific interactor styles. However, this approach is error-prone and unsustainable.

Additional context

This proposal was created following discussion with @LucasGandel while discussing the following issue:

• Consolidate InteractorStyle integration KitwareMedical/SlicerVirtualReality#123

[lassoan]

We got rid of interactor styles in Slicer (just kept a sham class that just forwards events to widgets) because they did not allow dynamic mapping of GUI events to actions; and the hardcoded event mappings often conflicted with event mappings in VTK widgets.

VTK VR developers realized the rigidity of interactor styles, too, and implemented a very basic event translation mechanism in vtkVRInteractorStyle::MapInputToAction. Slicer developers chose not to add a new event translation mechanism at the interactor style level, but switch to consistently do all event translation and processing in widgets. There is no need to think about if a feature should be implemented in a widget or in the interactor style: each group of related feature goes into a widget (with or without a visible representation). We would never need to put completely unrelated features, such as camera positioning, object positioning/transforming, data probing, clipping, teleportation, etc. into the interactor style, as it is done in the current VR interactor.

We also improved the widget event translation mechanism to allow specifying widget state for an event translation rule. For example, DEL key press event can be mapped to delete point when the current widget state is "point selected"; and it can be mapped to delete the entire markup if the state is "whole markup is selected").

We also added interaction context to event handling: this allows simultaneous manipulation of widgets (multiple widgets, or different components of the same widget) by different people, using different devices, all working on the same scene displayed in VR view(s) and in 2D and 3D views. Desktop mouse&keyboard is one context; each VR controller is a separate context. For example, the desktop user can select and move a markup control point with keyboard and mouse, while the VR user can drag one or two other control points of the same markup point list using the VR controllers.

Returning to the old event handling based on monolithic interaction styles classes would be a huge step backward in Slicer. I think the simplest would be to keep Slicer and VTK interaction handling separate (maybe factor out some common logic in helper classes). We just need to avoid past mistakes when VTK core developers kept refactoring virtual reality implementation in VTK in a way that it breaks Slicer. The problem may get better by itself, because as the implementation matures there should be less need for breaking changes. We could also facilitate keeping important APIs in VTK functional by adding VTK tests for features that we rely on in Slicer.

I'm really sorry for not being able to propose any simple way of harmonizing implementations for this in VTK and Slicer, but 3D widgets is one of the very few parts of VTK that I find unusable for anything more than toy examples (single-view manipulation of a few selected widget types). It is not just me. You can find that there are about 3 generations of 3D widgets in VTK, all with serious limitations, so it is quite obvious that VTK developers struggled with this. There were 4-5 attempts (and $500k+ spent) trying to use VTK widgets in Slicer without major redesign and all those attempts have failed. When we permitted ourselves to break away from the VTK design then suddenly we could implement features that seemed impossible before.