Wrong co-ordinates captured from PyVistaQt window in QStackedWidget

[colosteve]

I would like to use a simple point picker by right clicking on a mesh. The example here in issue 239 works well: pyvista/pyvista-support#239

However, when this code is incorporated into PyQt5 where the plotter has been embedded as a QStackedWidget, the captured mouse co-ordinates are wrong. I have added a line to the Issue 239 example above to plot the point of the mouse co-ordinates.

Using the existing function: enable_point_picking, also has this effect of capturing the mouse co-ordinates incorrectly.

I would expect the enable_point_picking function or issue 239 example to work when the plotter is embedded as a QStackedWidget in Pyqt5.

---

To Reproduce

Please include a code snippet to reproduce the bug in the block below:

# Insert code here
import sys
import pyvista as pv
from pyvista import examples
import pyvistaqt as pvqt
import numpy as np
from PyQt5 import QtCore, QtGui, QtWidgets     # pip3 install pyqt5
from PyQt5.QtCore import * #QFile, QFileInfo, Qt, QSortFilterProxyModel, QAbstractTableModel
from PyQt5.QtGui import * # QPen, QStandardItem, QStandardItemModel, QPushButton
from PyQt5.QtWidgets import * #QStyledItemDelegate, QApplication, QHeaderView, QTableView, QWidget, QHBoxLayout, QVBoxLayout, QAbstractItemView, QListWidget, QPushButton, QMenu, QMessageBox,QDialog


class Picker:
    def __init__(self, plotter, mesh):
        self.plotter = plotter
        self.mesh = mesh
        self._points = []
        
    @property
    def points(self):
        """To access all th points when done."""
        return self._points
        
    def __call__(self, *args):
        picked_pt = np.array(self.plotter.pick_mouse_position())

        # Line below added to show where mouse position is...
        w.plotter.add_mesh(pv.Sphere(radius=3, center=picked_pt), color='pink')

        direction = picked_pt - self.plotter.camera_position[0]
        direction = direction / np.linalg.norm(direction)
        start = picked_pt - 1000 * direction
        end = picked_pt + 10000 * direction
        point, ix = self.mesh.ray_trace(start, end, first_point=True)
        if len(point) > 0:
            self._points.append(point)
            q = w.plotter.add_mesh(pv.Sphere(radius=3, center=point),
                           color='red')
        return


class OpNote(QDialog):
    def __init__(self):
        super().__init__()  
        mesh = examples.load_airplane()
        self.tabs = QStackedWidget(self)
        self.tabs.setObjectName("tabs")
        self.tabs.setGeometry(QtCore.QRect(370, 210, 461, 441))

        self.tabs.setCursor(QCursor(QtCore.Qt.SizeAllCursor))

        self.plotter = pvqt.QtInteractor()
        self.plotter.add_mesh(mesh, show_edges=True, color='w')
        
        self.tabs.addWidget(self.plotter.interactor)
            
        picker = Picker(self.plotter, mesh)
        self.plotter.track_click_position(picker, side='right')
        self.plotter.add_text('Use right mouse click to pick points')


app=QtWidgets.QApplication(sys.argv)
w = OpNote()
w.show()
app.exec_()

Screenshots
If applicable, add screenshots to help explain your problem.

---

System Information:
Please run the following code wherever you are experiencing the bug and paste the output below. This report helps us track down bugs and it is critical to addressing your bug:

# Get system info
import pyvista as pv
print(pv.Report())

# Paste system info here
--------------------------------------------------------------------------------
  Date: Wed Jul 28 10:17:53 2021 BST

                OS : Darwin
            CPU(s) : 8
           Machine : x86_64
      Architecture : 64bit
       Environment : Python
        GPU Vendor : Apple
      GPU Renderer : Apple M1
       GPU Version : 4.1 Metal - 71.0.7

  Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:10:52)  [Clang 6.0
  (clang-600.0.57)]

           pyvista : 0.31.3
               vtk : 9.0.20210612
             numpy : 1.19.5
           imageio : 2.9.0
           appdirs : 1.4.4
            scooby : 0.5.7
            meshio : 4.4.6
        matplotlib : 3.3.3
         pyvistaqt : 0.5.0
             PyQt5 : 5.15.4
             scipy : 1.7.0
              tqdm : 4.61.1
--------------------------------------------------------------------------------

[akaszynski]

when this code is incorporated into PyQt5 where the plotter has been embedded as a QStackedWidget, the captured mouse co-ordinates are wrong

Strange indeed. I'm not sure what magic is going inside Qt, but my guess is it's passing events from the widget to VTK and the coordinates aren't being transferred properly. If you can figure out how they're being miss-translated, we could add a hook into pyvistaqt that could translate coordinates properly.

[2umbrella]

I have the same problem. Do you have a solution for this problem?
The problem is that the coordinates obtained by pick_click_position of pyvista are not accurate.

[joley-gh]

I am encountering the same issue for a DockedView.
I defined my own class, which inherits the QtInteractor class. It seems that the central widget needs to be the QtInteractor for the coordinates to work correctly(?).

I tried to remove the integration as a DockedWidget and simply called self.show() to display the plot in a new Window.
Here, the coordinates are still wrong. When closing the window, the same window reappears with a new title. Now, the interaction is much smoother and the picking works correctly.

Here is some code:

from pyvistaqt import QtInteractor
from pyvistaqt import BackgroundPlotter
import pyvista as pv
from PyQt6.QtGui import *
from PyQt6.QtWidgets import *
from PyQt6.QtCore import *
import numpy as np
import threading
from pyvista import examples


class Picker:
    def __init__(self, plotter, mesh):
        self.plotter = plotter
        self.mesh = mesh
        self._points = []

    @property
    def points(self):
        return self._points

    def __call__(self, *args):
        picked_pt = np.array(self.plotter.pick_click_position())
        direction = picked_pt - self.plotter.camera_position[0]
        direction = direction / np.linalg.norm(direction)
        start = picked_pt - 1000 * direction
        end = picked_pt + 1000 * direction
        self._points.append(picked_pt)
        w = self.plotter.add_mesh(pv.Sphere(radius=300, center=picked_pt), color="red")


class CustomInteractor(QtInteractor):
    def __init__(self, BasePlotter, root):
        
        self.actor = None
        self.mesh = None
        self.current_camera_pos = None
        self.scaler = 1.0

        super().__init__()

    def remesh(
        self, mesh=None):
       
        if mesh is not None:
            if self.current_camera_pos is not None:
                self.current_camera_pos = self.camera_position

            self.clear_actors()
           
            self.actor = self.add_mesh(
                mesh,
                smooth_shading=False,
                render_lines_as_tubes=True,
                show_scalar_bar=False,
                # pickable=True,
            )

            picker = Picker(self, self.mesh)
            self.track_click_position(picker, side="right")

        else:
            return

    def plot_preview(self):

        mesh = examples.load_airplane()

        # open new windows with the plot
        self.iren.reset_picker()
        if self.iren is not None:
            self.show()

It would be awesome to get to the second plot (after closing the first one) or simply wrapping the normal pyvista.plotter in a qtwindow, that can be closed without crashing the whole application.

When walking along the stack call I came to the same conclusion as @2umbrella, that pick_click_position() returns a wrong value. This happens in the vtk backend so there is not much we can really do. The coordinates of click_position are correct.

It would be awesome if someone with more insight could have a look at this problem.Being able to dock the plotting view and beaing able to use picking would be awesome for interacting in a complex ui.