Issue with Vuetify slider interactor in Trame project deployed on AWS #260

ferracioli · 2023-05-23T18:40:36Z

ferracioli
May 23, 2023

Hi everyone,

I'm currently working on a project that involves utilizing Trame on an AWS server. The deployment was done using the Trame Cookiecutter. The project incorporates a display section that showcases 3D medical image volumes for evaluation by specialists. To navigate through the image slices, we've implemented a Vuetify slider interactor.

Initially, everything was functioning properly. However, we encountered an unexpected issue where the slider interactor ceased to work as intended. Currently, it only changes the slices when we click on the VTK's window, without transitioning through the slices. We suspect that this problem may be related to the VTK version.

Our team has a Docker instance that was previously built, and in that instance, the transitions work fine. However, the new Docker instances no longer exhibit the expected behavior. We've noticed differences in the library versions between the new instances and the working one. Below, you'll find a comparison of the library versions:

Old version that works:

New version:

Based on this issue, I have two specific questions:

How can we revert to the previous version of the libraries that were working correctly? We haven't made any changes to the Dockerfile, yet the library versions seem to be changing. Is there a way to lock the versions we are currently using to prevent this problem in the future?
Do any of you have any insights or ideas about what could have caused this problem? It's worth noting that the VTK functionality only seems to work after interacting with the VTK window.

Bellow, I will provide a simplified version of our code.

Thank you in advance for your assistance.

Answered by jourdain

May 23, 2023

Thanks for providing some code to test and track such issue.

One way to fix and figure out which of the dependency introduce that behavior is to fix some of those dependencies.

I would suggest to start by fixing trame-vtk==2.3.5 into that file of your repo and rebuild the server directory (you may have to delete that directory before rebuilding the full image)

Please keep us in the loop on what you find.

Side note, are you sure you are not missing the () in ctrl.volume_update as that would do nothing if you don't execute the function.

View full answer

ferracioli · 2023-05-23T18:41:14Z

ferracioli
May 23, 2023
Author

Simplified source code:

main.py

from trame.app import get_server, dev
from . import ui


def _reload():
    server = get_server()
    dev.reload(ui)
    ui.initialize(server)


def main(server=None, **kwargs):
    # Get or create server
    if server is None:
        server = get_server()

    if isinstance(server, str):
        server = get_server(server)

    # Make UI auto reload
    server.controller.on_server_reload.add(_reload)

    # Init application
    # engine.initialize(server)
    ui.initialize(server)

    # Start server
    server.start(**kwargs)


if __name__ == "__main__":
    main()

ui.py

from trame.ui.vuetify import SinglePageWithDrawerLayout
from trame.ui.router import RouterViewLayout
from trame.widgets import vuetify, router, vtk, trame

# Required for interactor initialization
from vtkmodules.vtkInteractionStyle import vtkInteractorStyleSwitch  # noqa
import vtkmodules.vtkRenderingOpenGL2  # noqa

from . import volumes


def initialize(server):
    state, ctrl = server.state, server.controller
    state.trame__title = "Dashboard"

    # -----------------------------------------------------------------------------
    # Callbacks
    # -----------------------------------------------------------------------------

    vol_object = volumes.SingletonVolumes()

    # Precisa configurar o TextRenderer e o renderer
    vol_object.create_objects()

    window = vol_object.create_window()

    state.id_display = None
    state.id_special = 'id_00000000'
    state.cleaned = False
        

    @state.change("volume", "image_slice", "id_special")
    def update_volume(volume, image_slice, **kwargs):

        slice_n = image_slice / 100
        
            
        if volume and state.id_special and (state.id_display != state.id_special):

            vol_object.load_volumes(state.id_special)
            state.id_display = state.id_special
            vol_object.generate_image()
            ctrl.volume_update

        elif volume and state.id_special and (state.id_display == state.id_special):

            if state.cleaned:
                vol_object.generate_image()
                state.cleaned = False
            else:
                vol_object.updates_slice(slice_n=slice_n)
            ctrl.volume_update

        else:
            vol_object.clean_image()
            state.cleaned = True  
        

    # # Life cycle update
    ctrl.on_server_ready.add(ctrl.volume_update)

    # -----------------------------------------------------------------------------
    # GUI
    # -----------------------------------------------------------------------------
    
    row_style = {
    "classes": "d-flex flex-shrink-1",
    "style": "min-width: 0;",
    }


    # Foo route
    with RouterViewLayout(server, "/"):
        with vuetify.VRow(**row_style):
            with vuetify.VCard(classes="ma-4", style="width: 90vw; height: calc(100% - 120px);", align="center"):
                with vuetify.VCardTitle("Display"):
                    vuetify.VSpacer()
                    vuetify.VCheckbox(
                        v_model=("volume", False),
                        on_icon="mdi-eye",
                        off_icon="mdi-eye-off",
                        classes="mx-1",
                        hide_details=True,
                        dense=True,
                        hint="Button used to display the selected volume"
                    )
                with vuetify.VContainer(fluid=True):
                    with vuetify.VRow(**row_style):
                        vuetify.VSlider(
                            v_model=("image_slice", 0),
                            min=0, max=100,
                            dense=True,
                            hide_details=True,
                            style="width: 550px;",
                            label="Slices: ",
                        )
                with vuetify.VContainer(fluid=True):
                    with vuetify.VRow(**row_style):
                        with vuetify.VContainer(fluid=True, style="height: 70vh;"):
                            vuetify.VSpacer()
                            view = vtk.VtkRemoteView(window)
                            ctrl.on_server_ready.add(view.update)
                            ctrl.volume_update = view.update
                        vuetify.VSpacer()


    # Main page content
    with SinglePageWithDrawerLayout(server) as layout:
        layout.title.set_text("Dashboard")
        
        trame.ClientStateChange(
            value="graph_theme",
            immediate=True,
            trigger_on_create=True,
            change="$vuetify.theme.dark = graph_theme",
        )

        with layout.toolbar:
            vuetify.VSpacer()
            vuetify.VCheckbox(
                v_model=("graph_theme", True),
                change="$vuetify.theme.dark = $event",
                on_icon="mdi-lightbulb-off-outline",
                off_icon="mdi-lightbulb-outline",
                classes="mx-1",
                hide_details=True,
                dense=True,
            )

        with layout.content:
            with vuetify.VContainer():
                router.RouterView()

        # add router buttons to the drawer
        with layout.drawer:
            with vuetify.VList(shaped=True, v_model=("selectedRoute", 0)):
                vuetify.VSubheader("Options")
                        
                with vuetify.VListItem(to="/"):
                    with vuetify.VListItemIcon():
                        vuetify.VIcon("mdi-eye")
                    with vuetify.VListItemContent():
                        vuetify.VListItemTitle("Display")

volumes.py

import vtkmodules.vtkInteractionStyle  # noqa
import vtkmodules.vtkRenderingOpenGL2  # noqa
from vtkmodules.vtkInteractionStyle import vtkInteractorStyleSwitch, vtkInteractorStyleImage  # noqa
from vtkmodules.vtkCommonColor import vtkNamedColors
from vtkmodules.vtkCommonCore import vtkLookupTable
from vtkmodules.vtkCommonMath import vtkMatrix4x4
from vtkmodules.vtkIOImage import vtkNIFTIImageReader
from vtkmodules.vtkImagingCore import vtkImageReslice, vtkImageMapToColors
from vtkmodules.vtkRenderingCore import (
    vtkRenderWindow,
    vtkRenderWindowInteractor,
    vtkRenderer,
    vtkImageActor,
    vtkImageSliceMapper,
)


class SingletonMeta(type):
    """
    https://refactoring.guru/pt-br/design-patterns/singleton/python/example
    The Singleton class can be implemented in different ways in Python. Some
    possible methods include: base class, decorator, metaclass. We will use the
    metaclass because it is best suited for this purpose.
    """

    _instances = {}

    def __call__(cls, *args, **kwargs):
        """
        Possible changes to the value of the `__init__` argument do not affect
        the returned instance.
        """
        if cls not in cls._instances:
            instance = super().__call__(*args, **kwargs)
            cls._instances[cls] = instance
        return cls._instances[cls]
    

def reads_volume(path, reader):
    reader.SetFileName(path)
    reader.Update()

    return reader


def get_plane(reader):
    
    slice_n = 0.0

    (xMin, xMax, yMin, yMax, zMin, zMax) = reader.GetExecutive().GetWholeExtent(reader.GetOutputInformation(0))
    (xSpacing, ySpacing, zSpacing) = reader.GetOutput().GetSpacing()
    (x0, y0, z0) = reader.GetOutput().GetOrigin()

    center = [x0 + xSpacing * slice_n * (xMin + xMax),
              y0 + ySpacing * slice_n * (yMin + yMax),
              z0 + zSpacing * slice_n * (zMin + zMax)]

    # Matrices for axial, coronal, sagittal, oblique view orientations
    axial = vtkMatrix4x4()
    axial.DeepCopy((1, 0, 0, center[0],
                    0, -1, 0, center[1],
                    0, 0, 1, center[2],
                    0, 0, 0, 1))

    return axial


class SingletonVolumes(metaclass=SingletonMeta):

    def __init__(self):
        self.master_reader = None
        self.reader_02 = None
        self.reader_03 = None
        self.ground_brain_reader = None
        self.deep_brain_reader = None
        self.ground_ventr_reader = None
        self.deep_ventr_reader = None
        self.window = None
        self.master_renderer = None
        self.ground_renderer = None
        self.deep_renderer = None
        self.iren = None
        self.iren_style = None
        self.master_slicer = None
        self.ground_slicer = None
        self.deep_slicer = None
        self.master_actor = None
        self.ground_actor = None
        self.deep_actor = None
        self.patient_id = None
        self.master_plane = None
        self.actions = None
        self.zSpacing = None
        self.zMax = None
        self.delta_y = 0
        self.master_angle = 30
        self.s3 = None
        self.bucket_name = None
        self.dataset = None
        self.temp_folder = None
        self.remote_list = None
        self.round_number = None
        self.greyscale_table = None
        self.labels_table = None
        self.master_colors = None
        self.label02_colors = None
        self.label03_colors = None
        self.image_mapper = None


    # Changes the slice when clicked with the right button
    def RightButtonCallback(self, obj, event):
    
        if event == "RightButtonPressEvent":
            self.actions["Slicing"] = 1
        else:
            self.actions["Slicing"] = 0
            
    # While holding left or right button, implements the changes in the rendering
    def MouseMoveCallback(self, obj, event):

        # Gets the vertical movement
        lastY = obj.GetLastEventPosition()[1]
        mouseY = obj.GetEventPosition()[1]

        if self.actions["Slicing"] == 1:
            delta_y = (mouseY - lastY)/4
            (x, y, z) = self.master_slicer.GetResliceAxesOrigin()

            # Limits the slice range
            # Note: this only works for the axial perspective, adding new perspectives will
            # require a reconstruction of this part until the render
            if z + delta_y > self.zMax*self.zSpacing:
                delta_y = self.zMax*self.zSpacing - z
            elif z + delta_y < 0:
                delta_y = 0

            print(delta_y)
            # Implement the slice change
            self.master_slicer.SetResliceAxesOrigin(x, y, z+delta_y)
            self.ground_slicer.SetResliceAxesOrigin(x, y, z+delta_y)
            self.deep_slicer.SetResliceAxesOrigin(x, y, z+delta_y)

            self.window.Render()


        
    def create_label_lookup_table(self):
        
        self.greyscale_table.SetRange(-50, 250)  # image intensity range
        self.greyscale_table.SetValueRange(0.0, 1.0)  # from black to white
        self.greyscale_table.SetSaturationRange(0.0, 0.0)  # no color saturation
        self.greyscale_table.SetRampToLinear()
        self.greyscale_table.Build()

        self.master_colors.SetLookupTable(self.greyscale_table)
        

        self.labels_table.SetNumberOfColors(3)
        self.labels_table.SetTableRange(0, 2)
        self.labels_table.Build()
        self.labels_table.SetTableValue(0, self.colors.GetColor4d('Black'))
        self.labels_table.SetTableValue(1, self.colors.GetColor4d('green'))
        self.labels_table.SetTableValue(2, self.colors.GetColor4d('orange'))
        

        self.label02_colors.SetLookupTable(self.labels_table)
        self.label03_colors.SetLookupTable(self.labels_table)

        
    def load_volumes(self, patient_id):
        
        self.patient_id = patient_id
        
        master_image_local = "/work/MRHead.nii.gz"
        model_brain_local = "/work/MRHead-label.nii.gz"
        model_ventr_local = "/work/MRHead-label.nii.gz"
        truth_brain_local = "/work/MRHead-label.nii.gz"
        truth_ventr_local = "/work/MRHead-label.nii.gz"
        
        try:
            self.master_reader = reads_volume(master_image_local, self.master_reader)
        except:
            self.master_reader = None

        try:
            self.deep_brain_reader = reads_volume(model_brain_local, self.deep_brain_reader)
        except:
            self.deep_brain_reader = None

        try:
            self.deep_ventr_reader = reads_volume(model_ventr_local, self.deep_ventr_reader)
        except:
            self.deep_ventr_reader = None
        
        try:
            self.ground_brain_reader = reads_volume(truth_brain_local, self.ground_brain_reader)
        except:
            self.ground_brain_reader = None
        
        try:
            self.ground_ventr_reader = reads_volume(truth_ventr_local, self.ground_ventr_reader)
        except:
            self.ground_ventr_reader = None
        

    def select_region(self, region):

        if "Specialist" in region:
            self.reader_02 = self.ground_brain_reader
            self.reader_03 = self.ground_ventr_reader
        elif "Brain" in region:
            self.reader_02 = self.ground_brain_reader
            self.reader_03 = self.deep_brain_reader
        else:
            self.reader_02 = self.ground_ventr_reader
            self.reader_03 = self.deep_ventr_reader
        
        self.master_plane = get_plane(self.master_reader)
        
    def create_objects(self):
        
        self.master_reader = vtkNIFTIImageReader()
        self.ground_brain_reader = vtkNIFTIImageReader()
        self.deep_brain_reader = vtkNIFTIImageReader()
        self.ground_ventr_reader = vtkNIFTIImageReader()
        self.deep_ventr_reader = vtkNIFTIImageReader()
        
        self.master_renderer = vtkRenderer()
        self.ground_renderer = vtkRenderer()
        self.deep_renderer = vtkRenderer()   

        self.window = vtkRenderWindow()
        
        self.iren_style = vtkInteractorStyleImage()

        self.iren = vtkRenderWindowInteractor()

        self.master_slicer = vtkImageReslice()
        self.ground_slicer = vtkImageReslice()
        self.deep_slicer = vtkImageReslice()

        self.master_actor = vtkImageActor()
        self.ground_actor = vtkImageActor()
        self.deep_actor = vtkImageActor()
        
        self.image_mapper = vtkImageSliceMapper()
        
        self.greyscale_table = vtkLookupTable()
        self.labels_table = vtkLookupTable()
        
        self.colors = vtkNamedColors()
        
        self.master_colors = vtkImageMapToColors()
        self.label02_colors = vtkImageMapToColors()
        self.label03_colors = vtkImageMapToColors()
        
        self.create_label_lookup_table()

    def create_window(self):

        self.window.AddRenderer(self.master_renderer)
        self.window.AddRenderer(self.ground_renderer)
        self.window.AddRenderer(self.deep_renderer)
        
        self.ground_renderer.SetActiveCamera(self.master_renderer.GetActiveCamera())
        self.deep_renderer.SetActiveCamera(self.master_renderer.GetActiveCamera())
        
        self.iren_style.SetInteractionModeToImageSlicing()
        
        self.iren.SetInteractorStyle(self.iren_style)
        
        self.window.SetInteractor(self.iren)
        
        self.window.ShowWindowOff()
        self.iren.Initialize()

        return self.window

    def clean_image(self):

        self.master_renderer.RemoveAllViewProps()
        self.ground_renderer.RemoveAllViewProps()
        self.deep_renderer.RemoveAllViewProps()
        
        self.master_renderer.ResetCamera()
        
        self.window.Render()
        

    # Updates the slices without generating the complete scene again
    def updates_slice(self, slice_n):

        (x, y, z) = self.master_slicer.GetResliceAxesOrigin()

        # Updates the slice with setResliceAxesOrigin
        self.master_slicer.SetResliceAxesOrigin(x, y, slice_n*self.zMax*self.zSpacing)
        self.ground_slicer.SetResliceAxesOrigin(x, y, slice_n*self.zMax*self.zSpacing)
        self.deep_slicer.SetResliceAxesOrigin(x, y, slice_n*self.zMax*self.zSpacing)
        
        self.master_renderer.ResetCamera()

        self.window.Render()
        
        self.iren.ProcessEvents()
        


    def generate_image(self, region="Brain"):

        print('Entrou no generate_image')
        self.zMax = self.master_reader.GetDataExtent()[5]
        self.zSpacing = self.master_reader.GetOutput().GetSpacing()[2] 
        
        opacity = 1.0
        
        self.actions = dict()
        self.actions["Slicing"] = 0

        self.select_region(region)
        

        self.master_slicer.SetInputConnection(self.master_reader.GetOutputPort())
        self.master_slicer.SetOutputDimensionality(2)
        self.master_slicer.SetResliceAxes(self.master_plane)
        self.master_slicer.SetInterpolationModeToLinear()
        self.master_slicer.Update()
        
        self.master_colors.SetInputConnection(self.master_slicer.GetOutputPort())

        # Display the image
        self.master_actor.GetMapper().SetInputConnection(self.master_colors.GetOutputPort())
        
        # Extract a slice in the desired orientation
        self.ground_slicer.SetInputConnection(self.reader_02.GetOutputPort())
        self.ground_slicer.SetOutputDimensionality(2)
        self.ground_slicer.SetResliceAxes(self.master_plane)
        self.ground_slicer.Update()


        self.label02_colors.SetInputConnection(self.ground_slicer.GetOutputPort())

        # Display the image
        self.ground_actor.GetMapper().SetInputConnection(self.label02_colors.GetOutputPort())
        
        # Extract a slice in the desired orientation  
        self.deep_slicer.SetInputConnection(self.reader_03.GetOutputPort())
        self.deep_slicer.SetOutputDimensionality(2)
        self.deep_slicer.SetResliceAxes(self.master_plane)
        self.deep_slicer.Update()
        
        self.label03_colors.SetInputConnection(self.deep_slicer.GetOutputPort())

        # Display the image
        self.deep_actor.GetMapper().SetInputConnection(self.label03_colors.GetOutputPort())

        self.ground_actor.GetProperty().SetOpacity(opacity)
        
        self.deep_actor.GetProperty().SetOpacity(opacity)
        

        self.master_renderer.SetBackground(0, 0, 0)
        self.master_renderer.SetViewport(0, 0, 1 / 3, 1)
        self.master_renderer.ResetCamera()
        self.master_renderer.AddActor(self.master_actor)
        self.master_renderer.ResetCamera()

        self.ground_renderer.SetBackground(0, 0, 0)
        self.ground_renderer.SetViewport(1 / 3, 0, 2 / 3, 1)
        self.ground_renderer.AddActor(self.ground_actor)
        self.master_renderer.ResetCamera()
        self.ground_renderer.AddActor(self.master_actor)
        self.master_renderer.ResetCamera()
        
        self.deep_renderer.SetBackground(0, 0, 0)
        self.deep_renderer.SetViewport(2 / 3, 0, 1, 1)
        self.deep_renderer.AddActor(self.deep_actor)
        self.master_renderer.ResetCamera()
        self.deep_renderer.AddActor(self.master_actor)
        self.master_renderer.ResetCamera()

        self.master_renderer.SetAspect(1.0, 1.0)


        self.iren.RemoveObservers('RightButtonPressEvent')
        self.iren.RemoveObservers('RightButtonReleaseEvent')
        self.iren.AddObserver('RightButtonPressEvent', self.RightButtonCallback)
        self.iren.AddObserver('RightButtonReleaseEvent', self.RightButtonCallback)
        self.iren.AddObserver("MouseMoveEvent", self.MouseMoveCallback)

        self.window.Render() 
        
        self.iren.ProcessEvents()
        
        self.iren.Initialize()

0 replies

jourdain · 2023-05-23T18:49:52Z

jourdain
May 23, 2023
Maintainer

Thanks for providing some code to test and track such issue.

One way to fix and figure out which of the dependency introduce that behavior is to fix some of those dependencies.

I would suggest to start by fixing trame-vtk==2.3.5 into that file of your repo and rebuild the server directory (you may have to delete that directory before rebuilding the full image)

Please keep us in the loop on what you find.

Side note, are you sure you are not missing the () in ctrl.volume_update as that would do nothing if you don't execute the function.

3 replies

ferracioli May 23, 2023
Author

Thank you so much! The problem was exactly the missing (). For some reason, the code used to work in the previous Docker builds, depsite this error. My team tested running the display with this fix, and now the slider is working perfectly!

Best regards

jourdain May 23, 2023
Maintainer

Not sure if you can, but I would love to see what people are doing with trame. If you can share a screenshot publicly or privately that will be great. ([email protected])

ferracioli May 23, 2023
Author

It would be my pleasure to share our project. I will consult with my team, and most likely, we will be able to provide more information about our dashboard within this week. It will take some time to review the displayed data as we are working with CT data from our patients, but we will reach out to you soon!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with Vuetify slider interactor in Trame project deployed on AWS #260

{{title}}

Replies: 2 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Issue with Vuetify slider interactor in Trame project deployed on AWS #260

ferracioli May 23, 2023

Replies: 2 comments · 3 replies

ferracioli May 23, 2023 Author

jourdain May 23, 2023 Maintainer

ferracioli May 23, 2023 Author

jourdain May 23, 2023 Maintainer

ferracioli May 23, 2023 Author

ferracioli
May 23, 2023

Replies: 2 comments 3 replies

ferracioli
May 23, 2023
Author

jourdain
May 23, 2023
Maintainer

ferracioli May 23, 2023
Author

jourdain May 23, 2023
Maintainer

ferracioli May 23, 2023
Author