problem with live image display from pycromanager

[monfort314]

Hi, I'm trying the plugin for displaying the live image from
https://pycro-manager.readthedocs.io/en/latest/application_notebooks/pycro_manager_imjoy_tutorial.html that uses https://gist.github.com/oeway/f59c1d1c49c94a831e5e21ba4c6111dd

Specifically, I'm trying to run the code below, but it seems that after pushing 'start live' the function get_image() is called only twice (I checked it with a global counter)

Many thanks,

`import time

import numpy as np
from imjoy import api

from pycromanager import Bridge

class MyMicroscope:
async def setup(self):
bridge = Bridge()
self._core = bridge.get_core()
exposure = self._core.get_exposure()
api.showMessage("MMcore loaded, exposure: " + str(exposure))

def snap_image(self):
    if self._core.is_sequence_running():
        self._core.stop_sequence_acquisition()
    self._core.snap_image()
    return self.get_image()

def get_image(self):
    # we can also check remaining with getRemainingImageCount()
    tagged_image = self._core.get_tagged_image()
    image_array = np.reshape(
        tagged_image.pix,
        newshape=[-1, tagged_image.tags["Height"], tagged_image.tags["Width"]],
    )
    image_array = (image_array / image_array.max() * 255).astype("uint8")
    return image_array

def get_device_properties(self):
    core = self._core
    devices = core.get_loaded_devices()
    devices = [devices.get(i) for i in range(devices.size())]
    device_items = []
    for device in devices:
        names = core.get_device_property_names(device)
        props = [names.get(i) for i in range(names.size())]
        property_items = []
        for prop in props:
            value = core.get_property(device, prop)
            is_read_only = core.is_property_read_only(device, prop)
            if core.has_property_limits(device, prop):
                lower = core.get_property_lower_limit(device, prop)
                upper = core.get_property_upper_limit(device, prop)
                allowed = {
                    "type": "range",
                    "min": lower,
                    "max": upper,
                    "readOnly": is_read_only,
                }
            else:
                allowed = core.get_allowed_property_values(device, prop)
                allowed = {
                    "type": "enum",
                    "options": [allowed.get(i) for i in range(allowed.size())],
                    "readOnly": is_read_only,
                }
            property_items.append(
                {"device": device, "name": prop, "value": value, "allowed": allowed}
            )
            # print('===>', device, prop, value, allowed)
        if len(property_items) > 0:
            device_items.append(
                {
                    "name": device,
                    "value": "{} properties".format(len(props)),
                    "items": property_items,
                }
            )
    return device_items

async def run(self, ctx):
    mmcore_api = {
        "_rintf": True,
        "snapImage": self.snap_image,
        "getImage": self.get_image,
        "getDeviceProperties": self.get_device_properties,
        "getCameraDevice": self._core.get_camera_device,
        "setCameraDevice": self._core.set_camera_device,
        "startContinuousSequenceAcquisition": self._core.start_continuous_sequence_acquisition,
        "stopSequenceAcquisition": self._core.stop_sequence_acquisition,
        "setExposure": self._core.set_exposure,
        "getExposure": self._core.get_exposure,
        "setProperty": self._core.set_property,
        "getProperty": self._core.get_property,
    }
    viewer = await api.createWindow(
        src="https://gist.github.com/oeway/f59c1d1c49c94a831e5e21ba4c6111dd",
        data={"mmcore": mmcore_api},
    )

api.export(MyMicroscope())`

[oeway]

Hi @Olaa4 Thanks for the report.

Could you check if there is any error in the browser console when you run it. (This is how to open it: https://balsamiq.com/support/faqs/browserconsole/)

[monfort314]

Hi @oeway !
Thanks for an immediate feedback! Yes, actually there is an error in the browser (Please see below).
Strangely, anaconda console did not display any errors...

It looks like camera image buffer issue but I could not find any robust solution. Do you have any suggestions?
I tried Chrome and Firefox browsers - I got the same error (In MS IE and MS Edge ImJoy plugin to fail to run)
Thanks!

Traceback (most recent call last):
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\imjoy_rpc\rpc.py", line 391, in _call_method
result = method(*args, **kwargs)
File "", line 33, in get_image
tagged_image = self._core.get_tagged_image()
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 432, in
): instance._translate_call(signatures_list, args)
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 554, in _translate_call
return self._deserialize(self._socket.receive())
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 168, in receive
self._check_exception(message)
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 173, in _check_exception
raise Exception(response["value"])
Exception: java.lang.Exception: Camera image buffer read failed.
mmcorej.MMCoreJJNI.CMMCore_getImage__SWIG_1(Native Method)
mmcorej.CMMCore.getImage(CMMCore.java:936)
mmcorej.CMMCore.getTaggedImage(CMMCore.java:162)
mmcorej.CMMCore.getTaggedImage(CMMCore.java:167)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.micromanager.internal.zmq.ZMQServer.runMethod(ZMQServer.java:371)
org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:426)
org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:111)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

(anonymous) @ index.js?v=20211029164638:148
Comm._callback @ comm.js:194
Comm.handle_msg @ comm.js:202
(anonymous) @ comm.js:132
Promise.then (async)
CommManager.comm_msg @ comm.js:131
i @ jquery.min.js:2
Kernel._handle_iopub_message @ kernel.js:1238
Kernel._finish_ws_message @ kernel.js:1017
(anonymous) @ kernel.js:1008
Promise.then (async)
Kernel._handle_ws_message @ kernel.js:1008
i @ jquery.min.js:2
rpc.js:557 Error: Traceback (most recent call last):
File "", line 33, in get_image
tagged_image = self._core.get_tagged_image()
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 432, in
): instance._translate_call(signatures_list, args)
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 554, in _translate_call
return self._deserialize(self._socket.receive())
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 168, in receive
self._check_exception(message)
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 173, in _check_exception
raise Exception(response["value"])
Exception: java.lang.Exception: Camera image buffer read failed.
mmcorej.MMCoreJJNI.CMMCore_getImage__SWIG_1(Native Method)
mmcorej.CMMCore.getImage(CMMCore.java:936)
mmcorej.CMMCore.getTaggedImage(CMMCore.java:162)
mmcorej.CMMCore.getTaggedImage(CMMCore.java:167)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.micromanager.internal.zmq.ZMQServer.runMethod(ZMQServer.java:371)
org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:426)
org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:111)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

at a._decode (rpc.js:768)
at a._decode (rpc.js:812)
at a._unwrap (rpc.js:844)
at rpc.js:260
at Connection._fire (index.js?v=20211029164638:76)
at index.js?v=20211029164638:153
at Comm._callback (comm.js:194)
at Comm.handle_msg (comm.js:202)
at comm.js:132

_encode @ rpc.js:557
_encode @ rpc.js:682
_wrap @ rpc.js:829
i @ rpc.js:893
Promise.catch (async)
(anonymous) @ rpc.js:220
async function (async)
(anonymous) @ rpc.js:202
_fire @ utils.js:863
(anonymous) @ connection.js:65
rpc.js:768 Uncaught (in promise) Error: Error: Traceback (most recent call last):
File "", line 33, in get_image
tagged_image = self._core.get_tagged_image()
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 432, in
): instance._translate_call(signatures_list, args)
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 554, in _translate_call
return self._deserialize(self._socket.receive())
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 168, in receive
self._check_exception(message)
File "C:\Users\lab-nmsb\AppData\Roaming\Python\Python36\site-packages\pycromanager\core.py", line 173, in _check_exception
raise Exception(response["value"])
Exception: java.lang.Exception: Camera image buffer read failed.
mmcorej.MMCoreJJNI.CMMCore_getImage__SWIG_1(Native Method)
mmcorej.CMMCore.getImage(CMMCore.java:936)
mmcorej.CMMCore.getTaggedImage(CMMCore.java:162)
mmcorej.CMMCore.getTaggedImage(CMMCore.java:167)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.micromanager.internal.zmq.ZMQServer.runMethod(ZMQServer.java:371)
org.micromanager.internal.zmq.ZMQServer.parseAndExecuteCommand(ZMQServer.java:426)
org.micromanager.internal.zmq.ZMQServer.lambda$initialize$1(ZMQServer.java:111)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

at a._decode (rpc.js:768)
at a._decode (rpc.js:812)
at a._unwrap (rpc.js:844)
at rpc.js:260
at Connection._fire (utils.js:211)
at Connection.handleEvent (pluginIframe.js:73)

_decode @ rpc.js:768
_decode @ rpc.js:812
_unwrap @ rpc.js:844
(anonymous) @ rpc.js:260
_fire @ utils.js:211
handleEvent @ pluginIframe.js:73
async function (async)
updateImage @ VM27:58
updateImage @ VM27:72
async function (async)
updateImage @ VM27:58
toggleLive @ VM27:79
click @ VM28:3
t @ vue.min.js:6
Ir.t._withTask.i._withTask @ vue.min.js:6

[oeway]

Hi, thanks for the info.

Yes, it seems this is the cause of the issue: Exception: java.lang.Exception: Camera image buffer read failed..

And the error is from micro-manager itself, however, I have no idea how to address it.

I did a quick search and found this, maybe you can give it a try: LC-PolScope/LC-PolScope#26 (comment)

Otherwise, maybe you can ask in the micro-manager forum?