90 add enter and exit methods to qmi instrument

.github/workflows/reusable-ci-workflows.yml

@@ -37,7 +37,7 @@ jobs:
           sudo apt-get update -qy
           sudo apt-get install -y bc
           pip install --upgrade pip
-          pip install -e '.[dev]'
+          pip install '.[dev]'
 
       - name: Run pylint
         run: |

.github/workflows/scheduled-full-ci.yml

@@ -29,6 +29,7 @@ jobs:
       - name: Run unit tests and generate report
         if: always()
         run: |
+          pip install .
           pip install unittest-xml-reporting
           python -m xmlrunner --output-file testresults.xml discover --start-directory=tests --pattern="test_*.py"
 

CHANGELOG.md

@@ -6,6 +6,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## \[x.y.z] - Unreleased
 
+### Added
+- The `QMI_Instrument` and `QMI_TaskRunner` (which inherit from `QMI_RpcObject`) are now equipped with specific `__enter__` and `__exit__` methods, which in the case of `QMI_Instrument`
+  also open and close the instrument when run with a `with` context manager protocol. Meanwhile `QMI_TaskRunner` starts and stops then joins a QMI task thread. In practise, these context managers
+  can be used instead of the to-be-obsoleted `open_close` and `start_stop_join` context managers. The context manager protocol cannot be used for `QMI_RpcObject` directly.
+
 ### Changed
 - The CI pipelines are now using reusable workflows, placed in reusable-ci-workflows.yml.
 - The file names for the different pipeline actions were also changed to be more descriptive.

documentation/sphinx/source/design.rst

@@ -244,6 +244,78 @@ could lead into unexpected hardware responses and/or other kinds of issues, and
 Further, the proxies have the possibility of *locking* their objects to be controlled by a specific context only. The use of the ``lock()``,
 ``unlock()``, ``force_unlock()`` and ``is_locked()`` methods are illustrated in the Tutorial.
 
+**Context management**
+======================
+
+QMI offers a few context managers to facilitate better control of the QMI contexts, instruments, tasks and signals.
+
+QMI contexts can be started and stopped with a `start_stop` context manager, available in ``qmi.utils.context_managers`` module.
+The following code based on the ``with`` statement::
+
+    with start_stop(qmi, "name"):
+        custom_code_here ...
+
+has the same effect as::
+
+    qmi.start("name")
+    try:
+        custom_code_here ...
+    finally:
+        qmi.stop()
+
+both ensuring that ``qmi.stop()`` will be called even when an error occurs in the custom code.
+
+We can make instruments and tasks in the QMI context. For automatic opening and closing of an instrument driver instance
+based on `QMI_Instrument`, we can do::
+
+    with qmi.make_instrument("instrument_name", InstrumentClass, ...) as instr:
+        custom_code_here...
+
+which has the same effect as::
+
+    instr = qmi.make_instrument("instrument_name", InstrumentClass)
+    instr.open()
+    try:
+        custom_code_here...
+    finally:
+        instr.close()
+
+Alternatively, the `open_close` context manager,from ``qmi.utils.context_managers`` can be used, but this context manager
+will be obsoleted. That option requires making the instrument instance first and then giving it as an input to the context manager.
+
+For tasks we can use the context management protocol to automatically start the task thread when entering a task's `QMI_LoopTask` context,
+and stopping and joining to it at exit. Similar to the instrument, we can do::
+
+    with qmi.make_task("task_name", TaskClass, ...) as task:
+         task_code_here...
+
+And the task should be stopped and joined after the task is finished. In the ``qmi.utils.context_managers`` is also context manager
+`start_stop_join` to do this, but it will be obsoleted.
+
+Further context managers in ``qmi.utils.context_managers`` are `lock_unlock` and `subscribe_unsubscribe` context managers.
+The `lock_unlock` manager is meant for RPC objects that the user wants to lock while they are used by some script or task.
+Typical use::
+
+    some_instr = qmi.get_instrument(...)
+    with lock_unlock(some_instr):
+        priviledged_code_here...
+
+The `lock_unlock` context manager accepts also extra input arguments, so that `timeout` and `lock_token` arguments can
+also be given for the context manager.
+
+And the final `subscribe_unsubscribe` context manager is meant to be used with signals. For example, a task has signal
+named `sig_send_data` in the task's class. And we want to subscribe to it to receive data updates. If a task is f.ex.
+obtained from another context, and we want to receive in `data_receiver`::
+
+    signal_task = qmi.get_task(...)
+    data_receiver = QMI_SignalReceiver()
+    with subscribe_unsubscribe(signal_task.sig_send_data, data_receiver):
+        data = data_receiver.get_next_signal()
+
+If the task is 'running' and publishing data, the receiver should receive the data from it and then unsubscribe from
+the signal again. Forgetting to unsubscribe from the signal could possibly lead to memory issues if the receiver is
+still present, because then the published data could keep accumulating into the receiver queue.
+
 **Messaging**
 =============
 

documentation/sphinx/source/tutorial.rst

@@ -183,7 +183,7 @@ However, if you find yourself in a situation in which the locking proxy was lost
 >>> nsg2.is_locked()
 False
 
-It is also possible to unlock from another context proxy by providing the context name as well. See the example below how to connect from another context to an instrument.
+It is also possible to unlock from another instrument proxy by providing the context name as well. See the example below, how to get from another context the instrument proxy.
 
 >>> import qmi
 >>> qmi.start("client")
@@ -355,7 +355,7 @@ To set up a simple measurement script, create a file ``measure_demo.py`` with th
     #!/usr/bin/env python3
 
     import qmi
-    from qmi.utils.context_managers import start_stop, open_close
+    from qmi.utils.context_managers import start_stop
     from qmi.instruments.dummy.noisy_sine_generator import NoisySineGenerator
 
     def measure_data(nsg):
@@ -368,8 +368,7 @@ To set up a simple measurement script, create a file ``measure_demo.py`` with th
 
     def main():
         with start_stop(qmi, "measure_demo"):
-            nsg = qmi.make_instrument("nsg", NoisySineGenerator)
-            with open_close(nsg):
+            with qmi.make_instrument("nsg", NoisySineGenerator) as nsg:
                 measure_data(nsg)
 
     if __name__ == "__main__":
@@ -383,21 +382,8 @@ Note that the script uses :py:class:`qmi.utils.context_managers.start_stop` to
 start and stop the QMI framework.
 This is just a convenient way to make sure that ``qmi.start()`` and ``qmi.stop()``
 will always be called.
-The following code based on the ``with`` statement::
-
-    with start_stop(qmi, "name"):
-        custom_code_here ...
-
-has the same effect as::
-
-    qmi.start("name")
-    custom_code_here ...
-    qmi.stop()
-
-with the difference that the ``with`` mechanism ensures that ``qmi.stop()``
-will be called even when an error occurs in the custom code.
-Similarly, the script uses :py:class:`qmi.utils.context_managers.open_close`
-to open and close to represent the calls to ``nsg.open()`` and ``nsg.close()``.
+Similarly, the `QMI_Instrument` objects are equipped with context managers that open and close the
+the instrument, calling ``nsg.open()`` and ``nsg.close()`` at the creation and destruction of the instance.
 
 .. note::
     Some users prefer to invoke scripts from an interactive Python session,
@@ -464,14 +450,13 @@ the task and continues to perform other activities::
 
     import time
     import qmi
-    from qmi.utils.context_managers import start_stop, open_close
+    from qmi.utils.context_managers import start_stop
     from qmi.instruments.dummy.noisy_sine_generator import NoisySineGenerator
     from demo_task import DemoTask
 
     def main():
         with start_stop(qmi, "task_demo"):
-            nsg = qmi.make_instrument("nsg", NoisySineGenerator)
-            with open_close(nsg):
+            with qmi.make_instrument("nsg", NoisySineGenerator) as nsg:
                 task = qmi.make_task("task", DemoTask)
                 task.start()
                 print("the task has been started")
@@ -585,11 +570,10 @@ Try making the following class::
                 raise qmi.core.exceptions.QMI_TaskRunException("No such attribute in task: 'amplitude_factor'")
 
 And give it as the ``task_runner`` input when making the task, it is possible to change the value from outside the task.
-We now also switch to use the ``start_stop_join`` context manager for tasks::
+We now also switch to use the internal context manager for tasks::
 
     ...
-                task = qmi.make_task("task", DemoTask, task_runner=CustomTaskRunner)
-                with start_stop_join(task):
+                with qmi.make_task("task", DemoTask, task_runner=CustomTaskRunner) as task:
                     print("the task has been started")
                     time.sleep(1)
                     task.set_amplitude_factor(1.0)
@@ -631,7 +615,7 @@ of the task in the script instead of inside the task. We now rewrite the script
 
     import time
     import qmi
-    from qmi.utils.context_managers import start_stop, open_close
+    from qmi.utils.context_managers import start_stop
     from qmi.instruments.dummy.noisy_sine_generator import NoisySineGenerator
     from demo_task import DemoTask
 
@@ -644,10 +628,8 @@ of the task in the script instead of inside the task. We now rewrite the script
 
     def main_2():
         with start_stop(qmi, "task_demo"):
-            nsg = qmi.make_instrument("nsg", NoisySineGenerator)
-            with open_close(nsg):
-                task = qmi.make_task("task", DemoRpcControlTask, task_runner=CustomRpcControlTaskRunner)
-                with start_stop_join(task):
+            with qmi.make_instrument("nsg", NoisySineGenerator) as nsg:
+                with qmi.make_task("task", DemoRpcControlTask, task_runner=CustomRpcControlTaskRunner) as task:
                     print("the task has been started")
                     for i in range(5):
                         sample = nsg.get_sample()

qmi/core/instrument.py

@@ -47,7 +47,7 @@ class QMI_Instrument(QMI_RpcObject):
     (measurements, getting and setting of parameters).
 
     Driver should implement a method `reset()` when applicable.
-    This methods returns the instrument to its default settings.
+    This method returns the instrument to its default settings.
 
     Drivers should implement a method `get_idn()` when applicable.
     This method returns an instance of `QMI_InstrumentIdentification`.
@@ -66,6 +66,19 @@ def __init__(self, context: 'qmi.core.context.QMI_Context', name: str) -> None:
         super().__init__(context, name)
         self._is_open = False
 
+    @rpc_method
+    def __enter__(self) -> "QMI_Instrument":
+        """The `__enter__` methods is decorated as `rpc_method` so that `QMI_RpcProxy` can call it when using the
+        proxy with a `with` context manager. This method also opens the instrument."""
+        self.open()
+        return self
+
+    @rpc_method
+    def __exit__(self, *args, **kwargs) -> None:
+        """The `__exit__` methods is decorated as `rpc_method` so that `QMI_RpcProxy` can call it when using the
+        proxy with a `with` context manager. This method also closes the instrument."""
+        self.close()
+
     def release_rpc_object(self) -> None:
         """Give a warning if the instrument is removed while still open."""
         if self._is_open:

qmi/core/rpc.py

@@ -83,7 +83,7 @@ def square(self, x):
 Since QMI V0.29.1 it is also possible to lock and unlock with a custom token from
 other proxies as well, as long as the contexts for the other proxies have the same name.
 
-Example 1:
+Example 1::
         # Two proxies in same context.
         proxy1 = context.make_rpc_object("my_object", MyRpcTestClass)
         proxy2 = context.get_rpc_object_by_name("my_context.my_object")
@@ -94,7 +94,7 @@ def square(self, x):
         proxy2.unlock(lock_token=custom_token)
         proxy2.is_locked()  # Returns False
 
-Example 2:
+Example 2::
         # Three proxies in different contexts. The first one serves as an "object provider".
         c1 = QMI_Context("c1", config)
         c1.start()
@@ -557,7 +557,20 @@ def blocking_rpc_method_call(context: "qmi.core.context.QMI_Context",
 
 
 class QMI_RpcNonBlockingProxy:
-    """Proxy class for RPC objects that performs non-blocking calls."""
+    """Proxy class for RPC objects that performs non-blocking calls. Direct instantiation is not recommended.
+
+    This is always also instantiated in `QMI_RpcProxy` as `self.rpc_nonblocking` attribute. Typically, if user wants
+    to use a non-blocking call with an RPC object `rpc_proxy`, they would do:
+    ```python
+    future = rpc_proxy.rpc_nonblocking.some_rpc_command(args)
+    # Other stuff can be done here in the meanwhile, and the `proxy` is not blocked while waiting to return
+    retval = future.wait()
+    ```
+    Instead of the usual
+    ```python
+    retval = rpc_proxy.some_rpc_commands(args)
+    ```
+    """
 
     def __init__(self, context: "qmi.core.context.QMI_Context", descriptor: RpcObjectDescriptor) -> None:
 
@@ -602,7 +615,11 @@ def __repr__(self) -> str:
 
 
 class QMI_RpcProxy:
-    """Proxy class for RPC objects that performs blocking calls."""
+    """Proxy class for RPC objects that performs blocking calls. All RPC objects created return this proxy class to
+    enable RPC communication between objects.
+
+    Direct instantiation of this class is not meant to be done by users; internal use only!
+    """
 
     def __init__(self, context: "qmi.core.context.QMI_Context", descriptor: RpcObjectDescriptor) -> None:
 
@@ -659,6 +676,27 @@ def make_rpc_forward_function(method_name: str):
         # Add non-blocking proxy.
         self.rpc_nonblocking = QMI_RpcNonBlockingProxy(context, descriptor)
 
+    def __enter__(self) -> "QMI_RpcProxy":
+        """The context manager definition is needed for the proxy as it will always be returned from QMI contexts,
+        instead of the actual RPC object instance. Trying to use context management directly on this class will
+        cause recursion error, but for actual RPC objects not.
+
+        The with keyword checks if the `QMI_RpcProxy` class (not instance) implements the context manager protocol,
+        i.e. `__enter__` and `__exit__`, so they exist here as stubs. These stubs make an RPC to the actual `__enter__`
+        and `__exit__` methods on the relevant RPC object. If in those classes `__enter__` and `__exit__` are
+        decorated as `rpc_methods`, we do not see a recursion error. See for further details in:
+        https://docs.python.org/3/reference/datamodel.html#special-method-lookup
+
+        `rpc_method` decorated `__enter__` and `__exit__` methods are currently implemented in `QMI_Instrument` and
+        `QMI_TaskRunner` classes.
+        """
+        self.__enter__()
+        return self
+
+    def __exit__(self, *args, **kwargs):
+        self.__exit__()
+        return
+
     def __repr__(self) -> str:
         return "<rpc proxy for {} ({})>".format(self._rpc_object_address, self._rpc_class_fqn)
 
@@ -891,9 +929,13 @@ def __init__(self,
     def __repr__(self) -> str:
         return "{}({!r})".format(type(self).__name__, self._name)
 
+    @rpc_method
+    def __enter__(self):
+        raise NotImplementedError(f"{type(self)} is not meant to be used with context manager.")
+
     def lock(self, timeout: float = 0.0, lock_token: Optional[str] = None) -> bool:
-        """Lock the remote object. If timeout is given, try every 0.1s within the given timeout value. The remote object
-        can be locked with an optional custom lock token by giving a string into `lock_token` keyword argument.
+        """Lock the remote object. If timeout is given, try every 0.1s within the given timeout value. The remote
+        object can be locked with an optional custom lock token by giving a string into `lock_token` keyword argument.
 
         If successful, this proxy is the only proxy that can invoke RPC methods on the remote object; other proxies
         will receive an "object is locked" response. The return value indicates if the lock was granted; a denied lock
@@ -1354,7 +1396,6 @@ def make_proxy(self) -> QMI_RpcProxy:
 
         # This may raise an exception!
         rpc_object = self._rpc_thread.rpc_object()
-
         return QMI_RpcProxy(self._context, rpc_object.rpc_object_descriptor)
 
     def handle_message(self, message: QMI_Message) -> None:

qmi/core/task.py

@@ -262,7 +262,7 @@ def update_settings(self) -> bool:
         to this task, this method copies the new settings to `self.settings`
         and returns True.
 
-        Otherwise the settings remain the same and this method returns False.
+        Otherwise, the settings remain the same and this method returns False.
 
         Returns:
             True if there are new settings, False if the settings are unchanged.
@@ -602,6 +602,19 @@ def __init__(self,
         assert state == _TaskThread.State.READY_TO_RUN
         assert self._thread.task is not None
 
+    @rpc_method
+    def __enter__(self):
+        """The `__enter__` methods is decorated as `rpc_method` so that `QMI_RpcProxy` can call it when using the
+        proxy with a `with` context manager. This method also calls to start the task thread."""
+        return self.start()
+
+    @rpc_method
+    def __exit__(self, *args, **kwargs):
+        """The `__exit__` methods is decorated as `rpc_method` so that `QMI_RpcProxy` can call it when using the
+        proxy with a `with` context manager. This method also calls to stop and join the task thread."""
+        self.stop()
+        self.join()
+
     def release_rpc_object(self) -> None:
         """Ensure the task is joined before it is removed from the context."""
         if not self._joined: