24.7.18

24.7.18

Important

This release intentionally breaks the high-temp plugin, and temperature-expansion plugin. Don't update until you're prepared to update those (and the plugins have been updated, follow along on our forums

Enhancements

• improvements to the UI's experiment profile preview.
• hours_elapsed() is a function in profile expressions, which returns the hours since the profile started.
• unit() can be used in mqtt fetch expressions. Example: unit():stirring:target_rpm is identical to ::stirring:target_rpm. The latter can be seen as a shortened version of the former.
• experiment profiles can have a description in the job field (i.e. at the same level as actions).
• Updated Raspberry Pi OS image to 2024-07-04.
• Vendoring the TMP1075 library, which also fixes the RPi5 error.
• In places where the ipv4 is displayed (Inventory page, System tab, pio workers status, etc), all ipv4 addresses are displayed.

Breaking changes

• remove the temperature_control, dosing_control, and led_control abstractions. These were introduced early in the Pioreactor software as a way to quickly change automations, but they have been more of a wort than a win. While working on the internals of experiment profiles recently, it became more and more clear how poor this abstraction was. The removal of them has some consequences and some backward incompatibilities:

• updating experiment profiles: experiment profiles that have a *_control job will need to be updated to use *_automation, eventually. For now, we are allowing *_control in profiles: in the backend, we are renaming *_control to *_automations, but a warning will be produced. Later, we'll remove this renaming and profiles will need to be completely updated. Example:

  experiment_profile_name: start_temp_control
  
  metadata:
    author: Cam DP
  
  common:
    jobs:
      temperature_control:
        actions:
          - type: start
            hours_elapsed: 0
            options:
              automation_name: thermostat
              target_temperature: 30
          - type: stop
            hours_elapsed: 12
      temperature_automation:
        actions:
          - type: update
            hours_elapsed: 6
            options:
              target_temperature: 35

  becomes:

  experiment_profile_name: start_temp_control
  
  metadata:
    author: Cam DP
  
  common:
    jobs:
      temperature_automation:
        actions:
          - type: start
            hours_elapsed: 0
            options:
              automation_name: thermostat
              target_temperature: 30
          - type: stop
            hours_elapsed: 12
          - type: update
            hours_elapsed: 6
            options:
              target_temperature: 35

  • update plugins. For users using, specifically, the high-temp plugin, or temperature-expansion-kit plugin, new plugins will be released. Look on the forums, or documentation, for update instructions.

The benefits of removing this abstraction is much less code, less overhead, easier developer experience, and overall simplification. Later, we may create a new abstraction, but now we are moving abstractions back to level 0.

• log in experiment profiles now uses expressions instead of Python string formatting. For example: The unit {unit} is running {job} in experiment {experiment} should be replaced by expressions in the string: The unit ${{unit()}} is running ${{job_name()}} in the experiment ${{experiment}}. Note: {job} is now ${{job_name()}}.
• cycle_media and cycle_alt_media now publish dosing events, and will be recorded by dosing automations, and the db.

Bug fixes

• When pausing temperature automations, the heater now turns off and stays off until unpaused. This is the intended behaviour.

24.7.18rc0

Upcoming

Enhancements

• improvements to the UI's experiment profile preview.
• hours_elapsed() is a function in profile expressions, which returns the hours since the profile started.
• unit() can be used in mqtt fetch expressions. Example: unit():stirring:target_rpm is identical to ::stirring:target_rpm. The latter can be seen as a shortened version of the former.
• experiment profiles can have a description in the job field (i.e. at the same level as actions).
• Updated Raspberry Pi OS image to 2024-07-04.
• Vendoring the TMP1075 library, which also fixes the RPi5 error.
• In places where the ipv4 is displayed (Inventory page, System tab, pio workers status, etc), all ipv4 addresses are displayed.

Breaking changes

• remove the temperature_control, dosing_control, and led_control abstractions. These were introduced early in the Pioreactor software as a way to quickly change automations, but they have been more of a wort than a win. While working on the internals of experiment profiles recently, it became more and more clear how poor this abstraction was. The removal of them has some consequences and some backward incompatibilities:

• updating experiment profiles: experiment profiles that have a *_control job will need to be updated to use *_automation, eventually. For now, we are allowing *_control in profiles: in the backend, we are renaming *_control to *_automations, but a warning will be produced. Later, we'll remove this renaming and profiles will need to be completely updated. Example:

  experiment_profile_name: start_temp_control
  
  metadata:
    author: Cam DP
  
  common:
    jobs:
      temperature_control:
        actions:
          - type: start
            hours_elapsed: 0
            options:
              automation_name: thermostat
              target_temperature: 30
          - type: stop
            hours_elapsed: 12
      temperature_automation:
        actions:
          - type: update
            hours_elapsed: 6
            options:
              target_temperature: 35

  becomes:

  experiment_profile_name: start_temp_control
  
  metadata:
    author: Cam DP
  
  common:
    jobs:
      temperature_automation:
        actions:
          - type: start
            hours_elapsed: 0
            options:
              automation_name: thermostat
              target_temperature: 30
          - type: stop
            hours_elapsed: 12
          - type: update
            hours_elapsed: 6
            options:
              target_temperature: 35

  • update plugins. For users using, specifically, the high-temp plugin, or temperature-expansion-kit plugin, new plugins will be released. Look on the forums, or documentation, for update instructions.

The benefits of removing this abstraction is much less code, less overhead, easier developer experience, and overall simplification. Later, we may create a new abstraction, but now we are moving abstractions back to level 0.

• log in experiment profiles now uses expressions instead of Python string formatting. For example: The unit {unit} is running {job} in experiment {experiment} should be replaced by expressions in the string: The unit ${{unit()}} is running ${{job_name()}} in the experiment ${{experiment}}. Note: {job} is now ${{job_name()}}.
• cycle_media and cycle_alt_media now publish dosing events, and will be recorded by dosing automations, and the db.

Bug fixes

• When pausing temperature automations, the heater now turns off and stays off until unpaused. This is the intended behaviour.

24.7.7

24.7.5 & 24.7.6 & 24.7.7

Hotfix release for 24.7.3. This pins blinka to a specific version which does not install numpy.

24.7.5rc0

24.7.5rc0

Testing hotfix for numpy 2.0 incompatibility.

24.7.3

24.7.3

Enhancements

• A new live preview in the UI's experiment profile editor. This preview tool is useful for getting immediate feedback when writing a profile. We'll keep on adding to this to improve the edit-profile workflow - please send us feedback!
• new when action type in experiment profiles that will execute an action (or list of actions) when some expression is true. For example, start a chemostat when a threshold OD is first achieved, log a message when event is triggered, or monitor a bioreactor parameter and execute an action if it goes out of bounds.
• New config turbidostat.config that can be used to modify some internal turbidostat settings:

  [turbidostat.config]
  signal_channel=2
  od_smoothing_ema=0.5
  

• Better user interaction on the Pioreactors page when the assigned experiment and "viewing" experiment are different.
• Select / Deselect all Pioreactors to assign to an experiment faster.
• Added unit() function to experiment profiles expressions that returns the unit name the expression is evaluated for. Ex: if: ${{ unit() == worker01 }}.
• Added job_name() function to experiment profiles expressions that returns the job_name the expression is evaluated for. Ex: if: ${{ job_name() == stirring }}.
• Added experiment() function to experiment profiles expressions that returns the experiment the expression is evaluated for. Ex: if: ${{ experiment() == exp001 }}.

Breaking changes

• significant web backend API changes! See list of rules in docs.

Bug fixes

• Fix UI code editor from being unresponsive when all the text was removed.
• Experiment profiles won't be overwritten if providing the same filename as an existing profile.

24.7.3rc0

Upcoming

Highlights

• A new live preview in the UI's experiment profile editor. This preview tool is useful for getting immediate feedback when writing a profile. We'll keep on adding to this to improve the edit-profile workflow - please send us feedback!
• new when action type in experiment profiles that will execute an action (or list of actions) when some expression is true. For example, start a chemostat when a threshold OD is first achieved, log a message when event is triggered, or monitor a bioreactor parameter and execute an action if it goes out of bounds.

Enhancements

• New config turbidostat.config that can be used to modify some internal turbidostat settings:

  [turbidostat.config]
  signal_channel=2
  od_smoothing_ema=0.5
  

• Better user interaction on the Pioreactors page when the assigned experiment and "viewing" experiment are different.

• Select / Deselect all Pioreactors to assign to an experiment faster.

• Added unit() function to experiment profiles expressions that returns the unit name the expression is evaluated for. Ex: if: ${{ unit() == worker01 }}.

• Added job_name() function to experiment profiles expressions that returns the job_name the expression is evaluated for. Ex: if: ${{ job_name() == stirring }}.

• Added experiment() function to experiment profiles expressions that returns the experiment the expression is evaluated for. Ex: if: ${{ experiment() == exp001 }}.

Breaking changes

• significant web backend API changes! See list of rules in docs.

Bug fixes

• Fix UI code editor from being unresponsive when all the text was removed.
• Experiment profiles won't be overwritten if providing the same filename as an existing profile.

24.6.10

Important

We strongly recommend you be on 24.5.1 or higher before updating. If not, update to 24.5.1 first!

Enhancements

• we changed the "auto" algorithm for picking a good ir_led_intensity. We now try to maximize the intensity, up to some constraints around saturating ADCs, LED longevity, and signal. In general, we expect a higher IR intensity, but this will help with noise and detecting lower signals.
• More improvements on the Pioreactor-specific page: added charts and a logs table.
• Added a "retry failed tests" to the UI's self-test dialog.
• pio run self_test has a new flag --retry-failed to only retry tests that failed in the previous run (if any).
• better clean up when a worker is removed from a cluster.
• reduce the mosquitto logs to reduce writes to disk and speed up connections.
• Use lexicographical ordering for all displays of workers in the UI
• This only applies to new installed images, and not updates. Updated to the latest RPI image, 2024-03-15, -> linux kernel update to 6.6. Recent versions of linux have improved support for usb wifi devices.
• This only applies to new installed images, and not updates. leader-only images will install worker Python libraries.
• This only applies to new installed images, and not updates. all experiment data will be deleted when the experiment is deleted.
• performance improvements

Breaking changes

• Changed the web backend API endpoints for time-series, logs, shutdown, reboot, and plugins to be more RESTful. See docs for updated rules in the docs.

Bug fixes

• fix performing an "undo" when editing the config.ini and experiment profiles.
• fix Pioreactor v1.1 bug when change target temperature mid cycle causing the inferred temperature to change significantly.
• if a worker disconnected from the network, messages are queued in memory until the network reconnects. This has two problems. The first is that there is a finite amount of memory, and we don't want to OOM. The second is that when the worker(s) reconnect, there is a flurry of messages. For some jobs that use messages as events, this can cause multiple triggers in quick succession. We've added some logic that helps avoid these situations:
  1. we max the queue of unsent messages to 100 (arbitrary)
  2. in important jobs, like temperature automations, it will only respond to "recent" messages and not old messages.

24.6.10rc0

Upcoming

Enhancements

• we changed the "auto" algorithm for picking a good ir_led_intensity. We now try to maximize the intensity, up to some constraints around saturating ADCs, LED longevity, and signal. In general, we expect a higher IR intensity, but this will help with noise and detecting lower signals.
• More improvements on the Pioreactor-specific page: added charts and a logs table.
• Added a "retry failed tests" to the UI's self-test dialog.
• pio run self_test has a new flag --retry-failed to only retry tests that failed in the previous run (if any).
• better clean up when a worker is removed from a cluster.
• reduce the mosquitto logs to reduce writes to disk and speed up connections.
• Use lexicographical ordering for all displays of workers in the UI
• This only applies to new installed images, and not updates. Updated to the latest RPI image, 2024-03-15, -> linux kernel update to 6.6. Recent versions of linux have improved support for usb wifi devices.
• This only applies to new installed images, and not updates. leader-only images will install worker Python libraries.
• This only applies to new installed images, and not updates. all experiment data will be deleted when the experiment is deleted.
• performance improvements

Breaking changes

• Changed the web backend API endpoints for time-series, logs, shutdown, reboot, and plugins to be more RESTful. See docs for updated rules in the docs.

Bug fixes

• fix performing an "undo" when editing the config.ini and experiment profiles.
• fix Pioreactor v1.1 bug when change target temperature mid cycle causing the inferred temperature to change significantly.
• if a worker disconnected from the network, messages are queued in memory until the network reconnects. This has two problems. The first is that there is a finite amount of memory, and we don't want to OOM. The second is that when the worker(s) reconnect, there is a flurry of messages. For some jobs that use messages as events, this can cause multiple triggers in quick succession. We've added some logic that helps avoid these situations:
  1. we max the queue of unsent messages to 100 (arbitrary)
  2. in important jobs, like temperature automations, it will only respond to "recent" messages and not old messages.

24.5.31

24.5.31

Important

We strongly recommend you be on 24.5.1 or higher before updating. If not, update to 24.5.1 first!

Highlights

• New /pioreactor/worker-name page in the UI for a detailed view of an individual Pioreactor, including a realtime visualization of the Pioreactor!

Enhancements

• UI now supports external MQTT broker. This configuration lives in the same place as the exiting MQTT settings: in the config.ini, under [mqtt].
• Added groupings on the Experiment drop down to organize "Active" and "Inactive" experiments. An active experiment has >= 1 Pioreactor assigned to it.

Breaking changes

• New log topic that partitions by the level. This should make subscribers to the log topic slimmer (like the UI, who previosly would have to accept all messages and filter to what they needed). Should result in a performance increase.

Bug fixes

• Fix for Pioreactors page when no workers are added to the cluster.
• Fix for UI labels when trying to remove labels from Pioreactors.
• Improvements to REF self-tests.

24.5.31rc0

Upcoming

Highlights

• New /pioreactor/worker-name page in the UI for a detailed view of an individual Pioreactor, including a realtime visualization of the Pioreactor!

Enhancements

• UI now supports external MQTT broker. This configuration lives in the same place as the exiting MQTT settings: in the config.ini, under [mqtt].
• Added groupings on the Experiment drop down to organize "Active" and "Inactive" experiments. An active experiment has >= 1 Pioreactor assigned to it.

Breaking changes

• New log topic that partitions by the level. This should make subscribers to the log topic slimmer (like the UI, who previosly would have to accept all messages and filter to what they needed). Should result in a performance increase.

Bug fixes

• Fix for Pioreactors page when no workers are added to the cluster.
• Fix for UI labels when trying to remove labels from Pioreactors.
• Improvements to REF self-tests.