[Bug]: incompatible pymodbus version crashes integration

[Hammerhand17]

Describe the issue

Hi. Sometimes (maybe 1 of 3) when I reboot home assistant, connection with my inverter is lost and isn't recovered until several days before. This is very inconvenient because I lose a lot of information when I reboot to update something on HA.
This is the log:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 334, in _async_setup_component
    component = await integration.async_get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1031, in async_get_component
    self._component_future.result()
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1011, in async_get_component
    comp = await self.hass.async_add_import_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1071, in _get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/loop.py", line 200, in protected_loop_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/config/custom_components/huawei_solar/__init__.py", line 20, in <module>
    from huawei_solar import (
  File "/usr/local/lib/python3.12/site-packages/huawei_solar/__init__.py", line 3, in <module>
    from .bridge import HuaweiSolarBridge
  File "/usr/local/lib/python3.12/site-packages/huawei_solar/bridge.py", line 28, in <module>
    from .huawei_solar import (
  File "/usr/local/lib/python3.12/site-packages/huawei_solar/huawei_solar.py", line 17, in <module>
    from pymodbus.message.rtu import MessageRTU
ModuleNotFoundError: No module named 'pymodbus.message'

Describe your Huawei Solar Setup

Inverter Type: Huawei SUN2000 5KTL-L1
Inverter Firmware version: SUN2000L V200R001C00SPC130
sDongle present: No
Power meter present: single phase
Optimizers Present: No
Battery: None
Huawei Solar integration version: 1.4.1.

How do you connect to the inverter?

Via the SUN2000-<serial_no> WiFi

Upload your Diagnostics File

Drag & Drop your Diagnostics File here.

Upload your relevant debug logs

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 334, in _async_setup_component
    component = await integration.async_get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1031, in async_get_component
    self._component_future.result()
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1011, in async_get_component
    comp = await self.hass.async_add_import_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/loader.py", line 1071, in _get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/loop.py", line 200, in protected_loop_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/config/custom_components/huawei_solar/__init__.py", line 20, in <module>
    from huawei_solar import (
  File "/usr/local/lib/python3.12/site-packages/huawei_solar/__init__.py", line 3, in <module>
    from .bridge import HuaweiSolarBridge
  File "/usr/local/lib/python3.12/site-packages/huawei_solar/bridge.py", line 28, in <module>
    from .huawei_solar import (
  File "/usr/local/lib/python3.12/site-packages/huawei_solar/huawei_solar.py", line 17, in <module>
    from pymodbus.message.rtu import MessageRTU
ModuleNotFoundError: No module named 'pymodbus.message'

Please confirm the following:

• I'm running the latest release of Home Assistant.
• I'm running the latest release of this integration.
• I did not find an existing issue describing this problem.
• I did upload the diagnostics-file that I could retrieve from the 'Devices & Services Page'
• I did upload the relevant debug logs (via 'Enable Debug Logging'-feature or by manually configuring HA logging)

[wlcrs]

The error ModuleNotFoundError: No module named 'pymodbus.message' is caused by an incompatible/outdated version of pyModbus being installed on your HA. This is usually caused by another integration that has a dependency on that outdated pyModbus version. Depending on the order in which the integrations are loaded, an (in)correct version of pyModbus is installed.

Please disable the other integration causing the failure, or ask the developer to upgrade the required pyModbus library to 3.6.9 or higher.

Closing as this is not a bug in this integration. As such, there is nothing for me left to do.

[Hammerhand17]

The error ModuleNotFoundError: No module named 'pymodbus.message' is caused by an incompatible/outdated version of pyModbus being installed on your HA. This is usually caused by another integration that has a dependency on that outdated pyModbus version. Depending on the order in which the integrations are loaded, an (in)correct version of pyModbus is installed.

Please disable the other integration causing the failure, or ask the developer to upgrade the required pyModbus library to 3.6.9 or higher.

Closing as this is not a bug in this integration. As such, there is nothing for me left to do.

Digged in every other integration and I found no one that uses pymodbus (read manifest.json one by one)

Besides, I use pip list | grep pymodbus and I receive
pymodbus 3.7.4

So it seems this is not the issue.

[wlcrs]

Apparently there were some changes in the API of pyModbus between v3.6.9 and v3.7.0 . In the past that error message popped up because an outdated version of pyModbus was installed where that namespace did not exist yet. In v3.7.0 and above, it doesn't exist anymore.

The huawei-solar library explicitly pins the version of pyModbus to pymodbus>=3.6.9,<3.7, so in theory, it should not install that version that is "too new". Alas, this is an example of the Python dependency hell.

When trying to fix them, I notice that the author of pyModbus is changing the API once again already, meaning that if I would do the effort of fixing the huawei-solar library to support the latest pyModbus version, it will break on the next release.

I don't have much time at the moment to do the updates. Given that on the next release of pyModbus is 100% guaranteed to break again, I'm probably going to wait until things have stabilized again.

[wlcrs]

I've just had a second similar issue being reported here (#866 )

As stated in my reaction over there: my policy is that I intend to keep the required pymodbus version in sync with that of the native modbus integration.

I've done some most of the work yesterday to make the huawei-solar library compatible with v3.8.0 of pymodbus which was released a few days ago. It still needs some additional testing and polishing. I will only release a new version of this integration using that updated version when that version of pymodbus used by the HA modbus integration is also bumped to that version.

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[Hammerhand17]

Sorry, not sure if it finally was solved or not

[amaduain]

I've just had a second similar issue being reported here (#866 )

As stated in my reaction over there: my policy is that I intend to keep the required pymodbus version in sync with that of the native modbus integration.

I've done some most of the work yesterday to make the huawei-solar library compatible with v3.8.0 of pymodbus which was released a few days ago. It still needs some additional testing and polishing. I will only release a new version of this integration using that updated version when that version of pymodbus used by the HA modbus integration is also bumped to that version.

Thanks a lot for the integration, I have been using it for a year with very little problems.
I'm impacted for the issue with the pymodbus versions, because I also have a couple of other modbus devices around (TCP) and now I have to disable this to make the others work.
All the python dependencies can be a pain in the A**, maybe you already answer this, but are you planning or thinking to move this development to an Add-on approach, something like this https://github.com/owagner/modbus2mqtt or this https://github.com/Instathings/modbus2mqtt

Didn't read the codebase (sorry) having it on addon and using mqtt as connection link could be beneficial in terms of dependencies and have more time to adapt in case of changes in HA, like this case.
Just my 2 cents, if you want me to give more details maybe we can open another discussion thread.

[wlcrs]

Maintaining a project like this to a high standard is challenging enough as it is. I have no intention to do anything with an addon and/or MQTT, as:

• that approach would double the amount of maintenance required
• it has its own set of restrictions and pitfalls (ex. how to handle dynamic querying of registers depending on which entities are enabled in HA?)
• it makes it more complex to understand and setup for the average user

For a problem that doesn't affect 99% of the users of this integration, that is too much to ask.

As this is an open source project, everyone is invited to take on this challenge on it's own of course.

Closing as duplicate of #873 which is more active.

[amaduain]

Maintaining a project like this to a high standard is challenging enough as it is. I have no intention to do anything with an addon and/or MQTT, as:

* that approach would double the amount of maintenance required

* it has its own set of restrictions and pitfalls (ex. how to handle dynamic querying of registers depending on which entities are enabled in HA?)

* it makes it more complex to understand and setup for the average user

For a problem that doesn't affect 99% of the users of this integration, that is too much to ask.

As this is an open source project, everyone is invited to take on this challenge on it's own of course.

Closing as duplicate of #873 which is more active.

Makes sense, thanks a lot for the feedback.