nibe-mqtt-gateway

nibe-mqtt-gateway is an MQTT integration for Nibe heatpumps.

It is used to integrate a Nibe VVM310/S2125 into Home Assistant via Mosquitto as MQTT broker. Additionally, nibe-mqtt-gateway publishes heapump monitoring data as Prometheus metrics, provides 4 relays that can be used to control the Nibe AUX inputs and it can count the electrical energy consumption via an S0 interface.

Features

• connection to Nibe heatpump VVM310/S2125 via RS485
• wired Ethernet (no Wifi needed nor supported)
• direct connection to MQTT broker
• configurable set of of published Nibe registers
• supports Modbus Data Messages (fast reading of up to 20 registeres preconfigured by Modbus Manager, no 32 bit registers)
• supports writing to Nibe registers
• energy meter connected via S0 interface to OptIn1, persisted in NVS
• energy consumption metrics split by Nibe operation mode (register 43086 aka prio)
• exposes the 4 relays of the PRODIno ESP32 board via MQTT
• supports Home Assistant MQTT auto-discovery
• simple web UI for info and administration
• OTA updates (well, over Ethernet)
• upload of configuration files including the ModbusManager CSV file
• metrics via Prometheus endpoint
• nibe registers and other measurements as Prometheus metrics
• logging via MQTT topic (as alternative to serial interface)
• automatic safe-boot mode when ending up in crash loop

Prerequisites

• PRODINo ESP32 Ethernet v1 board - other boards might be possible but have not been tested
• USB adapter for debugging (via serial output, no real JTAG debugging unfortunately) and initial uploading of firmware
• Nibe heatpump VVM310/S2125 - other models might work but have not been tested
• ESP-IDF v5.3.2
• MQTT broker like Mosquitto
• energy meter with S0 interface, e.g. DRT 428D
• a safe, well-protected home network. nibe-mqtt-gateway is lacking even the most basic security measures. Never expose nibe-mqtt-gateway to the internet.

Additionally helpful:

• Visual Studio Code + ESP-IDF Visual Studio Code Extension

Installation

Wiring

Build Firmware

• install ESP-IDF v5.3.2
• clone this project
• adapt settings, especially idf.espIdfPath to point to ESP-IDF installation
• generate sdkconfig and adapt if necessary
• build project, see also Development section below

Result is a firmware file: ./build/nige-mqtt-gateway.bin

Installation

For initial installation, flash the firmware using a serial adapter.

For subsequent installations, the firmware can be uploaded via OTA: http://nibegw/update

Configuration

The configuration consists of:

• general configuration json config.json like MQTT broker url and credentials, heatpump registers to be polled, logging etc.
• a Nibe ModbusManager file that defines all available registers, nibe-modbus.csv
• the Energy Meter value can be set to adjust it with the real meter reading

When uploading a configuration file, nibe-mqtt-gateway stores it in flash memory and reboots to activate the configuration change.

General configuration:

• see config/config.json.template for format and configuration options, json file allows comments
• http://nibegw/config shows the current configuration as uploaded
• http://nibegw/config?runtime=true shows the current runtime configuration (internal data structures translated back to json, for debugging)
• upload config.json: curl -X POST -H "Content-Type: application/json" --data-binary @config.json http://nibegw/config

Nibe Modbus configuration:

• http://nibegw/config/nibe shows the current nibe modbus configuration. A csv file in Nibe ModbusManager format.
• use Nibe ModbusManager to get a CSV with all registers. Save e.g. as config/nibe-modbus-vvm310.csv.
• upload nibe-modbus-vvm310.csv: curl -F "[email protected]" http://nibegw/config/nibe

Energy Meter configuration (also via UI):

• adjusting energy meter
  • curl -X POST -H "Content-Type: application/json" -d <energy in Wh> http://nibegw/config/energymeter
  • to prevent misconfiguration: value can only be changed by +-10kWh
  • increases are set immediately
  • decreases are waited, i.e. energy counter stops counting for the diff to avoid breaking counter metrics
• set energy counter to an initial value without any checks - can break counter metrics
  • curl -X POST -H "Content-Type: application/json" -d <energy in wh> http://nibegw/config/energymeter?init=true
• no reboot when adjusting the energy meter

Trouble Shooting

The RGB multi-functional LED shows the status of nibe-mqtt-gateway:

• blue - initializing
• blue blinking - waiting for IP address
• green blinking - running ok, connected with MQTT broker
• red blinking - error, got IP address but e.g. not connected with MQTT broker, check http://nibegw or logs
• orange blinking - OTA firmware upload in progress

Web UI:

• http://nibegw - main page with some status info and several config options
• http://nibegw/update - OTA update and upload of file system
• http://nibegw/metrics - Prometheus endpoint with some insights like heap, uptime and execution times (in addition to heatpump metrics)

Depending on the configuration, logs are available via Serial interface or the MQTT topic nibegw/log. Show logging over MQTT:

mosquitto_sub --url 'mqtt://<user>:<password>@<broker>/nibegw/log'

Log levels can be configured via config.json (see above):

• standard ESP logging config applies, especially CONFIG_LOG_MAXIMUM_LEVEL
• default log level is info, unless changed in sdkconfig
• nibe-mqtt-gateway sources are complied with LOG_LOCAL_LEVEL=ESP_LOG_DEBUG to allow debug logging
• log levels can be temporarily changed via UI or curl
  • curl -X POST -H "Content-Type: application/x-www-form-urlencoded" -d "tag=<tag>&level=<none|error|war|info|debug|verbose>" http://nibegw/config/log
  • log levels are set back to config.json settings after reset

After 3 fast crashes in a row, nibe-mqtt-gateway boots into a safe-mode that should allow to upload a fixed/working firmware via OTA:

• only OTA upload is supported
• no mqtt, nibegw, energy meter, etc.
• Nibe RS485 protocol is maintained so that the heat pump should not go into alarm state (received data is acknowledged but ignored)
• init status is shown as 0x0001 (= InitStatus::SafeBoot)
• metrics other than nibegw_status_info are missing
• logs are only available via serial interface

MQTT Topics and Metrics

Description assumes standard configuration, see config.json.template:

• mqtt.rootTopic = nibegw
• discoveryPrefix = homeassistant
• logging.logTopic = nibegw/log

General

Description	MQTT topic	MQTT discovery	Metric	Comment
Device availability	nibegw/availability			last will topic
Logs	nibegw/log			see trouble shooting section
Status nibegw initialization			nibegw_status_info {category="init"}	0=OK, otherwise check logs
Status nibegw MQTT			nibegw_status_info {category="mqtt"}	0=OK, otherwise check logs
Time to poll Nibe registers			nibegw_task_runtime_seconds {task="publishNibeRegisters"}	~1s per polled register
Runtime for 30s cyclic task			nibegw_task_runtime_seconds {task="pollingTask"}	should be <1s
Free heap			nibegw_total_free_bytes
Minumum free heap			nibegw_minimum_free_bytes
Uptime			nibegw_uptime_seconds_total	reset on boot
Boot count			nibegw_boot_count	To detect crash loops, reset after 30s

Nibe Registers

Description	MQTT topic	MQTT discovery	Metric	Comment
Nibe register read	nibegw/nibe/<id>	homeassistant/sensor/nibegw/ nibe-<id>/config	nibe_<title> {register="<id>"}	Metric name is configurable
Nibe register write	nibegw/nibe/<id>/set	homeassistant/switch/nibegw/ nibe-<id>/config		only for R/W registers

Legend/Info

• <id> = Nibe register ID
• <title> = Nibe register title
• sensor and switch in MQTT discovery topic are example components and can be configured
• metric names are API and should be overridden in configuration, see config.json.template for examples

Energy Meter

Description	MQTT topic	MQTT discovery	Metric	Comment
Energy Meter	nibegw/energy-meter	homeassistant/sensor/nibegw/ energy-meter/config	nibe_energy_meter_wh_total	absolute value, stored in NVS
Energy consumption per Nibe operating mode/prio			nibe_energy_consumption_wh_total {mode="unknown|off|heating|hotwater|cooling"}	metric reset on reboot

Relays

Description	MQTT topic	MQTT discovery	Metric	Comment
Relay 1..4	nibegw/relay/<name>/state	homeassistant/switch/nibegw/ <name>/config	nibegw_relay_state {relay="1..4",name="<name>"}	relay state
Relay 1..4	nibegw/relay/<name>/set	homeassistant/switch/nibegw/ <name>/config		set state

Legend/Info

• <name> = relay name is configurable, defaults to relay-1..4

Development

Build

Visual Studio Code + ESP-IDF Visual Studio Code Extension is used as IDE.

• switch target to esp32
• delete sdkconfig file
• 'Full Clean' (delete build directory)
• 'Build' project
• Upload to Prodino board either via serial interface or OTA
• see above for configuration and trouble shooting

Unit Testing

Parts of the code base have unit tests using Unity that can run on the Linux/Mac (see ESP-IDF - Running Applications on Host).

How to run unit tests using vscode:

• switch target to linux
• 'Full Clean' (delete build directory)
• 'Build' project, ignore the error by esp_idf_size
• 'Monitor' runs the unit tests and prints results to console
  • attention: 'Monitor' doesn't build -> an explicit 'Build' is needed after any source code change

Implementation:

• tests are located in a special test component, this component picks source files from main that can be tested
• any code that depends on arduino-esp32 can't run on the host emulation and therefore is not testable easily, this includes any usage of the Arduino String class

Tips and Tricks

Home Assistant Switches for writing to Nibe

Nibe R/W registers have a state and a command topic. When a switch in HA sends a command, it takes some time until the new state is confirmed via the state topic depending on how the register is polled:

• ~20s for registers preconfigured by Modbus Manager (Nibe Data Messages are sent every 2s and 2 registers are published to MQTT for every received data message)
• 30s for registers configured in pollRegisters
• n * 30s for registers configured in pollRegistersSlow (n = length of pollRegistersSlow array)

As a result, the toggle button in HA UI may flicker/bounce. To prevent this, configured optimistic: true in homeassistantDiscoveryOverrides and add the following customization to your HA configuration.yaml:

homeassistant:
  customize_domain:
    # show switches as toggle and never as two lightning buttons
    switch:
      assumed_state: false

To Do

• smoother unit testing development cycle: vscode task, less Arduino dependencies
• debugging, at least for unit tests on host
• authentication for admin operations (OTA, config uploads etc.)
• replace arduino-esp32 by native esp-idf (less dependencies, should save up to 90k)

Credits

Used source code and libraries

• arduino-esp32 - Arduino core for the ESP32, LGPL v2.1
• ArduinoJson - JSON Library, MIT License
• esp-idf - Espressif IoT Development Framework, Apache License 2.0
• esp-idf-net-logging - Redirect esp-idf logging to the network, MIT License
• org.openhab.binding.nibeheatpump - nibegw, Eclipse Public License 2.0
• ProDinoESP32 - KMP ProDino ESP32 examples library
• Unity - Unit testing for C, MIT License

Inspiration and ideas

• nibe-mqtt - Nibe MQTT integration, GPL 3.0

License and Disclaimer

This project has been developed for personal use only. The code is provided AS IS under the Apache License v2.0 in the hope that it will be useful, but WITHOUT ANY WARRANTY whatsoever.

Nibe is registered mark of NIBE Energy Systems.