Merge pull request #133 from agittins/agittins-dev

Docs, Sensor Churn fixes, Unit conversions, Max_Velocity via ConfigFlow

- chore: Documentation updates for new features

    - updated strings in en.json
    - Big readme update
        - new screenshots for deviceinfo and sensor-info
        - Fixed spelling of Shelly and clarified it's the Shelly Plus devices that offer bluetooth proxy.
        - documented new settings, and clarified how some options work.
        - included `recorder` glob filtering tip from @jaymunro

- feat: Reduce sensor churn with update_interval state caching and others

    - Config setting for update_interval now controls how long we can cache or "sit on" an increasing distance sensor before updating the state machine. The backend still updates every second, and uses the most up-to-date values for calcs, but we rate-limit what we send to the UI to lessen database and browser memory impacts. Decreasing measurements are still sent immediately, so only "fading" distances are rate-limited.

    - fixed BermudaSensorEntity not being sub-classed from SensorEntity. This explains why I was having to implement the `state` property. Downside (perhaps) is that now we suffer the sensor class' history dialog which does 5-minute min/max/mean instead of actual data, but this might be better anyway for mobile and tablet clients as it reduces the browser memory footprint, I believe. We still get proper, granular data from the history tool, so no biggie.

    - cleaned up some dead code comments in sensor.py

    - replaced all `state` property implementations with `native_value` instead.

- fix: user-specified units now work, so those using the king's freedom units can feel at home :-)

- feat: Added MAX_VELOCITY to config flow

    - MAX_VELOCITY sets an upper bound on how fast a device can be seen as "travelling away" from a scanner. Readings that imply movement faster than this are ignored, as they are almost certainly noisy. The default of 3m/s (approx 10km/h or 6mph) is a sensible default, but can be tweaked if desired.

    - Added fallback for name in CONF_DEVICES select list to use mac address, in case it's causing items to not show for some users.

custom_components/bermuda/__init__.py

@@ -41,13 +41,15 @@
 from .const import CONF_DEVICES
 from .const import CONF_DEVTRACK_TIMEOUT
 from .const import CONF_MAX_RADIUS
+from .const import CONF_MAX_VELOCITY
 from .const import CONF_REF_POWER
 from .const import CONF_SMOOTHING_SAMPLES
 from .const import CONF_UPDATE_INTERVAL
 from .const import CONFDATA_SCANNERS
 from .const import DEFAULT_ATTENUATION
 from .const import DEFAULT_DEVTRACK_TIMEOUT
 from .const import DEFAULT_MAX_RADIUS
+from .const import DEFAULT_MAX_VELOCITY
 from .const import DEFAULT_REF_POWER
 from .const import DEFAULT_SMOOTHING_SAMPLES
 from .const import DEFAULT_UPDATE_INTERVAL
@@ -328,7 +330,6 @@ def update(
 
             # Verify the new reading is vaguely sensible. If it isn't, we
             # ignore it by duplicating the last cycle's reading.
-            MAX_VELOCITY = 3  # m/s for how fast a device can retreat.
             if len(self.hist_stamp) > 1:
                 # How far (away) did it travel in how long?
                 # we check this reading against the recent readings to find
@@ -369,10 +370,10 @@ def update(
 
             self.hist_velocity.insert(0, velocity)
 
-            if velocity > MAX_VELOCITY:
+            if velocity > self.options.get(CONF_MAX_VELOCITY):
                 if device_address.upper() in self.options[CONF_DEVICES]:
                     _LOGGER.debug(
-                        "This sparrow %s flies too fast (%dm/s), ignoring",
+                        "This sparrow %s flies too fast (%2fm/s), ignoring",
                         device_address,
                         velocity,
                     )
@@ -600,6 +601,7 @@ def __init__(
         # entries yet, so some users might not have this defined after an update.
         self.options[CONF_MAX_RADIUS] = DEFAULT_MAX_RADIUS
         self.options[CONF_SMOOTHING_SAMPLES] = DEFAULT_SMOOTHING_SAMPLES
+        self.options[CONF_MAX_VELOCITY] = DEFAULT_MAX_VELOCITY
 
         if hasattr(entry, "options"):
             # Firstly, on some calls (specifically during reload after settings changes)
@@ -612,6 +614,7 @@ def __init__(
                     CONF_DEVICES,
                     CONF_DEVTRACK_TIMEOUT,
                     CONF_MAX_RADIUS,
+                    CONF_MAX_VELOCITY,
                     CONF_REF_POWER,
                     CONF_SMOOTHING_SAMPLES,
                 ):

custom_components/bermuda/config_flow.py

@@ -14,12 +14,14 @@
 from .const import CONF_DEVICES
 from .const import CONF_DEVTRACK_TIMEOUT
 from .const import CONF_MAX_RADIUS
+from .const import CONF_MAX_VELOCITY
 from .const import CONF_REF_POWER
 from .const import CONF_SMOOTHING_SAMPLES
 from .const import CONF_UPDATE_INTERVAL
 from .const import DEFAULT_ATTENUATION
 from .const import DEFAULT_DEVTRACK_TIMEOUT
 from .const import DEFAULT_MAX_RADIUS
+from .const import DEFAULT_MAX_VELOCITY
 from .const import DEFAULT_REF_POWER
 from .const import DEFAULT_SMOOTHING_SAMPLES
 from .const import DEFAULT_UPDATE_INTERVAL
@@ -117,6 +119,7 @@ async def async_step_globalopts(self, user_input=None):
                 service_info.name
                 or service_info.advertisement.local_name
                 or service_info.device.name
+                or service_info.address
             )
             options.append(
                 {
@@ -162,6 +165,10 @@ async def async_step_globalopts(self, user_input=None):
                 CONF_MAX_RADIUS,
                 default=self.options.get(CONF_MAX_RADIUS, DEFAULT_MAX_RADIUS),
             ): vol.Coerce(float),
+            vol.Required(
+                CONF_MAX_VELOCITY,
+                default=self.options.get(CONF_MAX_VELOCITY, DEFAULT_MAX_VELOCITY),
+            ): vol.Coerce(float),
             vol.Required(
                 CONF_DEVTRACK_TIMEOUT,
                 default=self.options.get(

custom_components/bermuda/const.py

@@ -77,6 +77,11 @@
 CONF_MAX_RADIUS, DEFAULT_MAX_RADIUS = "max_area_radius", 20
 DOCS[CONF_MAX_RADIUS] = "For simple area-detection, max radius from receiver"
 
+CONF_MAX_VELOCITY, DEFAULT_MAX_VELOCITY = "max_velocity", 3
+DOCS[CONF_MAX_VELOCITY] = (
+    "In metres per second - ignore readings that imply movement away faster than this limit. 3m/s (10km/h) is good."  # fmt: skip
+)
+
 CONF_DEVTRACK_TIMEOUT, DEFAULT_DEVTRACK_TIMEOUT = "devtracker_nothome_timeout", 30
 DOCS[CONF_DEVTRACK_TIMEOUT] = (
     "Timeout in seconds for setting devices as `Not Home` / `Away`."  # fmt: skip

custom_components/bermuda/entity.py

@@ -3,14 +3,19 @@
 from __future__ import annotations
 
 from typing import TYPE_CHECKING
+from typing import Any
 
+from homeassistant.components.bluetooth import MONOTONIC_TIME
+from homeassistant.config_entries import ConfigEntry
 from homeassistant.core import callback
 from homeassistant.helpers import area_registry
 from homeassistant.helpers import device_registry as dr
 from homeassistant.helpers.update_coordinator import CoordinatorEntity
 
 from .const import ATTRIBUTION
 from .const import BEACON_IBEACON_DEVICE
+from .const import CONF_UPDATE_INTERVAL
+from .const import DEFAULT_UPDATE_INTERVAL
 from .const import DOMAIN
 
 if TYPE_CHECKING:
@@ -27,13 +32,46 @@ class BermudaEntity(CoordinatorEntity):
     """
 
     def __init__(
-        self, coordinator: BermudaDataUpdateCoordinator, config_entry, address: str
+        self,
+        coordinator: BermudaDataUpdateCoordinator,
+        config_entry: ConfigEntry,
+        address: str,
     ):
         super().__init__(coordinator)
         self.coordinator = coordinator
         self.config_entry = config_entry
         self._device = coordinator.devices[address]
         self.area_reg = area_registry.async_get(coordinator.hass)
+        self.bermuda_update_interval = config_entry.options.get(
+            CONF_UPDATE_INTERVAL, DEFAULT_UPDATE_INTERVAL
+        )
+        self.bermuda_last_state: Any = 0
+        self.bermuda_last_stamp: int = 0
+
+    def _cached_ratelimit(self, statevalue: Any, FastFalling=True, FastRising=False):
+        """Uses the CONF_UPDATE_INTERVAL and other logic to return either the given statevalue
+        or an older, cached value. Helps to reduce excess sensor churn without compromising latency.
+
+        Only suitable for MEASUREMENTS, as numerical comparison is used.
+        """
+
+        nowstamp = MONOTONIC_TIME()
+        if (
+            (
+                self.bermuda_last_stamp < nowstamp - self.bermuda_update_interval
+            )  # Cache is stale
+            or (self.bermuda_last_state is None)  # Nothing compares to you.
+            or (statevalue is None)  # or you.
+            or (FastFalling and statevalue < self.bermuda_last_state)  # (like Distance)
+            or (FastRising and statevalue > self.bermuda_last_state)  # (like RSSI)
+        ):
+            # Publish the new value and update cache
+            self.bermuda_last_stamp = nowstamp
+            self.bermuda_last_state = statevalue
+            return statevalue
+        else:
+            # Send the cached value, don't update cache
+            return self.bermuda_last_state
 
     @callback
     def _handle_coordinator_update(self) -> None:

custom_components/bermuda/sensor.py

@@ -8,6 +8,7 @@
 
 from homeassistant import config_entries
 from homeassistant.components.sensor import SensorDeviceClass
+from homeassistant.components.sensor import SensorEntity
 from homeassistant.components.sensor import SensorStateClass
 from homeassistant.const import SIGNAL_STRENGTH_DECIBELS_MILLIWATT
 from homeassistant.const import STATE_UNAVAILABLE
@@ -23,10 +24,6 @@
 from .const import SIGNAL_DEVICE_NEW
 from .entity import BermudaEntity
 
-# from .const import DEFAULT_NAME
-# from .const import ICON
-# from .const import SENSOR
-
 _LOGGER: logging.Logger = logging.getLogger(__package__)
 
 
@@ -82,7 +79,7 @@ def device_new(address: str, scanners: [str]) -> None:
     await coordinator.async_config_entry_first_refresh()
 
 
-class BermudaSensor(BermudaEntity):
+class BermudaSensor(BermudaEntity, SensorEntity):
     """bermuda Sensor class."""
 
     @property
@@ -103,7 +100,7 @@ def name(self):
         return "Area"
 
     @property
-    def state(self):
+    def native_value(self):
         """Return the state of the sensor."""
         # return self.coordinator.data.get("body")
         return self._device.area_name
@@ -116,11 +113,6 @@ def entity_registry_enabled_default(self) -> bool:
         else:
             return False
 
-    # @property
-    # def icon(self):
-    #    """Return the icon of the sensor."""
-    #    return ICON
-
     @property
     def device_class(self):
         """Return de device class of the sensor."""
@@ -160,7 +152,7 @@ def name(self):
         return "Nearest Scanner"
 
     @property
-    def state(self):
+    def native_value(self):
         return self._device.area_scanner
 
 
@@ -177,8 +169,10 @@ def name(self):
         return "Nearest RSSI"
 
     @property
-    def state(self):
-        return self._device.area_rssi
+    def native_value(self):
+        return self._cached_ratelimit(
+            self._device.area_rssi, FastFalling=False, FastRising=True
+        )
 
     @property
     def device_class(self):
@@ -209,29 +203,16 @@ def name(self):
 
     @property
     def native_value(self):
-        """Define the native value of the measurement."""
+        """Return the native value of the sensor"""
         distance = self._device.area_distance
         if distance is not None:
-            return round(distance, 3)
-        return None
-
-    @property
-    def state(self):
-        """Return the user-facing state of the sensor"""
-        distance = self._device.area_distance
-        if distance is not None:
-            return round(distance, 3)
+            return self._cached_ratelimit(round(distance, 1))
         return None
 
     @property
     def device_class(self):
         return SensorDeviceClass.DISTANCE
 
-    @property
-    def unit_of_measurement(self):
-        """Results are in Metres"""
-        return UnitOfLength.METERS
-
     @property
     def native_unit_of_measurement(self):
         """Results are in metres"""
@@ -275,18 +256,7 @@ def native_value(self):
         devscanner = self._device.scanners.get(self._scanner.address, {})
         distance = getattr(devscanner, "rssi_distance", None)
         if distance is not None:
-            return round(distance, 3)
-        return None
-
-    @property
-    def state(self):
-        """Expose distance to given scanner.
-
-        Don't break if that scanner's never heard of us!"""
-        devscanner = self._device.scanners.get(self._scanner.address, {})
-        distance = getattr(devscanner, "rssi_distance", None)
-        if distance is not None:
-            return round(distance, 3)
+            return self._cached_ratelimit(round(distance, 3))
         return None
 
     @property
@@ -315,19 +285,8 @@ def unique_id(self):
     def name(self):
         return "Unfiltered Distance to " + self._scanner.name
 
-    # @property
-    # def native_value(self):
-    #     """Expose distance to given scanner.
-
-    #     Don't break if that scanner's never heard of us!"""
-    #     devscanner = self._device.scanners.get(self._scanner.address, {})
-    #     distance = getattr(devscanner, "rssi_distance_raw", None)
-    #     if distance is not None:
-    #         return round(distance, 3)
-    #     return None
-
     @property
-    def state(self):
+    def native_value(self):
         """Expose distance to given scanner.
 
         Don't break if that scanner's never heard of us!"""

custom_components/bermuda/translations/en.json

@@ -22,18 +22,22 @@
       "globalopts": {
         "data": {
           "max_area_radius": "Max radius in metres for simple AREA detection",
-          "devtracker_nothome_timeout": "Timeout in seconds to consider a device as `Not Home`.",
-          "update_interval": "How often in seconds to update sensor readings data",
-          "attenuation": "Environment attenuation factor for distance calculation.",
-          "ref_power": "Default rssi at 1 metre distance.",
-          "configured_devices": "List of Bluetooth devices to specifically create tracking entities for.",
-          "smoothing_samples": "How many samples to use for smoothing distance readings."
+          "max_velocity": "Max Velocity in metres per second - ignore readings that imply movement away faster than this limit. 3m/s (10km/h) is good.",
+          "devtracker_nothome_timeout": "Devtracker Timeout in seconds to consider a device as `Not Home`.",
+          "update_interval": "Update Interval - How often (in seconds) to update sensor readings.",
+          "smoothing_samples": "Smoothing Samples - how many samples to use for smoothing distance readings.",
+          "attenuation": "Attenuation - Environment attenuation factor for distance calculation/calibration.",
+          "ref_power": "Reference Power - Default rssi at 1 metre distance, for distance calibration.",
+          "configured_devices": "Configured Devices - Select which Bluetooth devices or Beacons to track with Sensors."
         },
         "data_description": {
           "max_area_radius": "In the simple `AREA` feature, a device will be marked as being in the AREA of it's closest receiver, if inside this radius. If you set it small, devices will go to `unknown` between receivers, but if large devices will always appear as in their closest Area.",
-          "devtracker_nothome_timeout": "How quickly to mark device_tracker entities as `not_home` after we stop seeing advertisements. 30 seconds or more is probably good.",
-          "update_interval": "Distance readings indicating approaches will still trigger immediately, but increasing distances will be rate limited by this to reduce how much your database grows.",
-          "smoothing_samples": "How many samples to average distance smoothing. Bigger numbers make for slower distance increases. 10 or 20 seems good."
+          "max_velocity": "If a reading implies a device is moving away faster than this, we ignore that reading. Humans normally walk at 1.4m/s, if they're holding scissors they move at 3m/s.",
+          "devtracker_nothome_timeout": "How quickly to mark device_tracker entities as `not_home` after we stop seeing advertisements. 30 to 300 seconds is probably good.",
+          "update_interval": "Shortening distances will still trigger immediately, but increasing distances will be rate limited by this to reduce how much your database grows.",
+          "smoothing_samples": "How many samples to average distance smoothing. Bigger numbers make for slower distance increases. Shortening distances are not affected. 10 or 20 seems good.",
+          "attenuation": "After setting ref_power at 1 metre, adjust attenuation so that other distances read correctly - more or less.",
+          "ref_power": "Put your most-common beacon 1 metre (3.28') away from your most-common proxy / scanner. Adjust ref_power until the distance sensor shows a lowest (not average) distance of 1 metre."
         }
       }
     }