Skip to content

Commit

Permalink
Merge pull request #140 from agittins/agittins-dev
Browse files Browse the repository at this point in the history 
feat: iOS / iPhone / Private BLE Device / IRK Support!

- 🚀 feat: iOS / iPhone Support with IRK #135 
    
    - Integrated with the `Private BLE Device` integration. Any devices set up in Private BLE will automatically be configured as Bermuda trackers as well.
    
    - 🐛 Fixed error if no devices are saved for setup (ie, CONF_DEVICES is empty) as may happen if one uses *only* Private BLE Devices.
    
    - Added refresh calls to device registry listener
    
    - Improved some mac_format calls by forcing .lower(), since some "addresses" like private ble will not match a MAC-based pattern.
    
    - Fixed new_device signal dispatcher to use full list of devices and check for create_sensor rather than scanning CONF_DEVICES
    
    - Added Private BLE Devices to the iBeacon code, so both are treated as "meta-devices" in that they have data that is provided via the MAC address, but is not "of" the mac address.
    
    - Fixed case when _refresh_scanners gathers addresses.
    
    - Updated device_info in entity.py to ensure entities tied to private_ble devices get linked to those device_registry entries.

- fix: typedef for device_new callback

- 🚤 feat: Update areas in realtime if scanners are moved improves #137 fix.
    
    - If an esphome or other scanner/proxy device has its "area" modified, we immediately update the Areas without requiring a reload or restart.
    
    - A little comment-linting.

- 🐛 fix: matching of scanner IDs loaded from data (stuck area) fixes #137 
    
    - scanner entries were being saved in data with "our default lower-case" form, but when loading was comparing them to uppercased. This was causing the data stored values to prevent correct loading, so changing the area of a scanner never got applied.

- 📯  chore: Downgraged ibeacon stale source msg to debug
    
    - was set to warning, and #138 shows people are getting this on their systems. I'm not sure why, so adding extra params to log but downgrading to debug since it's spamming people's logs.
  • Loading branch information
@agittins
agittins authored Mar 24, 2024
2 parents 87af4b6 + 6dd494f commit e43ff85
Show file tree
Hide file tree
Showing 5 changed files with 185 additions and 39 deletions.
31 changes: 17 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Bermuda aims to let you track any bluetooth device, and have Homeassistant tell
- Area-based device location (ie, device-level room prescence) is working reasonably well.
- Creates sensors for Area and Distance for devices you choose
- Supports iBeacon devices, including those with randomised MAC addresses (like Android phones running HA Companion App)
- Supports IRK (resolvable keys) via the [Private BLE Device](https://www.home-assistant.io/integrations/private_ble_device/) core component. Once your iOS device (or Android!) is set up in Private BLE Device, it will automatically receive Bermuda sensors as well!
- Creates `device_tracker` entities for chosen devices, which can be linked to "Person"s for Home/Not Home tracking
- Configurable settings for rssi reference level, environmental attenuation, max tracking radius
- Provides a comprehensive json/yaml dump of devices and their distances from each bluetooth
Expand All @@ -51,13 +52,6 @@ Bermuda aims to let you track any bluetooth device, and have Homeassistant tell
out a bit and get a workable impression of where devices and scanners are in relation to each
other. Math / Geometry geeks are very welcome to assist, I am well out of my depth here!

- As yet it doesn't know how to handle iPhones with their rotating MAC addresses, hopefully
we can integrate with [Private BLE Device](https://www.home-assistant.io/integrations/private_ble_device/)
to solve that. We do now support iBeacon, so companion apps such as the one for Android
will now work, even with the rotating MAC-address. iBeacon apps on iOS behave oddly when
backgrounded (an iOS-enforced oddity), so we don't support that either currently. We will
have Private BLE working at some point though, watch this space.

## What you won't need (if this works for you)

- Bermuda provides equivalent functionality to `bluetooth_ble_tracker` by
Expand Down Expand Up @@ -144,10 +138,13 @@ for any person/user.

### Can I track my phone?

- Android: Yes! iPhone: Soon!? Bermuda now supports the iBeacon format, so if you can get your phone
to broadcast iBeacon packets, then yes. The Homeassistant comanion app for
Android does, so it works well.
iPhone will be supported soon by tying in to the `Private BLE Device` integration.
- Yes! Both Android and iOS devices (iPhone, iPad) are supported. iWatch should
also work, providing you can get them showing up in the
[Private BLE Device](https://www.home-assistant.io/integrations/private_ble_device/)
core integration.

- Android: you have the option of using IRK (Private BLE Device) or iBeacon support
via the HA Companion app.

- Bermuda's iBeacon support is rather simplistic and opinionated, reflecting the
author somewhat.
Expand All @@ -170,8 +167,12 @@ for any person/user.

### Why do my bluetooth devices have only the address and no name?

- you can tell your bluetooth proxies to send an inquiry in response to
advertisements, this _might_ cause names to show up.
- You can simply rename your entities if you like.

- You can also tell your bluetooth proxies to send an inquiry in response to
advertisements, this _might_ cause names to show up. Consider also though
that it means while your proxy is asking a device for its name it can't be
listening for BLE traffic to proxy.
In esphome, this is done by adding `active: true` to the
`esp32_ble_tracker` section (this is separate from the active property of
the `bluetooth_proxy` section, which controls outbound client connections).
Expand Down Expand Up @@ -216,7 +217,9 @@ So how does that help?

- There are three main factors.
- How often your beacon transmits advertisements. Most are less than 2 seconds.
- Bermuda only checks for new advertisements every second.
- Bermuda only checks for new advertisements every second. It will update sensors
immediately if devices get "closer", but it is more leisurely for devices that
appear to be "leaving" an area.
- The proxies might not catch every advertisement. In my esphome proxies I usually
use these settings to ensure we don't miss toooo many:
```yaml
Expand Down
149 changes: 128 additions & 21 deletions custom_components/bermuda/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,16 @@
from homeassistant.const import STATE_NOT_HOME
from homeassistant.const import STATE_UNAVAILABLE
from homeassistant.core import Config
from homeassistant.core import Event
from homeassistant.core import HomeAssistant
from homeassistant.core import SupportsResponse
from homeassistant.core import callback
from homeassistant.exceptions import ConfigEntryNotReady
from homeassistant.helpers import area_registry
from homeassistant.helpers import config_validation as cv
from homeassistant.helpers import device_registry as dr
from homeassistant.helpers import entity_registry as er
from homeassistant.helpers.device_registry import EVENT_DEVICE_REGISTRY_UPDATED
from homeassistant.helpers.device_registry import DeviceEntry
from homeassistant.helpers.device_registry import format_mac
from homeassistant.helpers.dispatcher import async_dispatcher_send
Expand All @@ -37,6 +42,8 @@
from .const import BEACON_IBEACON_DEVICE
from .const import BEACON_IBEACON_SOURCE
from .const import BEACON_NOT_A_BEACON
from .const import BEACON_PRIVATE_BLE_DEVICE
from .const import BEACON_PRIVATE_BLE_SOURCE
from .const import CONF_ATTENUATION
from .const import CONF_DEVICES
from .const import CONF_DEVTRACK_TIMEOUT
Expand All @@ -53,8 +60,10 @@
from .const import DEFAULT_REF_POWER
from .const import DEFAULT_SMOOTHING_SAMPLES
from .const import DEFAULT_UPDATE_INTERVAL
from .const import DEVICE_TRACKER
from .const import DISTANCE_TIMEOUT
from .const import DOMAIN
from .const import DOMAIN_PRIVATE_BLE_DEVICE
from .const import HIST_KEEP_COUNT
from .const import PLATFORMS
from .const import SIGNAL_DEVICE_NEW
Expand Down Expand Up @@ -198,7 +207,8 @@ def update_advertisement(
if hasattr(scandata.scanner, "_discovered_device_timestamps"):
# Found a remote scanner which has timestamp history...
self.scanner_sends_stamps = True
# FIXME: Doesn't appear to be any API to get this otherwise...
# There's no API for this, so we somewhat sneakily are accessing
# what is intended to be a protected dict.
# pylint: disable-next=protected-access
stamps = scandata.scanner._discovered_device_timestamps

Expand Down Expand Up @@ -396,7 +406,7 @@ def calculate_data(self):
self.hist_velocity.insert(0, velocity)

if velocity > self.options.get(CONF_MAX_VELOCITY):
if self.parent_device.upper() in self.options[CONF_DEVICES]:
if self.parent_device.upper() in self.options.get(CONF_DEVICES, []):
_LOGGER.debug(
"This sparrow %s flies too fast (%2fm/s), ignoring",
self.parent_device,
Expand Down Expand Up @@ -644,6 +654,43 @@ def __init__(
# forcing a scan of the captured info.
self._do_full_scanner_init = True

# First time go through the private ble devices to see if there's
# any there for us to track.
self._do_private_device_init = True

@callback
def handle_devreg_changes(ev: Event):
"""Update our scanner list if the device registry is changed.
This catches area changes (on scanners) and any new/changed
Private BLE Devices."""
# We could try filtering on "updates" and "area" but I doubt
# this will fire all that often, and even when it does the difference
# in cycle time appears to be less than 1ms.
_LOGGER.debug(
"Device registry has changed, we will reload scanners and Private BLE Devs. ev: %s",
ev,
)
# Mark so that we will rebuild scanner list on next update cycle.
self._do_full_scanner_init = True
# Same with Private BLE Device entities
self._do_private_device_init = True

# Let's kick off a scanner and private_ble_device scan/refresh/init
self._refresh_scanners([], self._do_full_scanner_init)
self.configure_beacons()

# If there are no `CONFIGURED_DEVICES` and the user only has private_ble_devices
# in their setup, then we might have done our init runs before that integration
# was up - in which case we'll get device registry changes. We should kick off
# the update in case it's not running yet (because of no subscribers yet being
# attached to the dataupdatecoordinator).
self.hass.add_job(self._async_update_data())

# Listen for changes to the device registry and handle them.
# Primarily for when scanners get moved to a different area.
hass.bus.async_listen(EVENT_DEVICE_REGISTRY_UPDATED, handle_devreg_changes)

self.options = {}

# TODO: This is only here because we haven't set up migration of config
Expand Down Expand Up @@ -718,7 +765,7 @@ def device_tracker_created(self, address):

def _get_device(self, address: str) -> BermudaDevice:
"""Search for a device entry based on mac address"""
mac = format_mac(address)
mac = format_mac(address).lower()
# format_mac tries to return a lower-cased, colon-separated mac address.
# failing that, it returns the original unaltered.
if mac in self.devices:
Expand All @@ -728,7 +775,7 @@ def _get_device(self, address: str) -> BermudaDevice:
def _get_or_create_device(self, address: str) -> BermudaDevice:
device = self._get_device(address)
if device is None:
mac = format_mac(address)
mac = format_mac(address).lower()
self.devices[mac] = device = BermudaDevice(
address=mac, options=self.options
)
Expand Down Expand Up @@ -877,6 +924,8 @@ async def _async_update_data(self):
if scanner_device is None:
# The receiver doesn't have a device entry yet, let's refresh
# all of them in this batch...
self._do_full_scanner_init = True
self._do_private_device_init = True
self._refresh_scanners(matched_scanners, self._do_full_scanner_init)
self._do_full_scanner_init = False
scanner_device = self._get_device(discovered.scanner.source)
Expand All @@ -897,8 +946,8 @@ async def _async_update_data(self):

# END of per-advertisement-by-device loop

# Scanner entries have been loaded up with latest data, now we can process data for all devices
# over all scanners.
# Scanner entries have been loaded up with latest data, now we can
# process data for all devices over all scanners.
for device in self.devices.values():
# Recalculate smoothed distances, last_seen etc
device.calculate_data()
Expand All @@ -921,30 +970,61 @@ async def _async_update_data(self):
# so let's ensure any devices that we create sensors for are set up ready to go.
# We don't do this sooner because we need to ensure we have every active scanner
# already loaded up.
for address in self.options.get(CONF_DEVICES, []):
device = self._get_device(format_mac(address.lower()))
if device is not None:
for address, device in self.devices.items():
if device.create_sensor:
if not device.create_sensor_done or not device.create_tracker_done:
_LOGGER.debug(
"Firing device_new for %s (%s)", device.name, device.address
)
_LOGGER.debug("Firing device_new for %s (%s)", device.name, address)
async_dispatcher_send(
self.hass, SIGNAL_DEVICE_NEW, device.address, self.scanner_list
self.hass, SIGNAL_DEVICE_NEW, address, self.scanner_list
)

# end of async update

def configure_beacons(self):
"""Create iBeacon and other meta-devices from the received advertisements
"""Create iBeacon, Private_BLE and other meta-devices from the received advertisements
Note that at this point all the distances etc should be fresh for
the source devices, so we can just copy values from them to the beacon metadevice.
"""

entreg = er.async_get(self.hass)
devreg = dr.async_get(self.hass)

# ### Seed the Private BLE Device entries from the other integration
if self._do_private_device_init:
# Iterate through the Private BLE Device integration's entities,
# and ensure for each "device" we create a source device.
self._do_private_device_init = False
_LOGGER.debug("Refreshing Private BLE Device list")
for entity_id in self.hass.states.async_entity_ids(DEVICE_TRACKER):
# The device_tracker entity in private ble has no suffix on its unique_id.
if f"{DEVICE_TRACKER}.{DOMAIN_PRIVATE_BLE_DEVICE}" in entity_id:
# We have a private_ble_device to track!
pb_entity = entreg.async_get(entity_id)
pb_device = devreg.async_get(pb_entity.device_id)
pb_state = self.hass.states.get(entity_id)
pb_address = pb_state.attributes.get(
"current_address", "broken_address"
).lower()
if pb_address:
pble_source_device = self._get_or_create_device(pb_address)
pble_source_device.name = (
pb_device.name_by_user or pb_device.name
)
pble_source_device.prefname = (
pb_device.name_by_user or pb_device.name
)
pble_source_device.beacon_type = BEACON_PRIVATE_BLE_SOURCE
pble_source_device.beacon_unique_id = pb_entity.unique_id

# First let's find the freshest device advert for each Beacon unique_id
# Start keeping a winners-list by beacon/pb id
freshest_beacon_sources: dict[str, BermudaDevice] = {}

# Iterate through each device to see if it has an advert for our target id
for device in self.devices.values():
if device.beacon_type == BEACON_IBEACON_SOURCE:
if device.beacon_type in [BEACON_IBEACON_SOURCE, BEACON_PRIVATE_BLE_SOURCE]:
# We found an advert for our beacon of interest...
if (
device.beacon_unique_id not in freshest_beacon_sources # first-find
or device.last_seen
Expand All @@ -955,12 +1035,20 @@ def configure_beacons(self):
# then we are the freshest!
freshest_beacon_sources[device.beacon_unique_id] = device

# We now have a dict of the freshest device for each beacon/pb id.
# Now let's go through the freshest adverts and set up those beacons.
for beacon_unique_id, device in freshest_beacon_sources.items():
# Copy this device's info to the meta-device for tracking the beacon

metadev = self._get_or_create_device(beacon_unique_id)
metadev.beacon_type = BEACON_IBEACON_DEVICE
if device.beacon_type == BEACON_IBEACON_SOURCE:
metadev.beacon_type = BEACON_IBEACON_DEVICE
elif device.beacon_type == BEACON_PRIVATE_BLE_SOURCE:
metadev.beacon_type = BEACON_PRIVATE_BLE_DEVICE
else:
_LOGGER.warning(
"Invalid beacon type for freshest beacon: %s", device.beacon_type
)

# anything that isn't already set to something interesting, overwrite
# it with the new device's data.
Expand Down Expand Up @@ -1010,7 +1098,16 @@ def configure_beacons(self):
if device.last_seen > metadev.last_seen:
metadev.last_seen = device.last_seen
elif device.last_seen < metadev.last_seen:
_LOGGER.warning("Using freshest advert but it's still too old!")
# FIXME: This is showing up for some people
# (see https://github.com/agittins/bermuda/issues/138)
# but I can't see why. Downgrading to debug since it
# doesn't seem to affect ops, and adding
# params so I can perhaps get more info later.
_LOGGER.debug(
"Using freshest advert from %s for %s but it's still too old!",
device.name,
metadev.name,
)
# else there's no newer advert

if device.address not in metadev.beacon_sources:
Expand All @@ -1020,7 +1117,13 @@ def configure_beacons(self):
del metadev.beacon_sources[HIST_KEEP_COUNT:]

# Check if we should set up sensors for this beacon
if metadev.address.upper() in self.options.get(CONF_DEVICES, []):
if (
# iBeacons need to be specifically enabled:
metadev.address.upper() in self.options.get(CONF_DEVICES, [])
# But Private BLE Devices have already been deliberately configured
# by the user, so we always just enable them
or metadev.beacon_type == BEACON_PRIVATE_BLE_DEVICE
):
# This is a meta-device we track. Flag it for set-up:
metadev.create_sensor = True

Expand Down Expand Up @@ -1105,14 +1208,14 @@ def _refresh_scanners(
update_scannerlist = False

for scanner in scanners:
addresses.add(scanner.scanner.source.upper())
addresses.add(scanner.scanner.source.lower())

# If we are doing a full scan, add all the known
# scanner addresses to the list, since that will cover
# the scanners that have been restored from config.data
if do_full_scan:
for address in self.scanner_list:
addresses.add(address)
addresses.add(address.lower())

if len(addresses) > 0:
# FIXME: Really? This can't possibly be a sensible nesting of loops.
Expand All @@ -1121,14 +1224,16 @@ def _refresh_scanners(
for dev_entry in self.hass.data["device_registry"].devices.data.values():
for dev_connection in dev_entry.connections:
if dev_connection[0] in ["mac", "bluetooth"]:
found_address = dev_connection[1].upper()
found_address = format_mac(dev_connection[1])
if found_address in addresses:
scandev = self._get_device(found_address)
if scandev is None:
# It's a new scanner, we will need to update our saved config.
_LOGGER.debug("New Scanner: %s", found_address)
update_scannerlist = True
scandev = self._get_or_create_device(found_address)
# Found the device entry and have created our scannerdevice,
# now update any fields that might be new from the device reg:
scandev_orig = scandev
scandev.area_id = dev_entry.area_id
scandev.entry_id = dev_entry.id
Expand All @@ -1145,6 +1250,8 @@ def _refresh_scanners(
scandev.name,
)
scandev.is_scanner = True
# If the scanner data we loaded from our saved data appears
# out of date, trigger a full rescan of seen scanners.
if scandev_orig != scandev:
# something changed, let's update the saved list.
_LOGGER.debug(
Expand Down
Loading

0 comments on commit e43ff85

Please sign in to comment.