Unable to open chests in area as host

[AENykorak]

BC Ver 2.18.3
SV ver 1.6

While host in multiplayer, have recently been unable to press B to open chests in the area I am in.
Requires a game restart to start working again.
Was working fine until recently.
Playing with SVE and Horse Overhaul, amongst other mods.
Finished Grandpas Farm, Desert Bus, and Stable.
Since that day, no longer works.
Not a keybind issue I can see, maybe a mod interaction issue.

EDIT - It occurs on sleep. On a new day host is unable to open chests. Can access inventory, just not chests.
EDIT - It occurs whenever someone takes a horse off the farm
EDIT - Disabling Saddlebags for Horse Overhaul allows access to the chest, this points to an issue with inventories that leave the local area then trying to access the inventory from the chest dropdown.