Suggestion: remove spanning tree

[Arceliar]

Is your feature request related to a problem? Please describe.
Not a problem exactly, but a bit of a design inconsistency. At present, the softstate branch only uses the spanning tree in snek next hop lookups, and only in cases where a snek path is not known. This is what makes the root reachable to the rest of the DHT, but outside of that 1 node it's strictly optional. Building and maintaining a spanning tree of the network (with per hop signatures) is a lot of extra work, and increases the potential attack surface, just to make 1 specific node reachable.

Describe the solution you'd like
Remove the spanning tree entirely, and replace it with broadcast snek bootstraps from the "root" (now a misnomer). Basically, if you try to look up a next hop for a bootstrap, and you don't find any snek path that would lead to a valid next address in keyspace, then send the snek bootstrap to all peers (minus the one you received it from).

Describe alternatives you've considered
You could keep using the spanning tree.

Additional context
This is something I've prototyped in ironwood. In the ironwood/yggdrasil context, we also use the tree to do opportunistic greedy routing through treespace, so we have a good reason to maintain and use all that extra hard state, which is why I'm not going this direction in my own project for now. Pinecone doesn't use greedy routing through treespace, and (as far as I've heard) has no immediate plans to implement that. If you're not explicitly using the tree as a tree (and just using it as a way to ensure the root is reachable), then there's no technical reason why you couldn't remove it.

There are a number of benefits to removing the tree:

1. The code is simpler. In my code base, there was maybe a dozen or so lines of code involved in broadcasting the "root" DHT bootstraps through the network, instead of forwarding them towards 1 other node. That can replace all of the tree related code. Pinecone may need slightly more code to handle this, or to slightly change some behaviors about how DHT state is stored to minimize the amount of "root"-specific / broadcast code. Either way, I think this will make the protocol much simpler to describe and re-implement for other homeservers.
2. This removes the remaining hard state. In particular, it means that pinecone would no longer have any protocol traffic which requires reliable or ordered transmission (even if both properties are still nice to have). That would make it safer to run over networks where ordering isn't guaranteed (bluetooth?) and may open the door to other transport protocols (e.g. even plain old UDP, if you can handle message fragmentation).
3. It's probably more secure. There are likely some attacks that a root could execute, which other nodes could not, due to the way the root is treated differently from all other nodes in the network (being reachable only because of the hard state tree). I'd rather not go into detail about these here, but suffice it to say that removing the tree solves basically all of these issues. To be honest, in the context of matrix-over-pinecone, this would cover every remaining attack that I know of that isn't addressable more easily on the matrix side.

There are also some potential disadvantages:

1. You don't have the tree, so you can't use opportunistic greedy routing through treespace. If that wasn't on the agenda anyway, then this is no real loss. The main incentive to use greedy routing through treespace is to reduce stretch. For the matrix-over-pinecone use case, I think that's largely not an issue, since most nodes will still probably have an internet connection available (so they can directly peer with the home servers they care about, and have one hop paths to those nodes).
2. The lack of a tree means you can't use "my treespace coordinates changed" as a lazy way for nodes to determine that they are likely to be mobile (in the "moving around the network" sense, not the "running on a phone" sense). That could be useful information to know, so you would need to find another way to detect it (maybe by monitoring how your node's descending path changes -- if it times out, then you may have become unreachable due to mobility, for example). As far as I know, this is also not something pinecone currently cares about.
3. It may be slightly more expensive or slightly less performant in mobile networks. I spent exactly 0 time trying to optimize that in my ironwood prototype, so it's something you'd probably need to look into and experiment with.

I may try to throw together a PR for this myself, if nobody else has interest / gets to it before me.