file distribution in a disrupted network

[anarcat]

one issue with managing a distributed network in a difficult environment like this is file distribution. we're talking about many different problem spaces here, but i think they could have common solutions:

• general file sharing (e.g. "send me that document" problem)
• software updates (e.g. "i need to deploy a new server" or "security upgrades")

For packaging and software updates, the current approach (docker and containers in general) is a little unreliable. It assumes a central Docker hub and while it's possible to run proxies and/or local registries, it's kind of a challenge to deploy those as those programs are yet another system to maintain and deploy. And while Docker containers are fundamentally tar files, they are rarely distributed as such, and even then, they are made of multiple tar files, which further complicates deployments.

We should instead focus on existing Linux distributions as a basis for software that will be deployed. Those institutions have decades of experience in shipping ready-made and integrated software across the world, from an era where internet connectivity was very limited. Think "smuggling a DVD" or "download an ISO over satellite" as solutions to this problem. This requires severely reconsidering the current technological choices but, in the case of Matrix, is not that big of a deal because Matrix is already packaged in Debian.

For the more general file sharing problem, I suggest you start sharing copies (or at least pointers to) the Syncthing project. It's a fully distributed system that could easily survive DNS and network outages. In case of a total network outages, you'd need to rewire some nodes into directory and relay servers to bypass NAT issues, but that's a somewhat solveable problem. Otherwise Syncthing does local peer discovery to find peers on the local network, which allows for very fast file sharing using only the local LAN.

The other project that might be worth considering here is the NNCP project, which goes back to the really old, old-school mechanisms of sharing files and emails (remember UUCP?). Its approach is basically to assume a completely disconnected network while still ensuring end-to-end trust. It could be built on top of basically anything, from syncthing to snail mail while including satellite links.

Anyways, that's what I got so far, I hope i'm not misusing that issue tracker. :)

[anarcat]

For packaging and software updates, the current approach (docker and containers in general) is a little unreliable. It assumes a central Docker hub

just to clarify this, what i mean here is the docker-compose.yml file has this line:

    image: equalitie/ceno-client:latest

which implicitly assumes the existence of hub.docker.com. now there might be way to change that default upstream to use another registry, there is, as far as I know, no direct way to tell docker-compose to use another registry, so that's already a huge SPOF in the architecture.