Replacement for bootstrap script in IPFS

[darkdrgn2k]

mDNS is available on libp2p but does not seem to work

Investigate if its functioning, if so see if we can craft something that will work over CJDNS

[benhylau]

I am confused by title of this ticket, is this ipfs specific?

[makew0rld]

@benhylau yes, it's for IPFS peering over CJDNS, using mDNS.

[benhylau]

@makeworld-the-better-one How can mDNS replace unicasting to cjdns peers? mDNS will only work in local networks, if you have multiple nodes in your home. We'd still need peering over cjdns interface address to link distant nodes.

I am imagining:

1. Offline network, like Meshstream demo or Workshop use case
2. Home node with IPFS peerings over Internet
3. Home node with IPFS over cjdns even without Internet

Which case(s) is this trying to solve?

[darkdrgn2k]

How can mDNS replace unicasting to cjdns peers

You dont replace mDNS.

mDNS is a service that uses Multicasts/Broadcasts to LAN connected devices. So you are right it mDNS packets would never reach CJDNS targets because they would never be sent to them

However if mDNS does work on a LAN (the receiving device can understand and act upon a mDNS packet) we can use that to our advantage.

From the device that receives the packets, it does not really care how the packet got there (if it was multicast or broadcast or even unicast) it just knows it received a packet.

The idea is to craft a valid mDNS packet but instead of multicasting it to a lan (that would not work) unicast it to each peered cjdns addresses.

But this is all speculation, needs to be looked into and tested but if successful we can use a built-in mechanism in IPFS to convince it to peer with its cjdns neighbors.

First step is to see if mDNS works over a LAN in IPFS.

[benhylau]

Thanks for clarifying @darkdrgn2k. I feel like @jedahan may be interested in this discussion too. There were brief conversations about a generalized module with plugin-based architecture that peers peer-to-peer apps (e.g. ssb, ipfs) over a cjdns-like network via crafting of mDNS packets that are really IPv6 unicast to direct cjdns peers, as we have done in Meshstream.

[benhylau]

Proposing Unicast to bootstrap IPFS over cjdns peers as title?

[jedahan]

1. For ipfs.service, we ask cjdns for a peerlist, and unicast the appropriate bencoded udp command for adding a peer.
2. For ssb.service, we ask cjdns for a peerlist, and unicast the appropriate json udp advertisement for adding a peer.

The tricky parts are that its kinda hard to hook into the cjdns peerlist. Maybe it would be a nice feature to be able to ask cjdns to give a streaming response of new peers when its up.

The more generic, complicated, and probably less useful version:

1. Have ipfs.service, ssb.service etc do mDNS broadcasts.
2. Create an mDNS responder for broadcasts, that forward them over cjdns unicast.
3. Have ipfs-mdns-listen.service, ssb-mdns-listen.service that translates mDNS records received over unicast to appropriate commands/advertisements locally.

[darkdrgn2k]

The tricky parts are that its kinda hard to hook into the cjdns peerlist.

Using existing tools

while read -r cjdns_peer; do
    cjdns_addr=$(sudo /opt/cjdns/publictoip6 $cjdns_peer)

done << `sudo nodejs /opt/cjdns/tools/peerStats 2>/dev/null | awk '{ if ($2 == "ESTABLISHED") print $1 }' | awk -F. '{ print $6".k" }' | xargs`

Hooking into API instead to get json

./cexec "InterfaceController_peerStats(0)"

Maybe it would be a nice feature to be able to ask cjdns to give a streaming response of new peers when its up.

Yes but it may be low on his priority list. But im thinking maybe a "spam" like ssb does would work

[darkdrgn2k]

First things we need to do is see if mDNS even WORKS with IPFS.

[jedahan]

I am proposing we write our own advertisements so we don't have to add support to cjdns

[darkdrgn2k]

Yes i was thinking about that as well. Writing our own UDP6 Unicast advertisement mechanism is trivial even in bash.

Server:

socat - udp6-listen:1234,reuseaddr,fork | while read LINE
do 
    echo "Do something with  ${LINE}"; 
done 

Broadcast

while read -r cjdns_peer; do
    cjdns_addr=$(sudo /opt/cjdns/publictoip6 $cjdns_peer)
    echo "hello i am $cjdns_peer do something with me" |  socat - UDP6-datagram:[$cjdns_addr]:1234
done <<< `sudo nodejs /opt/cjdns/tools/peerStats 2>/dev/null | awk '{ if ($2 == "ESTABLISHED") print $1 }' | awk -F. '{ print $6".k" }' | xargs`

Although this is my "last resort" way (better than what we currently do)

I'd rather have the "receiver" use some sort of native implementation, so that its not depended on our own services being installed.

That way if you run an plain vanilla IPFS without any of our "special sauce", I can still get it to peer.

Otherwise it would only work if our services are running on both.

This is why i want to investigate if

• mDNS receiver actually work in IFPS
• can we exploit mDNS packets to trigger ipfs to peer with us

Once you are able to complete your work on UDP6 Multicast SSB Component, we can use the same unicast code as above to push the unicast udp6 over cjdns to peer SSB that way.

[jedahan]

If we have the ipfs daemon running with admin port open, we wouldn't even need an mdns responder:

while read -r cjdns_peer; do
    cjdns_addr=$(sudo /opt/cjdns/publictoip6 $cjdns_peer)
    curl "http://${cjdns_peer}:5001/api/v0/bootstrap/add?arg=/ip6/${cjdns_addr}/tcp/1234"
done <<< `sudo nodejs /opt/cjdns/tools/peerStats 2>/dev/null | awk '{ if ($2 == "ESTABLISHED") print $1 }' | awk -F. '{ print $6".k" }' | xargs

which would maybe want me to just distribute in ipfs/contrib:

function pargs() {
  while read -r cjdns_peer; do
    cjdns_addr=$(sudo /opt/cjdns/publictoip6 $cjdns_peer)
    $*
  done <<< `sudo nodejs /opt/cjdns/tools/peerStats 2>/dev/null | awk '{ if ($2 == "ESTABLISHED") print $1 }' | awk -F. '{ print $6".k" }' | xargs
}

which could be called:

pargs curl "http://\${cjdns_peer}:5001/api/v0/bootstrap/add?arg=/ip6/\${cjdns_addr}/tcp/1234"

and for ssb:

secret_handshake=$(cd ~/mvd && node bin config | jq '.shs') # psuedocode
pargs  echo "net:\${cjdns_addr}:8008~shs:${secret_handshake} |  socat - UDP6-datagram:[$cjdns_peer]:8008

[darkdrgn2k]

If we have the ipfs daemon running with admin port open

ipfs says not to leave the api open to the world. I would agree!

[benhylau]

This Cabal use case is probably related to current discussion #213 (comment)

[darkdrgn2k]

Idea on multi hop advertising #385