Don't replicate zero length feeds

[cblgh]

from irc

10:30:17 <@telamohn> Let me make a couple of new code repos, i have an experimental core-store that let's you purge & delete/gc feeds
10:30:49 < cblgh> oh nice, so we could remove feeds that just joined but never posted anything?
10:30:59 <@telamohn> it's not to be confused with andrew-wosh's corestore which is more targeted towards being embedded in data-structures.
10:31:01 < cblgh> that's been kinda annoying tbh
10:31:08 <@telamohn> hmm
10:31:25 <@telamohn> cblgh: ofcourse, you actually don't even need to replicate them
10:32:33 <@telamohn> just implement the describe() and accept() function in cabal-core.
10:32:59 < cblgh> are those from multifeed?
10:33:01 <@telamohn> during describe attach the feed.length to meta.
10:33:17 <@telamohn> and during accept return false if feed.lenght is zero.
10:33:36 <@telamohn> *if meta.length is zero
10:34:01 <@telamohn> cblgh: well once the PR goes through they're gonna be in multifeed :)
10:34:49 <@telamohn> https://github.com/telamon/replic8#middleware-interface <-- i'm talking about this interface.
10:34:53 <@telamohn> sorry posted link a bit late

[hackergrrl]

Possible edge case: I connect to a cabal, read some backlog, then leave. I return the next day, still not having posted anything yet, and now nobody will ever replicate my feed again (because it's size=0).

[hackergrrl]

cc @telamon

[telamon]

@noffle yes, that pretty much sums it up, if you're a lurker then you won't unnecessarily tax the network you'll just act as another seed,
And if a user writes a message his feed will no longer be empty and thus it will no longer be filtered.

[hackergrrl]

Ah, but how will other users know that that person has a non-empty feed if they refuse to sync with them when it's length=0 locally? :)

[telamon]

@noffle The peer shouldn't be filtered, only zero length feeds would be rejected. So a user with a zero-length writer should behave like --seed mode until he appends an entry.

I'm thinking of letting other peers know there's a new feed that is qualified for replication the same way that the active feed forwarding works.

When an peer with established connections suddenly becomes aware of a new feed the _onFeedReplicated handler is invoked that forwards the feed to all other connected peers that dosen't have it yet.

So the same needs to be done on the peer that has a zero-length writer when the first entry is written.
Maybe cabal could invoke the above _onFeedReplicated() or startConversation() method which should reannounce previously filtered feeds.

But anyway, at the moment of writing, in order to manually notify connected peers that your writer is no-longer empty we could do the following:

const writer = multi.feed('local')
writer.on('append', () => {
  if (writer.length === 1) {
    multi.replicationManager.connections.forEach(conn => {
      multi.replicationManager._onFeedReplicated(conn, writer.key)
    })
  }
})

I'm adding to my TODO to expose an manager#advertise(key) method to wrap up the above loop in a nicer package.

Did i manage to answer the question?