AP: implement the (non-standard?) replies collection

[nanos]

You may be aware of the FediFetcher project where we pull missing replies from the fediverse into Mastodon instances.

I'm wondering if it's possible to pulling replies to bridged Bluesky posts in particular. Does the bridge expose any sort of API to fetch these?

Thanks!

[Tamschi]

Bridgy Fed does not currently expose the replies collection.

I suspect it could, but with the current architecture it would likely increase database fetches quite a lot.

[Tamschi]

Hm… Though this is something that's in theory cacheable nicely.

[nanos]

Obviously I'd love to see this, as I see increasing numbers of Bluesky posts in my timeline (which is amazing to see), but I also understand if this isn't a priority right now.

[snarfed]

Yes! Not a priority for us right now, but replies collection does sound like the way we'd do this. PRs are welcome!

...having said that though, @Tamschi do you know any more about a replies collection? I know Mastodon has it, but it's not in their @context, and it's not actually specified in AP at all: https://www.w3.org/TR/activitypub/#collections

https://docs.joinmastodon.org/spec/activitypub/#properties-used doesn't say anything about the standardization status, just:

replies: A Collection of statuses that are in reply to the current status. Up to 5 replies from the same server will be fetched upon discovery of a remote status, in order to resolve threads more fully. On Mastodon’s side, the first page contains self-replies, and additional pages contain replies from other people.

Here's an example Mastodon Note with it:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    {
      "ostatus": "http://ostatus.org#",
      "atomUri": "ostatus:atomUri",
      "inReplyToAtomUri": "ostatus:inReplyToAtomUri",
      "conversation": "ostatus:conversation",
      "sensitive": "as:sensitive",
      "toot": "http://joinmastodon.org/ns#",
      "votersCount": "toot:votersCount"
    }
  ],
  "type": "Note",
  "id": "https://indieweb.social/users/snarfed/statuses/113887531659429526",
  "url": "https://indieweb.social/@snarfed/113887531659429526",
  "attributedTo": "https://indieweb.social/users/snarfed",
  "published": "2025-01-25T06:17:02Z",
  "to": ["https://www.w3.org/ns/activitystreams#Public"],
  "cc": ["https://indieweb.social/users/snarfed/followers"],
  "conversation": "tag:indieweb.social,2025-01-25:objectId=165027761:objectType=Conversation",
  "content": "<p>\ud83e\uddf6</p>",
  "replies": {
    "id": "https://indieweb.social/users/snarfed/statuses/113887531659429526/replies",
    "type": "Collection",
    "first": {
      "type": "CollectionPage",
      "next": "https://indieweb.social/users/snarfed/statuses/113887531659429526/replies?min_id=113907866049275877&page=true",
      "partOf": "https://indieweb.social/users/snarfed/statuses/113887531659429526/replies",
      "items": ["https://indieweb.social/users/snarfed/statuses/113907866049275877"]
    }
  },
  ...
}

[snarfed]

cc #308, #309, #310

[nanos]

..having said that though, @Tamschi do you know any more about a replies collection?

Not Tamschi, but what I can tell you is that it's a mess! Indeed it's such a mess that on FediFetcher - which supports only AP implementations - I decided it's easier to consume various implementations' proprietary APIs to get replies, than it is to try and do it the AP way.

AFAICT none of the major AP implementations implements any sort of AP reply collection fully. (e.g. Mastodon only lists self-replies in the AP response, Friendica doesn't list replies at all, GoToSocial requires authentication to fetch replies).

It's really a minefield ...

[Tamschi]

...having said that though, @Tamschi do you know any more about a replies collection? I know Mastodon has it, but it's not in their @context, and it's not actually specified in AP at all: https://www.w3.org/TR/activitypub/#collections

https://docs.joinmastodon.org/spec/activitypub/#properties-used doesn't say anything about the standardization status, just:

replies: A Collection of statuses that are in reply to the current status. Up to 5 replies from the same server will be fetched upon discovery of a remote status, in order to resolve threads more fully. On Mastodon’s side, the first page contains self-replies, and additional pages contain replies from other people.

It's defined here: https://www.w3.org/TR/activitystreams-vocabulary/#dfn-replies

I suspect it's quite free-form. Just listing @ids of reply statuses (the objects, not the activity creating them) may work. Note that you have to take block relationships and reply permissions into account.

I think there's an argument for excluding non-bridged replies, since the original author never sees those either.

[snarfed]

Hah, ok then.

Mastodon only lists self-replies in the AP response

Hmm, have you tried fetching more pages? https://docs.joinmastodon.org/spec/activitypub/#properties-used says:

On Mastodon’s side, the first page contains self-replies, and additional pages contain replies from other people.

[nanos]

Hmm, have you tried fetching more pages? https://docs.joinmastodon.org/spec/activitypub/#properties-used says:

I don't think I did, as it became obvious that there are less implementations supporting replies in the AP way, than there are implementations being compatible with the mastodon API, so I capitulated ...