[bug] Inclusion of invalid JSON-LD @context entries

[erincandescent]

The @context array in JSON-LD may contain two things:

• A direct inlined JSON-LD context definition, or
• A URL referencing a JSON-LD document which itself contains an @context

GoToSocial includes "http://joinmastodon.org/ns" (and "http://schema.org"?). Neither serves a JSON-LD document, so they shouldn't be included:

$ curl -H "Accept: application/ld+json" http://joinmastodon.org/ns -v
*   Trying 2a04:4e42:a00::347:80...
* Connected to joinmastodon.org (2a04:4e42:a00::347) port 80 (#0)
> GET /ns HTTP/1.1
> Host: joinmastodon.org
> User-Agent: curl/7.85.0
> Accept: application/ld+json
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Connection: close
< Content-Length: 0
< Server: Varnish
< Retry-After: 0
< Location: https://joinmastodon.org/ns
< Accept-Ranges: bytes
< Date: Wed, 05 Jul 2023 11:16:03 GMT
< Via: 1.1 varnish
< X-Served-By: cache-bma1636-BMA
< X-Cache: HIT
< X-Cache-Hits: 0
< X-Timer: S1688555764.869779,VS0,VE1
< Strict-Transport-Security: max-age=300
< alt-svc: h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400

$ curl -H "Accept: application/ld+json" https://joinmastodon.org/ns -v
*   Trying 2a04:4e42:a00::347:443...
* Connected to joinmastodon.org (2a04:4e42:a00::347) port 443 (#0)
> GET /ns HTTP/2
> Host: joinmastodon.org
> user-agent: curl/7.85.0
> accept: application/ld+json
>
< HTTP/2 404
< cache-control: private, no-cache, no-store, max-age=0, must-revalidate
< content-type: text/html; charset=utf-8
< etag: "j6meoew0yap42"
< x-nextjs-cache: HIT
< x-powered-by: Next.js
< accept-ranges: bytes
< via: 1.1 varnish, 1.1 varnish
< date: Wed, 05 Jul 2023 11:16:30 GMT
< x-served-by: cache-fra-eddf8230083-FRA, cache-bma1628-BMA
< x-cache: MISS, MISS
< x-cache-hits: 0, 0
< x-timer: S1688555791.620471,VS0,VE28
< vary: Accept-Encoding
< strict-transport-security: max-age=300
< alt-svc: h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400

From looking at (your fork of) the go-fed/activity code, it appears this comes from a fundamental misunderstanding (the code believes that JSON-LD contexts work like XML namespaces, when they do not; they are sort of an overlay on top of them)

This was originally noticed as Takahe issue 223. They have a workaround, but this GTS behaviour is incorrect.

[tsmethurst]

relates to #240

[erincandescent]

BTW as a workaround for the time being you could just rehost the contexts you're referencing somewhere else. Although I'm not sure if that would terminally confuse the go-fed/activity library

[tsmethurst]

Mmm i'm also not sure... I wouldn't really like to try it tbh, it's probably just something we should fix in our fork.

[puckipedia]

I just hit this as well. http://schema.org is a valid JSON-LD context, through a Link header. But actually applying the schema.org context causes name and Person (among with other values) to be replaced by http://schema.org/name and http://schema.org/Person, which makes it unusable.

[tsmethurst]

Yup, makes sense! We've had this on the back burner for a while now as something to fix. It's been low priority because most other implementations don't really do anything with json-ld compaction or schemas or what have you, and fixing it requires faffing about in our go-fed/activity fork. Actually this would be a great candidate for 'bugs to fix in beta' this year.

[zotanmew]

I also just hit this, my implementation refused to fetch an actor due to the type being deserialized as http://schema.org/Person.

[zotanmew]

Adding to this issue that https://gotosocial.org/ns also does not resolve as a valid LD context.

[zotanmew]

You also seem to be missing "sensitive": "https://www.w3.org/ns/activitystreams#sensitive".

[zotanmew]

from reversing a payload, it looks like the gotosocial ns should be:

{
  "@context": {
    "gts": "https://gotosocial.org/ns#",
    "interactionPolicy": "gts:interactionPolicy",
    "canLike": "gts:canLike",
    "canReply": "gts:canReply",
    "canAnnounce": "gts:canAnnounce",
    "always": {
      "@id": "gts:always",
      "@type": "@id"
    },
    "approvalRequired": {
      "@id": "gts:approvalRequired",
      "@type": "@id"
    }
  }
}

can someone confirm whether this is correct?

[tsmethurst]

https://gotosocial.org/ns

Indeed, the interactionPolicy feature is still in development so we didn't put anything up at the namespace yet.

can someone confirm whether this is correct?

Thanks for taking a look :)

Always and approvalRequired should be arrays, does "@type": "@id" permit that?

There's also the approvedBy field on Likes, Announces, and anything considered to be "statusable" like Note, Article, etc, which should be a single URI.

[tsmethurst]

Btw, no need to reverse a payload, current documentation for the feature is here: https://docs.gotosocial.org/en/latest/federation/posts/#interaction-policy

[puckipedia]

I believe canLike, canReply, and canAnnounce should be "@type": "@id" too (and, thus, approvedBy should be marked as such as well) (anything that contains one or more objects or URIs).

All values in JSON-LD (and thus in ActivityPub) can be any amount of values; 0, 1, or many (either represented as an array or as a single item). null, [], and abeing elided are considered identical, iirc.

Indeed, the interactionPolicy feature is still in development so we didn't put anything up at the namespace yet.

The exact reason it's troubling to not host a context at the path is that any server that uses JSON-LD (Kroeg, and the way more popular Iceshrimp.NET) can now no longer read any objects that include this context.

[zotanmew]

Specifically, I noticed this because someone opened a support ticket about GTS federation being broken

[tsmethurst]

Ah right thanks, I'll see if I can put something up there for now then, so you can interact with instances running GtS main that are already sending messages with interactionPolicy. I've tried to follow json-ld multiple times now over the years and bounced off it every single time so these pointers are appreciated!

[zotanmew]

Thanks for the details, I think this should be the context then?

{
  "@context": {
    "gts": "https://gotosocial.org/ns#",
    "interactionPolicy": {
      "@id": "gts:interactionPolicy",
      "@type": "@id"
    },
    "canLike": {
      "@id": "gts:canLike",
      "@type": "@id"
    },
    "canReply": {
      "@id": "gts:canReply",
      "@type": "@id"
    },
    "canAnnounce": {
      "@id": "gts:canAnnounce",
      "@type": "@id"
    },
    "always": {
      "@id": "gts:always",
      "@type": "@id"
    },
    "approvalRequired": {
      "@id": "gts:approvalRequired",
      "@type": "@id"
    },
    "approvedBy": {
      "@id": "gts:approvedBy",
      "@type": "@id"
    }
  }
}

[tsmethurst]

Much appreciated :) I'll fiddle with gotosocial.org and get back to you when the document is up.

[zotanmew]

Perfect. I've added a hotfix that intercepts the context with the above payload for the time being.

[tsmethurst]

@zotanmew @puckipedia The document is now served at https://gotosocial.org/ns :)

[zotanmew]

Thank you! I'd leave this issue open, as the mastodon context is still invalid (it should be served inline as appropriate)

My reverse engineered version of that context is

{
  "@context": {
    "toot": "http://joinmastodon.org/ns#",
    "discoverable": "toot:discoverable",
    "indexable": "toot:indexable",
    "memorial": "toot:memorial",
    "votersCount": "toot:votersCount",
    "suspended": "toot:suspended",
    "devices": "toot:devices",
    "Emoji": "toot:Emoji",
    "blurhash": "toot:blurhash",
    "featured": {
      "@id": "toot:featured",
      "@type": "@id"
    },
    "featuredTags": {
      "@id": "toot:featuredTags",
      "@type": "@id"
    },
    "focalPoint": {
      "@id": "toot:focalPoint",
      "@container": "@list"
    }
  }
}

[tsmethurst]

Thanks :) I'll try to get around to updating our go-fed fork to use that context, when time permits.

[puckipedia]

(devices here should be @id too. I highly recommend staring at https://github.com/mastodon/mastodon/blob/main/app/helpers/context_helper.rb for this; this is a map of extension to context, and is merged together for each extension used)