Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC4133: Extending User Profile API with Key:Value Pairs #4133

Open
wants to merge 56 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
0e85b3b
Extended profile fields
tcpipuk Sep 1, 2024
0f373db
Unstable client features
tcpipuk Sep 1, 2024
c5a3015
Clarification on limits
tcpipuk Sep 5, 2024
c8a5a1a
Warning about avatar_url and displayname limits
tcpipuk Sep 5, 2024
3157982
Error code clarification
tcpipuk Sep 5, 2024
b8ed87a
Whitespace fixes
tcpipuk Sep 6, 2024
a81f21f
Adjusted size limits
tcpipuk Sep 11, 2024
e688eb1
Clarified Canonical JSON
tcpipuk Sep 11, 2024
39d5fa2
Clarifications
tcpipuk Sep 13, 2024
e160c71
Clarifications
tcpipuk Sep 13, 2024
a9546aa
Remove UTF-16
tcpipuk Sep 13, 2024
0c43f50
Clarify key persistence
tcpipuk Sep 13, 2024
613411a
Out of date line
tcpipuk Sep 13, 2024
4a9557f
Clarify only `u.*` namespace is limited to 512 bytes
tcpipuk Sep 16, 2024
fbb4e44
Include read-only fields in capability
tcpipuk Sep 16, 2024
591999f
Change missing capability behaviour
tcpipuk Sep 16, 2024
15a0bd1
Typo correction
tcpipuk Sep 17, 2024
26c59f7
Privacy clarification for T&S
tcpipuk Sep 24, 2024
30e82aa
Consolidation and rewrite
tcpipuk Sep 25, 2024
2966c85
Whitespace fix
tcpipuk Sep 25, 2024
d97189e
Safety and security updates
tcpipuk Sep 25, 2024
ae19725
Clarify servers can hide fields
tcpipuk Sep 25, 2024
23a3a62
References to MSC4201 and MSC4202
tcpipuk Sep 26, 2024
f32932c
Clarify redacted `m.room.member` events
tcpipuk Sep 30, 2024
4afb8b8
Error codes and redundant instructions
tcpipuk Oct 1, 2024
bb4fc76
Removing custom fields
tcpipuk Oct 2, 2024
09318e8
Typo fix
tcpipuk Oct 11, 2024
9b4741e
Clarifications and readability
tcpipuk Oct 16, 2024
068d44e
Correct key length error to match common grammar limit
tcpipuk Nov 4, 2024
fa381da
Remove PATCH/PUT.
clokep Dec 20, 2024
b373a55
Merge pull request #1 from clokep/user-prof-fields
tcpipuk Dec 20, 2024
3349123
Removed redundant sections after `PUT` and `PATCH` methods removed
tcpipuk Jan 3, 2025
9fa0096
Re-add client feature for managing profile fields
tcpipuk Jan 4, 2025
da0a791
Update proposals/4133-extended-profiles.md
tcpipuk Jan 5, 2025
a948f5d
Clarify 403 error scenarios
tcpipuk Jan 5, 2025
c82dab6
Add section on caching behaviour under S-S API
tcpipuk Jan 5, 2025
21ad83f
Link to Canonical JSON in current spec
tcpipuk Jan 5, 2025
1726ef3
Cut down instructions for clients on when to display content from fed…
tcpipuk Jan 5, 2025
30d203e
Revert c82dab67d114ded78ef4b6a5a1f9f264002d6666
tcpipuk Jan 5, 2025
43e1e1a
Clarify caching and freshness challenges
tcpipuk Jan 5, 2025
af87bbe
Adjust abuse section
tcpipuk Jan 5, 2025
f686090
Authentication/rate-limiting/guest access requirements
tcpipuk Jan 5, 2025
17cc30b
Un-revert accidental revert of af87bbeb26c6d33725d970676009b52056a87f96
tcpipuk Jan 5, 2025
d620259
Simplify caching recommendations
tcpipuk Jan 7, 2025
c1b419a
Up to clients whether they check capability
tcpipuk Jan 10, 2025
4b3a8ed
Technically correct is the best kind of correct
tcpipuk Jan 10, 2025
0f41c82
Link to current federation profile endpoint
tcpipuk Jan 10, 2025
1b98d40
Mention check for spec version when using profile fields
tcpipuk Jan 10, 2025
9b2918e
Attempt to clarify what servers should not enforce about key naming
tcpipuk Jan 10, 2025
e00d2e9
Fix line wrapping after 9b2918e3735f5aec02f6309fcbc81feaca985804
tcpipuk Jan 10, 2025
a11286a
Add `M_MISSING_PARAM` error
tcpipuk Jan 13, 2025
2a86235
Clarify where errors apply
tcpipuk Jan 13, 2025
c4fa474
Merge branch 'matrix-org:main' into main
tcpipuk Jan 20, 2025
7ca83db
Clarify optional ability to not always federate every field
tcpipuk Jan 20, 2025
c7bb382
Fix inconsistent line wrapping
tcpipuk Jan 20, 2025
3843f27
Offer suggestions for hiding extended fields when member event redacted
tcpipuk Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Clarify where errors apply
  • Loading branch information
tcpipuk authored Jan 13, 2025

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
commit 2a862350e6847297c420174576d6f87273dc1459
20 changes: 19 additions & 1 deletion proposals/4133-extended-profiles.md
turt2live marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving to a thread for discussion to take place - @MTRNord said:

So I am still not sure if this is a good idea.

I do like that we can represent timezones. I do not (and I am aware this was moved to $later) think this msc works well for many other fields. Pronouns for example are by nature freetext which means that MSC reintroduces one of the core issues which this MSC initially had. Imho dropping it here just delays this concerns but did not address it.

I am also not sure why there was no move towards instead “forking”/continuing the profiles as rooms instead. Why do we need to do it like this and compromise extremely on this? Why couldn't the time have been used on the other thing?

Missing is also how unstable/client specific namespaced fields can be moderated by room admins. This brings me back to my past concerns in my other comments. This again essentially allows unmoderated freetext fields which even more confusing may only show up in some clients. So it could a) target specific user groups and b) a room admin may not even see it and have people complain. This makes T&S quite a bit harder for room admins. For server admins it obviously is not an issue.

Last but not least I fear feature creep with this. Will we have now a UI related MSC for every field here? If I want to for example want to have my gender in this would this need to be another MSC because other want this? This feels like we are starting to micro manage the spec and I am not sure this is a good idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm failing to understand how this MSC creates problems for pronouns specifically. This MSC defines a generic key/value structure, where the value can be pretty much any type. This could be a user-supplied string, an object, or a boolean value depending on the key's requirements. If you have specific concerns with how the MSC is interfering with other MSCs, like pronouns, please leave inline comments for review.

This MSC also states it complements profiles as rooms, which is true: one of the concerns with profiles as rooms is it tries to do too much. By accepting this MSC, we're agreeing that arbitrary key/value pairs should be supported, which makes other MSCs easier to land. In essence, this is already a continuation of the work, just at an earlier stage than the current MSCs in the area. Forking the effort feels incorrect, given all the authors involved would like to work with each other towards a solution. If there's specific language which doesn't feel in line with this, please leave an inline comment for review.

As for room moderation: profile fields are deliberately kept at the user level in this MSC. A future MSC may introduce them to the room/member level as well. This makes the T&S model one where the server admin is responsible for the user's profile, not the rooms they are participating in. Room admins still have the option to eject users they don't feel fit their room's purpose, and can operate bots which scan profiles for potentially abusive content if they wish. Meanwhile, abusive users and their profiles should be reported to the homeserver admin, which this MSC calls out as the preferred mechanism for dealing with issues. A future, unrelated, MSC will make this easier. If there's specific areas which could be improved further, inline comments are appreciated for further review.

Creating an MSC for every feature may be what happens at the start of the transition towards this key/value system, but as we gain experience in reviewing these MSCs we'll be better able to calibrate the types of things which should be in the spec and which are best covered as unstable extensions. For example, we may prefer a general m.bio field over lots of fields for gender, websites, contact info, and interests. Or we may prefer to have a dedicated field for gender, and combine the rest under m.bio. These types of concerns are best expressed on the individual MSCs which get opened after this MSC lands - the influx is expected, and something we'd need to manage regardless.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am currently on the move and cant draft a full reply in the meanwhile many of my concerns also have been raised by gnuxie at #4133 (comment) which likely answer some of the questions. I will write a proper reply once i am back at a computer :)

Copy link

@TheArcaneBrony TheArcaneBrony Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am also not sure why there was no move towards instead forking/continuing profiles as rooms instead.

You mean #4201? Which exists just to give PaR another try.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gnuxie wrote:

Why is this MSC being considered in isolation of any specified custom profile field?

  • All fields contain user generated content.

  • No custom profile field has sufficiently demonstrated itself (via the spec process or otherwise).

  • There are no fields defined in this MSC

  • All fields used in the implementation are currently under review in other MSCs and have not demonstrated themselves.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whilst we do make an effort to avoid speccing stuff that doesn't have a demonstrated purpose, there is a balance to make between that, and allowing things to land in manageable pieces. Historically, some features have struggled to make it through the spec process because their scope was just too broad. (MSC1849 was an example of that.)

Reviewing large MSCs is really hard (as is reviewing a large PR of any kind), just because everybody has to keep lots of context in their head for weeks or even months, and breaking them up helps build stable foundations before moving onto the next storey.

Further: in the case of this particular MSC, I see it mostly as a generalisation of the existing API. Even if we never end up landing anything like MSC4175, it doesn't feel like this MSC is adding undue clutter to the API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gnuxie wrote:

The MSC does not propose anything in itself except a place to store user generated content... without demonstrating consideration for user generated content, especially free-text content.

  • Free-text content was originally part of the MSC, but got moved out to another proposal (MSC4208) to "resolve" numerous blocking concerns about it. But these concerns will appear in almost every single follow-up MSC related to specifying any field that builds on this proposal.

  • The MSC still has a provision to display and enter custom fields:

    Clients MAY provide a UI for users to view and enter custom
    fields, respecting the appropriate namespaces.

  • The demonstrated implementation uses MSC4175 which is a parsable field with a very narrow range of values. But this proposal is clearly intended to be followed up with fields for unparsable user generated content. And we expect there to be many more unparsable fields than restricted and narrowly defined ones.

I believe the MSC should be modified to address the concerns with free-text content. And also demonstrate implementation within a client that does add free-text fields.

The discussion will have to happen with subsequent MSCs unless they define fields that are both very limited and restricted, which is obviously not the intent of this proposal. So it does not make sense to discuss this proposal in isolation of those concerns.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I really understand the concern here. It feels like you're arguing against this MSC because you're concerned that people may, in the future, use it as a basis for other MSCs which bring problems. You've acknowledged yourself that such problems are not inherent by pointing to MSC4175, so it seems inappropriate to express your concerns on this MSC. If you have concerns with MSC4208, then they belong on MSC4208, not here.

  • Free-text content was originally part of the MSC, but got moved out to another proposal (MSC4208) to "resolve" numerous blocking concerns about it. But these concerns will appear in almost every single follow-up MSC related to specifying any field that builds on this proposal.

Yeeess, but then the time to discuss them is on those MSCs, not here.

  • The MSC still has a provision to display and enter custom fields:
    > Clients MAY provide a UI for users to view and enter custom
    > fields, respecting the appropriate namespaces.

To be clear, I'm reading this as: "clients MAY provide a UI for users to view and enter their own profile fields". On that basis, it's hard to see why this, on the face of it, is any different to, say, allowing users to send freetext events.

I must admit I'm not that familiar with the T&S space, so it's entirely possible I'm missing something here, but in discussions I've had with people who are more familiar with it than me, they've not been concerned about letting users set profile data: it's when clients start showing the profile data of other users that you need to be concerned. I'd be happy to be set straight here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gnuxie wrote:

Why is there urgency to merge this MSC?

  • Clients can currently use the proposal to implement custom fields under the unstable namespace without any changes or hindrance. Some of them already have done, so there should be no urgency to get this merged.
  • Similar Concerns have been suppressed in Matrix rooms as bikeshedding or blocking the MSC. All developers are free to implement the proposal as is under the unstable flag. When the proposal becomes part of the specification it does not change the number of implementations, nor does it allow dependant MSCs to make any further progress.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Firstly, I'm sorry if your concerns haven't been heard. I don't think I've seen the discussions you refer to, and I wasn't aware of them before you raised them here. This is one reason we ask that concerns on MSCs be raised as comments on the MSC itself: conclusions in Matrix rooms get lost. Thank you for doing so now.

Secondly: I don't really see this as "urgency". The MSC has been 9 months in the making, and it feels like a relatively incremental change to the existing API. Honestly, it felt ready to land. I'll refer you to my earlier points that it can be very helpful to land MSCs incrementally rather than having a large amount to review all at once.

As the SCT, we've also had feedback in the past that it can be unreasonably difficult to get simple changes to the spec through the MSC process. There are numerous anecdotes from people who have just found the MSC process too arduous and have walked away in frustration. We have a balance to pick between rushing things into the protocol without due diligence, and letting the protocol ossify because changing it is just to damn hard.

All that said, you're right that this isn't actively blocking anything, and it's certainly not our goal to steamroller it through against significant opposition. On that basis: thank you again for raising your concerns. This is the FCP working as it should: we've put the FCP timer at least on hold for now and we'll see what can be done to address the concerns.

Whilst we can't promise that everyone is going to be delighted by every MSC that is accepted, it is important to make sure that members of the Matrix community have the opportunity to voice their concerns and that we take the time to consider those concerns properly.

Original file line number Diff line number Diff line change
@@ -193,7 +193,8 @@ demonstrates the process of defining new fields in the `m.*` namespace.
}
```

- **`M_MISSING_PARAM`**: Required parameter is missing, e.g. if a client attempts to set a profile field, but neglects to include that named field in the request body.
- **`M_MISSING_PARAM`**: Required parameter is missing, e.g. if a client attempts to set a profile
field, but neglects to include that named field in the request body.

```json
{
@@ -249,6 +250,23 @@ A server may return this error in several scenarios:
}
```

### Applicability of Error Codes

Unless explicitly stated otherwise, all error codes described in this
section apply to all Client-Server and Server-Server endpoints introduced
by this MSC. For example:

1. `M_NOT_FOUND` applies to any attempt to retrieve a non-existent profile
field.
2. `M_PROFILE_TOO_LARGE` applies to any attempt to create or update profile
data exceeding the allowed size.

The Server-Server endpoints introduced in this MSC adhere to the existing
error structure for federation, as the federation access remains read-only
in this proposal. This means no new error codes or status code combinations
are introduced for Server-Server endpoints beyond what is already
documented in the specification.

tcpipuk marked this conversation as resolved.
Show resolved Hide resolved
## Propagation of Profile Fields

The existing fields, `avatar_url` and `displayname`, will continue to trigger state events in each
Loading