Skip to content

Commit

Permalink
docs: define basic sharding for Communities (#132)
Browse files Browse the repository at this point in the history
Changes here address the bulk of what is necessary for
waku-org/pm#266 and
status-im/status-go#5821 to define the new
shared-shard strategy for Communities.

This PR should be enough unblock implementation and, once merged, we can
go ahead with the pubsub topic changes in status-go.
I'm not yet closing the accompanying deliverable, as the spec still
requires a full list of all Communities-related messages with functional
scope explicitly assigned.
  • Loading branch information
jm-clius authored Feb 21, 2025
1 parent 235c1d5 commit 9bed57e
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 21 deletions.
20 changes: 16 additions & 4 deletions status/56/communities.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,13 +307,25 @@ message CommunityDescription {

Note: The usage of the clock is described in the [Clock](#clock) section.

### Pubsub topic or shard usage
### Functional scope and shard assignment

Status network uses static sharding as explained in [Relay Sharding](https://github.com/waku-org/specs/blob/master/standards/core/relay-sharding.md#static-sharding)
We define two special [functional scopes](../raw/status-app-protocols.md#functional-scope) for messages related to Status Communities:
1. Global community control

Check failure on line 313 in status/56/communities.md

View workflow job for this annotation

GitHub Actions / lint

Lists should be surrounded by blank lines

status/56/communities.md:313 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "1. Global community control"] https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md032.md
2. Global community content

All communities by default use the default shard `32` for most of their messages and use shard `64` for a few specific messages like community event messages.
All messages that relate to controlling communities MUST be assigned the _global community control_ scope.
All messages that carry user-generated content for communities MUST be assigned the _global community content_ scope.

A community can be assigned a dedicated shard (in the range of 1-1024) after creation in which case all messages of the community are sent on that specific shard except for the community event messages.
> *Note:* a previous iteration of Status Communities defined separate community-wide scopes for each community.

Check failure on line 319 in status/56/communities.md

View workflow job for this annotation

GitHub Actions / lint

Emphasis style

status/56/communities.md:319:3 MD049/emphasis-style Emphasis style [Expected: underscore; Actual: asterisk] https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md049.md

Check failure on line 319 in status/56/communities.md

View workflow job for this annotation

GitHub Actions / lint

Emphasis style

status/56/communities.md:319:9 MD049/emphasis-style Emphasis style [Expected: underscore; Actual: asterisk] https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md049.md
However, this model was deprecated and all communities now operate on a global, shared scope.
This implies that different communities will share shards on the routing layer.

The following [Waku transport layer](../raw/status-app-protocols.md#waku-transport-layer) allocations are reserved for communities:
As per [STATUS-SIMPLE-SCALING](https://rfc.vac.dev/status/raw/simple-scaling/#relay-shards), communities use the default cluster ID `16`
set aside for all Status app protocols.
Within this cluster, the following [shards](../raw/status-app-protocols.md#pubsub-topics-and-sharding) are reserved for the community functional scopes:
1. All messages with a _global community control_ scope MUST be published to shard `128`

Check failure on line 327 in status/56/communities.md

View workflow job for this annotation

GitHub Actions / lint

Lists should be surrounded by blank lines

status/56/communities.md:327 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "1. All messages with a _global..."] https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md032.md
2. All messages with a _global community content_ scope MUST be published to shard `256`

### Content topic level encryption

Expand Down
39 changes: 22 additions & 17 deletions status/raw/status-app-protocols.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,26 +74,31 @@ However, since all users of the Status app MUST be able to participate in 1:1 ch
the functional scope of messages enabling 1:1 chats MUST be a global scope.
Similarly, since private group chats can be set up between any subset of Status app users,
the functional scope for messages related to private group chats MUST be global.
As a counter-example, messages that originate within a community (and are _addressed_ to members of that community)
are only of interest to participants that are also members of that community.
Such messages MUST have a community-wide functional scope.
A third group of messages are addressed only to the participant that generated those messages itself.
Along the same principle, messages that originate within communities are of global interest
for all users who have an interest in the Status Communities feature.
Such messages MUST have a global functional scope,
that can be accessed by any app users interested in communities.
A different group of messages are addressed only to the participant that generated those messages itself.
These _self-addressed_ messages MUST have a local functional scope.

If we further make a distinction between "control" and "content" messages,
we can distinguish five distinct functional scopes.

All Status messages MUST have one of these functional scopes:

#### Global scope
#### Global general scope

1. _Global control_: messages enabling the basic functioning of the app to control features that all app users should be able to participate in. Examples include Contact Requests, Community Invites, global Status Updates, Group Chat Invites, etc.
1. _Global control_: messages enabling the basic functioning of the app to control general features that all app users should be able to participate in. Examples include Contact Requests, global Status Updates, Group Chat Invites, etc.
2. _Global content_: messages carrying user-generated content for global functions. Examples include 1:1 chat messages, images shared over private group chats, etc.

#### Community scope
#### Global community scope

1._Community control_: messages enabling the basic functioning of the app to control features _only relevant to members of a specific community_. Examples include Community Membership Updates, community Status Updates, etc.
2. _Community content_: messages carrying user-generated content _only for members of a specific community_.
1. _Global community control_: messages enabling the basic functioning of the app to control features related to communities. Examples include Community Invites, Community Membership Updates, community Status Updates, etc.
2. _Global community content_: messages carrying user-generated content for members of any community.

> *Note:* a previous iteration of the Status Communities feature defined separate community-wide scopes for each community.

Check failure on line 99 in status/raw/status-app-protocols.md

View workflow job for this annotation

GitHub Actions / lint

Emphasis style

status/raw/status-app-protocols.md:99:3 MD049/emphasis-style Emphasis style [Expected: underscore; Actual: asterisk] https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md049.md

Check failure on line 99 in status/raw/status-app-protocols.md

View workflow job for this annotation

GitHub Actions / lint

Emphasis style

status/raw/status-app-protocols.md:99:9 MD049/emphasis-style Emphasis style [Expected: underscore; Actual: asterisk] https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md049.md
However, this model was deprecated and all communities now operate on a global, shared scope.
This implies that different communities will share shards on the routing layer.

#### Local scope

Expand All @@ -104,7 +109,7 @@ It SHOULD however inform the underlying [transport layer sharding](#pubsub-topic
In general a Status client SHOULD subscribe to participate in:

- all global functions
- (only) the community functions for communities of which it is a member, and
- global community functions if it is interested in this feature, and
- its own local functions.

### Content topics
Expand Down Expand Up @@ -205,9 +210,9 @@ It is RECOMMENDED that separate pubsub topics be used for global control message

#### Community messages

The application SHOULD define at least one separate pubsub topic for each separate community's control and content messages.
The application MAY define a set of more than one pubsub topic per community to allow traffic sharding for scalability.
It is RECOMMENDED that separate pubsub topics be used for community control messages and community content messages.
The application SHOULD define at least one distinct pubsub topic for global community control messages and global community content messages.
The application MAY define a set of more than one pubsub topic for global community messages to allow traffic sharding for scalability.
It is RECOMMENDED that separate pubsub topics be used for global community control messages and global community content messages.

#### Large messages

Expand Down Expand Up @@ -270,8 +275,8 @@ Full clients SHOULD use relay protocol as preferred method to subscribe to pubsu

1. Global control
2. Global content
3. Community control, for each community of which the app user is a member
4. Community content, for each community of which the app user is a member
3. Global community control, if the client has activated the Status Communities feature
4. Global community content, if the client has activated the Status Communities feature

Light clients SHOULD use filter protocol to subscribe only to the content topics relevant to the user.

Expand All @@ -297,8 +302,8 @@ Full clients SHOULD use relay protocol to publish to pubsub topics matching the

1. Global control
2. Global content
3. Community control, for each community of which the app user is a member
4. Community content, for each community of which the app user is a member
3. Global community control, if the client has activated the Status Communities feature
4. Global community content, if the client has activated the Status Communities feature

Light clients SHOULD use lightpush protocol to publish control and content messages.

Expand Down

0 comments on commit 9bed57e

Please sign in to comment.