Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flesh out misbehavior reports #237

Merged
merged 5 commits into from
Jan 3, 2025
Merged

Flesh out misbehavior reports #237

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
36390a9
feat: add misbehavior report endpoints
richardhuaaa Dec 20, 2024
28eeb27
feat: add node signatures to misbehavior reports
richardhuaaa Dec 20, 2024
552ca3f
fix: fix up timestamps
richardhuaaa Dec 20, 2024
490aa3a
fix: fix cyclical dependency
richardhuaaa Dec 20, 2024
1813d80
refactor: use oneof for mutually exclusive fields
richardhuaaa Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions proto/xmtpv4/envelopes/envelopes.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,45 @@ message OriginatorEnvelope {
BlockchainProof blockchain_proof = 3;
}
}

enum Misbehavior {
MISBEHAVIOR_UNSPECIFIED = 0;
MISBEHAVIOR_UNRESPONSIVE_NODE = 1;
MISBEHAVIOR_SLOW_NODE = 2;
MISBEHAVIOR_FAILED_REQUEST = 3;
MISBEHAVIOR_OUT_OF_ORDER = 4;
MISBEHAVIOR_DUPLICATE_SEQUENCE_ID = 5;
MISBEHAVIOR_CAUSAL_ORDERING = 6;
MISBEHAVIOR_INVALID_PAYLOAD = 7;
MISBEHAVIOR_BLOCKCHAIN_INCONSISTENCY = 8;
}

message LivenessFailure {
uint32 response_time_ns = 1;
SubscribeEnvelopesRequest subscribe = 2;
QueryEnvelopesRequest query = 3;
PublishPayerEnvelopesRequest publish = 4;
}

message SafetyFailure {
repeated OriginatorEnvelope envelopes = 1;
}

message UnsignedMisbehaviorReport {
uint64 reporter_time_ns = 1;
uint32 misbehaving_node_id = 2;
Misbehavior type = 3;
LivenessFailure liveness = 4;
SafetyFailure safety = 5;
// Nodes must verify this field is false for client-submitted reports
bool submitted_by_node = 6;
}

message MisbehaviorReport {
// Server time when the report was stored. Used only for querying reports.
// This field is not signed.
uint64 server_time_ns = 1;
Copy link
Contributor Author

@richardhuaaa richardhuaaa Dec 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need this separately to the reporter time (which may have client clock skew), so that the query endpoint can query all payloads larger than a certain timestamp. Could have used a sequence ID or a timestamp, decided to use a timestamp so that it's simpler, while also avoiding confusion with cursors in the main replication protocol

bytes unsigned_misbehavior_report = 2;
// Signed by the node hosting the report
xmtp.identity.associations.RecoverableEcdsaSignature signature = 3;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have the node's signature here. Alternative approaches:

  • No signature - it makes it a bit tricky to handle cases where a node retracts or denies a report it previously made. It also seems useful to know which node client reports came in through, as opposed to them going into one big pool.
  • Add a client signature - I think in most cases clients will want to stay anonymous. Adding your public key to a set of group payloads may end up deanonymizing a group.

When we define incentives for submitting reports, some entities may want to be rewarded. When we get there, perhaps we could add a claimant_public_key field to the UnsignedMisbehaviorReport. Probably don't need an additional signature for that.

}
48 changes: 28 additions & 20 deletions proto/xmtpv4/message_api/message_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,6 @@ import "xmtpv4/envelopes/envelopes.proto";

option go_package = "github.com/xmtp/proto/v3/go/xmtpv4/message_api";

// Misbehavior types
enum Misbehavior {
MISBEHAVIOR_UNSPECIFIED = 0;
MISBEHAVIOR_UNAVAILABLE_NODE = 1;
MISBEHAVIOR_OUT_OF_ORDER_ORIGINATOR_SID = 2;
MISBEHAVIOR_DUPLICATE_ORIGINATOR_SID = 3;
MISBEHAVIOR_CYCLICAL_MESSAGE_ORDERING = 4;
}

// Reports node misbehavior, submittable by nodes or by clients
message MisbehaviorReport {
Misbehavior type = 1;
repeated xmtp.xmtpv4.envelopes.OriginatorEnvelope envelopes = 2;
}

// Query for envelopes, shared by query and subscribe endpoints
// Either topics or originator_node_ids may be set, but not both
message EnvelopesQuery {
Expand Down Expand Up @@ -83,37 +68,60 @@ message GetInboxIdsResponse {
repeated Response responses = 1;
}

// Replication API
message SubmitMisbehaviorReportRequest {
xmtp.xmtpv4.envelopes.UnsignedMisbehaviorReport report = 1;
}

message SubmitMisbehaviorReportResponse {}

message QueryMisbehaviorReportsRequest {
uint64 after_ns = 1;
Copy link
Contributor Author

@richardhuaaa richardhuaaa Dec 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the minimum set of parameters we should be able to query by, but in the future it may make sense to add queries by misbehaving_node_id or misbehavior type or excuding client reports

}

message QueryMisbehaviorReportsResponse {
repeated xmtp.xmtpv4.envelopes.MisbehaviorReport reports = 1;
}

service ReplicationApi {
// Subscribe to envelopes
rpc SubscribeEnvelopes(SubscribeEnvelopesRequest) returns (stream SubscribeEnvelopesResponse) {
option (google.api.http) = {
post: "/mls/v2/subscribe-envelopes"
body: "*"
};
}

// Query envelopes
rpc QueryEnvelopes(QueryEnvelopesRequest) returns (QueryEnvelopesResponse) {
option (google.api.http) = {
post: "/mls/v2/query-envelopes"
body: "*"
};
}

// Publish envelope
rpc PublishPayerEnvelopes(PublishPayerEnvelopesRequest) returns (PublishPayerEnvelopesResponse) {
option (google.api.http) = {
post: "/mls/v2/publish-payer-envelopes"
body: "*"
};
}

// Get inbox ids
rpc GetInboxIds(GetInboxIdsRequest) returns (GetInboxIdsResponse) {
option (google.api.http) = {
post: "/mls/v2/get-inbox-ids"
body: "*"
};
}

rpc SubmitMisbehaviorReport(SubmitMisbehaviorReportRequest) returns (SubmitMisbehaviorReportResponse) {
option (google.api.http) = {
post: "/mls/v2/submit-misbehavior-report"
body: "*"
};
}

rpc QueryMisbehaviorReports(QueryMisbehaviorReportsRequest) returns (QueryMisbehaviorReportsResponse) {
option (google.api.http) = {
post: "/mls/v2/query-misbehavior-reports"
body: "*"
};
}
}
Loading