Skip to content

Commit

Permalink
chore: deprecating named sharding (#2723)
Browse files Browse the repository at this point in the history
  • Loading branch information
gabrielmer authored Jul 9, 2024
1 parent d0980eb commit e1518cf
Show file tree
Hide file tree
Showing 31 changed files with 108 additions and 206 deletions.
2 changes: 1 addition & 1 deletion apps/chat2/config_chat2.nim
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ type

topics* {.
desc: "Default topics to subscribe to (space separated list).",
defaultValue: "/waku/2/default-waku/proto",
defaultValue: "/waku/2/rs/0/0",
name: "topics"
.}: string

Expand Down
2 changes: 1 addition & 1 deletion apps/chat2bridge/config_chat2bridge.nim
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ type Chat2MatterbridgeConf* = object

topics* {.
desc: "Default topics to subscribe to (space separated list)",
defaultValue: "/waku/2/default-waku/proto",
defaultValue: "/waku/2/rs/0/0",
name: "topics"
.}: string

Expand Down
4 changes: 2 additions & 2 deletions apps/liteprotocoltester/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ docker compose logs -f receivernode
| ---: | :--- | :--- |
| NUM_MESSAGES | Number of message to publish | 120 |
| DELAY_MESSAGES | Frequency of messages in milliseconds | 1000 |
| PUBSUB | Used pubsub_topic for testing | /waku/2/default-waku/proto |
| PUBSUB | Used pubsub_topic for testing | /waku/2/rs/0/0 |
| CONTENT_TOPIC | content_topic for testing | /tester/1/light-pubsub-example/proto |

### Lite Protocol Tester application cli options
Expand All @@ -67,7 +67,7 @@ docker compose logs -f receivernode
| --service-node| Address of the service node to use for lightpush and/or filter service | - |
| --num-messages | Number of message to publish | 120 |
| --delay-messages | Frequency of messages in milliseconds | 1000 |
| --pubsub-topic | Used pubsub_topic for testing | /waku/2/default-waku/proto |
| --pubsub-topic | Used pubsub_topic for testing | /waku/2/rs/0/0 |
| --content_topic | content_topic for testing | /tester/1/light-pubsub-example/proto |
| --cluster-id | Cluster id for the test | 0 |
| --config-file | TOML configuration file to fine tune the light waku node<br>Note that some configurations (full node services) are not taken into account | - |
Expand Down
2 changes: 1 addition & 1 deletion apps/liteprotocoltester/run_service_node.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,5 @@ exec /usr/bin/wakunode\
--metrics-server-address=0.0.0.0\
--nodekey=e3f5e64568b3a612dee609f6e7c0203c501dab6131662922bdcbcabd474281d5\
--nat=extip:${IP}\
--pubsub-topic=/waku/2/default-waku/proto\
--pubsub-topic=/waku/2/rs/0/0\
--cluster-id=0
2 changes: 1 addition & 1 deletion apps/liteprotocoltester/run_tester_node.sh
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ echo "Using service node: ${SERIVCE_NODE_ADDR}"
exec /usr/bin/liteprotocoltester\
--log-level=DEBUG\
--service-node="${SERIVCE_NODE_ADDR}"\
--pubsub-topic=/waku/2/default-waku/proto\
--pubsub-topic=/waku/2/rs/0/0\
--cluster-id=0\
--num-messages=${NUM_MESSAGES}\
--delay-messages=${DELAY_MESSAGES}\
Expand Down
2 changes: 1 addition & 1 deletion apps/liteprotocoltester/tester_config.nim
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import
export confTomlDefs, confTomlNet, confEnvvarDefs, confEnvvarNet

const
LitePubsubTopic* = PubsubTopic("/waku/2/default-waku/proto")
LitePubsubTopic* = PubsubTopic("/waku/2/rs/0/0")
LiteContentTopic* = ContentTopic("/tester/1/light-pubsub-example/proto")

type TesterFunctionality* = enum
Expand Down
4 changes: 2 additions & 2 deletions docs/operators/how-to/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ By default a nwaku node will:
See [this tutorial](./configure-key.md) if you want to generate and configure a persistent private key.
- listen for incoming libp2p connections on the default TCP port (`60000`)
- enable `relay` protocol
- subscribe to the default pubsub topic, namely `/waku/2/default-waku/proto`
- subscribe to the default pubsub topic, namely `/waku/2/rs/0/0`
- enable `store` protocol, but only as a client.
This implies that the nwaku node will not persist any historical messages itself,
but can query `store` service peers who do so.
Expand Down Expand Up @@ -107,7 +107,7 @@ enr=enr:-IO4QDxToTg86pPCK2KvMeVCXC2ADVZWrxXSvNZeaoa0JhShbM5qed69RQz1s1mWEEqJ3aok
## Typical configuration (relay node)

The typical configuration for a nwaku node is to run the `relay` protocol,
subscribed to the default pubsub topic `/waku/2/default-waku/proto`,
subscribed to the default pubsub topic `/waku/2/rs/0/0`,
and connecting to one or more existing peers.
We assume below that running nodes also participate in Discovery v5
to continually discover and connect to random peers for a more robust mesh.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial/onchain-rln-relay-chat2.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ At a high level, when a chat2 client is run with Waku-RLN-Relay mounted in on-ch
Under the hood, the chat2 client constantly listens to the membership contract and keeps itself updated with the latest state of the group.

In the following test setting, the chat2 clients are to be connected to the Waku test fleets as their first hop.
The test fleets will act as routers and are also set to run Waku-RLN-Relay over the same pubsub topic and content topic as chat2 clients i.e., the default pubsub topic of `/waku/2/default-waku/proto` and the content topic of `/toy-chat/3/mingde/proto`.
The test fleets will act as routers and are also set to run Waku-RLN-Relay over the same pubsub topic and content topic as chat2 clients i.e., the default pubsub topic of `/waku/2/rs/0/0` and the content topic of `/toy-chat/3/mingde/proto`.
Spam messages published on the said combination of topics will be caught by the test fleet nodes and will not be routed.
Note that spam protection does not rely on the presence of the test fleets.
In fact, all the chat2 clients are also capable of catching and dropping spam messages if they receive any.
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial/rln-chat-cross-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ As for the setup, please follow the tutorials below:
- [JS-chat](https://examples.waku.org/rln-js/)

Once you set up your chat client, it will be connected to the Waku v2 test fleets as its first hop.
Messages generated by the chat client are set to be published on a specific combination of pubsub and content topic i.e., the default pubsub topic of `/waku/2/default-waku/proto` and the content topic of `/toy-chat/3/mingde/proto`.
Messages generated by the chat client are set to be published on a specific combination of pubsub and content topic i.e., the default pubsub topic of `/waku/2/rs/0/0` and the content topic of `/toy-chat/3/mingde/proto`.
The test fleets also run Waku-RLN-Relay over the same pubsub topic and content topic.
Test fleets act as routers and enforce the message rate limit.
As such, any spam messages published by a chat client on the said combination of topics will be caught by the Waku v2 test fleet nodes and will not be routed.
Expand Down
2 changes: 1 addition & 1 deletion examples/cbindings/waku_example.c
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ int main(int argc, char** argv) {
userData) );

WAKU_CALL( waku_relay_subscribe(ctx,
"/waku/2/default-waku/proto",
"/waku/2/rs/0/0",
event_handler,
userData) );

Expand Down
2 changes: 1 addition & 1 deletion examples/filter_subscriber.nim
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ const
FilterPeer =
"/ip4/34.16.1.67/tcp/30303/p2p/16Uiu2HAmDCp8XJ9z1ev18zuv8NHekAsjNyezAvmMfFEJkiharitG"
# node-01.gc-us-central1-a.waku.test.statusim.net on waku.test
FilterPubsubTopic = PubsubTopic("/waku/2/default-waku/proto")
FilterPubsubTopic = PubsubTopic("/waku/2/rs/0/0")
FilterContentTopic = ContentTopic("/examples/1/light-pubsub-example/proto")

proc unsubscribe(
Expand Down
2 changes: 1 addition & 1 deletion examples/lightpush_publisher.nim
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ const
LightpushPeer =
"/ip4/178.128.141.171/tcp/30303/p2p/16Uiu2HAkykgaECHswi3YKJ5dMLbq2kPVCo89fcyTd38UcQD6ej5W"
# node-01.do-ams3.waku.test.statusim.net on waku.test
LightpushPubsubTopic = PubsubTopic("/waku/2/default-waku/proto")
LightpushPubsubTopic = PubsubTopic("/waku/2/rs/0/0")
LightpushContentTopic = ContentTopic("/examples/1/light-pubsub-example/proto")

proc publishMessages(
Expand Down
2 changes: 1 addition & 1 deletion examples/publisher.nim
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ proc setupAndPublish(rng: ref HmacDrbgContext) {.async.} =

# Make sure it matches the publisher. Use default value
# see spec: https://rfc.vac.dev/spec/23/
let pubSubTopic = PubsubTopic("/waku/2/default-waku/proto")
let pubSubTopic = PubsubTopic("/waku/2/rs/0/0")

# any content topic can be chosen
let contentTopic = ContentTopic("/examples/1/pubsub-example/proto")
Expand Down
2 changes: 1 addition & 1 deletion examples/subscriber.nim
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ proc setupAndSubscribe(rng: ref HmacDrbgContext) {.async.} =

# Make sure it matches the publisher. Use default value
# see spec: https://rfc.vac.dev/spec/23/
let pubSubTopic = PubsubTopic("/waku/2/default-waku/proto")
let pubSubTopic = PubsubTopic("/waku/2/rs/0/0")

# any content topic can be chosen. make sure it matches the publisher
let contentTopic = ContentTopic("/examples/1/pubsub-example/proto")
Expand Down
2 changes: 1 addition & 1 deletion tests/node/peer_manager/test_peer_manager.nim
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ suite "Peer Manager":

# When making an operation that triggers onPeerMetadata
discard await client.filterSubscribe(
some("/waku/2/default-waku/proto"), "waku/lightpush/1", serverRemotePeerInfo
some("/waku/2/rs/0/0"), "waku/lightpush/1", serverRemotePeerInfo
)
await sleepAsync(FUTURE_TIMEOUT)

Expand Down
2 changes: 1 addition & 1 deletion tests/test_wakunode.nim
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ suite "WakuNode":
node1 = newTestWakuNode(nodeKey1, parseIpAddress("0.0.0.0"), Port(61000))
nodeKey2 = generateSecp256k1Key()
node2 = newTestWakuNode(nodeKey2, parseIpAddress("0.0.0.0"), Port(61002))
pubSubTopic = "/waku/2/default-waku/proto"
pubSubTopic = "/waku/2/rs/0/0"
contentTopic = ContentTopic("/waku/2/default-content/proto")
payload = "hello world".toBytes()
message = WakuMessage(payload: payload, contentTopic: contentTopic)
Expand Down
2 changes: 1 addition & 1 deletion tests/testlib/wakunode.nim
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ proc defaultTestWakuNodeConf*(): WakuNodeConf =
maxConnections: 50,
maxMessageSize: "1024 KiB",
clusterId: 0,
pubsubTopics: @["/waku/2/rs/1/0"],
pubsubTopics: @["/waku/2/rs/0/0"],
relay: true,
storeMessageDbUrl: "sqlite://store.sqlite3",
)
Expand Down
41 changes: 19 additions & 22 deletions tests/waku_core/test_message_digest.nim
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@ suite "Waku Message - Deterministic hashing":
test "digest computation - empty meta field":
## Test vector:
##
## pubsub_topic = 0x2f77616b752f322f64656661756c742d77616b752f70726f746f
## pubsub_topic = 2f77616b752f322f72732f302f30
## waku_message.payload = 0x010203045445535405060708
## waku_message.content_topic = 0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f
## waku_message.meta = <empty>
## waku_message.ts = 0x175789bfa23f8400
##
## message_hash = 0xa2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8
## message_hash = 0xcccab07fed94181c83937c8ca8340c9108492b7ede354a6d95421ad34141fd37

## Given
let pubsubTopic = DefaultPubsubTopic # /waku/2/default-waku/proto
let pubsubTopic = DefaultPubsubTopic # /waku/2/rs/0/0
let message = fakeWakuMessage(
contentTopic = DefaultContentTopic, # /waku/2/default-content/proto
payload = "\x01\x02\x03\x04TEST\x05\x06\x07\x08".toBytes(),
Expand All @@ -29,29 +29,28 @@ suite "Waku Message - Deterministic hashing":

## Then
check:
byteutils.toHex(pubsubTopic.toBytes()) ==
"2f77616b752f322f64656661756c742d77616b752f70726f746f"
byteutils.toHex(pubsubTopic.toBytes()) == "2f77616b752f322f72732f302f30"
byteutils.toHex(message.contentTopic.toBytes()) ==
"2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f"
byteutils.toHex(message.payload) == "010203045445535405060708"
byteutils.toHex(message.meta) == ""
byteutils.toHex(toBytesBE(uint64(message.timestamp))) == "175789bfa23f8400"
messageHash.toHex() ==
"a2554498b31f5bcdfcbf7fa58ad1c2d45f0254f3f8110a85588ec3cf10720fd8"
"cccab07fed94181c83937c8ca8340c9108492b7ede354a6d95421ad34141fd37"

test "digest computation - meta field (12 bytes)":
## Test vector:
##
## pubsub_topic = 0x2f77616b752f322f64656661756c742d77616b752f70726f746f
## pubsub_topic = 0x2f77616b752f322f72732f302f30
## waku_message.payload = 0x010203045445535405060708
## waku_message.content_topic = 0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f
## waku_message.meta = 0x73757065722d736563726574
## waku_message.ts = 0x175789bfa23f8400
##
## message_hash = 0x64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05
## message_hash = 0xb9b4852f9d8c489846e8bfc6c5ca6a1a8d460a40d28832a966e029eb39619199

## Given
let pubsubTopic = DefaultPubsubTopic # /waku/2/default-waku/proto
let pubsubTopic = DefaultPubsubTopic # /waku/2/rs/0/0
let message = fakeWakuMessage(
contentTopic = DefaultContentTopic, # /waku/2/default-content/proto
payload = "\x01\x02\x03\x04TEST\x05\x06\x07\x08".toBytes(),
Expand All @@ -64,29 +63,28 @@ suite "Waku Message - Deterministic hashing":

## Then
check:
byteutils.toHex(pubsubTopic.toBytes()) ==
"2f77616b752f322f64656661756c742d77616b752f70726f746f"
byteutils.toHex(pubsubTopic.toBytes()) == "2f77616b752f322f72732f302f30"
byteutils.toHex(message.contentTopic.toBytes()) ==
"2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f"
byteutils.toHex(message.payload) == "010203045445535405060708"
byteutils.toHex(message.meta) == "73757065722d736563726574"
byteutils.toHex(toBytesBE(uint64(message.timestamp))) == "175789bfa23f8400"
messageHash.toHex() ==
"64cce733fed134e83da02b02c6f689814872b1a0ac97ea56b76095c3c72bfe05"
"b9b4852f9d8c489846e8bfc6c5ca6a1a8d460a40d28832a966e029eb39619199"

test "digest computation - meta field (64 bytes)":
## Test vector:
##
## pubsub_topic = 0x2f77616b752f322f64656661756c742d77616b752f70726f746f
## pubsub_topic = 0x2f77616b752f322f72732f302f30
## waku_message.payload = 0x010203045445535405060708
## waku_message.content_topic = 0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f
## waku_message.meta = 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f
## waku_message.ts = 0x175789bfa23f8400
##
## message_hash = 0x7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27"
## message_hash = 0x653460d04f66c5b11814d235152f4f246e6f03ef80a305a825913636fbafd0ba

## Given
let pubsubTopic = DefaultPubsubTopic # /waku/2/default-waku/proto
let pubsubTopic = DefaultPubsubTopic # /waku/2/rs/0/0
let message = fakeWakuMessage(
contentTopic = DefaultContentTopic, # /waku/2/default-content/proto
payload = "\x01\x02\x03\x04TEST\x05\x06\x07\x08".toBytes(),
Expand All @@ -99,30 +97,29 @@ suite "Waku Message - Deterministic hashing":

## Then
check:
byteutils.toHex(pubsubTopic.toBytes()) ==
"2f77616b752f322f64656661756c742d77616b752f70726f746f"
byteutils.toHex(pubsubTopic.toBytes()) == "2f77616b752f322f72732f302f30"
byteutils.toHex(message.contentTopic.toBytes()) ==
"2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f"
byteutils.toHex(message.payload) == "010203045445535405060708"
byteutils.toHex(message.meta) ==
"000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f"
byteutils.toHex(toBytesBE(uint64(message.timestamp))) == "175789bfa23f8400"
messageHash.toHex() ==
"7158b6498753313368b9af8f6e0a0a05104f68f972981da42a43bc53fb0c1b27"
"653460d04f66c5b11814d235152f4f246e6f03ef80a305a825913636fbafd0ba"

test "digest computation - zero length payload":
## Test vector:
##
## pubsub_topic = 0x2f77616b752f322f64656661756c742d77616b752f70726f746f
## pubsub_topic = 0x2f77616b752f322f72732f302f30
## waku_message.payload = []
## waku_message.content_topic = 0x2f77616b752f322f64656661756c742d636f6e74656e742f70726f746f
## waku_message.meta = 0x73757065722d736563726574
## waku_message.ts = 0x175789bfa23f8400
##
## message_hash = 0x483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de4
## message_hash = 0x0f6448cc23b2db6c696aa6ab4b693eff4cf3549ff346fe1dbeb281697396a09f

## Given
let pubsubTopic = DefaultPubsubTopic # /waku/2/default-waku/proto
let pubsubTopic = DefaultPubsubTopic # /waku/2/rs/0/0
let message = fakeWakuMessage(
contentTopic = DefaultContentTopic, # /waku/2/default-content/proto
payload = newSeq[byte](),
Expand All @@ -136,7 +133,7 @@ suite "Waku Message - Deterministic hashing":
## Then
check:
messageHash.toHex() ==
"483ea950cb63f9b9d6926b262bb36194d3f40a0463ce8446228350bd44e96de4"
"0f6448cc23b2db6c696aa6ab4b693eff4cf3549ff346fe1dbeb281697396a09f"

test "waku message - check meta size is enforced":
# create message with meta size > 64 bytes (invalid)
Expand Down
20 changes: 4 additions & 16 deletions tests/waku_core/test_namespaced_topics.nim
Original file line number Diff line number Diff line change
Expand Up @@ -134,17 +134,6 @@ suite "Waku Message - Content topics namespacing":
err.cause == "generation should be a numeric value"

suite "Waku Message - Pub-sub topics namespacing":
test "Stringify named sharding pub-sub topic":
## Given
var ns = NsPubsubTopic.named("waku-dev")

## When
let topic = $ns

## Then
check:
topic == "/waku/2/waku-dev"

test "Stringify static sharding pub-sub topic":
## Given
var ns = NsPubsubTopic.staticSharding(clusterId = 0, shardId = 2)
Expand All @@ -156,19 +145,18 @@ suite "Waku Message - Pub-sub topics namespacing":
check:
topic == "/waku/2/rs/0/2"

test "Parse named pub-sub topic string - Valid string":
test "Parse invalid pub-sub topic string":
## Given
let topic = "/waku/2/waku-dev"

## When
let nsRes = NsPubsubTopic.parse(topic)

## Then
check nsRes.isOk()

let ns = nsRes.get()
check nsRes.isErr()
let err = nsRes.tryError()
check:
ns.name == "waku-dev"
err.kind == ParsingErrorKind.InvalidFormat

test "Parse static sharding pub-sub topic string - Valid string":
## Given
Expand Down
24 changes: 0 additions & 24 deletions tests/waku_core/topics/test_pubsub_topic.nim
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,3 @@ suite "Static Sharding Functionality":
topic.clusterId == 0
topic.shardId == 1
topic == "/waku/2/rs/0/1"

suite "Automatic Sharding Mechanics":
test "Shard Selection Algorithm":
let
topic1 = NsPubsubTopic.parseNamedSharding("/waku/2/xxx").get()
topic2 = NsPubsubTopic.parseNamedSharding("/waku/2/123").get()
topic3 = NsPubsubTopic.parseNamedSharding("/waku/2/xxx123").get()

check:
# topic1.shardId == 1
# topic1.clusterId == 0
topic1 == NsPubsubTopic.staticSharding(0, 1)
# topic2.shardId == 1
# topic2.clusterId == 0
topic2 == NsPubsubTopic.staticSharding(0, 1)
# topic3.shardId == 1
# topic3.clusterId == 0
topic3 == NsPubsubTopic.staticSharding(0, 1)

test "Shard Selection Algorithm without topicName":
let topicResult = NsPubsubTopic.parseNamedSharding("/waku/2/")

check:
topicResult.isErr()
Loading

0 comments on commit e1518cf

Please sign in to comment.