Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: liteprotocoltester for simulation #2813

Merged
merged 12 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions apps/liteprotocoltester/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
START_PUBLISHING_AFTER=10
# can add some seconds delay before SENDER starts publishing

NUM_MESSAGES=0
# 0 for infinite number of messages

DELAY_MESSAGES=8000
# ms delay between messages


MIN_MESSAGE_SIZE=15Kb
MAX_MESSAGE_SIZE=145Kb

## for wakusim
PUBSUB=/waku/2/rs/66/0
CONTENT_TOPIC=/tester/2/light-pubsub-test/wakusim
CLUSTER_ID=66

## for status.prod
#PUBSUB=/waku/2/rs/16/32
#CONTENT_TOPIC=/tester/2/light-pubsub-test/fleet
#CLUSTER_ID=16

## for TWN
#PUBSUB=/waku/2/rs/1/4
#CONTENT_TOPIC=/tester/2/light-pubsub-test/twn
#CLUSTER_ID=1
36 changes: 36 additions & 0 deletions apps/liteprotocoltester/Dockerfile.liteprotocoltester
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# TESTING IMAGE --------------------------------------------------------------

## NOTICE: This is a short cut build file for ubuntu users who compiles nwaku in ubuntu distro.
## This is used for faster turnaround time for testing the compiled binary.
## Prerequisites: compiled liteprotocoltester binary in build/ directory

FROM ubuntu:noble AS prod

LABEL maintainer="[email protected]"
LABEL source="https://github.com/waku-org/nwaku"
LABEL description="Lite Protocol Tester: Waku light-client"
LABEL commit="unknown"
LABEL version="unknown"

# DevP2P, LibP2P, and JSON RPC ports
EXPOSE 30303 60000 8545

# Referenced in the binary
RUN apt-get update && apt-get install -y --no-install-recommends \
libgcc1 \
libpcre3 \
libpq-dev \
wget \
iproute2 \
&& rm -rf /var/lib/apt/lists/*

# Fix for 'Error loading shared library libpcre.so.3: No such file or directory'
RUN ln -s /usr/lib/libpcre.so /usr/lib/libpcre.so.3

COPY build/liteprotocoltester /usr/bin/
COPY apps/liteprotocoltester/run_tester_node.sh /usr/bin/

ENTRYPOINT ["/usr/bin/run_tester_node.sh", "/usr/bin/liteprotocoltester"]

# # By default just show help if called without arguments
CMD ["--help"]
4 changes: 2 additions & 2 deletions apps/liteprotocoltester/Dockerfile.liteprotocoltester.compile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
ARG NIMFLAGS
ARG MAKE_TARGET=liteprotocoltester
ARG NIM_COMMIT
ARG LOG_LEVEL=TRACE
ARG LOG_LEVEL=DEBUG

# Get build tools and required header files
RUN apk add --no-cache bash git build-base pcre-dev linux-headers curl jq
Expand All @@ -27,7 +27,7 @@

# PRODUCTION IMAGE -------------------------------------------------------------

FROM alpine:3.18 as prod
FROM alpine:3.18 AS prod

ARG MAKE_TARGET=liteprotocoltester

Expand Down
2 changes: 1 addition & 1 deletion apps/liteprotocoltester/Dockerfile.liteprotocoltester.copy
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
## This is used for faster turnaround time for testing the compiled binary.
## Prerequisites: compiled liteprotocoltester binary in build/ directory

FROM ubuntu:noble as prod
FROM ubuntu:noble AS prod

LABEL maintainer="[email protected]"
LABEL source="https://github.com/waku-org/nwaku"
Expand Down
126 changes: 123 additions & 3 deletions apps/liteprotocoltester/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ At this stage we can only configure number of messages and fixed frequency of th

### Phase 1

> NOTICE: This part is obsolate due integration with waku-simulator.
> It needs some rework to make it work again standalone.

Lite Protocol Tester application is built under name `liteprotocoltester` in apps/liteprotocoltester folder.

Starting from nwaku repository root:
Expand All @@ -48,16 +51,102 @@ docker compose up -d
docker compose logs -f receivernode
```

### Phase 2

> Integration with waku-simulator!

- For convenience, integration is done in cooperation with waku-simulator repository, but nothing is tightly coupled.
- waku-simulator must be started separately with its own configuration.
- To enable waku-simulator working without RLN currently a separate branch is to be used.
- When waku-simulator is configured and up and running, lite-protocol-tester composite docker setup can be started.

```bash

# Start waku-simulator

git clone https://github.com/waku-org/waku-simulator.git ../waku-simulator
cd ../waku-simulator
git checkout chore-integrate-liteprotocoltester

# optionally edit .env file

docker compose -f docker-compose-norln.yml up -d

# navigate localhost:30001 to see the waku-simulator dashboard

cd ../{your-repository}

make LOG_LEVEL=DEBUG liteprotocoltester

cd apps/liteprotocoltester

# optionally edit .env file

docker compose -f docker-compose-on-simularor.yml build
docker compose -f docker-compose-on-simularor.yml up -d
docker compose -f docker-compose-on-simularor.yml logs -f receivernode
```
#### Current setup

- waku-simulator is configured to run with 25 full node
- liteprotocoltester is configured to run with 3 publisher and 1 receiver
- liteprotocoltester is configured to run 1 lightpush service and a filter service node
- light clients are connected accordingly
- publishers will send 250 messages in every 200ms with size between 1KiB and 120KiB
- Notice there is a configurable wait before start publishing messages as it is noticed time is needed for the service nodes to get connected to full nodes from simulator
- light clients will print report on their and the connected service node's connectivity to the network in every 20 secs.

### Phase 3

> Run independently on a chosen waku fleet

This option is simple as is just to run the built liteprotocoltester binary with run_tester_node.sh script.

Syntax:
`./run_tester_node.sh <path-to-liteprotocoltester-binary> <SENDER|RECEIVER> <service-node-address>`

How to run from you nwaku repository:
```bash
cd ../{your-repository}

make LOG_LEVEL=DEBUG liteprotocoltester

cd apps/liteprotocoltester

# optionally edit .env file

# run publisher side
./run_tester_node.sh ../../build/liteprotocoltester SENDER [chosen service node address that support lightpush]

# or run receiver side
./run_tester_node.sh ../../build/liteprotocoltester RECEIVER [chosen service node address that support filter service]
```

#### Recommendations

In order to run on any kind of network, it is recommended to deploy the built `liteprotocoltester` binary with the `.env` file and the `run_tester_node.sh` script to the desired machine.

Select a lightpush service node and a filter service node from the targeted network, or you can run your own. Note down the selected peers peer_id.

Run a SENDER role liteprotocoltester and a RECEIVER role one on different terminals. Depending on the test aim, you may want to redirect the output to a file.

> RECEIVER side will periodically print statistics to standard output.

## Configure

### Environment variables for docker compose runs

| Variable | Description | Default |
| ---: | :--- | :--- |
| NUM_MESSAGES | Number of message to publish | 120 |
| NUM_MESSAGES | Number of message to publish, 0 means infinite | 120 |
| DELAY_MESSAGES | Frequency of messages in milliseconds | 1000 |
| PUBSUB | Used pubsub_topic for testing | /waku/2/rs/0/0 |
| PUBSUB | Used pubsub_topic for testing | /waku/2/rs/66/0 |
| CONTENT_TOPIC | content_topic for testing | /tester/1/light-pubsub-example/proto |
| CLUSTER_ID | cluster_id of the network | 16 |
| START_PUBLISHING_AFTER | Delay in seconds before starting to publish to let service node connected | 5 |
| MIN_MESSAGE_SIZE | Minimum message size in bytes | 1KiB |
| MAX_MESSAGE_SIZE | Maximum message size in bytes | 120KiB |


### Lite Protocol Tester application cli options

Expand All @@ -67,7 +156,10 @@ docker compose logs -f receivernode
| --service-node| Address of the service node to use for lightpush and/or filter service | - |
| --num-messages | Number of message to publish | 120 |
| --delay-messages | Frequency of messages in milliseconds | 1000 |
| --pubsub-topic | Used pubsub_topic for testing | /waku/2/rs/0/0 |
| --min-message-size | Minimum message size in bytes | 1KiB |
| --max-message-size | Maximum message size in bytes | 120KiB |
| --start-publishing-after | Delay in seconds before starting to publish to let service node connected in seconds | 5 |
| --pubsub-topic | Used pubsub_topic for testing | /waku/2/default-waku/proto |
| --content_topic | content_topic for testing | /tester/1/light-pubsub-example/proto |
| --cluster-id | Cluster id for the test | 0 |
| --config-file | TOML configuration file to fine tune the light waku node<br>Note that some configurations (full node services) are not taken into account | - |
Expand All @@ -82,6 +174,34 @@ docker compose logs -f receivernode

### Docker image notice

#### Building for docker compose runs on simulator or standalone
Please note that currently to ease testing and development tester application docker image is based on ubuntu and uses the externally pre-built binary of 'liteprotocoltester'.
This speeds up image creation. Another dokcer build file is provided for proper build of boundle image.

> `Dockerfile.liteprotocoltester.copy` will create an image with the binary copied from the build directory.

> `Dockerfile.liteprotocoltester.compile` will create an image completely compiled from source. This can be quite slow.

#### Creating standalone runner docker image

To ease the work with lite-proto-tester, a docker image is possible to build.
With that image it is easy to run the application in a container.

> `Dockerfile.liteprotocoltester` will create an ubuntu image with the binary copied from the build directory. You need to pre-build the application.

Here is how to build and run:
```bash
cd <your-repository>
make liteprotocoltester

cd apps/liteprotocoltester
docker build -t liteprotocoltester:latest -f Dockerfile.liteprotocoltester ../..

# alternatively you can push it to a registry

# edit and adjust .env file to your needs and for the network configuration

docker run --env-file .env liteprotocoltester:latest RECEIVER <service-node-ip4-peer-address>

docker run --env-file .env liteprotocoltester:latest SENDER <service-node-ip4-peer-address>
```
96 changes: 96 additions & 0 deletions apps/liteprotocoltester/diagnose_connections.nim
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
when (NimMajor, NimMinor) < (1, 4):
{.push raises: [Defect].}
else:
{.push raises: [].}

import
std/[options, strutils, os, sequtils, net, strformat],
chronicles,
chronos,
metrics,
libbacktrace,
system/ansi_c,
libp2p/crypto/crypto,
confutils,
libp2p/wire

import
../../waku/common/logging,
../../waku/factory/waku,
../../waku/factory/external_config,
../../waku/node/health_monitor,
../../waku/node/waku_metrics,
../../waku/waku_api/rest/builder as rest_server_builder,
../../waku/node/peer_manager,
../../waku/waku_lightpush/common,
../../waku/waku_relay,
../../waku/waku_filter_v2,
../../waku/waku_api/rest/client,
../../waku/waku_api/rest/admin/client,
./tester_config,
./lightpush_publisher,
./filter_subscriber

logScope:
topics = "diagnose connections"

proc logSelfPeersLoop(pm: PeerManager, interval: Duration) {.async.} =
trace "Starting logSelfPeersLoop diagnosis loop"
while true:
let selfLighpushPeers = pm.peerStore.getPeersByProtocol(WakuLightPushCodec)
let selfRelayPeers = pm.peerStore.getPeersByProtocol(WakuRelayCodec)
let selfFilterPeers = pm.peerStore.getPeersByProtocol(WakuFilterSubscribeCodec)

let printable = catch:
"""*------------------------------------------------------------------------------------------*
| Self ({pm.switch.peerInfo}) peers:
*------------------------------------------------------------------------------------------*
| Lightpush peers({selfLighpushPeers.len()}): ${selfLighpushPeers}
*------------------------------------------------------------------------------------------*
| Filter peers({selfFilterPeers.len()}): ${selfFilterPeers}
*------------------------------------------------------------------------------------------*
| Relay peers({selfRelayPeers.len()}): ${selfRelayPeers}
*------------------------------------------------------------------------------------------*""".fmt()

if printable.isErr():
echo "Error while printing statistics: " & printable.error().msg
else:
echo printable.get()

await sleepAsync(interval)

proc logServiceRelayPeers(
pm: PeerManager, codec: string, interval: Duration
) {.async.} =
trace "Starting service node connectivity diagnosys loop"
while true:
echo "*------------------------------------------------------------------------------------------*"
echo "| Service peer connectivity:"
let selfLighpushPeers = pm.selectPeer(codec)
if selfLighpushPeers.isSome():
let ma = selfLighpushPeers.get().addrs[0]
var serviceIp = initTAddress(ma).valueOr:
echo "Error while parsing multiaddress: " & $error
continue

serviceIp.port = Port(8645)
let restClient = newRestHttpClient(initTAddress($serviceIp))

let getPeersRes = await restClient.getPeers()

if getPeersRes.status == 200:
let nrOfPeers = getPeersRes.data.len()
echo "Service node (@" & $ma & ") peers: " & $getPeersRes.data
else:
echo "Error while fetching service node (@" & $ma & ") peers: " &
$getPeersRes.data
else:
echo "No service node peers found"

echo "*------------------------------------------------------------------------------------------*"

await sleepAsync(interval)

proc startPeriodicPeerDiagnostic*(pm: PeerManager, codec: string) {.async.} =
asyncSpawn logSelfPeersLoop(pm, chronos.seconds(60))
# asyncSpawn logServiceRelayPeers(pm, codec, chronos.seconds(20))
Loading
Loading