From b85906f05c1a3f4b1c13b1bc48e950c3392b25f5 Mon Sep 17 00:00:00 2001
From: lrq619 <lrq619@sjtu.edu.cn>
Date: Tue, 11 Jul 2023 09:40:09 +0800
Subject: [PATCH] Add docs for detailed add new benchmark in vSwarm

Signed-off-by: lrq619 <lrq619@sjtu.edu.cn>
---
 docs/adding_new_benchmark.md | 66 ++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)
 create mode 100644 docs/adding_new_benchmark.md

diff --git a/docs/adding_new_benchmark.md b/docs/adding_new_benchmark.md
new file mode 100644
index 00000000..7c89d701
--- /dev/null
+++ b/docs/adding_new_benchmark.md
@@ -0,0 +1,66 @@
+This is a detailed document of how to add a new benchmark in vSwarm.
+# Add Standalone Benchmarks in vSwarm
+## Python
+### 1. Add protobuf
+The first thing we need to do is to add protobuf in [vSwarm-proto](https://github.com/vhive-serverless/vSwarm-proto).
+
+First, clone [vSwarm-proto](https://github.com/vhive-serverless/vSwarm-proto), operations below should be taken in your local `vSwarm-proto/` folder.
+
+Add a new folder to `proto/` named by your benchmark `{bench}`, add file `{bench}.proto` to this `proto/{bench}/` folder, which should define the protobuf of your benchmark.
+
+In `{bench}.proto` file, define the needed grpc service. Note that invoker and most of vHive work with the simple [`HelloWorld`](https://grpc.io/docs/what-is-grpc/introduction/) service. However, you can implement arbitrary services for your function as the relay, sitting in between the invoker and your function, can be used to make translations. Refer to [aes.proto](https://github.com/vhive-serverless/vSwarm-proto/blob/main/proto/aes/aes.proto) and the corresponding [translation](https://github.com/vhive-serverless/vSwarm-proto/blob/main/grpcclient/aes_client.go) as an example of a custom service. 
+```
+// The greeting service definition.
+service Greeter {
+  // Sends a greeting
+  rpc SayHello (HelloRequest) returns (HelloReply) {}
+}
+```
+After defining your `HelloRequest` and `HelloReply`, do the following to generate protobuf:
+1. Repeat the steps in `vSwarm-proto/.github/.workflows/build-tests.yml` to install `pipenv` and `protoc`.
+2. Add a line in  `go.mod` file's `replace` module:
+```
+github.com/vhive-serverless/vSwarm-proto/proto/{bench} => ./proto/{bench}
+```
+3. Run the following two commands to generate the `protocbuf` for your benchmark:
+```
+cd vSwarm-proto/proto/{bench}
+pipenv install 
+pipenv run python3 -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. ./proto/bert/*.proto
+```
+This generates protobuf files for python
+```
+protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative ./proto/bert/*.proto
+```
+This generates protobuf files for go.
+
+Alternatively, run `make proto-all` in the root of vSwarm-proto, which does both steps for all protocols automatically.
+
+Then, in `vSwarm-proto/grpcclient` folder, add a file named `{bench}_client.go`. This is the client that the relay will select to invoke your function. Each client must implement [`GrpcClient`](https://github.com/vhive-serverless/vSwarm-proto/blob/70781f2339083f8c6d298ce0be13e7b7869c1409/grpcclient/grpcclient.go#L79) interface. 
+`Init` and `Close` are used to set up and close the connection to the function. 
+`GetGenerator` will must return an generator for creating new input values for your function.
+`Request` is the actual invocation of the function.
+Refer to other `*_client.go` clients for more details about how to implement those functions.
+
+Finally, in `vSwarm-proto/grpcclient/getclient.go`, add your `{bench}` to `func FindServiceName` and `func FindGrpcClient`
+
+After these are done, commit and push to the upstream branch.
+
+Below operations should be taken under `vSwarm` instead of `vSwarm-proto`
+### 2.Modify relay
+Assume you have changed `vSwarm-proto` but the change hasn't been merged into main branch, if you want to test its functionality locally, you should build a local relay docker image to replace the one on the cloud, so:
+```
+cd tools/bin/relay
+# commit id is the commit you have made to vSwarm-proto repo
+go get github.com/vhive-serverless/vSwarm-proto@{commit id}
+make all
+```
+
+Now, you have a local relay image that uses your modified version of protoc.
+Remember **DO NOT** commit changes in `tools/relay` to the remote repo
+
+### 3.Dockerize your benchmark
+This is a case-by-case process, guidelines on how to do this will be given in the future.
+
+### 4.Adding CI
+Guidelines on how to do this will be given in the future