diff --git a/benchmarks/data-analytics/README.md b/benchmarks/data-analytics/README.md index 6616389a6..660968d4d 100644 --- a/benchmarks/data-analytics/README.md +++ b/benchmarks/data-analytics/README.md @@ -12,28 +12,28 @@ Supported tags and their respective `Dockerfile` links: - [`Base`][basedocker]: This image contains the hadoop base which is needed for both master and slave images. - [`Master`][masterdocker]: This image contains the main benchmark (hadoop master node, and mahout). - - [`Slave`][slavedocker]: This image contains the hadoop slave image. + - [`Slave`][slavedocker]: This image contains the hadoop slave image. - [`Data`][datasetdocker]: This image contains the dataset used by the benchmark. -These images are automatically built using the mentioned Dockerfiles available on `cloudsuite/benchmarks/data-analytics/` [GitHub repo][repo]. +These images are automatically built using the mentioned Dockerfiles available on `ParsaLab/cloudsuite` [GitHub repo][repo]. ## Starting the volume image ## This benchmark uses a Wikipedia dataset of ~30GB. We prepared a dataset image for training dataset, to download it once, and use it to run the benchmark. You can pull this image from Docker Hub. - $ docker pull cloudsuite/dataanalytics/dataset + $ docker pull cloudsuite/data-analytics:dataset The following command will start the volume image, making the data available for other docker images on the host: - $ docker create --name data cloudsuite/dataanalytics/dataset + $ docker create --name data cloudsuite/data-analytics:dataset ## Starting the Master ## To start the master you first have to `pull` the master image. - $ docker pull cloudsuite/dataanalytics/master + $ docker pull cloudsuite/data-analytics:master Then, run the benchmark with the following command: - $ docker run -d -t --dns 127.0.0.1 -P --name master -h master.cloudsuite.com --volumes-from data cloudsuite/dataanalytics/master + $ docker run -d -t --dns 127.0.0.1 -P --name master -h master.cloudsuite.com --volumes-from data cloudsuite/data-analytics:master ## Starting the Slaves ## @@ -41,7 +41,7 @@ If you want to have a single-node cluster, please skip this step. To have more than one node, you need to start the slave containers. In order to do that, you first need to `pull` the slave image. - $ docker pull cloudsuite/dataanalytics/slave + $ docker pull cloudsuite/data-analytics:slave To connect the slave containers to the master, you need the master IP. @@ -49,9 +49,9 @@ To connect the slave containers to the master, you need the master IP. Then, run as many slave containers as you want: - $ docker run -d -t --dns 127.0.0.1 -P --name slave$i -h slave$i.cloudsuite.com -e JOIN_IP=$FIRST_IP cloudsuite/dataanalytics/slave + $ docker run -d -t --dns 127.0.0.1 -P --name slave$i -h slave$i.cloudsuite.com -e JOIN_IP=$FIRST_IP cloudsuite/data-analytics:slave -Where `$i` is the slave number, you should start with 1 (i.e., slave1, slave1.cloudsuite.com, slave2, slave2.cloudsuite.com, ...). +Where `$i` is the slave number, you should start with 1 (i.e., slave1, slave1.cloudsuite.com, slave2, slave2.cloudsuite.com, ...). ## Running the benchmark ## @@ -64,16 +64,15 @@ Then, run the benchmark with the following command: $ ./run.sh -It asks you to enter the number of slaves, if you have a single-node cluster, please enter 0. +It asks you to enter the number of slaves, if you have a single-node cluster, please enter 0. After entering the slave number, it prepares hadoop, downloads the dataset (it takes a lot of time to download this dataset) and runs the benchmark. After the benchmark finishes, the model will be available in HDFS, under the wikipediamodel directory. -[basedocker]: https://github.com/CloudSuite-EPFL/DataAnalytics/blob/master/Dockerfile "Base Dockerfile" -[masterdocker]: https://github.com/CloudSuite-EPFL/DataAnalytics/blob/master/Dockerfile "Master Dockerfile" -[slavedocker]: https://github.com/CloudSuite-EPFL/DataAnalytics/blob/master/Dockerfile "Slave Dockerfile" +[basedocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-analytics/master/Dockerfile "Base Dockerfile" +[masterdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-analytics/master/Dockerfile "Master Dockerfile" +[slavedocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-analytics/slave/Dockerfile "Slave Dockerfile" +[datasetdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-analytics/dataset/Dockerfile "Dataset Dockerfile" -[datasetdocker]: https://github.com/CloudSuite-EPFL/DataAnalytics/blob/master/dataset/Dockerfile "Dataset Dockerfile" - -[repo]: https://github.com/ParsaLab/cloudsuite/tree/master/benchmarks/data-analytics "GitHub Repo" -[dhrepo]: https://hub.docker.com/r/cloudsuite/dataanalytics/ "DockerHub Page" -[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/dataanalytics.svg "Go to DockerHub Page" -[dhstars]: https://img.shields.io/docker/stars/cloudsuite/dataanalytics.svg "Go to DockerHub Page" +[repo]: https://github.com/ParsaLab/cloudsuite "GitHub Repo" +[dhrepo]: https://hub.docker.com/r/cloudsuite/data-analytics/ "DockerHub Page" +[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/data-analytics.svg "Go to DockerHub Page" +[dhstars]: https://img.shields.io/docker/stars/cloudsuite/data-analytics.svg "Go to DockerHub Page" diff --git a/benchmarks/data-caching/README.md b/benchmarks/data-caching/README.md index 0761fe6fe..5651ff993 100644 --- a/benchmarks/data-caching/README.md +++ b/benchmarks/data-caching/README.md @@ -18,7 +18,7 @@ Supported tags and their respective `Dockerfile` links: - [`server`][serverdocker]: This represents the Memcached server running as a daemon. - [`client`][clientdocker]: This represents the client which requests to access the server's data. -These images are automatically built using the mentioned Dockerfiles available on `CloudSuite-EPFL/DataCaching` [GitHub repo][repo]. +These images are automatically built using the mentioned Dockerfiles available on `ParsaLab/cloudsuite` [GitHub repo][repo]. ### Preparing a network between the server(s) and the client @@ -31,32 +31,32 @@ We will attach the launched containers to this newly created docker network. ### Starting the Server #### To start the server you have to first `pull` the server image and then run it. To `pull` the server image use the following command: - $ docker pull cloudsuite/datacaching:server + $ docker pull cloudsuite/data-caching:server It takes some time to download the image, but this is only required the first time. The following command will start the server with four threads and 4096MB of dedicated memory, with a minimal object size of 550 bytes listening on port 11211 as default: - $ docker run --name dc-server --net caching_network -d cloudsuite/datacaching:server -t 4 -m 4096 -n 550 + $ docker run --name dc-server --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550 We assigned a name to this server to facilitate linking it with the client. We also used `--net` option to attach the container to our prepared network. As mentioned before, you can have multiple instances of the Memcached server, just remember to give each of them a unique name. For example, the following commands create four Memcached server instances: - $ docker run --name dc-server1 --net caching_network -d cloudsuite/datacaching:server -t 4 -m 4096 -n 550 - $ docker run --name dc-server2 --net caching_network -d cloudsuite/datacaching:server -t 4 -m 4096 -n 550 - $ docker run --name dc-server3 --net caching_network -d cloudsuite/datacaching:server -t 4 -m 4096 -n 550 - $ docker run --name dc-server4 --net caching_network -d cloudsuite/datacaching:server -t 4 -m 4096 -n 550 + $ docker run --name dc-server1 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550 + $ docker run --name dc-server2 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550 + $ docker run --name dc-server3 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550 + $ docker run --name dc-server4 --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550 ### Starting the Client #### To start the client you have to first `pull` the client image and then run it. To `pull` the server image use the following command: - $ docker pull cloudsuite/datacaching:client + $ docker pull cloudsuite/data-caching:client It takes some time to download the image, but this is only required the first time. To start the client container use the following command: - $ docker run -it --name dc-client --net caching_network cloudsuite/datacaching:client bash + $ docker run -it --name dc-client --net caching_network cloudsuite/data-caching:client bash This boots up the client container and you'll be logged in as the `memcache` user. Note that by using the `--net` option, you can easily make these containers visible to each other. @@ -133,11 +133,11 @@ and the client on different sockets of the same machine [memcachedWeb]: http://memcached.org/ "Memcached Website" - [serverdocker]: https://github.com/CloudSuite-EPFL/DataCaching/blob/master/server/Dockerfile "Server Dockerfile" + [serverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-caching/server/Dockerfile "Server Dockerfile" - [clientdocker]: https://github.com/CloudSuite-EPFL/DataCaching/blob/master/client/Dockerfile "Client Dockerfile" + [clientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-caching/client/Dockerfile "Client Dockerfile" - [repo]: https://github.com/CloudSuite-EPFL/DataCaching "GitHub Repo" - [dhrepo]: https://hub.docker.com/r/cloudsuite/datacaching/ "DockerHub Page" - [dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/datacaching.svg "Go to DockerHub Page" - [dhstars]: https://img.shields.io/docker/stars/cloudsuite/datacaching.svg "Go to DockerHub Page" + [repo]: https://github.com/ParsaLab/cloudsuite "GitHub Repo" + [dhrepo]: https://hub.docker.com/r/cloudsuite/data-caching/ "DockerHub Page" + [dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/data-caching.svg "Go to DockerHub Page" + [dhstars]: https://img.shields.io/docker/stars/cloudsuite/data-caching.svg "Go to DockerHub Page" diff --git a/benchmarks/data-caching/client/README.md b/benchmarks/data-caching/client/README.md index 7e2038729..d20789160 100644 --- a/benchmarks/data-caching/client/README.md +++ b/benchmarks/data-caching/client/README.md @@ -4,5 +4,5 @@ This `Dockerfile` creates an ubuntu (latest) image representing the Memcached cl Example: - $ docker pull cloudsuite/datacaching:client - $ docker run -it --name dc-client --link=dc-server cloudsuite/datacaching:client bash + $ docker pull cloudsuite/data-caching:client + $ docker run -it --name dc-client --net caching_network cloudsuite/data-caching:client bash diff --git a/benchmarks/data-caching/server/README.md b/benchmarks/data-caching/server/README.md index 1b74348bc..30db202ef 100644 --- a/benchmarks/data-caching/server/README.md +++ b/benchmarks/data-caching/server/README.md @@ -4,5 +4,5 @@ This `Dockerfile` creates an ubuntu (latest) image containing the latest version Memcached will be started as a daemon with the passed parameters. Example: - $ docker pull cloudsuite/datacaching:server - $ docker run --name dc-server -d cloudsuite/datacaching:server -t 4 -m 4096 -n 550 + $ docker pull cloudsuite/data-caching:server + $ docker run --name dc-server --net caching_network -d cloudsuite/data-caching:server -t 4 -m 4096 -n 550 diff --git a/benchmarks/data-serving/README.md b/benchmarks/data-serving/README.md index f8b460add..dd725b8a9 100644 --- a/benchmarks/data-serving/README.md +++ b/benchmarks/data-serving/README.md @@ -17,10 +17,10 @@ To facilitate the communication between the client and the server(s), we build a We will attach the launched containers to this newly created docker network. -###Server Container +### Server Container Start the server container: - $ docker run -it --name cassandra-server --net serving_network cloudsuite/dataserving:server bash + $ docker run -it --name cassandra-server --net serving_network cloudsuite/data-serving:server bash In order to create a keyspace and a column family, you can use the following commands after connecting to the server with the cassandra-cli under the directory in which Cassandra is unpacked. (A link to a basic tutorial with cassandra-cli: http://wiki.apache.org/cassandra/CassandraCli) @@ -49,13 +49,13 @@ You can use other commands in the cassandra-cli to verify the correctness of the If you make a mistake you can use the *drop* command and try again: $ drop keyspace usertable; - -###Multiple Server Containers + +### Multiple Server Containers For a cluster setup with multiple servers, we need to instantiate a seed server: ``` -$ docker run -it --name cassandra-server-seed --net serving_network cloudsuite/dataserving:server bash +$ docker run -it --name cassandra-server-seed --net serving_network cloudsuite/data-serving:server bash ``` Then we prepare the server as previously. @@ -63,16 +63,16 @@ Then we prepare the server as previously. The other server containers are instantiated as follows: ``` -$ docker run -it --name cassandra-server(id) --net serving_network -e CASSANDRA_SEEDS=cassandra-server-seed cloudsuite/dataserving:server bash +$ docker run -it --name cassandra-server(id) --net serving_network -e CASSANDRA_SEEDS=cassandra-server-seed cloudsuite/data-serving:server bash ``` You can find more details at the websites: http://wiki.apache.org/cassandra/GettingStarted and https://hub.docker.com/_/cassandra/. -###Client Container(s) +### Client Container(s) After successfully creating the aforementioned schema, you are ready to benchmark with YCSB. Start the client container: - $ docker run -it --name cassandra-client --link cassandra-server:server cloudsuite/dataserving:client bash + $ docker run -it --name cassandra-client --link cassandra-server:server cloudsuite/data-serving:client bash Change to the ycsb directory: ``` @@ -86,7 +86,7 @@ or, for a "one seed - one normal server" setup: ``` $ export HOSTS="cassandra-server-seed,cassandra-server1" ``` -Load dataset on ycsb: +Load dataset on ycsb: ``` $ ./bin/ycsb load cassandra-10 -p hosts=$HOSTS -P workloads/workloada ``` @@ -113,15 +113,14 @@ Running the benchmark --------------------- After you install and run the server, install the YCSB framework files and populate Cassandra, you are one step away from running the benchmark. To specify the runtime parameters for the client, a good practice is to create a settings file. You can keep the important parameters (e.g., *target*, *threadcount*, *hosts*, *operationcount*, *recordcount*) in this file. -The *settings.dat* file defines the IP address(es) of the node(s) running Cassandra, in addition to the *recordcount* parameter (which should be less than or equal to the number specified in the data generation step to avoid potential errors). +The *settings.dat* file defines the IP address(es) of the node(s) running Cassandra, in addition to the *recordcount* parameter (which should be less than or equal to the number specified in the data generation step to avoid potential errors). -The *operationcount* parameter sets the number of operations to be executed on the data store. +The *operationcount* parameter sets the number of operations to be executed on the data store. The *run.command* file takes the *settings.dat* file as an input and runs the following command: $ /ycsb/bin/ycsb run cassandra-10 -p hosts=server -P /ycsb/workloads/workloada -[dhrepo]: https://hub.docker.com/r/cloudsuite/dataserving/ "DockerHub Page" -[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/dataserving.svg "Go to DockerHub Page" -[dhstars]: https://img.shields.io/docker/stars/cloudsuite/dataserving.svg "Go to DockerHub Page" - +[dhrepo]: https://github.com/ParsaLab/cloudsuite/tree/master/benchmarks/data-serving "DockerHub Page" +[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/data-serving.svg "Go to DockerHub Page" +[dhstars]: https://img.shields.io/docker/stars/cloudsuite/data-serving.svg "Go to DockerHub Page" diff --git a/benchmarks/graph-analytics/README.md b/benchmarks/graph-analytics/README.md index 580258548..447264c2a 100644 --- a/benchmarks/graph-analytics/README.md +++ b/benchmarks/graph-analytics/README.md @@ -19,19 +19,19 @@ Supported tags and their respective `Dockerfile` links: - [`spark-worker`][sparkworkerdocker] This builds an image for the Spark worker node. You may spawn several workers. - [`spark-client`][sparkclientdocker] This builds an image with the Spark client node. The client is used to start the benchmark. -These images are automatically built using the mentioned Dockerfiles available on [`CloudSuite-EPFL/GraphAnalytics`][repo] and [`CloudSuite-EPFL/spark-base`][sparkrepo]. +These images are automatically built using the mentioned Dockerfiles available on [`ParsaLab/cloudsuite`][repo]. ### Starting the volume images ### The first step is to create the volume images that contain the binaries and the dataset of the Graph Analytics benchmark. First `pull` the volume images, using the following command: - $ docker pull cloudsuite/GraphAnalytics:data - $ docker pull cloudsuite/GraphAnalytics:benchmark + $ docker pull cloudsuite/graph-analytics:data + $ docker pull cloudsuite/graph-analytics:benchmark The following command will start the volume images, making both the data and the binaries available for other docker images on the host: - $ docker create --name data cloudsuite/GraphAnalytics:data - $ docker create --name bench cloudsuite/GraphAnalytics:benchmark + $ docker create --name data cloudsuite/graph-analytics:data + $ docker create --name bench cloudsuite/graph-analytics:benchmark ### Starting the master node ### @@ -71,13 +71,16 @@ To run the benchmark from the interactive container, use the following command: $ bash /benchmark/graph_analytics/run_benchmark.sh -[benchmarkdocker]: https://github.com/CloudSuite-EPFL/GraphAnalytics/blob/master/benchmarks/Dockerfile "Benchmark volume Dockerfile" -[datadocker]: https://github.com/CloudSuite-EPFL/GraphAnalytics/blob/master/data/Dockerfile "Data volume Dockerfile" -[sparkmasterdocker]: https://github.com/CloudSuite-EPFL/spark-base/blob/master/spark-master/Dockerfile "Spark Master Node Dockerfile" -[sparkworkerdocker]: https://github.com/CloudSuite-EPFL/spark-base/blob/master/spark-worker/Dockerfile "Spark Worker Dockerfile" -[sparkclientdocker]: https://github.com/CloudSuite-EPFL/spark-base/blob/master/spark-client/Dockerfile "Spark Client Dockerfile" -[repo]: https://github.com/CloudSuite-EPFL/GraphAnalytics "Graph Analytics GitHub Repo" -[sparkrepo]: https://github.com/CloudSuite-EPFL/spark-base "Spark Base GitHub Repo" -[dhrepo]: https://hub.docker.com/r/cloudsuite/graphanalytics/ "DockerHub Page" -[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/graphanalytics.svg "Go to DockerHub Page" -[dhstars]: https://img.shields.io/docker/stars/cloudsuite/graphanalytics.svg "Go to DockerHub Page" +[benchmarkdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/graph-analytics/benchmark/Dockerfile "Benchmark volume Dockerfile" +[datadocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/graph-analytics/data/Dockerfile "Data volume Dockerfile" +[sparkmasterdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/spark-base/spark-master/Dockerfile "Spark Master Node Dockerfile" +[sparkworkerdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/spark-base/spark-worker/Dockerfile "Spark Worker Dockerfile" +[sparkclientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/spark-base/spark-client/Dockerfile "Spark Client Dockerfile" +[repo]: https://github.com/ParsaLab/cloudsuite "GitHub Repo" +[dhrepo]: https://hub.docker.com/r/cloudsuite/graph-analytics/ "DockerHub Page" +[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/graph-analytics.svg "Go to DockerHub Page" +[dhstars]: https://img.shields.io/docker/stars/cloudsuite/graph-analytics.svg "Go to DockerHub Page" + +[serverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-caching/server/Dockerfile "Server Dockerfile" + +[clientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/data-caching/client/Dockerfile "Client Dockerfile" diff --git a/benchmarks/media-streaming/README.md b/benchmarks/media-streaming/README.md index 46c183004..9c50fff85 100644 --- a/benchmarks/media-streaming/README.md +++ b/benchmarks/media-streaming/README.md @@ -1,4 +1,4 @@ -# MediaStreaming +# Media Streaming [![Pulls on DockerHub][dhpulls]][dhrepo] [![Stars on DockerHub][dhstars]][dhrepo] @@ -16,7 +16,7 @@ Supported tags and their respective `Dockerfile` links: - [`client`][clientdocker]: This represents the httperf client. - [`dataset`][datasetdocker]: This represents the video files dataset for the streaming server. -These images are automatically built using the mentioned Dockerfiles available on the `CloudSuite-EPFL/MediaStreaming` [GitHub repo][repo]. +These images are automatically built using the mentioned Dockerfiles available on the `ParsaLab/cloudsuite` [GitHub repo][repo]. ### Dataset @@ -24,11 +24,11 @@ The streaming server requires a video dataset to serve. We generate a synthetic To set up the dataset you have to first `pull` the dataset image and then run it. To `pull` the dataset image use the following command: - $ docker pull cloudsuite/mediastreaming:dataset + $ docker pull cloudsuite/media-streaming:dataset The following command will create a dataset container that exposes the video dataset volume, which will be used by the streaming server: - $ docker run -d --name streaming_dataset cloudsuite/mediastreaming:dataset + $ docker run -d --name streaming_dataset cloudsuite/media-streaming:dataset ### Creating a network between the server and the client(s) @@ -42,34 +42,34 @@ We will attach the launched containers to this newly created docker network. ### Starting the Server #### To start the server you have to first `pull` the server image and then run it. To `pull` the server image use the following command: - $ docker pull cloudsuite/mediastreaming:server + $ docker pull cloudsuite/media-streaming:server The following command will start the server, mount the dataset volume, and attach it to the *streaming_network* network: - $ docker run -it --name=streaming_server --volumes-from streaming_dataset --net streaming_network cloudsuite/mediastreaming:server + $ docker run -it --name=streaming_server --volumes-from streaming_dataset --net streaming_network cloudsuite/media-streaming:server ### Starting the Client #### To start the client you have to first `pull` the client image and then run it. To `pull` the client image use the following command: - $ docker pull cloudsuite/mediastreaming:client + $ docker pull cloudsuite/media-streaming:client To start the client container and connect it to the *streaming_network* network use the following command: - $ docker run -it --name=streaming_client --volumes-from streaming_dataset --net streaming_network cloudsuite/mediastreaming:client + $ docker run -it --name=streaming_client --volumes-from streaming_dataset --net streaming_network cloudsuite/media-streaming:client To start the client, navigate to the /videoperf/run directory in the client container and launch the *benchmark.sh* script. This script is configured to launch a client process that issues a mix of requests for different videos of various qualities and performs a binary search of experiments to find the peak request rate the client can sustain while keeping the failure rate acceptable. At the end of the script's execution, the client's log files can be found under the /videoperf/run/output directory. - [datasetdocker]: https://github.com/CloudSuite-EPFL/MediaStreaming/blob/master/dataset/Dockerfile "Dataset Dockerfile" + [datasetdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/media-streaming/dataset/Dockerfile "Dataset Dockerfile" - [serverdocker]: https://github.com/CloudSuite-EPFL/MediaStreaming/blob/master/server/Dockerfile "Server Dockerfile" + [serverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/media-streaming/server/Dockerfile "Server Dockerfile" - [clientdocker]: https://github.com/CloudSuite-EPFL/MediaStreaming/blob/master/client/Dockerfile "Client Dockerfile" + [clientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/media-streaming/client/Dockerfile "Client Dockerfile" - [repo]: https://github.com/CloudSuite-EPFL/MediaStreaming "GitHub Repo" - [dhrepo]: https://hub.docker.com/r/cloudsuite/mediastreaming/ "DockerHub Page" - [dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/mediastreaming.svg "Go to DockerHub Page" - [dhstars]: https://img.shields.io/docker/stars/cloudsuite/mediastreaming.svg "Go to DockerHub Page" + [repo]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/media-streaming "GitHub Repo" + [dhrepo]: https://hub.docker.com/r/cloudsuite/media-streaming/ "DockerHub Page" + [dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/media-streaming.svg "Go to DockerHub Page" + [dhstars]: https://img.shields.io/docker/stars/cloudsuite/media-streaming.svg "Go to DockerHub Page" [nginx_repo]: https://github.com/nginx/nginx "Nginx repo" [httperf_repo]: https://github.com/httperf/httperf "httperf repo" diff --git a/benchmarks/spark-base/README.md b/benchmarks/spark-base/README.md index 35ce90a24..4bd69146e 100644 --- a/benchmarks/spark-base/README.md +++ b/benchmarks/spark-base/README.md @@ -1,5 +1,9 @@ Spark Base Image for Cloudsuite ========== + +[![Pulls on DockerHub][dhpulls]][dhrepo] +[![Stars on DockerHub][dhstars]][dhrepo] + This repository contains the docker image with a base Spark image for the CloudSuite workloads. ## Building the images ## @@ -11,7 +15,7 @@ Supported tags and their respective `Dockerfile` links: - [`spark-worker`][sparkworkerdocker] This builds an image with the Spark worker node. You may spawn clusters of several workers. - [`spark-client`][sparkclientdocker] This builds an image with the Spark client node. The client is used to start the benchmark. -These images are automatically built using the mentioned Dockerfiles available on [`CloudSuite-EPFL/spark-base`][sparkrepo]. +These images are automatically built using the mentioned Dockerfiles available on [`ParsaLab/cloudsuite`][repo]. ### Starting the volume images ### @@ -21,7 +25,7 @@ The `data` container contains the dataset that is necessary for the benchmark to The `bench` container hosts the Java Spark binaries and scripts necessary to run the benchmark. The client `Entrypoint` script looks for a folder with the same name as the command line argument passed to the `docker run` command and runs the `run_benchmark.sh` script in that folder. -Assuming all the volume images are pulled, the following command will start the volume images, making both the data and the binaries avaliable for other docker images in the host: +Assuming all the volume images are pulled, the following command will start the volume images, making both the data and the binaries available for other docker images in the host: $ docker create --name data [data-volume-image-tag] $ docker create --name bench [binary-volume-image-tag] @@ -64,7 +68,12 @@ To run the benchmark from the interactive container, use the following command: $ bash /benchmark/[benchmark-name]/run_benchmark.sh -[sparkmasterdocker]: https://github.com/CloudSuite-EPFL/spark-base/blob/master/spark-master/Dockerfile "Spark Master Node Dockerfile" -[sparkworkerdocker]: https://github.com/CloudSuite-EPFL/spark-base/blob/master/spark-worker/Dockerfile "Spark Worker Dockerfile" -[sparkclientdocker]: https://github.com/CloudSuite-EPFL/spark-base/blob/master/spark-client/Dockerfile "Spark Client Dockerfile" -[sparkrepo]: https://github.com/CloudSuite-EPFL/spark-base "Spark Base GitHub Repo" +[sparkmasterdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/spark-base/spark-master/Dockerfile "Spark Master Node Dockerfile" +[sparkworkerdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/spark-base/spark-worker/Dockerfile "Spark Worker Dockerfile" +[sparkclientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/spark-base/spark-client/Dockerfile "Spark Client Dockerfile" + +[repo]: https://github.com/ParsaLab/cloudsuite/ "GitHub Repo" + +[dhrepo]: https://hub.docker.com/r/cloudsuite/spark-base/ "DockerHub Page" +[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/spark-base.svg "Go to DockerHub Page" +[dhstars]: https://img.shields.io/docker/stars/cloudsuite/spark-base.svg "Go to DockerHub Page" diff --git a/benchmarks/web-search/README.md b/benchmarks/web-search/README.md index 841c8408f..ad973d397 100644 --- a/benchmarks/web-search/README.md +++ b/benchmarks/web-search/README.md @@ -16,7 +16,7 @@ Supported tags and their respective `Dockerfile` links: - [`server`][serverdocker] This builds an image for the Apache Solr index nodes. You may spawn several nodes. - [`client`][clientdocker] This builds an image with the client node. The client is used to start the benchmark and query the index nodes. -These images are automatically built using the mentioned Dockerfiles available on [`CloudSuite-EPFL/WebSearch`][repo]. +These images are automatically built using the mentioned Dockerfiles available on [`https://github.com/ParsaLab/cloudsuite/tree/master/benchmarks/web-search`][repo]. ### Creating a network between the server(s) and the client(s) @@ -30,12 +30,12 @@ We will attach the launched containers to this newly created docker network. To start the server you have to first `pull` the server image and then run it. To `pull` the server image, use the following command: - $ docker pull cloudsuite/websearch:server + $ docker pull cloudsuite/web-search:server The following command will start the server and forward port 8983 to the host, so that the Apache Solr's web interface can be accessed from the web browser using the host's IP address. More information on Apache Solr's web interface can be found [here][solrui]. The first parameter past to the image indicates the memory allocated for the JAVA process. The pregenerated Solr index occupies 12GB of memory, and therefore we use `12g` to avoid disk accesses. The second parameter indicates the number of Solr nodes. Because the index is for a single node only, the aforesaid parameter should be `1` always. - $ docker run -it --name server --net search_network -p 8983:8983 cloudsuite/websearch:server 12g 1 - + $ docker run -it --name server --net search_network -p 8983:8983 cloudsuite/web-search:server 12g 1 + At the end of the server booting process, the container prints the `server_address` of the index node. This address is used in the client container. The `server_address` message in the container should look like this (note that the IP address might change): $ Index Node IP Address: 172.19.0.2 @@ -44,11 +44,11 @@ At the end of the server booting process, the container prints the `server_addre To start a client you have to first `pull` the client image and then run it. To `pull` the client image, use the following command: - $ docker pull cloudsuite/websearch:client + $ docker pull cloudsuite/web-search:client The following command will start the client node and run the benchmark. The `server_address` refers to the IP address, in brackets (e.g., "172.19.0.2"), of the index node that receives the client requests. The four numbers after the server address refer to: the scale, which indicates the number of concurrent clients (50); the ramp-up time in seconds (90), which refers to the time required to warm up the server; the steady-state time in seconds (60), which indicates the time the benchmark is in the steady state; and the rump-down time in seconds (60), which refers to the time to wait before ending the benchmark. Tune these parameters accordingly to stress your target system. - $ docker run -it --name client --net search_network cloudsuite/websearch:client server_address 50 90 60 60 + $ docker run -it --name client --net search_network cloudsuite/web-search:client server_address 50 90 60 60 The output results will show on the screen after the benchmark finishes. @@ -56,8 +56,8 @@ The output results will show on the screen after the benchmark finishes. - The target response time requires that 99% of the requests are serviced within 200ms. -- The throughput statistic, operations per second, is shown as: - +- The throughput statistic, operations per second, is shown as: + `25.133` - The response time statistics, average, maximun, minimum, 90-th, and 99-th, are shown as: @@ -86,16 +86,15 @@ The output results will show on the screen after the benchmark finishes. ``` More information about Solr can be found [here][solrmanual]. -[datadocker]: https://github.com/CloudSuite-EPFL/WebSearch/tree/master/data/Dockerfile "Data volume Dockerfile" -[serverdocker]: https://github.com/CloudSuite-EPFL/WebSearch/tree/master/server/Dockerfile "Server Dockerfile" -[clientdocker]: https://github.com/CloudSuite-EPFL/WebSearch/tree/master/client/Dockerfile "Client Dockerfile" +[datadocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-search/data/Dockerfile "Data volume Dockerfile" +[serverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-search/server/Dockerfile "Server Dockerfile" +[clientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-search/client/Dockerfile "Client Dockerfile" [solrui]: https://cwiki.apache.org/confluence/display/solr/Overview+of+the+Solr+Admin+UI "Apache Solr UI" [solrmanual]: https://cwiki.apache.org/confluence/display/solr/Apache+Solr+Reference+Guide "Apache Solr Manual" [nutchtutorial]: https://wiki.apache.org/nutch/NutchTutorial "Nutch Tutorial" [apachesolr]: https://github.com/apache/solr "Apache Solr" [apachenutch]: https://github.com/apache/nutch "Apache Nutch" -[repo]: https://github.com/CloudSuite-EPFL/WebSearch "Web Search GitHub Repo" -[dhrepo]: https://hub.docker.com/r/cloudsuite/websearch/ "DockerHub Page" -[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/websearch.svg "Go to DockerHub Page" -[dhstars]: https://img.shields.io/docker/stars/cloudsuite/websearch.svg "Go to DockerHub Page" - +[repo]: https://github.com/ParsaLab/cloudsuite/tree/master/benchmarks/web-search "Web Search GitHub Repo" +[dhrepo]: https://hub.docker.com/r/cloudsuite/web-search/ "DockerHub Page" +[dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/web-search.svg "Go to DockerHub Page" +[dhstars]: https://img.shields.io/docker/stars/cloudsuite/web-search.svg "Go to DockerHub Page" diff --git a/benchmarks/web-serving/README.md b/benchmarks/web-serving/README.md index 4bf5ece7c..f288fd1f9 100644 --- a/benchmarks/web-serving/README.md +++ b/benchmarks/web-serving/README.md @@ -1,4 +1,4 @@ -# WebServing +# Web Serving [![Pulls on DockerHub][dhpulls]][dhrepo] [![Stars on DockerHub][dhstars]][dhrepo] @@ -15,9 +15,9 @@ Supported tags and their respective `Dockerfile` links: - [`web_server`][webserverdocker]: This represents the web server. - [`memcached_server`][memcacheserverdocker]: This represents the memcached server. - [`db_server`][mysqlserverdocker]: This represents the database server, which runs MySQL. - - [`fabanclient`][clientdocker]: This represents the faban client. + - [`faban_client`][clientdocker]: This represents the faban client. -These images are automatically built using the mentioned Dockerfiles available on the `CloudSuite-EPFL/WebServing` [GitHub repo][repo]. +These images are automatically built using the mentioned Dockerfiles available on the `ParsaLab/cloudsuite` [GitHub repo][repo]. ### Creating a network between the servers and the client(s) @@ -30,39 +30,39 @@ We will attach the launched containers to this newly created docker network. ### Starting the web server #### To start the web server, you first have to `pull` the server image. To `pull` the server image use the following command: - $ docker pull cloudsuite/webserving:web_server + $ docker pull cloudsuite/web-serving:web_server The following command will start the web server, and attach it to the *my_net* network: - $ docker run -d -t --net=my_net --privileged=true --name=web_server cloudsuite/webserving:web_server /etc/bootstrap.sh + $ docker run -d -t --net=my_net --privileged=true --name=web_server cloudsuite/web-serving:web_server /etc/bootstrap.sh ### Starting the database server #### To start the database server, you have to first `pull` the server image. To `pull` the server image use the following command: - $ docker pull cloudsuite/webserving:db_server + $ docker pull cloudsuite/web-serving:db_server The following command will start the database server, and attach it to the *my_net* network: - $ docker run -d -t --net=my_net --privileged=true --name=mysql_server cloudsuite/webserving:db_server + $ docker run -d -t --net=my_net --privileged=true --name=mysql_server cloudsuite/web-serving:db_server ### Starting the memcached server #### To start the memcached server, you have to first `pull` the server image. To `pull` the server image use the following command: - $ docker pull cloudsuite/webserving:memcached_server + $ docker pull cloudsuite/web-serving:memcached_server The following command will start the memcached server, and attach it to the *my_net* network: - $ docker run -d -t --net=my_net --privileged=true --name=memcache_server cloudsuite/webserving:memcached_server + $ docker run -d -t --net=my_net --privileged=true --name=memcache_server cloudsuite/web-serving:memcached_server ### Starting the client #### To start the client, you have to first `pull` the client image. To `pull` the client image use the following command: - $ docker pull cloudsuite/webserving:faban_client + $ docker pull cloudsuite/web-serving:faban_client To start the client container and connect it to the *my_net* network use the following command: - $ docker run -d -t --net=my_net --privileged=true --name=faban_client cloudsuite/webserving:faban_client /etc/bootstrap.sh -bash + $ docker run -d -t --net=my_net --privileged=true --name=faban_client cloudsuite/web-serving:faban_client /etc/bootstrap.sh -bash ### Running the benchmark ### @@ -73,12 +73,12 @@ To start the client, you need to run the run.sh file. You can find the summary of the results in the summary.xml file. - [webserverdocker]: https://github.com/CloudSuite-EPFL/WebServing/blob/master/web_server/Dockerfile "WebServer Dockerfile" - [memcacheserverdocker]: https://github.com/CloudSuite-EPFL/WebServing/blob/master/memcached_server/Dockerfile "MemcacheServer Dockerfile" - [mysqlserverdocker]: https://github.com/CloudSuite-EPFL/WebServing/blob/master/db_server/Dockerfile "MysqlServer Dockerfile" - [clientdocker]: https://github.com/CloudSuite-EPFL/WebServing/blob/master/faban_client/Dockerfile "Client Dockerfile" + [webserverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-serving/web_server/Dockerfile "WebServer Dockerfile" + [memcacheserverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-serving/memcached_server/Dockerfile "MemcacheServer Dockerfile" + [mysqlserverdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-serving/db_server/Dockerfile "MysqlServer Dockerfile" + [clientdocker]: https://github.com/ParsaLab/cloudsuite/blob/master/benchmarks/web-serving/faban_client/Dockerfile "Client Dockerfile" - [repo]: https://github.com/CloudSuite-EPFL/WebServing "GitHub Repo" - [dhrepo]: https://hub.docker.com/r/cloudsuite/webserving/ "DockerHub Page" - [dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/webserving.svg "Go to DockerHub Page" - [dhstars]: https://img.shields.io/docker/stars/cloudsuite/webserving.svg "Go to DockerHub Page" + [repo]: https://github.com/ParsaLab/cloudsuite/tree/master/benchmarks/web-serving "GitHub Repo" + [dhrepo]: https://hub.docker.com/r/cloudsuite/web-serving/ "DockerHub Page" + [dhpulls]: https://img.shields.io/docker/pulls/cloudsuite/web-serving.svg "Go to DockerHub Page" + [dhstars]: https://img.shields.io/docker/stars/cloudsuite/web-serving.svg "Go to DockerHub Page"