Skip to content

Commit

Permalink
[BRING BACK TO MASTER] Update result and fix sanity check bugs.
Browse files Browse the repository at this point in the history
  • Loading branch information
@rkooo567
rkooo567 committed Aug 12, 2020
1 parent 56810e1 commit 89fd21f
Show file tree
Hide file tree
Showing 7 changed files with 91 additions and 7 deletions.
13 changes: 11 additions & 2 deletions doc/dev/bin/pip_download_test.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
# This script automatically download ray and run the sanity check (sanity_check.py)
# in various Python version. This script requires conda command to exist.

unset RAY_ADDRESS
export RAY_HASH=$RAY_HASH
export RAY_VERSION=$RAY_VERSION

if [[ -z "$RAY_HASH" ]]; then
echo "RAY_HASH env var should be provided"
exit 1
Expand All @@ -27,7 +31,7 @@ pip install --upgrade pip
# This is required to use conda activate
source "$(conda info --base)/etc/profile.d/conda.sh"

for PYTHON_VERSION in "3.5" "3.6" "3.7" "3.8"
for PYTHON_VERSION in "3.6" "3.7" "3.8"
do
env_name="${RAY_VERSION}-${PYTHON_VERSION}-env"
conda create -y -n "${env_name}" python=${PYTHON_VERSION}
Expand All @@ -41,7 +45,12 @@ do
printf "\n\n\n"

pip install redis==3.3.2
pip install msgpack==0.6.2
pip install msgpack==1.0.0
pip install aioredis
pip install colorful
pip install prometheus-client==0.7.1
pip install opencensus
pip install gpustat
pip install ray
pip uninstall -y ray
pip install --index-url https://test.pypi.org/simple/ ray
Expand Down
9 changes: 6 additions & 3 deletions doc/dev/bin/ray-project/cluster.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ cluster_name: sanity-check

# The maximum number of workers nodes to launch in addition to the head
# node. This takes precedence over min_workers. min_workers defaults to 0.
max_workers: 1
max_workers: 0

# Cloud-provider specific configuration.
provider:
Expand All @@ -23,13 +23,16 @@ head_setup_commands:
- wget --quiet https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh || true
- bash Anaconda3-5.0.1-Linux-x86_64.sh -b -p $HOME/anaconda3 || true
- echo 'export PATH="$HOME/anaconda3/bin:$PATH"' >> ~/.bashrc
- echo 'RAY_VERSION=0.8.7' >> ~/.bashrc
- echo 'RAY_HASH=56810e136587432ed4758d882bf8894b930355e3' >> ~/.bashrc
- chmod +x ./pip_download_test.sh || true
- pip install -U pip

# Custom commands that will be run on worker nodes after common setup.
worker_setup_commands: []

# Command to start ray on the head node. You don't need to change this.
head_start_ray_commands: []
head_start_ray_commands: [ray stop]

# Command to start ray on worker nodes. You don't need to change this.
worker_start_ray_commands: []
worker_start_ray_commands: [ray stop]
2 changes: 0 additions & 2 deletions doc/dev/download_wheels.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,10 @@ if [[ -z "$RAY_VERSION" ]]; then
fi

# Make sure Linux wheels are downloadable without errors.
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp35-cp35m-manylinux1_x86_64.whl"
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp36-cp36m-manylinux1_x86_64.whl"
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp37-cp37m-manylinux1_x86_64.whl"
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp38-cp38-manylinux1_x86_64.whl"
# Make sure macos wheels are downloadable without errors.
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp35-cp35m-macosx_10_13_intel.whl"
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp36-cp36m-macosx_10_13_intel.whl"
wget "https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp37-cp37m-macosx_10_13_intel.whl"
# Wheel name convention has been changed from Python 3.8.
Expand Down
41 changes: 41 additions & 0 deletions doc/dev/release_logs/0.8.7/rllib_regression.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
== Status ==
Memory usage on this node: 9.5/480.3 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/64 CPUs, 0.0/8 GPUs, 0.0/325.78 GiB heap, 0.0/99.07 GiB objects
Result logdir: /home/ubuntu/ray_results/apex
Result logdir: /home/ubuntu/ray_results/atari-a2c
Result logdir: /home/ubuntu/ray_results/atari-basic-dqn
Result logdir: /home/ubuntu/ray_results/atari-impala
Result logdir: /home/ubuntu/ray_results/atari-ppo-tf
Result logdir: /home/ubuntu/ray_results/atari-ppo-torch
Number of trials: 24 (24 TERMINATED)
+-------------------------------------------+------------+-------+--------+------------------+---------+----------+
| Trial name | status | loc | iter | total time (s) | ts | reward |
|-------------------------------------------+------------+-------+--------+------------------+---------+----------|
| IMPALA_BreakoutNoFrameskip-v4_b13b3_00000 | TERMINATED | | 355 | 3601.74 | 7491750 | 381.68 |
| IMPALA_BreakoutNoFrameskip-v4_b13b3_00001 | TERMINATED | | 355 | 3603.97 | 7482000 | 367.12 |
| IMPALA_BreakoutNoFrameskip-v4_b13b3_00002 | TERMINATED | | 356 | 3607.98 | 7528500 | 375.77 |
| IMPALA_BreakoutNoFrameskip-v4_b13b3_00003 | TERMINATED | | 355 | 3605.97 | 7411000 | 328.21 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00004 | TERMINATED | | 664 | 3603.23 | 3320000 | 31.47 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00005 | TERMINATED | | 906 | 3603.53 | 4530000 | 41.31 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00006 | TERMINATED | | 897 | 3600.83 | 4485000 | 60.32 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00007 | TERMINATED | | 913 | 3600.05 | 4565000 | 51.28 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00008 | TERMINATED | | 492 | 3606.93 | 2460000 | 35.75 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00009 | TERMINATED | | 480 | 3600.3 | 2400000 | 13.71 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00010 | TERMINATED | | 416 | 3606.47 | 2080000 | 7.49 |
| PPO_BreakoutNoFrameskip-v4_b13b3_00011 | TERMINATED | | 422 | 3602.06 | 2110000 | 13.29 |
| APEX_BreakoutNoFrameskip-v4_b13b3_00012 | TERMINATED | | 106 | 3614.12 | 5371360 | 48.41 |
| APEX_BreakoutNoFrameskip-v4_b13b3_00013 | TERMINATED | | 103 | 3618.46 | 7777760 | 82.33 |
| APEX_BreakoutNoFrameskip-v4_b13b3_00014 | TERMINATED | | 94 | 3627.11 | 6155840 | 62.79 |
| APEX_BreakoutNoFrameskip-v4_b13b3_00015 | TERMINATED | | 92 | 3615.04 | 6159520 | 63.5 |
| A2C_BreakoutNoFrameskip-v4_b13b3_00016 | TERMINATED | | 352 | 3606.65 | 3135000 | 62.67 |
| A2C_BreakoutNoFrameskip-v4_b13b3_00017 | TERMINATED | | 351 | 3605.31 | 3098000 | 124.24 |
| A2C_BreakoutNoFrameskip-v4_b13b3_00018 | TERMINATED | | 351 | 3607.44 | 3113500 | 129.7 |
| A2C_BreakoutNoFrameskip-v4_b13b3_00019 | TERMINATED | | 353 | 3601.78 | 4086000 | 271.04 |
| DQN_BreakoutNoFrameskip-v4_b13b3_00020 | TERMINATED | | 24 | 3682.43 | 250000 | 11.27 |
| DQN_BreakoutNoFrameskip-v4_b13b3_00021 | TERMINATED | | 24 | 3681.4 | 250000 | 15.9 |
| DQN_BreakoutNoFrameskip-v4_b13b3_00022 | TERMINATED | | 24 | 3677.74 | 250000 | 11.62 |
| DQN_BreakoutNoFrameskip-v4_b13b3_00023 | TERMINATED | | 24 | 3664.08 | 250000 | 7.47 |
+-------------------------------------------+------------+-------+--------+------------------+---------+----------+


14 changes: 14 additions & 0 deletions doc/dev/release_logs/0.8.7/stress_tests/application_stress_test.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
== Status ==
Memory usage on this node: 22.0/480.3 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/640 CPUs, 0/8 GPUs, 0.0/1906.54 GiB heap, 0.0/566.21 GiB objects (0/1.0 GPUType:V100)
Result logdir: /home/ubuntu/ray_results/atari-impala
Number of trials: 4 (4 TERMINATED)
+------------------------------------------------+------------+-------+-----------------------------+--------+------------------+----------+----------+
| Trial name | status | loc | env | iter | total time (s) | ts | reward |
|------------------------------------------------+------------+-------+-----------------------------+--------+------------------+----------+----------|
| IMPALA_BreakoutNoFrameskip-v4_b783d_00000 | TERMINATED | | BreakoutNoFrameskip-v4 | 746 | 7521.77 | 30009000 | 462.43 |
| IMPALA_BeamRiderNoFrameskip-v4_b783d_00001 | TERMINATED | | BeamRiderNoFrameskip-v4 | 761 | 7671.4 | 30024500 | 2318.14 |
| IMPALA_QbertNoFrameskip-v4_b783d_00002 | TERMINATED | | QbertNoFrameskip-v4 | 739 | 7457.03 | 30022000 | 10025.5 |
| IMPALA_SpaceInvadersNoFrameskip-v4_b783d_00003 | TERMINATED | | SpaceInvadersNoFrameskip-v4 | 766 | 7718.84 | 30037000 | 786.8 |
+------------------------------------------------+------------+-------+-----------------------------+--------+------------------+----------+----------+
4 changes: 4 additions & 0 deletions doc/dev/release_logs/0.8.7/stress_tests/test_dead_actors.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Finished in: 456.6799967288971s
Average iteration time: 4.566798083782196s
Max iteration time: 10.93368911743164s
Min iteration time: 1.0819299221038818s
15 changes: 15 additions & 0 deletions doc/dev/release_logs/0.8.7/stress_tests/test_many_tasks.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Stage 0 results:
Total time: 22.120722770690918
Stage 1 results:
Total time: 174.56333804130554
Average iteration time: 17.456315088272095
Max iteration time: 18.66864323616028
Min iteration time: 15.951467514038086
Stage 2 results:
Total time: 1324.5760805606842
Average iteration time: 264.91476254463197
Max iteration time: 278.47786712646484
Min iteration time: 248.30649852752686
Stage 3 results:
Actor creation time: 3.133239269256592
Total time: 2001.6567075252533

0 comments on commit 89fd21f

Please sign in to comment.