Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Colab Env Check & Install #9

Open
wants to merge 45 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
d1c1a00
fix spacing
gumdropsteve Oct 17, 2019
9d27e40
new script to check GPU type of colab and provide next step guidance …
gumdropsteve Oct 17, 2019
83853c3
Create colab-env-check.sh
gumdropsteve Oct 17, 2019
6d0dc43
up to current breaking point
gumdropsteve Oct 21, 2019
28385a9
formatted thru installs; current issues displayed w/ details
gumdropsteve Oct 21, 2019
4a5c430
Merge pull request #1 from gumdropsteve/feature_utils-colab-env-check
gumdropsteve Oct 21, 2019
59c4732
Switch to pynvml from !nvidia-smi
gumdropsteve Oct 21, 2019
299a416
Merge pull request #2 from gumdropsteve/feature_utils-colab-env-check
gumdropsteve Oct 21, 2019
72e37e4
w/ temp fix for column names, runs up to installing cuML
gumdropsteve Oct 25, 2019
7b48188
installing cuml issues
gumdropsteve Oct 25, 2019
5498066
cuml installed; new error with null values
gumdropsteve Oct 25, 2019
8a75074
add taxi colab button
gumdropsteve Oct 27, 2019
45fd58b
too much data (RuntimeError: Exception occured! file=/conda/conda-bld…
gumdropsteve Oct 27, 2019
7a45cd3
formatted and runs e2e with temp fixes (issue: column names)
gumdropsteve Oct 28, 2019
1f5f0af
Create colab_env.py
gumdropsteve Nov 11, 2019
90f9980
env check Merge branch 'feature_utils' of https://github.com/gumdrops…
gumdropsteve Nov 11, 2019
7989b43
Merge pull request #5 from gumdropsteve/feature/utils
gumdropsteve Nov 11, 2019
a5ba9e5
Merge pull request #6 from gumdropsteve/feature_utils
gumdropsteve Nov 11, 2019
463fe6f
colab env check
gumdropsteve Nov 11, 2019
1f95713
Update colab_env.py
gumdropsteve Nov 12, 2019
69b0e4f
Update colab_env.py
gumdropsteve Nov 12, 2019
133c20d
via Adam + colab_env.py
gumdropsteve Nov 13, 2019
34cb23a
Update bsql-colab.sh
gumdropsteve Nov 13, 2019
6694e86
Update colab_env.py
gumdropsteve Nov 13, 2019
346a176
Update README.md
gumdropsteve Nov 13, 2019
e34a4d4
Update colab_env.py
gumdropsteve Nov 13, 2019
e5e7653
Create set_paths.py
gumdropsteve Nov 13, 2019
b4ddc23
Update bsql-colab.sh
gumdropsteve Nov 13, 2019
e3a6118
Update bsql-colab.sh
gumdropsteve Nov 13, 2019
f16046c
Update and rename set_paths.py to set_paths.sh
gumdropsteve Nov 13, 2019
35e2991
Delete set_paths.sh
gumdropsteve Nov 13, 2019
27ccfea
Delete taxi_fare_prediction.ipynb
gumdropsteve Nov 13, 2019
0fe2c63
Delete taxi_fare_prediction.ipynb
gumdropsteve Nov 13, 2019
b3cb7e8
Try bsql script from ref colab
gumdropsteve Nov 13, 2019
92d49e9
Back to Felipe's channel
gumdropsteve Nov 13, 2019
b3a600e
Remove env check
gumdropsteve Nov 13, 2019
6efa6f5
Update colab_env.py
gumdropsteve Nov 13, 2019
ee1848d
sp
gumdropsteve Nov 13, 2019
77315eb
clear success output
gumdropsteve Nov 13, 2019
f4b7156
Merge branch 'master' into feature/utils
gumdropsteve Nov 14, 2019
d81571e
convert to include latest notebooks-contrib
gumdropsteve Nov 15, 2019
b0e6347
Update bsql-colab.sh
gumdropsteve Nov 15, 2019
62cffe9
fi
gumdropsteve Nov 15, 2019
62a5837
Create blazingsql.sh (#11)
gumdropsteve Nov 18, 2019
079340e
Update bsql-colab.sh (#12)
gumdropsteve Nov 18, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions utils/blazingsql.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
conda install -y --prefix /usr/local -c blazingsql/label/cuda10.0 -c blazingsql -c rapidsai -c conda-forge -c defaults blazingsql-calcite blazingsql-orchestrator blazingsql-ral blazingsql-python

pip install flatbuffers
37 changes: 37 additions & 0 deletions utils/colab_env.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import pynvml
"""
check the colab environment for BlazingSQL and RAPIDS AI compatiblity
prereqs:
> gpu with pascal architecture & compute capability >= 6.0
more info: http://bit.ly/rapids-gpus-friend
> colab translation: Tesla P100 or T4
more info: http://bit.ly/rapids_in_colab_friend_link
"""
# focus GPU type (check colab settings are correct)
try:
# start up
pynvml.nvmlInit()
# identify handle then extract GPU name
handle = pynvml.nvmlDeviceGetHandleByIndex(0)
gpu_name = pynvml.nvmlDeviceGetName(handle)
# not on gpu hardware acceleration
except:
# error: hardware accelerator
raise Exception("\nPlease make sure you've configured Colab to request a GPU instance type.\n\n"
"At top of Colab, try: Runtime -> Change runtime type -> Hardware accelerator -> GPU -> Save\n")
# check if allocated K80 instance
if gpu_name == b'Tesla K80':
# error: compute capability < 6.0
raise Exception("\nYou've been allocated a K80 instance\n\n"
"Unfortunately, colab demos require a T4 or P100 instance\n\n"
"At top of Colab, try: Runtime -> Reset all runtimes...\n")
# compatable GPUs that have been available in Colab
compatable_gpus = [b'Tesla P100-PCIE-16GB', b'Tesla T4']
# check if allocated gpu is in list of known compatable GPUs
if gpu_name in compatable_gpus:
# we're blazing
print(f'\n\n***********************************\nGPU = {gpu_name}\nWoo! You got the right kind of GPU!\n***********************************\n\n')
# unacknowledged outcome
else:
# please let us know, thank you
raise Exception(f"\nYou've achieved wizardy.\nyour GPU is {my_gpu}\nPlease inform [email protected]")
25 changes: 0 additions & 25 deletions utils/env-check.py

This file was deleted.