plotly extension + user lib installation bug #65
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #65 +/- ##
=======================================
Coverage 50.00% 50.00%
=======================================
Files 4 4
Lines 118 118
=======================================
Hits 59 59
Misses 59 59 ☔ View full report in Codecov by Sentry. |
Dockerfile
Outdated
| # install pipenv | ||
| RUN pip3 install pipenv | ||
|
|
||
| # install python dependencies | ||
| COPY Pipfile* ./ | ||
| RUN pipenv sync --system | ||
|
|
||
| # Update permissions on the Bitnami directory after Pipenv installs | ||
| # to ensure the Spark user can install packages properly within notebook. | ||
| RUN chown -R spark_user:spark /opt/bitnami |
There was a problem hiding this comment.
In my hands the bitnami chown takes a really long time and now will rerun anytime we change the python packages. Why does it need to be moved forward in the build? pip shouldn't touch it, right?
Same question for the yarn setup below, although that's fast
There was a problem hiding this comment.
I think pipenv run by root changes owners of some of python lib dir. As a result, users sometimes failed use pip install.
There was a problem hiding this comment.
The python lib dir is in /opt/bitnami?
There was a problem hiding this comment.
What if we ran pipenv with the spark user?
There was a problem hiding this comment.
Cannot run pipenv with spark user. python lib is in /opt/bitnami. Maybe docker works even differently on linux and mac? chown is cached with mac and it only takes less than a minutes to build for me.
There was a problem hiding this comment.
How is that possible when that directory is chowned to spark?
There was a problem hiding this comment.
no clue. That's why I think the safest way is to run RUN chown -R spark_user:spark /opt/bitnami after we done modifying that dir.
There was a problem hiding this comment.
This makes no sense whatsoever. Why would setting $HOME to /opt/bitnami fix a permissions error?
There was a problem hiding this comment.
Ok, how about we leave home as /opt/bitnami for now but make a ticket to look into this weirdness at some point. At least we'll have a record if something similar starts happening
Enable Plotly plot + Users can now install any package they need using pip.