Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swancustomenvironments: Remove swan middle layer #421

Merged
merged 2 commits into from
Dec 9, 2024

Conversation

rodrigo-sobral
Copy link
Contributor

@rodrigo-sobral rodrigo-sobral commented Dec 5, 2024

Due to the fact venv-pack (used by nxcals) doesn't support editable packages to be shipped with the environment, and the swan middle layer was based on the use of a .pth file, which is seen as a source of editable packages and therefore makes venv-pack fail. For this reason, the swan middle layer has been removed and the extra packages have been added directly to the environment. In spite of this, the requirements file must not be modified.

@@ -229,7 +229,7 @@ <h2 id="modal-title" style="text-align: left;"></h2>
else {
outputBox.innerHTML += message_line;
outputBox.scrollTop = outputBox.scrollHeight;
if (message_line.includes('ERROR')) {
if (message_line.toUpperCase().includes('ERROR')) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need the uppercase? Isn't ERROR in lowercase your signal for when something is wrong? Is it to catch other errors that you do not signal?

@@ -1,27 +1,10 @@
#!/bin/bash

# Create a middle layer for installing ipykernel, putting it apart from the user environment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit message still needs to reflect the fact that venv-pack does not support editable packages, and the middle layer was based on the use of a .pth file, which is seen as a source of editable packages and therefore makes venv-pack fail.

@@ -41,15 +24,28 @@ if [ "${RESOLVED_REQ}" = true ]; then
# Use the same pip configuration as the Acc-Py default pip
ACCPY_PIP_CONF="-i $(pip config get global.index-url) --allow-insecure-host $(pip config get global.trusted-host)"
uv pip install ${ACCPY_PIP_CONF} -r "${REQ_PATH}" 2>&1
uv pip install ${ACCPY_PIP_CONF} ${IPYKERNEL} 2>&1
Copy link
Contributor

@etejedor etejedor Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and below, we need a comment that says:

# Enforce installation of our version of ipykernel and its dependencies

USER_PACKAGES_PATH=$(python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
echo ${SWAN_PACKAGES_PATH} > ${USER_PACKAGES_PATH}/$(basename $SWAN_ENV).pth
if [ -n "${USE_NXCALS}" ]; then
# For NXCALS, install the Spark packages and their dependencies, using the according tool for its resolution
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion:

# For NXCALS, enforce installation of our Spark extensions and their dependencies at certain versions

@@ -24,14 +17,11 @@ eval "${ACTIVATE_ENV_CMD}"
_log "Installing packages from ${REQ_PATH}..."
if [ "${RESOLVED_REQ}" = true ]; then
uv pip install -r "${REQ_PATH}" 2>&1
uv pip install ${IPYKERNEL} 2>&1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above

@rodrigo-sobral rodrigo-sobral force-pushed the master branch 2 times, most recently from 1025e6d to e39bd19 Compare December 9, 2024 15:36
Keep ipykernel package definition consistent with the remaining extra packages.
Due to the fact `venv-pack` (used by nxcals) doesn't support editable packages to be shipped with the environment, and the swan middle layer was based on the use of a .pth file, which is seen as a source of editable packages and therefore makes venv-pack fail. For this reason, the swan middle layer has been removed and the extra packages have been added directly to the environment. In spite of this, the requirements file must not be modified.
…irements

Prevents UV errors like `No solution found when resolving dependencies:`
@etejedor etejedor merged commit f7e254e into swan-cern:master Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants