Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix conda builds #1776

Merged
merged 29 commits into from
May 20, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
3f5d922
test conda packages in a test environment as part of CI
eberrigan Apr 16, 2024
30692dc
do not test sleap import using conda build
eberrigan Apr 16, 2024
bb5403a
use github environment variables to define build path for each OS in …
eberrigan Apr 16, 2024
4d43542
figure out paths one OS at a time
eberrigan Apr 16, 2024
6ceb574
github environment variables work in subsequent steps not current step
eberrigan Apr 16, 2024
2a6b9d8
use local builds first
eberrigan Apr 16, 2024
31d1618
print env info
eberrigan Apr 17, 2024
10537aa
try simple environment creation
eberrigan Apr 18, 2024
67c2a24
try conda instead of mamba
eberrigan Apr 19, 2024
de8d3fb
fix windows build path
eberrigan Apr 19, 2024
9802717
fix windows build path
eberrigan Apr 19, 2024
793f2e1
Merge branch 'develop' into elizabeth/Fix-ubuntu-conda-build
eberrigan Apr 21, 2024
f9f8887
add comment to reference pull request
eberrigan Apr 21, 2024
92d10a9
remove test stage from conda build for macs and test instead by creat…
eberrigan Apr 22, 2024
b4fdc74
test workflow by pushing to current branch
eberrigan Apr 22, 2024
72bc341
test conda package on macos runner
eberrigan Apr 23, 2024
967838a
Mac build does not need nvidia channel
eberrigan Apr 23, 2024
26526c5
qudida and albumentations are conda installed now
eberrigan Apr 23, 2024
61ba13d
add comment with original issue
eberrigan Apr 23, 2024
c26cc56
use python 3.9
eberrigan Apr 23, 2024
9c31c28
use conda match specifications syntax
eberrigan Apr 23, 2024
e8d3f35
make print statements more readable for troubleshooting python versio…
eberrigan Apr 23, 2024
5d576ac
clean up build file
eberrigan Apr 24, 2024
519c90e
Merge branch 'develop' into elizabeth/fix-mac-conda-build
eberrigan May 16, 2024
4b84171
update version for pre-release
eberrigan May 16, 2024
b6cbdba
add TODO
eberrigan May 16, 2024
a72202a
add tests for conda packages before uploading
eberrigan May 17, 2024
b98594c
update ci comments and branches
eberrigan May 17, 2024
d084f74
remove macos test of pip wheel since python 3.9 is not supported by s…
eberrigan May 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .conda/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ requirements:
- conda-forge::ndx-pose
- conda-forge::importlib-metadata ==4.11.4

test:
imports:
- sleap
# This no longer works so we have moved it to the build workflow
# https://github.com/talmolab/sleap/pull/1744
# test:
# imports:
# - sleap
3 changes: 1 addition & 2 deletions .conda_mac/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,10 @@

# Install anything that didn't get conda installed via pip.
# We need to turn pip index back on because Anaconda turns it off for some reason.

export PIP_NO_INDEX=False
export PIP_NO_DEPENDENCIES=False
export PIP_IGNORE_INSTALLED=False

pip install --no-cache-dir -r requirements.txt --no-binary qudida,albumentations
pip install --no-cache-dir -r requirements.txt

python setup.py install --single-version-externally-managed --record=record.txt
1 change: 0 additions & 1 deletion .conda_mac/condarc.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# This file is not used at the moment, but when github actions can be used to build the package, it needs to be listed.
# https://github.com/github/roadmap/issues/528

channels:
Expand Down
10 changes: 5 additions & 5 deletions .conda_mac/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ source:

requirements:
host:
- conda-forge::python ~=3.9
- conda-forge::python >=3.9.0, <3.10.0
- anaconda::numpy >=1.19.5,<1.23.0
- conda-forge::setuptools
- conda-forge::packaging
Expand Down Expand Up @@ -59,7 +59,7 @@ requirements:
- conda-forge::ndx-pose

run:
- conda-forge::python ~=3.9
- conda-forge::python >=3.9.0, <3.10.0
- conda-forge::attrs >=21.2.0
- conda-forge::cattrs ==1.1.1
- conda-forge::h5py
Expand Down Expand Up @@ -89,6 +89,6 @@ requirements:
- conda-forge::albumentations
- conda-forge::ndx-pose

test:
imports:
- sleap
# test:
# imports:
# - sleap
68 changes: 68 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,74 @@ jobs:
run: |
conda build .conda_mac --output-folder build

# Test built conda package (Ubuntu and Windows)
- name: Test built conda package (Ubuntu and Windows)
if: matrix.os != 'macos-14'
shell: bash -l {0}
run: |
echo "Current build path: $BUILD_PATH"
conda deactivate

echo "Python executable before activating environment:"
which python
echo "Python version before activating environment:"
python --version
echo "Conda info before activating environment:"
conda info

echo "Creating and testing conda environment with sleap package..."
conda create -y -n sleap_test -c file://$BUILD_PATH -c sleap/label/dev -c conda-forge -c nvidia -c anaconda sleap
conda activate sleap_test

echo "Python executable after activating sleap_test environment:"
which python
echo "Python version after activating sleap_test environment:"
python --version
echo "Conda info after activating sleap_test environment:"
conda info
echo "List of installed conda packages in the sleap_test environment:"
conda list
echo "List of installed pip packages in the sleap_test environment:"
pip list

echo "Testing sleap package installation..."
sleap_version=$(python -c "import sleap; print(sleap.__version__)")
echo "Test completed using sleap version: $sleap_version"

# Test built conda package (Mac)
- name: Test built conda package (Mac)
if: matrix.os == 'macos-14'
shell: bash -l {0}
run: |
echo "Current build path: $BUILD_PATH"
conda deactivate

echo "Python executable before activating environment:"
which python
echo "Python version before activating environment:"
python --version
echo "Conda info before activating environment:"
conda info

echo "Creating and testing conda environment with sleap package..."
conda create -y -n sleap_test -c file://$BUILD_PATH -c conda-forge -c anaconda sleap
conda activate sleap_test

echo "Python executable after activating sleap_test environment:"
which python
echo "Python version after activating sleap_test environment:"
python --version
echo "Conda info after activating sleap_test environment:"
conda info
echo "List of installed conda packages in the sleap_test environment:"
conda list
echo "List of installed pip packages in the sleap_test environment:"
pip list

echo "Testing sleap package installation..."
sleap_version=$(python -c "import sleap; print(sleap.__version__)")
echo "Test completed using sleap version: $sleap_version"

# Login to conda (Ubuntu)
- name: Login to Anaconda (Ubuntu)
if: matrix.os == 'ubuntu-22.04'
Expand Down
77 changes: 74 additions & 3 deletions .github/workflows/build_manual.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ on:
- '.github/workflows/build_manual.yml'
branches:
# - develop
- fakebranch
# - talmo/fix-mac-v140
# - fakebranch
- elizabeth/fix-mac-conda-build

jobs:
build:
Expand Down Expand Up @@ -83,20 +83,91 @@ jobs:
shell: bash -l {0}
run: |
conda build .conda --output-folder build
echo "BUILD_PATH=$(pwd)/build" >> "$GITHUB_ENV"

# Build conda package (Windows)
# Build conda package (Windows)
- name: Build conda package (Windows)
if: matrix.os == 'windows-2022'
shell: powershell
run: |
conda build .conda --output-folder build
echo "BUILD_PATH=\$(pwd)\build" >> "$env:GITHUB_ENV"

# Build conda package (Mac)
- name: Build conda package (Mac)
if: matrix.os == 'macos-14'
shell: bash -l {0}
run: |
conda build .conda_mac --output-folder build
echo "BUILD_PATH=$(pwd)/build" >> "$GITHUB_ENV"

# Test built conda package (Ubuntu and Windows)
- name: Test built conda package (Ubuntu and Windows)
if: matrix.os != 'macos-14'
shell: bash -l {0}
run: |
echo "Current build path: $BUILD_PATH"
conda deactivate

echo "Python executable before activating environment:"
which python
echo "Python version before activating environment:"
python --version
echo "Conda info before activating environment:"
conda info

echo "Creating and testing conda environment with sleap package..."
conda create -y -n sleap_test -c file://$BUILD_PATH -c sleap/label/dev -c conda-forge -c nvidia -c anaconda sleap
conda activate sleap_test

echo "Python executable after activating sleap_test environment:"
which python
echo "Python version after activating sleap_test environment:"
python --version
echo "Conda info after activating sleap_test environment:"
conda info
echo "List of installed conda packages in the sleap_test environment:"
conda list
echo "List of installed pip packages in the sleap_test environment:"
pip list

echo "Testing sleap package installation..."
sleap_version=$(python -c "import sleap; print(sleap.__version__)")
echo "Test completed using sleap version: $sleap_version"

# Test built conda package (Mac)
- name: Test built conda package (Mac)
if: matrix.os == 'macos-14'
shell: bash -l {0}
run: |
echo "Current build path: $BUILD_PATH"
conda deactivate

echo "Python executable before activating environment:"
which python
echo "Python version before activating environment:"
python --version
echo "Conda info before activating environment:"
conda info

echo "Creating and testing conda environment with sleap package..."
conda create -y -n sleap_test -c file://$BUILD_PATH -c conda-forge -c anaconda sleap
conda activate sleap_test

echo "Python executable after activating sleap_test environment:"
which python
echo "Python version after activating sleap_test environment:"
python --version
echo "Conda info after activating sleap_test environment:"
conda info
echo "List of installed conda packages in the sleap_test environment:"
conda list
echo "List of installed pip packages in the sleap_test environment:"
pip list

echo "Testing sleap package installation..."
sleap_version=$(python -c "import sleap; print(sleap.__version__)")
echo "Test completed using sleap version: $sleap_version"

# # Login to conda (Ubuntu)
# - name: Login to Anaconda (Ubuntu)
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# This file contains the minimal requirements to be installed via pip when using conda.

# No conda packages for these
imgstore<0.3.0 # 0.3.3 results in https://github.com/O365/python-o365/issues/591
imgstore<0.3.0 # 0.3.3 results in https://github.com/O365/python-o365/issues/591 which is from https://github.com/regebro/tzlocal/issues/112 when tzlocal is v3.0
nixio>=1.5.3 # Constrain put on by @jgrewe from G-Node
qimage2ndarray # ==1.9.0
segmentation-models
Expand Down
2 changes: 1 addition & 1 deletion sleap/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"""


__version__ = "1.4.0"
__version__ = "1.4.0a0"


def versions():
Expand Down
Loading