From b1dd115f30ac9a89fe6430d906d7c3ad1ee48296 Mon Sep 17 00:00:00 2001 From: Neil Shephard Date: Wed, 27 Nov 2024 09:33:30 +0000 Subject: [PATCH 1/4] ci: Use --no-build-isolation to install cgat As [suggested](https://github.com/cgat-developers/cgat-apps/issues/134#issuecomment-2497400890) attempting to install dependencies (and in one instance `IsoSLAM` itself) with the `--no-build-isolation` flag in the hope that NumPy is correctly detected by `cgat` when it is built and installed. Further in light of [investigations](https://github.com/cgat-developers/cgat-apps/issues/134#issuecomment-2503431974) also explicitly install the build requirements (`numpy` / `Cython` / `pysam` / `wheel`) prior to installing. --- .github/workflows/sphinx_docs_to_gh_pages.yaml | 8 +++++++- .github/workflows/tests.yaml | 10 +++++++--- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/.github/workflows/sphinx_docs_to_gh_pages.yaml b/.github/workflows/sphinx_docs_to_gh_pages.yaml index 7550757..8d09f5e 100644 --- a/.github/workflows/sphinx_docs_to_gh_pages.yaml +++ b/.github/workflows/sphinx_docs_to_gh_pages.yaml @@ -24,7 +24,13 @@ jobs: python-version: 3.11 - name: Installing the Documentation requirements run: | - pip3 install .[docs] + # Install build dependencies for now (see + # https://github.com/cgat-developers/cgat-apps/issues/134#issuecomment-2503431974) + pip install numpy + pip install Cython + pip install pysam + pip install wheel + pip3 install --no-build-isolation .[docs] - name: Running Sphinx to gh-pages Action # Uncomment if only tagged releases are to have documentation built # if: startsWith(github.ref, 'refs/tags') diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index 08eebc6..1fc3ab0 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -40,10 +40,14 @@ jobs: pip install --upgrade virtualenv pip install --upgrade pip setuptools virtualenv --upgrade-embed-wheels + # Install build dependencies for now (see + # https://github.com/cgat-developers/cgat-apps/issues/134#issuecomment-2503431974) pip install numpy - pip show numpy - pip install cgatcore - pip install cgat + pip install Cython + pip install pysam + pip install wheel + pip install --no-build-isolation cgatcore + pip install --no-build-isolation cgat pip install -e .[tests] - name: Test with pytest run: | From e26cf6329c7922b171cf0193d59ddb3441568379 Mon Sep 17 00:00:00 2001 From: Neil Shephard Date: Wed, 27 Nov 2024 11:50:56 +0000 Subject: [PATCH 2/4] ci: install cgat from GitHub v0.7.4 --- .github/workflows/sphinx_docs_to_gh_pages.yaml | 1 + .github/workflows/tests.yaml | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/sphinx_docs_to_gh_pages.yaml b/.github/workflows/sphinx_docs_to_gh_pages.yaml index 8d09f5e..6cf9855 100644 --- a/.github/workflows/sphinx_docs_to_gh_pages.yaml +++ b/.github/workflows/sphinx_docs_to_gh_pages.yaml @@ -30,6 +30,7 @@ jobs: pip install Cython pip install pysam pip install wheel + pip install --no-build-isolation git+https://github.com/cgat-developers/cgat-apps@v0.7.4 pip3 install --no-build-isolation .[docs] - name: Running Sphinx to gh-pages Action # Uncomment if only tagged releases are to have documentation built diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index 1fc3ab0..e588a37 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -47,8 +47,8 @@ jobs: pip install pysam pip install wheel pip install --no-build-isolation cgatcore - pip install --no-build-isolation cgat - pip install -e .[tests] + pip install --no-build-isolation git+https://github.com/cgat-developers/cgat-apps@v0.7.4 + pip install --no-build-isolation -e .[tests] - name: Test with pytest run: | pytest --cov=isoslam --cov-report=xml --mpl -x From d858c9f676063b4f0302806cf740e1fad886cfc8 Mon Sep 17 00:00:00 2001 From: Neil Shephard Date: Wed, 27 Nov 2024 12:15:42 +0000 Subject: [PATCH 3/4] ci: Remove Windows from os matrix of tests.yaml There is no wheel for pysam on the Windows release (see [files](https://pypi.org/project/pysam/#files)) so removing windows from the os matrix of tests. --- .github/workflows/tests.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index e588a37..282e6bb 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -27,7 +27,7 @@ jobs: runs-on: ${{ matrix.os }} strategy: matrix: - os: ["ubuntu-latest", "macos-latest", "windows-latest"] + os: ["ubuntu-latest", "macos-latest"] # "windows-latest"] python-version: ["3.9", "3.10", "3.11", "3.12"] steps: - uses: actions/checkout@v4 From a0c94e5f35e72f3411c4f1d8ce9f6440682f90a9 Mon Sep 17 00:00:00 2001 From: Neil Shephard Date: Thu, 28 Nov 2024 11:11:39 +0000 Subject: [PATCH 4/4] ci: trying newer cgatcore package setup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [updated setup.py and pyproject.toml](https://github.com/cgat-developers/cgat-apps/pull/136/) moves some of the configuration options of `cgat-apps` from `setup.py` to `pyproject.toml`, testing to see if they install. Previously I've found that there were some errors building pysam for windows (see [log](https://github.com/sudlab/IsoSLAM/actions/runs/12049902667/job/33597647405?pr=77)) there are no wheels for pysam for windows so it attempts to build from source so I've removed Windows from the build matrix but then hit errors trying to compile [`v0.7.4`](https://github.com/sudlab/IsoSLAM/actions/runs/12050103589/job/33598275747?pr=77) ``` cgat/BamTools/bamtools.c: In function ‘__pyx_pw_4cgat_8BamTools_8bamtools_7bam2bam_filter_bam’: cgat/BamTools/bamtools.c:41504:8: warning: ‘__pyx_v_remove_contig_tids’ may be used uninitialized in this function [-Wmaybe-uninitialized] 41504 | int *__pyx_v_remove_contig_tids; | ^~~~~~~~~~~~~~~~~~~~~~~~~~ cgat/BamTools/bamtools.c: In function ‘__pyx_pf_4cgat_8BamTools_8bamtools_2bam2stats_count.constprop’: cgat/BamTools/bamtools.c:1204:40: warning: ‘__pyx_v_fastq_nreads’ may be used uninitialized in this function [-Wmaybe-uninitialized] 1204 | #define PyInt_FromLong PyLong_FromLong | ^~~~~~~~~~~~~~~ cgat/BamTools/bamtools.c:27028:11: note: ‘__pyx_v_fastq_nreads’ was declared here 27028 | int64_t __pyx_v_fastq_nreads; | ^~~~~~~~~~~~~~~~~~~~ cgat/BamTools/bamtools.c:33541:31: warning: ‘__pyx_v_fastq_counts’ may be used uninitialized in this function [-Wmaybe-uninitialized] 33541 | __pyx_v_fastq_count = (&(__pyx_v_fastq_counts[__pyx_v_read_index])); | ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ gcc -shared -Wl,--rpath=/opt/hostedtoolcache/Python/3.12.7/x64/lib -Wl,--rpath=/opt/hostedtoolcache/Python/3.12.7/x64/lib build/temp.linux-x86_64-cpython-312/cgat/BamTools/bamtools.o -L/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/pysam -L/opt/hostedtoolcache/Python/3.12.7/x64/lib -L/opt/hostedtoolcache/Python/3.12.7/x64/lib -lhts -lctabixproxies.cpython-312-x86_64-linux-gnu -lcfaidx.cpython-312-x86_64-linux-gnu -lcsamfile.cpython-312-x86_64-linux-gnu -lcvcf.cpython-312-x86_64-linux-gnu -lcbcf.cpython-312-x86_64-linux-gnu -lctabix.cpython-312-x86_64-linux-gnu -lchtslib.cpython-312-x86_64-linux-gnu -o build/lib.linux-x86_64-cpython-312/cgat/BamTools/bamtools.cpython-312-x86_64-linux-gnu.so -Wl,-rpath,/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/pysam -Wl,-rpath,/opt/hostedtoolcache/Python/3.12.7/x64/lib -Wl,-rpath,/opt/hostedtoolcache/Python/3.12.7/x64/lib /usr/bin/ld: cannot find -lhts: No such file or directory collect2: error: ld returned 1 exit status error: command '/usr/bin/gcc' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for cgat Failed to build cgat ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cgat) ``` This may be a system library that is required though :thinking: --- .github/workflows/sphinx_docs_to_gh_pages.yaml | 2 +- .github/workflows/tests.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/sphinx_docs_to_gh_pages.yaml b/.github/workflows/sphinx_docs_to_gh_pages.yaml index 6cf9855..dbc43ea 100644 --- a/.github/workflows/sphinx_docs_to_gh_pages.yaml +++ b/.github/workflows/sphinx_docs_to_gh_pages.yaml @@ -30,7 +30,7 @@ jobs: pip install Cython pip install pysam pip install wheel - pip install --no-build-isolation git+https://github.com/cgat-developers/cgat-apps@v0.7.4 + pip install --no-build-isolation git+https://github.com/cgat-developers/cgat-apps@AC-modifybuild pip3 install --no-build-isolation .[docs] - name: Running Sphinx to gh-pages Action # Uncomment if only tagged releases are to have documentation built diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index 282e6bb..1aad1d9 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -47,7 +47,7 @@ jobs: pip install pysam pip install wheel pip install --no-build-isolation cgatcore - pip install --no-build-isolation git+https://github.com/cgat-developers/cgat-apps@v0.7.4 + pip install --no-build-isolation git+https://github.com/cgat-developers/cgat-apps@AC-modifybuild pip install --no-build-isolation -e .[tests] - name: Test with pytest run: |