Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ A read_geotiff function for reading GeoTIFF into ndarray #3

Merged
merged 14 commits into from
Feb 28, 2024
Merged

✨ A read_geotiff function for reading GeoTIFF into ndarray #3

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d3881eb
:heavy_plus_sign: Add tiff
weiji14 Feb 26, 2024
e6b37b1
:heavy_plus_sign: Add ndarray
weiji14 Feb 26, 2024
6e53a0e
:heavy_plus_sign: Add numpy
weiji14 Feb 26, 2024
d77db44
:sparkles: A read_geotiff function for reading GeoTIFF into ndarray
weiji14 Feb 26, 2024
7e74ee7
:triangular_flag_on_post: Add pyo3 extension-module feature
weiji14 Feb 27, 2024
89ca974
:heavy_plus_sign: Add tempfile
weiji14 Feb 27, 2024
7f01407
:recycle: Refactor to allow read_geotiff Rust fn to read from buffer
weiji14 Feb 27, 2024
de7bd60
:white_check_mark: Add unit test for reading a BigTIFF file
weiji14 Feb 27, 2024
2cf92eb
:white_check_mark: Add unit test for reading a single-band GeoTIFF file
weiji14 Feb 27, 2024
7cad40d
:heavy_plus_sign: Add numpy as a python dependency
weiji14 Feb 27, 2024
b54ab26
:green_heart: Install NumPy build requirements
weiji14 Feb 27, 2024
41259a1
:green_heart: Install compilers and build libraries needed for NumPy
weiji14 Feb 27, 2024
9dfa03a
:green_heart: Install ninja-build
weiji14 Feb 27, 2024
a30ec8f
:memo: Add more crate and module level docs
weiji14 Feb 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ jobs:
install: |
apt-get update
apt-get install -y --no-install-recommends python3 python3-pip
apt build-dep numpy
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original error on s390x build at https://github.com/weiji14/cog3pio/actions/runs/8071580043/job/22051501437#step:7:410 before this line was added:

    error: subprocess-exited-with-error
    
    × Preparing metadata (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [20 lines of output]
        + /usr/bin/python3 /tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b/vendored-meson/meson/meson.py setup /tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b /tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b/.mesonpy-q6jzljl7 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b/.mesonpy-q6jzljl7/meson-python-native-file.ini
        The Meson build system
        Version: 1.2.99
        Source dir: /tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b
        Build dir: /tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b/.mesonpy-q6jzljl7
        Build type: native build
        Project name: NumPy
        Project version: 1.26.4
        
        ../meson.build:1:0: ERROR: Unknown compiler(s): [['cc'], ['gcc'], ['clang'], ['nvc'], ['pgcc'], ['icc'], ['icx']]
        The following exception(s) were encountered:
        Running `cc --version` gave "[Errno 2] No such file or directory: 'cc'"
        Running `gcc --version` gave "[Errno 2] No such file or directory: 'gcc'"
        Running `clang --version` gave "[Errno 2] No such file or directory: 'clang'"
        Running `nvc --version` gave "[Errno 2] No such file or directory: 'nvc'"
        Running `pgcc --version` gave "[Errno 2] No such file or directory: 'pgcc'"
        Running `icc --version` gave "[Errno 2] No such file or directory: 'icc'"
        Running `icx --version` gave "[Errno 2] No such file or directory: 'icx'"
        
        A full log can be found at /tmp/pip-install-6l_88pqi/numpy_281cd73ebd0a445c8bd7bbbb7479992b/.mesonpy-q6jzljl7/meson-logs/meson-log.txt
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: metadata-generation-failed
  
  × Encountered error while generating package metadata.
  ╰─> See above for output.
  
  note: This is an issue with the package mentioned above, not pip.
  hint: See above for details.
  25h::error::The process '/home/runner/work/_actions/uraimo/run-on-arch-action/517085f0367c8256bcfa753e3e13e1550af09954/src/run-on-arch.sh' failed with exit code 1

So at b54ab26, I added this apt build-dep numpy line following https://numpy.org/devdocs/building/#system-level-dependencies, but got E: You must put some 'deb-src' URIs in your sources.list at https://github.com/weiji14/cog3pio/actions/runs/8071955898/job/22052723659?pr=3#step:7:330.

Maybe best to just install the compilers individually instead of messing with /etc/apt/sources.list

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, added the NumPy build deps manually at 41259a1, and also needed to apt install ninja-build to get NumPy to compile on armv7/s390x/ppc64le (9dfa03a). The CI on those three platforms take ~24min to run though, compared to <5min for linux/aarch64 🙃

pip3 install -U pip pytest
run: |
set -e
Expand Down
Loading
Loading