-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update install to rely on mamba / conda envs only #214
Changes from 1 commit
2a913fe
ad4a771
06330cd
9ca6cc2
14e428b
a18b1f1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
name: Minimal CI | ||
|
||
on: | ||
push: | ||
branches: | ||
- "**" | ||
paths-ignore: | ||
- README.md | ||
- CHANGELOG.md | ||
- LICENSE | ||
- CONTRIBUTING.md | ||
- docs/** | ||
- mkdocs.yml | ||
|
||
jobs: | ||
test: | ||
uses: arup-group/actions-city-modelling-lab/.github/workflows/python-install-lint-test.yml@main | ||
with: | ||
os: ubuntu-latest | ||
py3version: "11" | ||
notebook_kernel: genet | ||
lint: false | ||
additional_mamba_args: coin-or-cbc "proj>=9.3" | ||
|
||
aws-upload: | ||
needs: test | ||
if: needs.test.result == 'success' | ||
uses: arup-group/actions-city-modelling-lab/.github/workflows/aws-upload.yml@main | ||
secrets: | ||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
AWS_S3_CODE_BUCKET: ${{ secrets.AWS_S3_CODE_BUCKET }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,69 +1,36 @@ | ||
name: Daily GeNet CI Build | ||
name: Daily CI | ||
|
||
on: | ||
schedule: | ||
- cron: '37 14 * * 1-5' | ||
- cron: '23 14 * * 1-5' | ||
|
||
jobs: | ||
build: | ||
get-date: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v3 | ||
|
||
- uses: actions/setup-python@v1 | ||
with: | ||
python-version: 3.11 | ||
|
||
- uses: actions/cache@v1 | ||
with: | ||
path: ~/.cache/pip | ||
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} | ||
restore-keys: | | ||
${{ runner.os }}-pip- | ||
|
||
- name: Install dependencies | ||
run: | | ||
export ACCEPT_EULA=Y | ||
sudo apt-get update | ||
python -m pip install --upgrade pip | ||
sudo apt-get install -y python3-pip libgdal-dev locales | ||
sudo apt-get install -y libspatialindex-dev | ||
sudo apt-get install -y coinor-cbc | ||
export CPLUS_INCLUDE_PATH=/usr/include/gdal | ||
export C_INCLUDE_PATH=/usr/include/gdal | ||
sudo apt-get install ca-certificates | ||
export CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt | ||
pip install GDAL==3.4.1 | ||
pip install -e '.[dev]' | ||
|
||
- name: Install jupyter kernel | ||
run: python -m ipykernel install --user --name genet | ||
|
||
- name: Run tests | ||
run: pytest | ||
|
||
- name: Send build success notification | ||
if: success() | ||
uses: rtCamp/[email protected] | ||
env: | ||
SLACK_MESSAGE: ${{ github.repository }} Daily scheduled CI Build ${{ github.run_number }} has succeeded | ||
SLACK_TITLE: Daily Scheduled CI Build Success | ||
SLACK_CHANNEL: city-modelling-feeds | ||
SLACK_USERNAME: GitHub Build Bot | ||
SLACK_ICON: https://slack-files2.s3-us-west-2.amazonaws.com/avatars/2017-12-19/288981919427_f45f04edd92902a96859_512.png | ||
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }} | ||
|
||
- name: Send build failure notification | ||
if: failure() | ||
uses: rtCamp/[email protected] | ||
env: | ||
SLACK_COLOR: '#FF0000' | ||
SLACK_LINK_NAMES: true | ||
SLACK_MESSAGE: '<!here> ${{ github.repository }} Daily scheduled CI Build ${{ github.run_number }} has failed' | ||
SLACK_TITLE: Daily Scheduled CI Build Failure! | ||
SLACK_CHANNEL: city-modelling-feeds | ||
SLACK_USERNAME: GitHub Build Bot | ||
SLACK_ICON: https://slack-files2.s3-us-west-2.amazonaws.com/avatars/2017-12-19/288981919427_f45f04edd92902a96859_512.png | ||
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }} | ||
- name: Add date to github output env | ||
run: echo "DATE=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT | ||
|
||
test: | ||
needs: get-date | ||
uses: arup-group/actions-city-modelling-lab/.github/workflows/python-install-lint-test.yml@main | ||
with: | ||
os: ubuntu-latest | ||
py3version: "11" | ||
notebook_kernel: genet | ||
pytest_args: '--no-cov' # ignore coverage | ||
cache_mamba_env: false | ||
lint: false | ||
mamba_env_name: daily-ci | ||
additional_mamba_args: coin-or-cbc "proj>=9.3" | ||
|
||
slack-notify-ci: | ||
needs: test | ||
if: always() | ||
uses: arup-group/actions-city-modelling-lab/.github/workflows/slack-notify.yml@main | ||
secrets: | ||
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }} | ||
with: | ||
result: needs.test.result | ||
channel: genet-feed | ||
message: Daily CI action |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
name: Pull Request CI | ||
|
||
on: | ||
pull_request: | ||
KasiaKoz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
branches: | ||
- main | ||
paths-ignore: | ||
- README.md | ||
- CHANGELOG.md | ||
- LICENSE | ||
- CONTRIBUTING.md | ||
- docs/** | ||
- mkdocs.yml | ||
|
||
jobs: | ||
test: | ||
strategy: | ||
matrix: | ||
os: [windows-latest, ubuntu-latest, macos-latest] | ||
py3version: ["9", "11"] | ||
fail-fast: false | ||
uses: arup-group/actions-city-modelling-lab/.github/workflows/python-install-lint-test.yml@main | ||
with: | ||
os: ${{ matrix.os }} | ||
py3version: ${{ matrix.py3version }} | ||
notebook_kernel: genet | ||
lint: false | ||
pytest_args: '--no-cov' # ignore coverage | ||
upload_to_codecov: false | ||
additional_mamba_args: coin-or-cbc "proj>=9.3" | ||
|
||
test-coverage: | ||
uses: arup-group/actions-city-modelling-lab/.github/workflows/python-install-lint-test.yml@main | ||
with: | ||
os: ubuntu-latest | ||
py3version: "11" | ||
notebook_kernel: genet | ||
lint: false | ||
pytest_args: 'tests/' # ignore example notebooks | ||
upload_to_codecov: true | ||
additional_mamba_args: coin-or-cbc "proj>=9.3" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,18 +1,11 @@ | ||
FROM python:3.11.4-bullseye | ||
FROM mambaorg/micromamba:1.5.3-bullseye-slim | ||
|
||
RUN apt-get update && \ | ||
apt-get upgrade -y && \ | ||
apt-get -y install gcc git libgdal-dev libgeos-dev libspatialindex-dev curl coinor-cbc && \ | ||
rm -rf /var/lib/apt/lists/* | ||
COPY --chown=$MAMBA_USER:$MAMBA_USER . ./src | ||
|
||
RUN curl -sL https://deb.nodesource.com/setup_17.x | bash - | ||
RUN apt-get -y install nodejs && node --version && npm --version && \ | ||
rm -rf /var/lib/apt/lists/* | ||
RUN micromamba install -y -n base -c conda-forge -c city-modelling-lab python=3.11 "proj>=9.3" pip coin-or-cbc --file src/requirements/base.txt && \ | ||
micromamba clean --all --yes | ||
ARG MAMBA_DOCKERFILE_ACTIVATE=1 | ||
|
||
RUN /usr/local/bin/python -m pip install --no-cache-dir --compile --upgrade pip | ||
|
||
COPY . . | ||
|
||
RUN pip3 install cmake>=3 | ||
RUN pip3 install --no-cache-dir --compile -e . && pip cache purge | ||
RUN pip install --no-deps ./src | ||
|
||
ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
import pyproj | ||
|
||
__version__ = "4.0.0" | ||
|
||
from genet.auxiliary_files import AuxiliaryFile | ||
|
@@ -19,3 +21,5 @@ | |
from genet.schedule_elements import Route, Schedule, Service, Stop | ||
from genet.use.road_pricing import Toll | ||
from genet.utils import elevation, google_directions, graph_operations | ||
|
||
pyproj.network.set_network_enabled(False) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What does this do? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It stops it from trying to connect to the network to download transformation gridfiles on-the-fly (see #213). If it does try, there is a bug with handling self-signed SSL certificates that will cause an error. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This wasn't needed before because the PROJ environment variable associated with this is set to OFF when downloading from homebrew/apt, but is ON when downloading from conda-forge (because in the former it automatically downloads the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See OSGeo/PROJ#3977 and pyproj4/pyproj#1233 for conversations I've initiated on it in upstream projects |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I INSIST ON RUNNING THIS ON THE 37th MINUTE!!! 🤣
(Actually, thinking about it, 37 actually does carry a very small advantage over 23, in that it makes clear that this is the minute, rather than the hour, but that's admittedly a marginal gain)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You know what, I did think it was hours 😅 maybe I should change it back to 37...