Upgrade to Ubuntu 22.04, Python 3.10 #3631
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Using a branch installation for GWLF-E until WikiWatershed/gwlf-e#95 is published to PyPI.
This was not used anywhere anymore.
Installing Firefox was difficult in the new Ubuntu VM. Chromium is a better representation of our user base.
rachelekm
reviewed
Jul 6, 2024
There was a problem hiding this comment.
I tested this branch's deployment to staging (confirmed via version.txt) but had trouble with the following -- not sure if these errors are expected or a side effect of the open delineating watersheds bug though, but noting here just in case:
I was able to analyze a predefined boundary but the free draw and square km would display error on analysis:
Both a drawn area and a predefined boundary selection were displaying error for Site Storm Model for me:
I was able to run through the commands to destroy and recreate the VMs locally 🎉 However when testing locally in the browser it appears that none of my imports seemed to work. Looking back at the logs I seemed to get an error with numpy, I re-ran the install command and included the logs below. Is this something you bumped into?
Show logs with error message
% vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -dmpq'
++ basename ./scripts/aws/setupdb.sh
+ usage='setupdb.sh [-h] [options] \n
--Sets up a postgresql database for MMW \n
\n
where options are one or more of: \n
-h show this help text\n
-b load/reload boundary data\n
-f load a named boundary sql.gz\n
-s load/reload stream data\n
-S load/reload Hi Res stream data (very large)\n
-d load/reload DRB stream data\n
-m load/reload mapshed data\n
-p load/reload DEP data\n
-c load/reload nhdplus catchment data\n
-q load/reload water quality data\n
-X purge s3 cache as well\n
-x purge s3 cache for given path\n
'
+ FILE_HOST=https://s3.amazonaws.com/data.mmw.azavea.com
+ load_boundary=false
+ file_to_load=
+ load_stream=false
+ load_hires_stream=false
+ load_mapshed=false
+ load_water_quality=false
+ load_catchment=false
+ should_purge_cache=false
+ getopts :hbsSdpmqcf:x: opt
+ case $opt in
+ load_drb_streams=true
+ getopts :hbsSdpmqcf:x: opt
+ case $opt in
+ load_mapshed=true
+ getopts :hbsSdpmqcf:x: opt
+ case $opt in
+ load_dep=true
+ getopts :hbsSdpmqcf:x: opt
+ case $opt in
+ load_water_quality=true
+ getopts :hbsSdpmqcf:x: opt
++ cat /etc/mmw.d/env/MMW_DB_HOST
+ export PGHOST=33.33.34.30
+ PGHOST=33.33.34.30
++ cat /etc/mmw.d/env/MMW_DB_NAME
+ export PGDATABASE=mmw
+ PGDATABASE=mmw
++ cat /etc/mmw.d/env/MMW_DB_USER
+ export PGUSER=mmw
+ PGUSER=mmw
++ cat /etc/mmw.d/env/MMW_DB_PASSWORD
+ export PGPASSWORD=mmw
+ PGPASSWORD=mmw
++ cat /etc/mmw.d/env/MMW_PUBLIC_HOSTED_ZONE_NAME
cat: /etc/mmw.d/env/MMW_PUBLIC_HOSTED_ZONE_NAME: No such file or directory
+ export PUBLIC_HOSTED_ZONE_NAME=
+ PUBLIC_HOSTED_ZONE_NAME=
+ psql -c 'CREATE EXTENSION IF NOT EXISTS postgis;'
NOTICE: extension "postgis" already exists, skipping
CREATE EXTENSION
+ psql -c 'ALTER TABLE spatial_ref_sys OWNER TO mmw;'
ALTER TABLE
+ psql -c 'CREATE EXTENSION IF NOT EXISTS pg_trgm;'
NOTICE: extension "pg_trgm" already exists, skipping
CREATE EXTENSION
+ envdir /etc/mmw.d/env /opt/app/manage.py migrate
RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe
Unable to import compiled InitSnow_inner, using slower version
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/InitSnow.py", line 6, in <module>
from .InitSnow_f_inner_compiled import InitSnow_f_inner
ImportError: numpy.core.multiarray failed to import
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/app/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 89, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 75, in handle
self.check(databases=[database])
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 419, in check
all_issues = checks.run_checks(
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/registry.py", line 76, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
all_namespaces = _load_all_namespaces(resolver)
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
url_patterns = getattr(resolver, 'url_patterns', [])
File "/usr/local/lib/python3.10/dist-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python3.10/dist-packages/django/urls/resolvers.py", line 602, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python3.10/dist-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python3.10/dist-packages/django/urls/resolvers.py", line 595, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/app/mmw/urls.py", line 14, in <module>
re_path(r'^api/', include('apps.geoprocessing_api.urls')),
File "/usr/local/lib/python3.10/dist-packages/django/urls/conf.py", line 34, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/app/apps/geoprocessing_api/urls.py", line 4, in <module>
from apps.modeling.views import get_job
File "/opt/app/apps/modeling/views.py", line 33, in <module>
from apps.modeling import tasks, geoprocessing
File "/opt/app/apps/modeling/tasks.py", line 25, in <module>
from gwlfe import gwlfe, Parser
File "/usr/local/lib/python3.10/dist-packages/gwlfe/gwlfe.py", line 20, in <module>
from . import AFOS_old
File "/usr/local/lib/python3.10/dist-packages/gwlfe/AFOS_old.py", line 7, in <module>
from .MultiUse_Fxns.Discharge.StreamFlow_1 import StreamFlow_1_f
File "/usr/local/lib/python3.10/dist-packages/gwlfe/MultiUse_Fxns/Discharge/StreamFlow_1.py", line 5, in <module>
from gwlfe.MultiUse_Fxns.Discharge.Flow import Flow
File "/usr/local/lib/python3.10/dist-packages/gwlfe/MultiUse_Fxns/Discharge/Flow.py", line 3, in <module>
from gwlfe.Input.WaterBudget.GrFlow import GrFlow
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/GrFlow.py", line 3, in <module>
from gwlfe.Input.WaterBudget.Percolation import Percolation
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/Percolation.py", line 4, in <module>
from gwlfe.Input.WaterBudget.Infiltration import Infiltration
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/Infiltration.py", line 4, in <module>
from gwlfe.Input.WaterBudget.Water import Water
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/Water.py", line 3, in <module>
from gwlfe.Input.WaterBudget.Melt_1 import Melt_1, Melt_1_f
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/Melt_1.py", line 4, in <module>
from gwlfe.Input.WaterBudget.InitSnow import InitSnow
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/InitSnow.py", line 9, in <module>
from gwlfe.Input.WaterBudget.InitSnow_inner import InitSnow_f_inner
File "/usr/local/lib/python3.10/dist-packages/gwlfe/Input/WaterBudget/InitSnow_inner.py", line 4, in <module>
cc = CC('InitSnow_f_inner_compiled')
File "/usr/local/lib/python3.10/dist-packages/numba/pycc/cc.py", line 65, in __init__
self._toolchain = Toolchain()
File "/usr/local/lib/python3.10/dist-packages/numba/pycc/platform.py", line 78, in __init__
self._raise_external_compiler_error()
File "/usr/local/lib/python3.10/dist-packages/numba/pycc/platform.py", line 121, in _raise_external_compiler_error
raise RuntimeError(msg)
RuntimeError: Attempted to compile AOT function without the compiler used by `numpy.distutils` present. If using conda try:
#> conda install gcc_linux-64 gxx_linux-64
|
The geoprocessing service should be fixed in 45e4f4d. I haven't seen the other error before, running that command now to see if I encounter it. |
|
Destroy and recreated the |
4 tasks
|
The numpy issues should be fixed in 4c1fe4f and be5b2c0, as already reviewed in WikiWatershed/gwlf-e#101 (review). Since you've already provisioned the VM for the review above, try running the failing command above again: vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -dmpq'If it succeeds, then we can probably approve this PR for now, especially because this version is also deployed to staging and is testing well there: https://staging.modelmywatershed.org/. You'll have to do reinstall the other datasets for the future, but that is a time taking operation and can be deferred till after this review: vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -bc' &&
vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -sS' |
The docker_compose module is no longer supported by Ansible, and switching to the docker_compose_v2 module required updating to a new version of Docker which does not work with the v1 Docker Image format of the RWD image. By using a service with inline docker cli commands, we replicate the settings in the docker_compose block, and also bring it more in line with the Geoprocessing service.
The Azavea role does not work with Ubuntu 22.04. It tries to add an nginx PPA which is not required in Jammy.
This is the recommended way of doing things in Ansible.
Previously the subprocess output would be reported as byte strings, which in Python 3.10 would add a b'' prefix, interfering with the output. By interpreting them as proper UTF-8 strings, we ensure that the original functionality is restored.
Now that the Python 3.10 supporting version of gwlf-e has been published to PyPI, we use that instead of the direct GitHub link.
Similarly, we use the PyPI published version of omgeo. This has no updates since 6.2.1, was necessary as part of the publishing.
syslog has been deprecated in favor of journal, which is used by default. The recommendation is to remove these settings entirely.
This makes explicit the current default behavior which will change in the next version of Celery.
rajadain
force-pushed
the
tt/3629/upgrade-ubuntu-python
branch
from
be5b2c0 to
362c3c2
Compare
|
Thanks for reviewing! Squashed fixups, will merge once green. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Upgrades the development and runtime VMs to Ubuntu 22.04, and the environment to Python 3.10. This is done in preparation for #3628 which needs Python 3.10 for
pystac-client.The big consequences of this change are:
nginxrole is now inlinedmodel-my-watershed.nginxinstead ofazavea.nginxbecause with new Ubuntu we don't need the custom PPA that theazavea.nginxrole was installing, and inlining it here was easier than updating it there.Chromiuminstead ofFirefoxfor JavaScript tests, because installingFirefoxwas difficult, and becauseChromiumis a better representative of our userbasedockeras a system service rather than through the outdateddocker-composePython wrapperdockerto v26 which is not compatible with the old RWD Docker v1 image. This causes RWD to fail. We have AWS 2-4: Publish New Docker Image rapid-watershed-delineation#83 to fix this as a follow up.Closes #3629
Demo
Here's this branch deployed to staging: https://staging.modelmywatershed.org/
Notes
This required the updating of GWLF-E to work with Python 3.10 WikiWatershed/gwlf-e#95 and subsequent release to PyPI https://pypi.org/project/gwlf-e/3.2.0/
This also required significant changes / upgrades to the CivicCI01 Jenkins server. Model My Watershed is the only active project on that server, which made making changes a little safer. All the CI jobs are now passing: http://civicci01.phl.corp.element84.com/view/mmw/
Testing Instructions
Note
Delineating watersheds will not work on staging right now. This is a known issue WikiWatershed/rapid-watershed-delineation#83
developWarning
These instructions will take multiple hours to run and will fully engage your computer. Reviewers are advised to trigger the commands during off hours and check back later to see progress.