Merge pull request #172 from Pale-Blue-Dot-97/minerva-v0.24rc

`minerva v0.24`

README.md

@@ -1,6 +1,6 @@
 # Minerva
 
-![GitHub release (latest by date)](https://img.shields.io/github/v/release/Pale-Blue-Dot-97/Minerva?) ![GitHub](https://img.shields.io/github/license/Pale-Blue-Dot-97/Minerva?) ![GitHub Pipenv locked Python version](https://img.shields.io/github/pipenv/locked/python-version/Pale-Blue-Dot-97/Minerva?)  ![GitHub contributors](https://img.shields.io/github/contributors/Pale-Blue-Dot-97/Minerva?) ![Snyk Vulnerabilities for GitHub Repo](https://img.shields.io/snyk/vulnerabilities/github/Pale-Blue-Dot-97/Minerva?) ![tests](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/tests.yml/badge.svg) [![Read the Docs](https://img.shields.io/readthedocs/smp?)](https://pale-blue-dot-97.github.io/Minerva/) [![Qodana](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml/badge.svg)](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml) [![CircleCI](https://dl.circleci.com/status-badge/img/gh/Pale-Blue-Dot-97/Minerva/tree/main.svg?style=svg&circle-token=7c738d256a0d8df674b2682daeb2f4b52381ced4)](https://dl.circleci.com/status-badge/redirect/gh/Pale-Blue-Dot-97/Minerva/tree/main) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Coverage Status](https://coveralls.io/repos/github/Pale-Blue-Dot-97/Minerva/badge.svg?t=ZycdOW)](https://coveralls.io/github/Pale-Blue-Dot-97/Minerva) [![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit)[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Pale-Blue-Dot-97/Minerva/main.svg?badge_token=3TtyNVaESs6kij02bGEejA)](https://results.pre-commit.ci/latest/github/Pale-Blue-Dot-97/Minerva/main?badge_token=3TtyNVaESs6kij02bGEejA)
+![GitHub release (latest by date)](https://img.shields.io/github/v/release/Pale-Blue-Dot-97/Minerva?) ![GitHub](https://img.shields.io/github/license/Pale-Blue-Dot-97/Minerva?) ![GitHub Pipenv locked Python version](https://img.shields.io/github/pipenv/locked/python-version/Pale-Blue-Dot-97/Minerva?)  ![GitHub contributors](https://img.shields.io/github/contributors/Pale-Blue-Dot-97/Minerva?) [![CodeFactor](https://www.codefactor.io/repository/github/pale-blue-dot-97/minerva/badge)](https://www.codefactor.io/repository/github/pale-blue-dot-97/minerva) ![Snyk Vulnerabilities for GitHub Repo](https://img.shields.io/snyk/vulnerabilities/github/Pale-Blue-Dot-97/Minerva?) ![tests](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/tests.yml/badge.svg) [![Read the Docs](https://img.shields.io/readthedocs/smp?)](https://pale-blue-dot-97.github.io/Minerva/) [![Qodana](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml/badge.svg)](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml) [![CircleCI](https://dl.circleci.com/status-badge/img/gh/Pale-Blue-Dot-97/Minerva/tree/main.svg?style=svg&circle-token=7c738d256a0d8df674b2682daeb2f4b52381ced4)](https://dl.circleci.com/status-badge/redirect/gh/Pale-Blue-Dot-97/Minerva/tree/main) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Coverage Status](https://coveralls.io/repos/github/Pale-Blue-Dot-97/Minerva/badge.svg?t=ZycdOW)](https://coveralls.io/github/Pale-Blue-Dot-97/Minerva) [![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit) [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Pale-Blue-Dot-97/Minerva/main.svg)](https://results.pre-commit.ci/latest/github/Pale-Blue-Dot-97/Minerva/main)
 
 <!-- PROJECT LOGO -->
 <br />
@@ -9,7 +9,7 @@
     <img src="docs/images/Minerva_logo.png" alt="Logo" width="" height="400">
   </a>
   <p align="center">
-    <b style="font-size:26px;"> v0.23-beta</b>
+    <b style="font-size:26px;"> v0.24</b>
     <br />
     <a href="https://pale-blue-dot-97.github.io/Minerva/"><strong>Explore the docs »</strong></a>
     <br />
@@ -145,6 +145,6 @@ Proceedings of the 2019 Conference on Computer Vision and Pattern Recognition (C
 
 ## Project Status 🔴🟡🟢
 
-This project is in a *beta* state. Expect bugs and breaking changes in future versions.
+This project is now in release *beta* state. Still expect some bugs and there may be breaking changes in future versions.
 
 <p align="right">(<a href="#top">back to top</a>)</p>

banner.txt

@@ -4,4 +4,4 @@
    *       *     / /  / // // /|  / /___/ _, _/| |/ / ___ |               *
     *           /_/  /_/___/_/ |_/_____/_/ |_| |___/_/  |_|     *   *
                THE REMOTE SENSING MACHINE LEARNING FRAMEWORK
-               ==============> v0.23.4-beta <===============
+               =================> v0.24.0 <=================

docs/docs_readme.md

@@ -1,6 +1,6 @@
 # Minerva
 
-![GitHub release (latest by date)](https://img.shields.io/github/v/release/Pale-Blue-Dot-97/Minerva?) ![GitHub](https://img.shields.io/github/license/Pale-Blue-Dot-97/Minerva?) ![GitHub Pipenv locked Python version](https://img.shields.io/github/pipenv/locked/python-version/Pale-Blue-Dot-97/Minerva?)  ![GitHub contributors](https://img.shields.io/github/contributors/Pale-Blue-Dot-97/Minerva?) ![Snyk Vulnerabilities for GitHub Repo](https://img.shields.io/snyk/vulnerabilities/github/Pale-Blue-Dot-97/Minerva?) ![tests](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/tests.yml/badge.svg) [![Read the Docs](https://img.shields.io/readthedocs/smp?)](https://pale-blue-dot-97.github.io/Minerva/) [![Qodana](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml/badge.svg)](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml) [![CircleCI](https://dl.circleci.com/status-badge/img/gh/Pale-Blue-Dot-97/Minerva/tree/main.svg?style=svg&circle-token=7c738d256a0d8df674b2682daeb2f4b52381ced4)](https://dl.circleci.com/status-badge/redirect/gh/Pale-Blue-Dot-97/Minerva/tree/main) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Coverage Status](https://coveralls.io/repos/github/Pale-Blue-Dot-97/Minerva/badge.svg?t=ZycdOW)](https://coveralls.io/github/Pale-Blue-Dot-97/Minerva) [![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit)[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Pale-Blue-Dot-97/Minerva/main.svg?badge_token=3TtyNVaESs6kij02bGEejA)](https://results.pre-commit.ci/latest/github/Pale-Blue-Dot-97/Minerva/main?badge_token=3TtyNVaESs6kij02bGEejA)
+![GitHub release (latest by date)](https://img.shields.io/github/v/release/Pale-Blue-Dot-97/Minerva?) ![GitHub](https://img.shields.io/github/license/Pale-Blue-Dot-97/Minerva?) ![GitHub Pipenv locked Python version](https://img.shields.io/github/pipenv/locked/python-version/Pale-Blue-Dot-97/Minerva?)  ![GitHub contributors](https://img.shields.io/github/contributors/Pale-Blue-Dot-97/Minerva?) [![CodeFactor](https://www.codefactor.io/repository/github/pale-blue-dot-97/minerva/badge)](https://www.codefactor.io/repository/github/pale-blue-dot-97/minerva) ![Snyk Vulnerabilities for GitHub Repo](https://img.shields.io/snyk/vulnerabilities/github/Pale-Blue-Dot-97/Minerva?) ![tests](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/tests.yml/badge.svg) [![Read the Docs](https://img.shields.io/readthedocs/smp?)](https://pale-blue-dot-97.github.io/Minerva/) [![Qodana](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml/badge.svg)](https://github.com/Pale-Blue-Dot-97/Minerva/actions/workflows/code_quality.yml) [![CircleCI](https://dl.circleci.com/status-badge/img/gh/Pale-Blue-Dot-97/Minerva/tree/main.svg?style=svg&circle-token=7c738d256a0d8df674b2682daeb2f4b52381ced4)](https://dl.circleci.com/status-badge/redirect/gh/Pale-Blue-Dot-97/Minerva/tree/main) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Coverage Status](https://coveralls.io/repos/github/Pale-Blue-Dot-97/Minerva/badge.svg?t=ZycdOW)](https://coveralls.io/github/Pale-Blue-Dot-97/Minerva) [![security: bandit](https://img.shields.io/badge/security-bandit-yellow.svg)](https://github.com/PyCQA/bandit) [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/Pale-Blue-Dot-97/Minerva/main.svg)](https://results.pre-commit.ci/latest/github/Pale-Blue-Dot-97/Minerva/main)
 
 <!-- PROJECT LOGO -->
 <br />
@@ -9,7 +9,7 @@
     <img src="docs/images/Minerva_logo.png" alt="Logo" width="" height="400">
   </a>
   <p align="center">
-    <b style="font-size:26px;"> v0.23-beta</b>
+    <b style="font-size:26px;"> v0.24</b>
     <br />
     <a href="https://pale-blue-dot-97.github.io/Minerva/"><strong>Explore the docs »</strong></a>
     <br />
@@ -22,8 +22,7 @@
 
 ## About 🔎
 
-Minerva is a package to aid in the building, fitting and testing of neural network models on geo-spatial
-rasterised land cover data.
+Minerva is a package to aid in the building, fitting and testing of neural network models on multi-spectral geo-spatial data.
 
 ## Getting Started ▶
 
@@ -106,7 +105,7 @@ Don't forget to give the project a star! Thanks again!
 
 ## License 🔏
 
-Minerva is distributed under a [GNU LGPLv3 License](https://choosealicense.com/licenses/lgpl-3.0/).
+Minerva is distributed under a [MIT License](https://choosealicense.com/licenses/mit/).
 
 <p align="right">(<a href="#top">back to top</a>)</p>
 
@@ -122,11 +121,19 @@ Contributions also provided by:
 - [Isabel Sargent](https://github.com/PenguinJunk)
 - [Steve Coupland](https://github.com/scoupland-os)
 - [Joe Guyatt](https://github.com/joeguyatt97)
+- [Ben Dickens](https://github.com/BenDickens)
+- [Kitty Varghese](https://github.com/kittyvarghese)
 
 ## Acknowledgments 📢
 
-I'd like to acknowledge the invaluable supervision and contributions of Prof Jonathon Hare and
-Dr Isabel Sargent towards this work.
+I'd like to acknowledge the invaluable supervision and contributions of [Prof Jonathon Hare](https://github.com/jonhare) and
+[Dr Isabel Sargent](https://github.com/PenguinJunk) towards this work.
+
+The following modules are adapted from open source third-parites:
+| Module | Original Author | License | Link |
+|:-------|:----------------|:--------|:-----|
+| `pytorchtools` | [Noah Golmant](https://github.com/noahgolmant) | MIT | https://github.com/noahgolmant/pytorch-lars/blob/master/lars.py |
+| `optimisers` | [Bjarte Mehus Sunde](https://github.com/Bjarten) | MIT | https://github.com/Bjarten/early-stopping-pytorch |
 
 This repositry also contains two small ``.tiff`` exercpts from the [ChesapeakeCVPR](https://lila.science/datasets/chesapeakelandcover) dataset used for unit testing purposes. Credit for this data goes to:
 
@@ -138,6 +145,6 @@ Proceedings of the 2019 Conference on Computer Vision and Pattern Recognition (C
 
 ## Project Status 🔴🟡🟢
 
-This project is in a *beta* state. Expect bugs and breaking changes in future versions.
+This project is now in release *beta* state. Still expect some bugs and there may be breaking changes in future versions.
 
 <p align="right">(<a href="#top">back to top</a>)</p>

minerva/__init__.py

@@ -42,7 +42,7 @@
     Proceedings of the 2019 Conference on Computer Vision and Pattern Recognition (CVPR 2019)
 """
 
-__version__ = "0.23.4"
+__version__ = "0.24.0"
 __author__ = "Harry Baker"
 __contact__ = "[email protected]"
 __license__ = "MIT License"

minerva/datasets.py

@@ -167,7 +167,7 @@ def __getitem__(  # type: ignore[override]
             queries[1]
         )
 
-    def __and__(self, other: "PairedDataset") -> IntersectionDataset:
+    def __and__(self, other: "PairedDataset") -> IntersectionDataset:  # type: ignore[override]
         """Take the intersection of two :class:`PairedDataset`.
 
         Args:
@@ -191,7 +191,7 @@ def __and__(self, other: "PairedDataset") -> IntersectionDataset:
             self, other, collate_fn=utils.pair_collate(concat_samples)
         )
 
-    def __or__(self, other: "PairedDataset") -> "PairedUnionDataset":
+    def __or__(self, other: "PairedDataset") -> "PairedUnionDataset":  # type: ignore[override]
         """Take the union of two :class:`PairedDataset`.
 
         Args:
@@ -307,7 +307,7 @@ def __init__(self, *args, **kwargs) -> None:
 
         self.datasets = new_datasets
 
-    def __getitem__(
+    def __getitem__(  # type: ignore[override]
         self, query: Tuple[BoundingBox, BoundingBox]
     ) -> Tuple[Dict[str, Any], Dict[str, Any]]:
         """Retrieve image and metadata indexed by query.

minerva/trainer.py

@@ -39,7 +39,6 @@
 #                                                     IMPORTS
 # =====================================================================================================================
 import os
-import warnings
 from contextlib import nullcontext
 from pathlib import Path
 from typing import (
@@ -62,7 +61,6 @@
 from alive_progress import alive_bar, alive_it
 from inputimeout import TimeoutOccurred, inputimeout
 from nptyping import Int, NDArray
-from packaging.version import Version
 from torch import Tensor
 from torch.nn.modules import Module
 from torch.nn.parallel import DistributedDataParallel as DDP
@@ -416,16 +414,6 @@ def __init__(
             )
             self.model = MinervaDataParallel(self.model, DDP, device_ids=[gpu])
 
-        # Wraps the model in `torch.compile` to speed up computation time.
-        # Python 3.11+ is not yet supported though, hence the exception clause.
-        if Version(torch.__version__) > Version("2.0.0"):  # pragma: no cover
-            try:
-                _compiled_model = torch.compile(self.model)
-                assert isinstance(_compiled_model, (MinervaModel, MinervaDataParallel))
-                self.model = _compiled_model
-            except RuntimeError as err:
-                warnings.warn(str(err))
-
     def init_wandb_metrics(self) -> None:
         """Setups up separate step counters for :mod:`wandb` logging of train, val, etc."""
         if isinstance(self.writer, Run):

minerva/utils/runner.py

@@ -61,10 +61,10 @@
 import torch
 import torch.distributed as dist
 import torch.multiprocessing as mp
+import wandb
 from wandb.sdk.lib import RunDisabled
 from wandb.sdk.wandb_run import Run
 
-import wandb
 from minerva.utils import CONFIG, MASTER_PARSER, utils
 
 # =====================================================================================================================

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "minerva"
-version = "0.23.4"
+version = "0.24.0"
 description = "Package to build, train and test neural network models on geospatial data"
 readme = "README.md"
 authors = [

tests/__init__.py

@@ -1,19 +1,25 @@
 # -*- coding: utf-8 -*-
-# Copyright (C) 2023 Harry Baker
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with this program in LICENSE.txt. If not,
-# see <https://www.gnu.org/licenses/>.
+# MIT License
+
+# Copyright (c) 2023 Harry Baker
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
 #
 # @org: University of Southampton
 # Created under a project funded by the Ordnance Survey Ltd.

tests/conftest.py

@@ -1,19 +1,25 @@
 # -*- coding: utf-8 -*-
-# Copyright (C) 2023 Harry Baker
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Lesser General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public License
-# along with this program in LICENSE.txt. If not,
-# see <https://www.gnu.org/licenses/>.
+# MIT License
+
+# Copyright (c) 2023 Harry Baker
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
 #
 # @org: University of Southampton
 # Created under a project funded by the Ordnance Survey Ltd.
@@ -158,7 +164,9 @@ def exp_cnn(x_entropy_loss, rgbi_input_size) -> MinervaModel:
 
 @pytest.fixture
 def random_mask(small_patch_size, std_n_classes) -> NDArray[Shape["32, 32"], Int]:
-    return np.random.randint(0, std_n_classes - 1, size=small_patch_size)
+    mask = np.random.randint(0, std_n_classes - 1, size=small_patch_size)
+    assert isinstance(mask, np.ndarray)
+    return mask
 
 
 @pytest.fixture
@@ -206,7 +214,9 @@ def random_mask_batch(
 
 @pytest.fixture
 def random_scene_classification_batch(std_batch_size, std_n_classes) -> LongTensor:
-    return torch.randint(0, std_n_classes - 1, size=(std_batch_size,))
+    batch = torch.randint(0, std_n_classes - 1, size=(std_batch_size,))
+    assert isinstance(batch, LongTensor)
+    return batch
 
 
 @pytest.fixture