Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downstream Tasks Module #275

Merged
merged 138 commits into from
Nov 13, 2023
Merged
Changes from 1 commit
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
8a236f4
Added the `tasks` sub-package
Pale-Blue-Dot-97 Sep 12, 2023
189f52d
NEW: `MinervaTask`
Pale-Blue-Dot-97 Sep 12, 2023
987fe83
Added module for KNN
Pale-Blue-Dot-97 Sep 12, 2023
8059f28
Added more attributes to `MinervaTask`
Pale-Blue-Dot-97 Sep 13, 2023
4cfafdd
Added `__init__`
Pale-Blue-Dot-97 Sep 13, 2023
4ca13b5
`step` now requires `mode` input
Pale-Blue-Dot-97 Sep 15, 2023
56b8796
Feature bank creation --> `generate_feature_bank`
Pale-Blue-Dot-97 Sep 15, 2023
9bfa8cf
singlular `loader` --> `loaders`
Pale-Blue-Dot-97 Sep 15, 2023
92674eb
Make loaders from parameters
Pale-Blue-Dot-97 Sep 15, 2023
8ac940b
Added abstract `adapt_model` method
Pale-Blue-Dot-97 Sep 15, 2023
270fb7d
Updated docstrings
Pale-Blue-Dot-97 Sep 15, 2023
c5a3dbf
Added `gpu` to arguments
Pale-Blue-Dot-97 Sep 18, 2023
e975398
Added `StandardEpoch` task
Pale-Blue-Dot-97 Sep 18, 2023
3f882e8
Added `get_task`
Pale-Blue-Dot-97 Sep 18, 2023
c808f9a
Converting `fit` to `MinervaTask` use
Pale-Blue-Dot-97 Sep 18, 2023
14b5d8e
Converted for `MinervaTask` use
Pale-Blue-Dot-97 Sep 18, 2023
eb41631
Integrated `MinervaTask` into `trainer`
Pale-Blue-Dot-97 Sep 20, 2023
c97d97a
Removed now unused imports
Pale-Blue-Dot-97 Sep 20, 2023
c9cd54f
Added `exp_fn` to args
Pale-Blue-Dot-97 Sep 20, 2023
d7fae47
Removed unused imports
Pale-Blue-Dot-97 Sep 20, 2023
25260da
Added `TSNEVis` task
Pale-Blue-Dot-97 Sep 20, 2023
b95c597
Removed `adapt_model` abstract method
Pale-Blue-Dot-97 Sep 20, 2023
9b85efd
Adapted existing TSNE method for `TSNEVis`
Pale-Blue-Dot-97 Sep 20, 2023
b5c0a8f
Added missing call to `task` in `tsne_cluster`
Pale-Blue-Dot-97 Sep 21, 2023
9676f28
Name simplifications
Pale-Blue-Dot-97 Sep 21, 2023
56498cd
Ensured `SummaryWriter` is defined
Pale-Blue-Dot-97 Sep 21, 2023
b9c9bcb
Added docstring to `TSNEVis`
Pale-Blue-Dot-97 Sep 21, 2023
ce73ffc
Removed methods for getting logger and IO function
Pale-Blue-Dot-97 Sep 21, 2023
e0ab5b6
Decouple `Trainer` from metrics, logging, datasets
Pale-Blue-Dot-97 Sep 22, 2023
b5ae4fa
Moving loggers into metric loggers
Pale-Blue-Dot-97 Sep 22, 2023
4b44546
Integrated a `MinervaLogger` into `MinervaMetrics` classes
Pale-Blue-Dot-97 Sep 24, 2023
b610634
Created `logging` sub-package
Pale-Blue-Dot-97 Sep 25, 2023
de7e677
`metrics` --> `logging.tasklog`
Pale-Blue-Dot-97 Sep 25, 2023
6299fb5
`logger` --> `logging.steplog`
Pale-Blue-Dot-97 Sep 25, 2023
3114bcd
Added docstring to `logging`
Pale-Blue-Dot-97 Sep 25, 2023
cbd6f53
Updated `logging` `__all__`
Pale-Blue-Dot-97 Sep 25, 2023
c2f2633
Updated `steplog` docstring
Pale-Blue-Dot-97 Sep 25, 2023
3ce676c
Updated `tasklog` docstring
Pale-Blue-Dot-97 Sep 25, 2023
56516e1
`mode` --> `train` boolean
Pale-Blue-Dot-97 Sep 26, 2023
0dd171f
`mode` --> `train` bool and added `name` arg
Pale-Blue-Dot-97 Sep 26, 2023
61c6ec7
Streamlined arguments between task and step loggers
Pale-Blue-Dot-97 Sep 26, 2023
d920bfa
Updated tests for `steplog.py`
Pale-Blue-Dot-97 Sep 26, 2023
ee66edf
Made `model_type` an arg
Pale-Blue-Dot-97 Sep 26, 2023
d97bee0
Fixed bugs
Pale-Blue-Dot-97 Sep 26, 2023
ce0757b
Removed disused imports
Pale-Blue-Dot-97 Sep 26, 2023
7806049
Added (blank) test modules for `tasks`
Pale-Blue-Dot-97 Sep 26, 2023
1a0c23a
Merged logging tests into `test_logging`
Pale-Blue-Dot-97 Sep 26, 2023
4d9eaf6
Use `check_substrings_in_string` for `model_type` comparisons
Pale-Blue-Dot-97 Sep 27, 2023
20cbb33
Fixed bugs
Pale-Blue-Dot-97 Sep 27, 2023
6fe0900
Add `test_get_task`
Pale-Blue-Dot-97 Sep 27, 2023
8189891
`MinervaTask.__repr__` now includes the task name
Pale-Blue-Dot-97 Sep 27, 2023
9e46559
Attempting to add `test_standard_epoch`
Pale-Blue-Dot-97 Sep 27, 2023
c29670d
Use `task_name` in `make_loaders`
Pale-Blue-Dot-97 Sep 28, 2023
0cce9b3
`make_loaders` can now use `task_params` and `params`
Pale-Blue-Dot-97 Sep 28, 2023
de936ff
Use `fallback_params` in `factory`
Pale-Blue-Dot-97 Sep 29, 2023
c31f34a
`batch_size` from params
Pale-Blue-Dot-97 Sep 29, 2023
f0369d6
Added default `modelio_cls` to tasks
Pale-Blue-Dot-97 Oct 2, 2023
b8fac29
Ensure `model_params` in `task_params` are updated
Pale-Blue-Dot-97 Oct 2, 2023
b3013d0
Made `modelio` methods static
Pale-Blue-Dot-97 Oct 2, 2023
5fc1f61
Assume single mode if "sampler" in first level of `dataset_params`
Pale-Blue-Dot-97 Oct 2, 2023
eb7c376
Fixed syntax error
Pale-Blue-Dot-97 Oct 2, 2023
913094f
Added `n_classes` as optional arg
Pale-Blue-Dot-97 Oct 2, 2023
08500bb
Ensure `output_shape` is always a `tuple`
Pale-Blue-Dot-97 Oct 2, 2023
ca23945
Setup model properly
Pale-Blue-Dot-97 Oct 2, 2023
23575cc
Fixed `test_depreciated` for tuple model output shape
Pale-Blue-Dot-97 Oct 3, 2023
73d6170
Added missing type hints
Pale-Blue-Dot-97 Oct 3, 2023
1457945
Use `exp_fcn` fixture
Pale-Blue-Dot-97 Oct 3, 2023
bda7902
Use fixtures, fix output shape assertions
Pale-Blue-Dot-97 Oct 3, 2023
a5f2ca2
Fixed `make_manifest` for tasks
Pale-Blue-Dot-97 Oct 3, 2023
544fd08
Fixed manifest making
Pale-Blue-Dot-97 Oct 3, 2023
767f8c4
Added `refresh_step_logger` method
Pale-Blue-Dot-97 Oct 9, 2023
7def636
Added `set_criterion` method
Pale-Blue-Dot-97 Oct 9, 2023
c457cc5
Renamed `mode` --> `task_name`
Pale-Blue-Dot-97 Oct 9, 2023
4860b7d
`Trainer.fit` now works
Pale-Blue-Dot-97 Oct 9, 2023
f4b8d55
Moved plotting to `MinervaTask`
Pale-Blue-Dot-97 Oct 9, 2023
34620d7
`compute_classification_report` --> `MinervaTask`
Pale-Blue-Dot-97 Oct 11, 2023
3a58741
Updated `TSNEVis` to new `MinervaTask` format
Pale-Blue-Dot-97 Oct 11, 2023
4c15321
Updated `WeightedKNN` signature
Pale-Blue-Dot-97 Oct 11, 2023
393125b
Added `save_metrics` to `MinervaTask`
Pale-Blue-Dot-97 Oct 11, 2023
e354bb8
Fixed bugs in `test_visutils`
Pale-Blue-Dot-97 Oct 11, 2023
fe6d6e8
Fixed incorrect task names in `test_factory`
Pale-Blue-Dot-97 Oct 11, 2023
7cd7302
Fixed task name in `test_epoch`
Pale-Blue-Dot-97 Oct 11, 2023
5342101
Updated example configs to new `tasks` structure
Pale-Blue-Dot-97 Oct 11, 2023
87cd7ab
Bump `torchgeo` --> `0.5.0`
Pale-Blue-Dot-97 Oct 12, 2023
8c44347
Renamed `root` --> `paths`
Pale-Blue-Dot-97 Oct 12, 2023
7c1d31a
Ensured kwargs are optional
Pale-Blue-Dot-97 Oct 12, 2023
793c110
Fixed `get_io_func` bug
Pale-Blue-Dot-97 Oct 12, 2023
2afdda8
Fixed bug with `step_logger_params`
Pale-Blue-Dot-97 Oct 12, 2023
4cc0022
Use `fallback_params` for `make_logger`
Pale-Blue-Dot-97 Oct 12, 2023
3f9ef0b
Use `features` and `test` for KNN modes
Pale-Blue-Dot-97 Oct 12, 2023
5ee0f63
Added missing call to `calc_metrics`
Pale-Blue-Dot-97 Oct 12, 2023
48c4385
Fixed `step_logger_params` use
Pale-Blue-Dot-97 Oct 12, 2023
a80a58d
Removed debug print statement
Pale-Blue-Dot-97 Oct 12, 2023
6ee2999
Fixed path issues
Pale-Blue-Dot-97 Oct 13, 2023
1f199e5
Fixed the length size of samplers
Pale-Blue-Dot-97 Oct 13, 2023
e11d80a
Fixed tests for TSNE Visualisation
Pale-Blue-Dot-97 Oct 13, 2023
fca87fd
Bypass class balancing and elimination
Pale-Blue-Dot-97 Oct 14, 2023
c957fb6
Merge branch 'beta' into downstream-tasks-module
Pale-Blue-Dot-97 Oct 14, 2023
8b07c00
Renamed `logging` --> `logger` to avoid inbuilt conflict
Pale-Blue-Dot-97 Oct 15, 2023
de6dfd4
mypy fixes
Pale-Blue-Dot-97 Oct 15, 2023
34ac069
Merge branch 'downstream-tasks-module' of github.com:Pale-Blue-Dot-97…
Pale-Blue-Dot-97 Oct 15, 2023
aefc969
Use `staticmethod` wrapper rather than decorator
Pale-Blue-Dot-97 Oct 15, 2023
6b74f2d
`mode` --> `train`
Pale-Blue-Dot-97 Oct 15, 2023
a7d724f
Use `np.prod` to get product of `self.output_size`
Pale-Blue-Dot-97 Oct 15, 2023
e668b25
Ensure logged logged float values are floats
Pale-Blue-Dot-97 Oct 15, 2023
f1ee454
Fixed/ ignored `mypy` issues
Pale-Blue-Dot-97 Oct 15, 2023
f3a9796
Fixed `staticmethod not callable` error
Pale-Blue-Dot-97 Oct 15, 2023
69cb571
Added `check_substrings_in_string` and `fallback_params` to `__all__`
Pale-Blue-Dot-97 Oct 15, 2023
085f3a7
Reincorporated `save` and `show` arg use
Pale-Blue-Dot-97 Oct 15, 2023
2abc10b
Added `get_logger` to `__all__`
Pale-Blue-Dot-97 Oct 15, 2023
3db7895
`qodana` fixes
Pale-Blue-Dot-97 Oct 15, 2023
07fa25b
Use `step` not `__call__`
Pale-Blue-Dot-97 Oct 15, 2023
7a23092
Added missing `task_name` param to docstrings
Pale-Blue-Dot-97 Oct 15, 2023
e9b732d
Updated `Qodana` baseline
Pale-Blue-Dot-97 Oct 15, 2023
e23f10c
Move backbone extraction to `TSNEVis`
Pale-Blue-Dot-97 Oct 16, 2023
09f43f1
Correct syntax error
Pale-Blue-Dot-97 Oct 16, 2023
f42ccd3
Use `forward` rather than `step`
Pale-Blue-Dot-97 Oct 16, 2023
9d5f65b
Remove duplicate calls to `calc_metrics`
Pale-Blue-Dot-97 Oct 23, 2023
69c91c2
Added `log_null` so that val logs same length as train
Pale-Blue-Dot-97 Oct 23, 2023
92b2b68
`log_epoch_number` no longer abstract
Pale-Blue-Dot-97 Oct 23, 2023
591997b
Move logging epoch number into `calc_metrics`
Pale-Blue-Dot-97 Oct 23, 2023
0e2b03b
Update requirements
Pale-Blue-Dot-97 Oct 24, 2023
051fee4
Updated documentation
Pale-Blue-Dot-97 Oct 24, 2023
8936ce0
Remove and replace broken fudge
Pale-Blue-Dot-97 Oct 27, 2023
2512a41
Use `SSL4E0-S12` data for training in example
Pale-Blue-Dot-97 Oct 27, 2023
23c345b
New test for the `SSL4E0S12Sentinel2` dataset class
Pale-Blue-Dot-97 Oct 27, 2023
36c0831
Fixced test with `crs` and `bands` args
Pale-Blue-Dot-97 Oct 27, 2023
2d57793
`flake8` fixes
Pale-Blue-Dot-97 Oct 27, 2023
ebb50ba
Added `compile_dataset_paths`
Pale-Blue-Dot-97 Oct 30, 2023
f966cda
Added glob compilation
Pale-Blue-Dot-97 Oct 30, 2023
cb74421
Fixed `TypeError`
Pale-Blue-Dot-97 Oct 30, 2023
27b7f78
Debug
Pale-Blue-Dot-97 Oct 30, 2023
54ada83
Fixed bug
Pale-Blue-Dot-97 Oct 30, 2023
f3b638b
`mypy` and `flake8` fixes
Pale-Blue-Dot-97 Nov 1, 2023
6ae8a71
Update requirements
Pale-Blue-Dot-97 Nov 2, 2023
9c986e4
Use `torchmetrics` for cosine similarity
Pale-Blue-Dot-97 Nov 10, 2023
0ca1788
`torchgeo` -->`0.5.1`
Pale-Blue-Dot-97 Nov 10, 2023
4ae4398
Remove Qodana
Pale-Blue-Dot-97 Nov 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added docstring to TSNEVis
Pale-Blue-Dot-97 committed Sep 21, 2023
commit b9c9bcb7c0d1834a42eda3b0e03da9da5045d501
6 changes: 6 additions & 0 deletions minerva/tasks/tsne.py
Original file line number Diff line number Diff line change
@@ -55,6 +55,12 @@
# CLASSES
# =====================================================================================================================
class TSNEVis(MinervaTask):
"""TSNE clustering task.

Passes a batch of data through the model in eval mode to get the embeddings.
Passes these embeddings to :mod:`visutils` to train a TSNE algorithm and then visual the cluster.
"""

def __init__(
self,
model: Union[MinervaModel, MinervaDataParallel],