Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

print to logger #448

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
dd2414b
Adding in `logger` commands instead of `print`
kallewesterling Jun 26, 2024
31eb638
Adding in clarifying comments
kallewesterling Jun 26, 2024
042c969
Adding in more clarifying comments
kallewesterling Jun 26, 2024
cc5a1ca
One last comment
kallewesterling Jun 26, 2024
75b810c
Dropping `cprint`
kallewesterling Jul 25, 2024
acc1a49
Replacing remaining relevant print statements with logger.info/logger…
kallewesterling Jul 25, 2024
8b4bfca
Fixing more print statements and testing side of things
kallewesterling Jul 26, 2024
2fbe079
merge rw changes
rwood-97 Jul 26, 2024
9642558
one extra fix
rwood-97 Jul 26, 2024
6a938cb
Merge branch 'main' into kallewesterling/issue27
kallewesterling Jul 26, 2024
4409edb
Adding a logger to base class Runner
kallewesterling Jul 26, 2024
638892a
Adding the correct URLs (see also #465)
kallewesterling Jul 26, 2024
a557f3f
Merge branch 'kallewesterling/issue27' of https://github.com/Living-w…
rwood-97 Aug 2, 2024
6722623
Merge branch 'main' into kallewesterling/issue27
rwood-97 Aug 2, 2024
181eebb
Merge branch 'main' into kallewesterling/issue27
rwood-97 Aug 2, 2024
bf79cff
delete test files
rwood-97 Aug 2, 2024
3a7d959
Resolving conflicts and fixing test errors
kallewesterling Aug 2, 2024
9a7e36a
adding temp checkpoints for tests to gitignore
kallewesterling Aug 2, 2024
7c6d1e6
Merge branch 'kallewesterling/issue27' into rw/issue27
kallewesterling Aug 2, 2024
b392189
fix missing "type" arguments
rwood-97 Aug 5, 2024
64c59ef
Update test_sheet_downloader.py
rwood-97 Aug 5, 2024
4f01220
Update test_sheet_downloader.py
rwood-97 Aug 8, 2024
bdb55d5
Update test_sheet_downloader.py
rwood-97 Aug 8, 2024
15e664c
Merge pull request #473 from Living-with-machines/rw/issue27
rwood-97 Aug 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion mapreader/annotate/annotator.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import functools
import hashlib
import json
import logging
import os
import random
import string
Expand All @@ -20,12 +21,15 @@

from ..load.loader import load_patches

# Ignore warnings
warnings.filterwarnings("ignore", category=UserWarning)

_CENTER_LAYOUT = widgets.Layout(
display="flex", flex_flow="column", align_items="center"
)

# Set up logging
logger = logging.getLogger(__name__)

class Annotator:
"""
Expand Down Expand Up @@ -218,7 +222,7 @@

# Test for existing patch annotation file
if os.path.exists(annotations_file):
print("[INFO] Loading existing patch annotations.")
logger.info("Loading existing patch annotations.")

Check warning on line 225 in mapreader/annotate/annotator.py

View check run for this annotation

Codecov / codecov/patch

mapreader/annotate/annotator.py#L225

Added line #L225 was not covered by tests
patch_df = self._load_annotations(
patch_df=patch_df,
annotations_file=annotations_file,
Expand Down
9 changes: 7 additions & 2 deletions mapreader/annotate/utils.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python
from __future__ import annotations

import logging

Check warning on line 4 in mapreader/annotate/utils.py

View check run for this annotation

Codecov / codecov/patch

mapreader/annotate/utils.py#L4

Added line #L4 was not covered by tests
import os
import random
import sys
Expand All @@ -25,10 +26,14 @@

from mapreader import load_patches, loader

# Ignore warnings
warnings.filterwarnings("ignore")
# warnings.filterwarnings(
# (
# "ignore", message="Pandas doesn't allow columns to be created via a new attribute name")

# Set up logging
logger = logging.getLogger(__name__)

Check warning on line 35 in mapreader/annotate/utils.py

View check run for this annotation

Codecov / codecov/patch

mapreader/annotate/utils.py#L35

Added line #L35 was not covered by tests


def prepare_data(
df: pd.DataFrame,
Expand Down Expand Up @@ -708,4 +713,4 @@
print(f"[INFO] {new_labels} labels were not already stored")
print(f"[INFO] Total number of saved annotations: {len(image_df)}")
else:
print("[INFO] No annotations to save!")
logger.info("No annotations to save!")

Check warning on line 716 in mapreader/annotate/utils.py

View check run for this annotation

Codecov / codecov/patch

mapreader/annotate/utils.py#L716

Added line #L716 was not covered by tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. here, should we replace all prints?

14 changes: 9 additions & 5 deletions mapreader/classify/classifier.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from __future__ import annotations

import copy
import logging
import os
import random
import socket
Expand Down Expand Up @@ -29,6 +30,9 @@
# from tqdm.autonotebook import tqdm
# from torch.nn.modules.module import _addindent

# Set up logging
logger = logging.getLogger(__name__)


class ClassifierContainer:
def __init__(
Expand Down Expand Up @@ -147,7 +151,7 @@
self.labels_map = labels_map

# set up model and move to device
print("[INFO] Initializing model.")
logger.info("Initializing model.")
if isinstance(model, nn.Module):
self.model = model.to(self.device)
self.input_size = input_size
Expand Down Expand Up @@ -733,12 +737,12 @@
print_info_batch_freq=print_info_batch_freq,
)
except KeyboardInterrupt:
print("[INFO] Exiting...")
logger.info("Exiting...")

Check warning on line 740 in mapreader/classify/classifier.py

View check run for this annotation

Codecov / codecov/patch

mapreader/classify/classifier.py#L740

Added line #L740 was not covered by tests
if os.path.isfile(self.tmp_save_filename):
print(f'[INFO] Loading "{self.tmp_save_filename}" as model.')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. here

self.load(self.tmp_save_filename, remove_after_load=remove_after_load)
else:
print("[INFO] No checkpoint file found - model has not been updated.")
logger.info("No checkpoint file found - model has not been updated.")

Check warning on line 745 in mapreader/classify/classifier.py

View check run for this annotation

Codecov / codecov/patch

mapreader/classify/classifier.py#L745

Added line #L745 was not covered by tests

def train_core(
self,
Expand Down Expand Up @@ -834,7 +838,7 @@
print(
"[WARNING] Could not import ``SummaryWriter`` from torch.utils.tensorboard" # noqa
)
print("[WARNING] Continuing without tensorboard.")
logger.warning("Continuing without tensorboard.")

Check warning on line 841 in mapreader/classify/classifier.py

View check run for this annotation

Codecov / codecov/patch

mapreader/classify/classifier.py#L841

Added line #L841 was not covered by tests
tensorboard_path = None

start_epoch = self.last_epoch + 1
Expand Down Expand Up @@ -1812,7 +1816,7 @@
The number of worker threads to use for loading data, by default 0.
"""
if sampler and shuffle:
print("[INFO] ``sampler`` is defined so train dataset will be unshuffled.")
logger.info("``sampler`` is defined so train dataset will be unshuffled.")

Check warning on line 1819 in mapreader/classify/classifier.py

View check run for this annotation

Codecov / codecov/patch

mapreader/classify/classifier.py#L1819

Added line #L1819 was not covered by tests

dataloader = DataLoader(
dataset,
Expand Down
12 changes: 8 additions & 4 deletions mapreader/classify/load_annotations.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env python
from __future__ import annotations

import logging
import os
from decimal import Decimal
from typing import Callable
Expand All @@ -16,6 +17,9 @@

from .datasets import PatchContextDataset, PatchDataset

# Set up logging
logger = logging.getLogger(__name__)


class AnnotationsLoader:
def __init__(self):
Expand Down Expand Up @@ -452,7 +456,7 @@
if user_input_ids.lower() in ["exit", "end", "stop"]:
break

print("[INFO] Exited.")
logger.info("Exited.")

Check warning on line 459 in mapreader/classify/load_annotations.py

View check run for this annotation

Codecov / codecov/patch

mapreader/classify/load_annotations.py#L459

Added line #L459 was not covered by tests

def show_sample(self, label_to_show: str, num_samples: int | None = 9) -> None:
"""Show a random sample of images with the specified label (tar_label).
Expand Down Expand Up @@ -629,7 +633,7 @@
self.datasets = datasets
self.dataset_sizes = dataset_sizes

print("[INFO] Number of annotations in each set:")
logger.info("Number of annotations in each set:")
for set_name in datasets.keys():
print(f" - {set_name}: {dataset_sizes[set_name]}")

Expand Down Expand Up @@ -761,15 +765,15 @@

if isinstance(sampler, str):
if sampler == "default":
print("[INFO] Using default sampler.")
logger.info("Using default sampler.")
sampler = self._define_sampler()
else:
raise ValueError(
'[ERROR] ``sampler`` can only be a PyTorch sampler, ``"default"`` or ``None``.'
)

if sampler and shuffle:
print("[INFO] ``sampler`` is defined so train dataset will be un-shuffled.")
logger.info("``sampler`` is defined so train dataset will be un-shuffled.")

Check warning on line 776 in mapreader/classify/load_annotations.py

View check run for this annotation

Codecov / codecov/patch

mapreader/classify/load_annotations.py#L776

Added line #L776 was not covered by tests

dataloaders = {
set_name: DataLoader(
Expand Down
6 changes: 5 additions & 1 deletion mapreader/download/sheet_downloader.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from __future__ import annotations

import json
import logging
import os
import re
import shutil
Expand All @@ -21,6 +22,9 @@
from .tile_loading import DEFAULT_TEMP_FOLDER, TileDownloader
from .tile_merging import TileMerger

# Set up logging
logger = logging.getLogger(__name__)


class SheetDownloader:
"""
Expand Down Expand Up @@ -504,7 +508,7 @@
self.get_polygons()

if len(self.found_queries) == 0:
print("[INFO] No query results found/saved.")
logger.info("No query results found/saved.")

Check warning on line 511 in mapreader/download/sheet_downloader.py

View check run for this annotation

Codecov / codecov/patch

mapreader/download/sheet_downloader.py#L511

Added line #L511 was not covered by tests
else:
divider = 14 * "="
print(f"{divider}\nQuery results:\n{divider}")
Expand Down
2 changes: 2 additions & 0 deletions mapreader/download/tile_loading.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@

from .data_structures import GridBoundingBox, GridIndex

# Set up logging
logger = logging.getLogger(__name__)

# Default values (constants)
DEFAULT_TEMP_FOLDER = "_tile_cache/" # must end with a "/"
DEFAULT_IMG_DOWNLOAD_FORMAT = "png"

Expand Down
2 changes: 2 additions & 0 deletions mapreader/download/tile_merging.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@
from .data_structures import GridBoundingBox, GridIndex
from .tile_loading import DEFAULT_IMG_DOWNLOAD_FORMAT, DEFAULT_TEMP_FOLDER

# Set up logging
logger = logging.getLogger(__name__)

# Default values (constants)
DEFAULT_OUT_FOLDER = "./"
DEFAULT_IMG_STORE_FORMAT = ("png", "PNG")

Expand Down
9 changes: 7 additions & 2 deletions mapreader/load/geo_utils.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
#!/usr/bin/env python
from __future__ import annotations

import logging

import numpy as np
import rasterio
from geopy.distance import geodesic, great_circle
from pyproj import Transformer

# Set up logging
logger = logging.getLogger(__name__)


def extractGeoInfo(image_path):
"""Extract geographic information (shape, CRS and coordinates) from GeoTiff files
Expand Down Expand Up @@ -34,7 +39,7 @@ def extractGeoInfo(image_path):
tiff_coord = tuple(tiff_src.bounds)

print(f"[INFO] Shape: {tiff_shape}. \n[INFO] CRS: {tiff_proj}.")
print("[INFO] Coordinates: {:.4f} {:.4f} {:.4f} {:.4f}".format(*tiff_coord))
logger.info("Coordinates: {:.4f} {:.4f} {:.4f} {:.4f}".format(*tiff_coord))

return tiff_shape, tiff_proj, tiff_coord

Expand Down Expand Up @@ -62,7 +67,7 @@ def reproject_geo_info(image_path, target_crs="EPSG:4326", calc_size_in_m=False)
transformer = Transformer.from_crs(tiff_proj, target_crs, always_xy=True)
coord = transformer.transform_bounds(*tiff_coord)
print(f"[INFO] New CRS: {target_crs}")
print("[INFO] Reprojected coordinates: {:.4f} {:.4f} {:.4f} {:.4f}".format(*coord))
logger.info("Reprojected coordinates: {:.4f} {:.4f} {:.4f} {:.4f}".format(*coord))

height, width, _ = tiff_shape

Expand Down
8 changes: 6 additions & 2 deletions mapreader/load/images.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
except ImportError:
pass

import logging
import os
import random
import re
Expand Down Expand Up @@ -37,6 +38,9 @@
# Ignore warnings
warnings.filterwarnings("ignore")

# Set up logging
logger = logging.getLogger(__name__)


class MapImages:
"""
Expand Down Expand Up @@ -518,7 +522,7 @@ def add_shape(self, tree_level: str | None = "parent") -> None:
self._add_shape_id(image_id=image_id)

def add_coords_from_grid_bb(self, verbose: bool = False) -> None:
print("[INFO] Adding coordinates, tree level: parent")
logger.info("Adding coordinates, tree level: parent")

parent_list = self.list_parents()

Expand Down Expand Up @@ -552,7 +556,7 @@ def add_coord_increments(self, verbose: bool | None = False) -> None:
pixel-wise delta longitude (``dlon``) and delta latitude (``dlat``)
for the image and adds the data to it.
"""
print("[INFO] Add coord-increments, tree level: parent")
logger.info("Add coord-increments, tree level: parent")

parent_list = self.list_parents()

Expand Down
6 changes: 5 additions & 1 deletion mapreader/spot_text/deepsolo_runner.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these methods be deleted or is this a merge thing?

Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from __future__ import annotations

import logging
import os
import pathlib
import pickle
Expand Down Expand Up @@ -32,6 +33,9 @@
"Please install DeepSolo from the following link: https://github.com/rwood-97/DeepSolo"
)

# Set up logging
logger = logging.getLogger(__name__)

Check warning on line 37 in mapreader/spot_text/deepsolo_runner.py

View check run for this annotation

Codecov / codecov/patch

mapreader/spot_text/deepsolo_runner.py#L37

Added line #L37 was not covered by tests


class DeepSoloRunner:
def __init__(
Expand Down Expand Up @@ -471,7 +475,7 @@
raise ValueError("[ERROR] Please provide a `parent_df`")

if self.parent_predictions == {}:
print("[INFO] Converting patch pixel bounds to parent pixel bounds.")
logger.info("Converting patch pixel bounds to parent pixel bounds.")

Check warning on line 478 in mapreader/spot_text/deepsolo_runner.py

View check run for this annotation

Codecov / codecov/patch

mapreader/spot_text/deepsolo_runner.py#L478

Added line #L478 was not covered by tests
_ = self.convert_to_parent_pixel_bounds()

for parent_id, prediction in self.parent_predictions.items():
Expand Down
6 changes: 5 additions & 1 deletion mapreader/spot_text/dptext_detr_runner.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from __future__ import annotations

import logging
import os
import pathlib

Expand Down Expand Up @@ -31,6 +32,9 @@
"Please install DPText-DETR from the following link: https://github.com/rwood-97/DPText-DETR"
)

# Set up logging
logger = logging.getLogger(__name__)

Check warning on line 36 in mapreader/spot_text/dptext_detr_runner.py

View check run for this annotation

Codecov / codecov/patch

mapreader/spot_text/dptext_detr_runner.py#L36

Added line #L36 was not covered by tests


class DPTextDETRRunner:
def __init__(
Expand Down Expand Up @@ -290,7 +294,7 @@
raise ValueError("[ERROR] Please provide a `parent_df`")

if self.parent_predictions == {}:
print("[INFO] Converting patch pixel bounds to parent pixel bounds.")
logger.info("Converting patch pixel bounds to parent pixel bounds.")

Check warning on line 297 in mapreader/spot_text/dptext_detr_runner.py

View check run for this annotation

Codecov / codecov/patch

mapreader/spot_text/dptext_detr_runner.py#L297

Added line #L297 was not covered by tests
_ = self.convert_to_parent_pixel_bounds()

for parent_id, prediction in self.parent_predictions.items():
Expand Down
Loading