Merge pull request #502 from maps-as-data/search_text

Search text

CHANGELOG.md

@@ -25,6 +25,7 @@ _ADD NEW CHANGES HERE_
 - Loading of dataframes from GeoJSON files now supported in many file loading methods (e.g. `add_metadata`, `Annotator.__init__`, `AnnotationsLoader.load`, etc.) ([#495](https://github.com/maps-as-data/MapReader/pull/495))
 - `load_frames.py` added to `mapreader.utils`. This has functions for loading from various file formats (e.g. CSV, Excel, GeoJSON, etc.) and converting to GeoDataFrames ([#495](https://github.com/maps-as-data/MapReader/pull/495))
 - Added tests for text spotting code ([#500](https://github.com/maps-as-data/MapReader/pull/500))
+- Added `search_preds`, `show_search_results` and `save_search_results_to_geojson` methods to text spotting code ([#502](https://github.com/maps-as-data/MapReader/pull/502))
 
 ### Changed
 

docs/source/using-mapreader/step-by-step-guide/6-spot-text.rst

@@ -112,15 +112,15 @@ e.g. for the ``DPTextDETRRunner``, if you choose the "ArT/R_50_poly.yaml", you s
 
 e.g. for the ``DeepSoloRunner``, if you choose the "R_50/IC15/finetune_150k_tt_mlt_13_15_textocr.yaml", you should download the "ic15_res50_finetune_synth-tt-mlt-13-15-textocr.pth" model weights file from the DeepSolo repo.
 
-e.g. for the ``MapTextPipeline``, if you choose the "ViTAEv2_S/rumsey/final_rumsey.yaml", you should download the "rumsey-finetune.pth" model weights file from the MapTextPipeline repo.
+e.g. for the ``MapTextRunner``, if you choose the "ViTAEv2_S/rumsey/final_rumsey.yaml", you should download the "rumsey-finetune.pth" model weights file from the MapTextPipeline repo.
 
 .. note:: We recommend using the "ViTAEv2_S/rumsey/final_rumsey.yaml" configuration and "rumsey-finetune.pth" weights from the ``MapTextPipeline``. But you should choose based on your own use case.
 
 For the DPTextDETRRunner, use:
 
 .. code-block:: python
 
-    from map_reader import DPTextDETRRunner
+    from mapreader import DPTextDETRRunner
 
     #EXAMPLE
     my_runner = DPTextDETR(
@@ -146,7 +146,7 @@ For the DeepSoloRunner, use:
 
 .. code-block:: python
 
-    from map_reader import DeepSoloRunner
+    from mapreader import DeepSoloRunner
 
     #EXAMPLE
     my_runner = DeepSoloRunner(
@@ -158,14 +158,14 @@ For the DeepSoloRunner, use:
 
 or, you can load your patch/parent dataframes from CSV/GeoJSON files as shown for the DPTextRunner (above).
 
-For the MapTextPipeline, use:
+For the MapTextRunner, use:
 
 .. code-block:: python
 
-    from map_reader import MapTextPipeline
+    from mapreader import MapTextRunner
 
     #EXAMPLE
-    my_runner = MapTextPipeline(
+    my_runner = MapTextRunner(
         patch_df,
         parent_df,
         cfg_file = "MapTextPipeline/configs/ViTAEv2_S/rumsey/final_rumsey.yaml",
@@ -182,7 +182,7 @@ You can explicitly set this using the ``device`` argument:
 .. code-block:: python
 
     #EXAMPLE
-    my_runner = MapTextPipeline(
+    my_runner = MapTextRunner(
         "./patch_df.csv",
         "./parent_df.csv",
         cfg_file = "MapTextPipeline/configs/ViTAEv2_S/rumsey/final_rumsey.yaml",
@@ -322,10 +322,77 @@ If you maps are georeferenced in your ``parent_df``, you can also convert the pi
 
     geo_preds_df = my_runner.convert_to_coords(return_dataframe=True)
 
-Again, you can save these to a csv file as above, or, you can save them to a geojson file for loading into GIS software:
+Again, you can save these to a csv file (as shown above), or, you can save them to a geojson file for loading into GIS software:
 
 .. code-block:: python
 
     my_runner.save_to_geojson("text_preds.geojson")
 
 This will save the predictions to a geojson file, with each text prediction as a separate feature.
+
+Search predictions
+------------------
+
+If you are using the DeepSoloRunner or the MapTextRunner, you will have recognized text outputs.
+You can search these predictions using the ``search_preds`` method:
+
+.. code-block:: python
+
+    search_results = my_runner.search_preds("search term")
+
+e.g To find all predictions containing the word "church" and ignoring the case:
+
+.. code-block:: python
+
+    # EXAMPLE
+    search_results = my_runner.search_preds("church")
+
+By default, this will return a dictionary containing the search results.
+If you'd like to return a dataframe instead, use the ``return_dataframe`` argument:
+
+.. code-block:: python
+
+    # EXAMPLE
+    search_results_df = my_runner.search_preds("church", return_dataframe=True)
+
+You can also ignore the case of the search term by setting the ``ignore_case`` argument:
+
+.. code-block:: python
+
+    # EXAMPLE
+    search_results_df = my_runner.search_preds("church", return_dataframe=True, ignore_case=True)
+
+
+The search accepts regex patterns so you can use these to search for more complex patterns.
+
+e.g. To search for all predictions containing the word "church" or "chapel", you could use the pattern "church|chapel":
+
+.. code-block:: python
+
+    # EXAMPLE
+    search_results_df = my_runner.search_preds("church|chapel", return_dataframe=True, ignore_case=True)
+
+Once you have your search results, you can view them on your map using the ``show_search_results`` method.
+
+.. code-block:: python
+
+    my_runner.show_search_results("map_74488689.png")
+
+This will show the map with the search results.
+
+As with the ``show`` method, you can use the ``border_color``, ``text_color`` and ``figsize`` arguments to customize the appearance of the image.
+
+Save search results
+~~~~~~~~~~~~~~~~~~~
+
+If your maps are georeferenced, you can also save your search results using the ``save_search_results_to_geojson`` method:
+
+.. code-block:: python
+
+    my_runner.save_search_results_to_geojson("search_results.geojson")
+
+This will save the search results to a geojson file, with each search result as a separate feature.
+
+These can then be loaded into GIS software for further analysis/exploration.
+
+If your maps are not georeferenced, you can save the search results to a csv file using the pandas ``to_csv`` method (as shown above).

mapreader/spot_text/deepsolo_runner.py

@@ -11,7 +11,6 @@
     )
 
 import geopandas as gpd
-import numpy as np
 import pandas as pd
 import torch
 from adet.config import get_cfg
@@ -21,18 +20,16 @@
 except ImportError:
     raise ImportError("[ERROR] Please install Detectron2")
 
-from shapely import LineString, MultiPolygon, Polygon
-
 # first assert we are using the deep solo version of adet
 if adet.__version__ != "0.2.0-deepsolo":
     raise ImportError(
         "[ERROR] Please install DeepSolo from the following link: https://github.com/rwood-97/DeepSolo"
     )
 
-from .runner_base import Runner
+from .rec_runner_base import RecRunner
 
 
-class DeepSoloRunner(Runner):
+class DeepSoloRunner(RecRunner):
     def __init__(
         self,
         patch_df: pd.DataFrame | gpd.GeoDataFrame | str | pathlib.Path,
@@ -68,6 +65,7 @@ def __init__(
         self.patch_predictions = {}
         self.parent_predictions = {}
         self.geo_predictions = {}
+        self.search_results = {}
 
         # setup the config
         cfg = get_cfg()  # get a fresh new config
@@ -231,50 +229,6 @@ def __init__(
         # setup the predictor
         self.predictor = DefaultPredictor(cfg)
 
-    def get_patch_predictions(
-        self,
-        outputs: dict,
-        return_dataframe: bool = False,
-        min_ioa: float = 0.7,
-    ) -> dict | pd.DataFrame:
-        """Post process the model outputs to get patch predictions.
-
-        Parameters
-        ----------
-        outputs : dict
-            The outputs from the model.
-        return_dataframe : bool, optional
-            Whether to return the predictions as a pandas DataFrame, by default False
-        min_ioa : float, optional
-            The minimum intersection over area to consider two polygons the same, by default 0.7
-
-        Returns
-        -------
-        dict or pd.DataFrame
-            A dictionary containing the patch predictions or a DataFrame if `as_dataframe` is True.
-        """
-        # key for predictions
-        image_id = outputs["image_id"]
-        self.patch_predictions[image_id] = []
-
-        # get instances
-        instances = outputs["instances"].to("cpu")
-        ctrl_pnts = instances.ctrl_points.numpy()
-        scores = instances.scores.tolist()
-        recs = instances.recs
-        bd_pts = np.asarray(instances.bd)
-
-        self._post_process(image_id, ctrl_pnts, scores, recs, bd_pts)
-        self._deduplicate(image_id, min_ioa=min_ioa)
-
-        if return_dataframe:
-            return self._dict_to_dataframe(self.patch_predictions, geo=False)
-        return self.patch_predictions
-
-    def _process_ctrl_pnt(self, pnt):
-        points = pnt.reshape(-1, 2)
-        return points
-
     def _ctc_decode_recognition(self, rec):
         last_char = "###"
         s = ""
@@ -291,83 +245,3 @@ def _ctc_decode_recognition(self, rec):
             else:
                 last_char = "###"
         return s
-
-    def _post_process(self, image_id, ctrl_pnts, scores, recs, bd_pnts, alpha=0.4):
-        for ctrl_pnt, score, rec, bd in zip(ctrl_pnts, scores, recs, bd_pnts):
-            # draw polygons
-            if bd is not None:
-                bd = np.hsplit(bd, 2)
-                bd = np.vstack([bd[0], bd[1][::-1]])
-                polygon = Polygon(bd).buffer(0)
-
-                if isinstance(polygon, MultiPolygon):
-                    polygon = polygon.convex_hull
-
-            # draw center lines
-            line = self._process_ctrl_pnt(ctrl_pnt)
-            line = LineString(line)
-
-            # draw text
-            text = self._ctc_decode_recognition(rec)
-            if self.voc_size == 37:
-                text = text.upper()
-            # text = "{:.2f}: {}".format(score, text)
-            text = f"{text}"
-            score = f"{score:.2f}"
-
-            self.patch_predictions[image_id].append([polygon, text, score])
-
-    @staticmethod
-    def _dict_to_dataframe(
-        preds: dict,
-        geo: bool = False,
-        parent: bool = False,
-    ) -> pd.DataFrame:
-        """Convert the predictions dictionary to a pandas DataFrame.
-
-        Parameters
-        ----------
-        preds : dict
-            A dictionary of predictions.
-        geo : bool, optional
-            Whether the dictionary is georeferenced coords (or pixel bounds), by default True
-        parent : bool, optional
-            Whether the dictionary is at parent level, by default False
-
-        Returns
-        -------
-        pd.DataFrame
-            A pandas DataFrame containing the predictions.
-        """
-        if geo:
-            columns = ["geometry", "crs", "text", "score"]
-        else:
-            columns = ["geometry", "text", "score"]
-
-        if parent:
-            columns.append("patch_id")
-
-        preds_df = pd.concat(
-            pd.DataFrame(
-                preds[k],
-                index=np.full(len(preds[k]), k),
-                columns=columns,
-            )
-            for k in preds.keys()
-        )
-
-        if geo:
-            # get the crs (should be the same for all)
-            if not preds_df["crs"].nunique() == 1:
-                raise ValueError("[ERROR] Multiple crs found in the predictions.")
-            crs = preds_df["crs"].unique()[0]
-
-            preds_df = gpd.GeoDataFrame(
-                preds_df,
-                geometry="geometry",
-                crs=crs,
-            )
-
-        preds_df.index.name = "image_id"
-        preds_df.reset_index(inplace=True)  # reset index to get image_id as a column
-        return preds_df

mapreader/spot_text/dptext_detr_runner.py

@@ -98,7 +98,7 @@ def get_patch_predictions(
         Returns
         -------
         dict or pd.DataFrame
-            A dictionary containing the patch predictions or a DataFrame if `as_dataframe` is True.
+            A dictionary containing the patch predictions or a DataFrame if `return_dataframe` is True.
         """
         # key for predictions
         image_id = outputs["image_id"]