feat: add T4 dataset updater with the specifeid FastLabel annotations (…

…#172) * feat: add T4 updater from FastLabel annotations Signed-off-by: ktro2828 <[email protected]> * fix: update conversion Signed-off-by: ktro2828 <[email protected]> * test: update tests Signed-off-by: ktro2828 <[email protected]> * feat: update FastLabel merger (#178) * update configs Signed-off-by: Shunsuke Miura <[email protected]> * add label converter to FastLabel2dToT4Converter Signed-off-by: Shunsuke Miura <[email protected]> * fix bug in from_json of surface_ann Signed-off-by: Shunsuke Miura <[email protected]> * add automatic_annotation key to sample_annotation/object_ann/surface_ann Signed-off-by: Shunsuke Miura <[email protected]> * accelerate default object_mask generation Signed-off-by: Shunsuke Miura <[email protected]> * feat: improve T4 updater functionality and directory handling Signed-off-by: Shunsuke Miura <[email protected]> * feat: support polygon-to-bbox conversion and improve polygon-to-RLE conversion * update object config Signed-off-by: Shunsuke Miura <[email protected]> * feat: update annotation loader * feat: only convert to bbox for objects labels Signed-off-by: Shunsuke Miura <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: handling automatic_annotation entry Signed-off-by: Shunsuke Miura <[email protected]> * fix: bug fix in rle_from_points Signed-off-by: Shunsuke Miura <[email protected]> * fix: add default value for automatic_annotation Signed-off-by: Shunsuke Miura <[email protected]> --------- Signed-off-by: Shunsuke Miura <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat: enhance surface category handling in AnnotationFilesGenerator and DeepenToT4Converter (#180) * feat: enhance surface category handling in AnnotationFilesGenerator and DeepenToT4Converter Signed-off-by: Shunsuke Miura <[email protected]> * delete unnecessary import Signed-off-by: Shunsuke Miura <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Shunsuke Miura <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor: parallelizate fl annotation processing (#182) * feat: enhance surface category handling in AnnotationFilesGenerator and DeepenToT4Converter Signed-off-by: Shunsuke Miura <[email protected]> # Conflicts: # perception_dataset/deepen/deepen_to_t4_converter.py # perception_dataset/fastlabel_to_t4/fastlabel_2d_to_t4_converter.py # perception_dataset/t4_dataset/annotation_files_generator.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor: parallelizate fl annotation processing Signed-off-by: Shunsuke Miura <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Shunsuke Miura <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * remove duplicate key in object.yaml Signed-off-by: Shunsuke Miura <[email protected]> * feat(fastalbel_to_t4dataset): fix keyframe handling (#187) * merge _load_annotation_jsons function to one Signed-off-by: Shunsuke Miura <[email protected]> * add default value for automatic_annotation Signed-off-by: Shunsuke Miura <[email protected]> * change to non-keyframe for no-annotation frame Signed-off-by: Shunsuke Miura <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused import Signed-off-by: Shunsuke Miura <[email protected]> --------- Signed-off-by: Shunsuke Miura <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * change import path Signed-off-by: Shunsuke Miura <[email protected]> * Revert "refactor: parallelizate fl annotation processing (#182)" This reverts commit 3872c99. * refactor: FastLabel annotation processing for parallel execution Signed-off-by: Shunsuke Miura <[email protected]> * supress no-3D annotation warning Signed-off-by: Shunsuke Miura <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ktro2828 <[email protected]> Signed-off-by: Shunsuke Miura <[email protected]> Co-authored-by: Shunsuke Miura <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shunsuke Miura <[email protected]>
tier4 · Jan 2, 2025 · 93f763b · 93f763b
1 parent 06a3b53
commit 93f763b
Show file tree

Hide file tree

Showing 31 changed files with 971 additions and 83 deletions.
diff --git a/config/label/object.yaml b/config/label/object.yaml
@@ -4,6 +4,7 @@ bus: [bus, BUS, vehicle.bus, vehicle.bus (bendy & rigid)]
 car:
   [
     car,
+    cars,
     CAR,
     vehicle.car,
     vehicle.construction,
@@ -55,3 +56,38 @@ stroller: [stroller, STROLLER, pedestrian.stroller]
 police_officer: [police_officer, POLICE_OFFICER, pedestrian.police_officer]
 wheelchair: [wheelchair, WHEELCHAIR, pedestrian.wheelchair]
 forklift: [forklift, FORKLIFT]
+train: [train, TRAIN]
+cone: [cone]
+curb: [curb]
+gate: [gate]
+guide_post: [guide_post]
+construction: [constructions, construction]
+traffic_sign: [traffic_sign]
+road_debris: [road_debris]
+other_obstacle: [other_obstacle]
+obstacle_others: [obstacle_others]
+laneline_solid_white: [laneline_solid_white]
+laneline_dash_white: [laneline_dash_white]
+laneline_solid_yellow: [laneline_solid_yellow]
+laneline_dash_yellow: [laneline_dash_yellow]
+laneline_solid_green: [laneline_solid_green]
+laneline_solid_red: [laneline_solid_red]
+deceleration_line: [deceleration_line]
+dashed_lane_marking: [dashed_lane_markings, dash_lane_markings, dashed_lane_marking, dash_white_merge, dash_white_branch]
+stopline: [stopline]
+crosswalk: [crosswalk]
+marking_character: [marking_character]
+marking_arrow: [marking_arrow]
+striped_road_marking: [striped_road_markings, striped_road_marking]
+parking_lot: [parking_lot]
+marking_other: [marking_other]
+road: [road]
+road_paint_lane_solid_white: [road_paint_lane_solid_white]
+road_paint_lane_dash_white: [road_paint_lane_dash_white]
+sidewalk: [sidewalk]
+building: [building, buildling] # typo
+wall/fence: [wall_fence, wall/fence]
+pole: [pole]
+vegetation/terrain: [vegetation_terrain, vegetation/terrain]
+sky: [sky]
+traffic_light: [traffic_light]
diff --git a/config/label/surface.yaml b/config/label/surface.yaml
@@ -4,8 +4,10 @@
 - sidewalk
 - building
 - wall_fence
+- wall/fence
 - pole
 - vegetation_terrain
+- vegetation/terrain
 - sky
 - road_paint_lane_solid_white
 - road_paint_lane_dash_white
diff --git a/config/update_t4_with_fastlabel_sample.yaml b/config/update_t4_with_fastlabel_sample.yaml
@@ -0,0 +1,23 @@
+task: update_t4_with_fastlabel
+description:
+  visibility:
+    full: "No occlusion of the object."
+    most: "Object is occluded, but by less than 50%."
+    partial: "The object is occluded by more than 50% (but not completely)."
+    none: "The object is 90-100% occluded and no points/pixels are visible in the label."
+  camera_index:
+    CAM_FRONT_NARROW: 0
+    CAM_FRONT_WIDE: 1
+    CAM_FRONT_RIGHT: 2
+    CAM_BACK_RIGHT: 3
+    CAM_BACK_NARROW: 4
+    CAM_BACK_WIDE: 5
+    CAM_BACK_LEFT: 6
+    CAM_FRONT_LEFT: 7
+  surface_categories: ./config/label/surface.yaml
+
+conversion:
+  make_t4_dataset_dir: false # If true, the output directory includes t4_dataset directory (such as "scene_dir"/t4_dataset/data|annotation). If false, "scene_dir"/data|annotation.
+  input_base: ./data/input_t4_format # could be non_annotated_t4_format or t4_format_3d_annotated
+  input_anno_base: ./data/fastlabel
+  output_base: ./data/output_t4_format # currently, this only includes the 2D annotations
diff --git a/perception_dataset/convert.py b/perception_dataset/convert.py
@@ -370,6 +370,33 @@ def main():
         converter.convert()
         logger.info(f"[END] Converting Fastlabel data ({input_base}) to T4 data ({output_base})")
 
+    elif task == "update_t4_with_fastlabel":
+        from perception_dataset.fastlabel_to_t4.fastlabel_2d_to_t4_updater import (
+            FastLabel2dToT4Updater,
+        )
+
+        input_base = config_dict["conversion"]["input_base"]
+        output_base = config_dict["conversion"]["output_base"]
+        input_anno_base = config_dict["conversion"]["input_anno_base"]
+        description = config_dict["description"]
+        make_t4_dataset_dir = config_dict["conversion"]["make_t4_dataset_dir"]
+
+        converter = FastLabel2dToT4Updater(
+            input_base=input_base,
+            output_base=output_base,
+            input_anno_base=input_anno_base,
+            overwrite_mode=args.overwrite,
+            description=description,
+            make_t4_dataset_dir=make_t4_dataset_dir,
+        )
+        logger.info(
+            f"[BEGIN] Updating T4 dataset ({input_base}) with FastLabel {input_anno_base} into T4 data ({output_base})"
+        )
+        converter.convert()
+        logger.info(
+            f"[DONE] Updating T4 dataset ({input_base}) with FastLabel {input_anno_base} into T4 data ({output_base})"
+        )
+
     elif task == "merge_2d_t4dataset_to_3d":
         from perception_dataset.t4_dataset.t4_dataset_2d3d_merger import T4dataset2D3DMerger
 

diff --git a/perception_dataset/deepen/deepen_to_t4_converter.py b/perception_dataset/deepen/deepen_to_t4_converter.py
@@ -6,6 +6,7 @@
 from typing import Any, Dict, List, Optional, Union
 
 from nuscenes.nuscenes import NuScenes
+import yaml
 
 from perception_dataset.abstract_converter import AbstractConverter
 from perception_dataset.deepen.deepen_annotation import (
@@ -20,7 +21,9 @@
 )
 from perception_dataset.rosbag2.rosbag2_converter import Rosbag2Converter
 from perception_dataset.t4_dataset.annotation_files_generator import AnnotationFilesGenerator
-from perception_dataset.t4_dataset.keyframe_consistency_resolver import KeyFrameConsistencyResolver
+from perception_dataset.t4_dataset.resolver.keyframe_consistency_resolver import (
+    KeyFrameConsistencyResolver,
+)
 from perception_dataset.utils.logger import configure_logger
 import perception_dataset.utils.misc as misc_utils
 
@@ -57,6 +60,11 @@ def __init__(
         self._label_info: Optional[LabelInfo] = label_info
 
         self._topic_list_yaml: Union[List, Dict] = topic_list
+        if description.get("surface_categories"):
+            with open(description["surface_categories"], "r") as f:
+                self._surface_categories: List[str] = yaml.safe_load(f)
+        else:
+            self._surface_categories = []
 
     def convert(self):
         camera_index: Dict[str, int] = self._description["camera_index"]
@@ -111,7 +119,10 @@ def convert(self):
         for t4data_name, dataset_id in self._t4data_name_to_deepen_dataset_id.items():
             output_dir = osp.join(self._output_base, t4data_name, self._t4_dataset_dir_name)
             input_dir = osp.join(self._input_base, t4data_name)
-            annotation_files_generator = AnnotationFilesGenerator(description=self._description)
+
+            annotation_files_generator = AnnotationFilesGenerator(
+                description=self._description, surface_categories=self._surface_categories
+            )
             annotation_files_generator.convert_one_scene(
                 input_dir=input_dir,
                 output_dir=output_dir,