Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replicator hot fix #900

Merged
merged 3 commits into from
Sep 23, 2024
Merged

Replicator hot fix #900

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f04ffe1
Hot fix for semantic seg - instance seg mismatch
hang-yin Sep 23, 2024
6701101
Fix sensor test
hang-yin Sep 23, 2024
d17d95e
Make sure report always gets compiled
hang-yin Sep 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 4 additions & 26 deletions omnigibson/sensors/vision_sensor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -444,35 +444,13 @@ def _remap_instance_segmentation(self, img, id_to_labels, semantic_img, semantic
self._register_instance(value, id=id)
replicator_mapping[key] = value

# Handle the cases for MicroPhysicalParticleSystem (FluidSystem, GranularSystem).
# They show up in the image, but not in the info (id_to_labels).
# We identify these values, find the corresponding semantic label (system name), and add the mapping.
# This is a temporary fix for the problem where some small number of pixels show up in the image, but not in the info (id_to_labels).
# We identify these values and mark them as unlabelled.
image_keys = th.unique(img)
for key in image_keys:
if str(key.item()) not in id_to_labels:
semantic_label = semantic_img[img == key].unique().item()
assert (
semantic_label in semantic_labels
), f"Semantic map value {semantic_label} is not in the semantic labels!"
category_name = semantic_labels[semantic_label]
if category_name in self.scene.available_systems.keys():
value = category_name
self._register_instance(value, id=id)
# If the category name is not in the registered systems,
# which happens because replicator sometimes returns segmentation map and id_to_labels that are not in sync,
# we will label this as "unlabelled" for now
# This only happens with a very small number of pixels, e.g. 0.1% of the image
else:
num_of_pixels = (img == key).sum().item()
resolution = (self._load_config["image_width"], self._load_config["image_height"])
percentage = (num_of_pixels / (resolution[0] * resolution[1])) * 100
if percentage > 2:
log.warning(
f"Marking {category_name} as unlabelled due to image & id_to_labels mismatch!"
f"Percentage of pixels: {percentage}%"
)
value = "unlabelled"
self._register_instance(value, id=id)
value = "unlabelled"
self._register_instance(value, id=id)
replicator_mapping[key.item()] = value

registry = VisionSensor.INSTANCE_ID_REGISTRY if id else VisionSensor.INSTANCE_REGISTRY
Expand Down
Loading