Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CollisionTest criteria json does not include information about the collision traffic event. #1056

Open
paulbourelly999 opened this issue Feb 19, 2024 · 0 comments

Comments

@paulbourelly999
Copy link

paulbourelly999 commented Feb 19, 2024

Describe our question or idea
Using the metrics module in scenario runner, I set up a scenario with a CollisionTest criteria. The expectation being that the metrics module would create a criteria json file indicating that a given run failed the collision test criteria along with information about the collisions or traffic events that caused it to fail. Instead I get the following json
"CollisionTest": { "children": [], "feedback_message": "", "blackbox_level": 4, "_terminate_on_failure": false, "test_status": "FAILURE", "expected_value_success": 0, "expected_value_acceptable": null, "actual_value": 1, "optional": false, "_collision_sensor": null, "other_actor": null, "other_actor_type": null, "registered_collisions": [], "last_id": null, "collision_time": 27.05000040307641, "terminate_on_failure": false },
Expected behavior
Looking through the code, the CollisionTest criteria should populate other attributes here including a collection of traffic events. While it seems to have recognized a collision took place based on the collision_time and failure state, but it seems to have not recorded any information about the collision.

Looking at the code, it seems to skip over any attributes it fails to serialize as JSON strings.

try:
key_dict = {key: criterion_dict[key]}
json.dump(key_dict, fp, sort_keys=False, indent=4)
criteria_dict[criterion.name].update(key_dict)
except TypeError:
pass

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: Ubuntu 18:04 (Docker)
  • CARLA Version : 0.9.10
  • Python version :3.6
  • Version 0.9.10
    Note: We are currently unable to upgrade to any newer version of CARLA due to the cost of integration incurred for updating. Would prefer solutions that do not require an update.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant