Skip to content

Commit

Permalink
Add CollisionTest and RedLighRunningTest to scenario (#317)
Browse files Browse the repository at this point in the history
<!-- Thanks for the contribution, this is awesome. -->

# PR Details
## Description
Adding CollisionTest and RedLightRunning test to scenario definition.
CollisionTest adds collision detector to vehicle
([atomic_criteria.py](https://github.com/carla-simulator/scenario_runner/blob/0ff0ebe7fb0ee2de1f157b11ed450e7b981398bf/srunner/scenariomanager/scenarioatomics/atomic_criteria.py#L313)).
Both test criteria are evaluated for the run and generate a JSON file
indicating success or failure as well as information about any
collisions or events in which we run a red light
```json
{
    "CollisionTest": {
        "children": [],
        "feedback_message": "",
        "blackbox_level": 4,
        "_terminate_on_failure": false,
        "test_status": "FAILURE",
        "expected_value_success": 0,
        "expected_value_acceptable": null,
        "actual_value": 1,
        "optional": false,
        "_collision_sensor": null,
        "other_actor": null,
        "other_actor_type": null,
        "registered_collisions": [],
        "last_id": null,
        "collision_time": 27.05000040307641,
        "terminate_on_failure": false
    },
    "RunningRedLightTest": {
        "children": [],
        "feedback_message": "",
        "blackbox_level": 4,
        "_terminate_on_failure": false,
        "test_status": "SUCCESS",
        "expected_value_success": 0,
        "expected_value_acceptable": null,
        "actual_value": 0,
        "optional": false,
        "list_traffic_events": [],
        "_last_red_light_id": null,
        "debug": false,
        "terminate_on_failure": false
    }
}
```
**NOTE** : While the collision test seems to be correctly recognizing
collisions and the time they occur at it is not currently providing data
about the collisions likes actors involved, type of collision, kinematic
information.

This is likely a bug in CarlaScenarioRunner. Investigation for this
information will be considered outside of this PR
<!--- Describe your changes in detail -->

## Related Issue
CDAR-746
<!--- This project only accepts pull requests related to open issues -->
<!--- If suggesting a new feature or change, please discuss it in an
issue first -->
<!--- If fixing a bug, there should be an issue describing it with steps
to reproduce -->
<!--- Please link to the issue here: -->

## Motivation and Context
Collect data on collisions and traffic violations
<!--- Why is this change required? What problem does it solve? -->

## How Has This Been Tested?
CDASim deployment
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Types of changes

<!--- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Defect fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that cause existing functionality
to change)

## Checklist:

<!--- Go over all the following points, and put an `x` in all the boxes
that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->

- [ ] I have added any new packages to the sonar-scanner.properties file
- [x] My change requires a change to the documentation.
- [x] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
[CARMA Contributing
Guide](https://github.com/usdot-fhwa-stol/carma-platform/blob/develop/Contributing.md)
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.
  • Loading branch information
paulbourelly999 authored Feb 16, 2024
1 parent cc06ab0 commit 850b4d0
Show file tree
Hide file tree
Showing 3 changed files with 50 additions and 9 deletions.
40 changes: 39 additions & 1 deletion ail_vru_uc1_scenario/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,43 @@ The Virtual Signal Controller is built locally and currently only available to l

## Data Collection
### Carla Recorder
This **CARMA Config** includes volumes and images that will use **Carla's** [recorder](https://carla.readthedocs.io/en/0.9.10/adv_recorder/) functionality to record **CARLA** simulation data into a `carla-recorder/` directory. Included will be a `Trb2024_1.json` file and a `Trb2024_1.log` file. The name of the file comes from the scenario name defined in the **scenario-runner** image. The `.json` file is a criteria file created by the **scenario runner** image and the `.log` file the carla simulation recording created by the carla server running in the **CDASim** image(see https://carla-scenariorunner.readthedocs.io/en/latest/metrics_module/). Using the metrics module in carla-scenario-runner we can define metrics in carla and evaluate the metrics from the carla recordings. This works for collision monitoring and will be prototyped for a custom defined near miss metric.
This **CARMA Config** includes volumes and images that will use **Carla's** [recorder](https://carla.readthedocs.io/en/0.9.10/adv_recorder/) functionality to record **CARLA** simulation data into a `carla-recorder/` directory. Included will be a `Trb2024_1.json` file and a `Trb2024_1.log` file. The name of the file comes from the scenario name defined in the **scenario-runner** image. The `.json` file is a criteria file created by the **scenario runner** image and the `.log` file the carla simulation recording created by the carla server running in the **CDASim** image(see https://carla-scenariorunner.readthedocs.io/en/latest/metrics_module/).The criteria json for the VulnerableRoadUser scenarion should look something like this :
```json
{
"CollisionTest": {
"children": [],
"feedback_message": "",
"blackbox_level": 4,
"_terminate_on_failure": false,
"test_status": "FAILURE",
"expected_value_success": 0,
"expected_value_acceptable": null,
"actual_value": 1,
"optional": false,
"_collision_sensor": null,
"other_actor": null,
"other_actor_type": null,
"registered_collisions": [],
"last_id": null,
"collision_time": 27.05000040307641,
"terminate_on_failure": false
},
"RunningRedLightTest": {
"children": [],
"feedback_message": "",
"blackbox_level": 4,
"_terminate_on_failure": false,
"test_status": "SUCCESS",
"expected_value_success": 0,
"expected_value_acceptable": null,
"actual_value": 0,
"optional": false,
"list_traffic_events": [],
"_last_red_light_id": null,
"debug": false,
"terminate_on_failure": false
}
}
```
This JSON file will provide information on whether any Collisions and RedLightRunning events took place during the run. Both are considered conditions for a test failure.The JSON should also provide information about the event that caused test failures like time, actor involved, and other event information althoguth this functionality seems to be currently broken, likely due to Carla Scenario Runner bugs.

Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@
KeepVelocity,
WaypointFollower,
)
from srunner.scenariomanager.scenarioatomics.atomic_criteria import CollisionTest
from srunner.scenariomanager.scenarioatomics.atomic_criteria import (
CollisionTest,
RunningRedLightTest
)
from srunner.scenariomanager.scenarioatomics.atomic_trigger_conditions import (
DriveDistance,
InTriggerDistanceToVehicle,
Expand Down Expand Up @@ -132,7 +135,7 @@ def _create_behavior(self):
:return: Behavior tree root
"""
carma_vehicle = None
self.carma_vehicle = None
for actor in CarlaDataProvider.get_world().get_actors():
if "role_name" in actor.attributes:
print(f"ACTOR ROLE: {actor.attributes['role_name']}")
Expand All @@ -142,15 +145,15 @@ def _create_behavior(self):
"role_name" in actor.attributes
and actor.attributes["role_name"] == "carma_1"
):
carma_vehicle = actor
self.carma_vehicle = actor
break

CarlaDataProvider.register_actor(actor)
crossing_person = self.other_actors_dict["crossing_person"]

start_condition = Idle(5, name="start_condition")
start_condition = InTriggerDistanceToVehicle(
crossing_person, carma_vehicle, WALKING_PERSON_TRIGGER_WALKING_DISTANCE_IN_METERS
crossing_person, self.carma_vehicle, WALKING_PERSON_TRIGGER_WALKING_DISTANCE_IN_METERS
)

walk_across_street = KeepVelocity(
Expand All @@ -164,7 +167,7 @@ def _create_behavior(self):
actor_behaviors = py_trees.composites.Parallel(name="actor_behaviors")
actor_behaviors.add_child(walk_across_street)

end_condition = DriveDistance(carma_vehicle, 100)
end_condition = DriveDistance(self.carma_vehicle, 100)

root = py_trees.composites.Sequence(name="root_sequence")
root.add_child(start_condition)
Expand All @@ -182,9 +185,8 @@ def _create_test_criteria(self) -> list:
:return: List of test criteria
"""
return [
# This is an example usage for including test criteria in
# ScenarioRunner.
# CollisionTest(self.carma_vehicle)
CollisionTest(self.carma_vehicle),
RunningRedLightTest(self.carma_vehicle)
]

def __del__(self):
Expand Down
1 change: 1 addition & 0 deletions ail_vru_uc1_scenario/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,7 @@ services:
&& python3 scenario_runner.py
--scenario VulnerableRoadUserCollision_LeftTurn
--record metrics/data/
--debug
--host 172.2.0.2"
networks:
Expand Down

0 comments on commit 850b4d0

Please sign in to comment.