'KeyError: Agent is None while performing tracking using CARLA Dataset'

[MaxBial]

Hi

Hope you are doing great. I am currently facing some issues while trying the tracking with CARLA dataset and hope you can give me some insights. I will explain what i did step-by-step:

1. I followed the tutorial from here (https://github.com/avstack-lab/avstack-docs/tree/update) where you explained about the perception and tracking with KITTI dataset. My project scope is with CARLA dataset.

2. I collected the CARLA dataset based on the tutorial and post process it using the python script. After post processing the contents of the folder looks like below.

3. Now i changed the codes in the tutorial to accommodate the CARLA, instead of KITTI

import os
from tqdm import tqdm
 
from avapi.kitti import KittiScenesManager
from avapi.carla import CarlaScenesManager
 
%load_ext autoreload
%autoreload 2
 
data_base = "../../../"
obj_data_dir_k = os.path.join(data_base, "KITTI/object")
raw_data_dir_k = os.path.join(data_base, "KITTI/raw")
obj_data_dir_c = os.path.join(data_base, "CARLA_Data/new_data_set1")
 
CSM = CarlaScenesManager(obj_data_dir_c)
CDM = CSM.get_scene_dataset_by_index(0)
 
#DM = KDM  # let's use KITTI for this one
DM = CDM

Here 'carla-vehicle' is used as now i am not concerning about others.

import inspect
from avstack.modules.perception.object3d import MMDetObjectDetector3D
from avstack.modules.tracking.tracker3d import BasicBoxTracker3D
 
result_folder_dets = "results_lidar/detections"
result_folder_trks = "results_lidar/tracks"
 
P = MMDetObjectDetector3D(
    model="pointpillars",
    dataset="carla-vehicle",
    threshold=0.7,
    output_folder=result_folder_dets,
    post_hooks=[{"type": "DetectionsLogger"}],
)
T = BasicBoxTracker3D(
    output_folder=result_folder_trks, post_hooks=[{"type": "TracksLogger"}]
)

Till here there was no errors and was working. In the next step, i am getting error while executing the below code.

# detection and tracking
for frame in tqdm(DM.frames):
    ego_reference = DM.get_ego_reference(frame=frame)
    pc = DM.get_lidar(frame, sensor="lidar-0")
    #pc = DM.get_lidar(frame)
 
    # -- perception
    dets = P(data=pc, frame=frame)
 
    # -- tracking
    tracks = T(detections=dets, platform=ego_reference)

The error i am encountering is:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[33], line 4
      2 for frame in tqdm(DM.frames):
      3     ego_reference = DM.get_ego_reference(frame=frame)
----> 4     pc = DM.get_lidar(frame, sensor="lidar-0")
      5     #pc = DM.get_lidar(frame)
      6 
      7     # -- perception
      8     dets = P(data=pc, frame=frame)
 
File ~/Documents/New CARLA-Sandbox/carla-sandbox/submodules/lib-avstack-api/avapi/_dataset.py:224, in BaseSceneDataset.get_lidar(self, frame, sensor, agent, filter_front, min_range, max_range, with_panoptic)
    221 sensor = self.get_sensor_name(sensor, agent)
    222 print('SENSOR:', sensor)
--> 224 ts = self.get_timestamp(frame, sensor, agent=agent)
    225 print('TS:', ts)
    226 calib = self.get_calibration(frame, sensor, agent=agent)
 
File ~/Documents/New CARLA-Sandbox/carla-sandbox/submodules/lib-avstack-api/avapi/_dataset.py:315, in BaseSceneDataset.get_timestamp(self, frame, sensor, agent, utime)
    313 def get_timestamp(self, frame, sensor=None, agent=None, utime=False):
    314     sensor = self.get_sensor_name(sensor, agent=agent)
--> 315     return self._load_timestamp(frame, sensor=sensor, agent=agent, utime=utime)
 
File ~/Documents/New CARLA-Sandbox/carla-sandbox/submodules/lib-avstack-api/avapi/carla/dataset.py:317, in CarlaSceneDataset._load_timestamp(self, frame, sensor, agent, utime)
    315 print('sensor:', sensor)
    316 print('FRame:', frame)
--> 317 return self.sensor_frame_to_ts[agent][sensor][frame]
 
KeyError: None
 

Doubts/Observations:

1. I am using the sensor name (here lidar-0) correct and there are many lidars available.
2. While calling DM.get_lidar(frame, sensor="lidar-0"), do we need to provide argument 'agent' too or may be use the get_agent_set function inside the get_lidar?
3. Lastly, can you please share the working version of carla-sandbox which goes with the documentation ?

I think that agent is always None and its not possible to access from the dictionary self.sensor_frame_to_ts[agent][sensor][frame].

Hope you can give me some feedback.
Best Regards,
Max