nuscenes

CoBEVT nuScenes Track

This repository contains the source code and data for our CoBEVT nuScenes track. The whole pipeline is based on CVT(CVPR2022)

Installation

# Clone repo
git clone https://github.com/DerrickXuNu/CoBEVT.git

cd CoBEVT/nuScenes

# Setup conda environment
conda create -y --name sinbevt python=3.8

conda activate sinbevt
conda install pytorch==1.11.0 torchvision==0.12.0 cudatoolkit=11.3 -c pytorch

# Install dependencies
pip install -r requirements.txt
pip install -e .

Data

Documentation:

• Dataset setup
• Label generation (optional)

Download the original datasets and our generated map-view labels

	Dataset	Labels
nuScenes	keyframes + map expansion (60 GB)	cvt_labels_nuscenes.tar.gz (361 MB)

The structure of the extracted data should look like the following

/datasets/
├─ nuscenes/
│  ├─ v1.0-trainval/
│  ├─ v1.0-mini/
│  ├─ samples/
│  ├─ sweeps/
│  └─ maps/
│     ├─ basemap/
│     └─ expansion/
└─ cvt_labels_nuscenes/
   ├─ scene-0001/
   ├─ scene-0001.json
   ├─ ...
   ├─ scene-1000/
   └─ scene-1000.json

When everything is setup correctly, check out the dataset with

python3 scripts/view_data.py \
  data=nuscenes \
  data.dataset_dir=/media/datasets/nuscenes \
  data.labels_dir=/media/datasets/cvt_labels_nuscenes \
  data.version=v1.0-mini \
  visualization=nuscenes_viz \
  +split=val

Training

             

An average job of 50k training iterations takes ~8 hours.
We trained model both on four A5000 and two A100(80G), and we found a total batch size of 16 will get the best results.

To train a model,

python scripts/train.py \
  +experiment=cvt_pyramid_axial_nuscenes_vehicle
  data.dataset_dir=/media/datasets/nuscenes \
  data.labels_dir=/media/datasets/cvt_labels_nuscenes

For more information, see

• config/config.yaml - base config
• config/model/cvt_pyramid_axial.yaml - model architecture (CVT + Pyramid FAX)
• config/experiment/cvt_pyramid_axial_nuscenes_vehicle.yaml - additional overrides

Benchmarking

To benchmark the inference speed, run the following command:

python scripts/benchmark.py \
  +experiment=cvt_pyramid_axial_nuscenes_vehicle
  data.dataset_dir=/media/datasets/nuscenes \
  data.labels_dir=/media/datasets/cvt_labels_nuscenes

Additional Information

Acknoledgement

We would like to sinsere thank CVT(CVPR2022) for their awesome training pipeline, which makes our implementation much easier. c