Project Plan

Work Packages

WP1 Literature Study

• WP1.1 Conduct a comprehensive literature review of state-of-the-art methods for object detection under adverse weather conditions using multiple modalities
• WP1.2 Analyze and compare various fusion strategies for exploiting the complementary characteristics of different sensors
• WP1.3 Search for suitable public datasets with adverse weather conditions and multimodal sensors data

WP2 Data Collection and Preparation

• WP2.1 Acquire the necessary datasets for multimodal object detection under adverse weather conditions, such as K-radar, DENSE, and aiMotive
• WP2.2 Develop tools for pre-processing and augmenting the datasets to enhance the performance of the models
• WP2.3 Perform statistical analysis to identify the main characteristics and challenges of the datasets, including data imbalance and class imbalance

WP3 Model Design and Implementation

• WP3.1 Design and implement an existing multimodal object detection architectures that integrates camera, LiDAR, and radar data
• WP3.2 Investigate various fusion strategies, such as concatenation, mixture of experts, attention-based fusion etc, to determine the most effective approach
• WP3.3 Explore deep learning architectures, such as CNNs, RNNs, and Transformers, to improve the performance of the multimodal model
• WP3.4 Optimize the model’s hyperparameters and train the model on the acquired datasets

WP4 Model Evaluation and Validation

• WP4.1 Evaluate the performance of the developed multimodal object detection model on the acquired datasets under adverse weather conditions
• WP4.2 Compare the proposed model’s results to existing state-of-the-art or baseline methods and analyze the strengths and weaknesses of each approach
• WP4.3 Identify the limitations of the proposed model and suggest possible future improvements

WP5 Project Report

• WP5.1 Write a detailed report that includes the research problem, objectives, methodology, results, and conclusion
• WP5.2 Present the research findings in a clear and concise manner, highlighting the contributions and limitations of the proposed multimodal object detection model
• WP5.3 Discuss possible future research directions based on the outcomes of the study

Milestones

M1 Literature review completed and best practice identified

M2 Data collection and preprocessing completed, including cleaning and augmentation

M3 Initial model development and testing completed

M4 Evaluation and optimization of the model completed

M5 Final model development and testing completed, including comparison with existing state-of-the-art methods and analysis of strengths and weaknesses of each approach.

M6 Project report completed

Deliverables

• Minimum Viable
  • Conduct a comprehensive literature review on state-of-the-art multimodal object detection methods and their fusion strategies
  • Develop and test existing models for object detection
  • Perform a comparative analysis of at least two methods on one dataset
  • Produce a project report that summarizes the work done and the results obtained
• Expected
  • Compare the performance of more advanced methods with the baseline methods
  • Complete the final development and testing of the model, including comparison with existing state-of-the-art methods and analysis of the strengths and weaknesses of each approach.
  • Produce a more extensive project report that details the methodology, experimental setup, results, and analysis.
• Desired
  • Compare the developed models’ performance on one or more additional datasets.
  • Propose improvements to the baseline fusion methods