Self-driving Car Simulator

This project is a modification tailored to the requirements of the UIT Car Racing contest (e.g., fine-tuning, model changes). The edited code is primarily based on the Self-driving Car Simulator.

Introduction

Pipeline

Detection Task

We compared different models based on the number of parameters:

Model	Params (M)	[email protected]	[email protected]:.95
YOLOv11n	21.2	0.993	0.861
YOLOv8n	3.2	0.992	0.887

Fine-tuning for model training: We leveraged Roboflow processing to generate cases with lighting changes and noise. The flip feature was avoided due to incorrect labels in flipped images.

model.train(data="../trafficsign-2-1/data.yaml", epochs=300, imgsz=(640, 384), fliplr=0.0, flipud=0.0)

Based on our results, YOLOv8 Nano was adopted for this project. Below are some example images predicted using YOLOv8 Nano. The model successfully predicted small objects, which is crucial for the control task.

Datasets

• Traffic Sign Detection
• Road Segmentation

Segmentation Task

We initially used YOLOv8m, which met our requirements.

Controller

A PID controller (Proportional Integral Derivative controller) is a widely used control loop mechanism. It continuously calculates an error value as the difference between a desired setpoint and a measured process variable, applying corrections based on proportional, integral, and derivative terms (denoted P, I, and D). The controller minimizes the error over time by adjusting a control variable using a weighted sum of the control terms.

Pseudocode for Control

# Find A and B
C = (A + B) * 2.5 / 3.5  # Right center point
error = IC = HI - HC
angle = PID(error, p, i, d)
speed = f(angle)

# OH: Hyper-parameter (e.g., 1/3 or 1/4 of image height)

Pseudocode for Handling Traffic Signs

# Pre-turning
for detected_class in yolo_outputs:
    if detected_class in traffic_signs:
        majority_classes.add(detected_class)

        if len(majority_classes) == 10:
            majority_class = find_majority(majority_classes)
            turning = True

# Turning
while turning_counter <= max_turning_counter:
    switch case for majority_class:
        angle = constant
        speed = constant
        
        turning_counter += 1

Usage

1. Download weights and place them in the pretrain directory.
2. Start Unity.
3. Run the script:

python main.py

Demo

Video demo available: Delulu team here.

Result

• Top 3 in the qualifiers round.
• Could not progress to the final round due to FPS discrepancies causing incorrect fine-tuned parameters.

References

• YOLOv8
• YOLOv11
• PIDNet
• CEEC

CarRacing-ver1