Image Processing
The Image Processing module is at the core of the Auto-Aiming System, enabling real-time detection and segmentation of the target. The chosen pipeline balances simplicity and effectiveness, consisting of the following steps:
-
Color Filtering:
- Enhances the red channel to isolate the target.
-
Thresholding:
- Segments the target by separating it from the background.
-
Morphological Filtering:
- Applies erosion followed by dilation (opening operation) to remove noise and refine the target's shape.
-
Centroid Calculation:
- Computes the target's position for feedback control.
This approach ensures reliable segmentation and precise centroid computation, driving the system's control loop.
Color filtering emphasizes specific channels in an image to isolate the object of interest. In this project, the red channel is enhanced to distinguish the target (a Pokéball) from the background.
// Filter colors
cv::Mat filteredFrame = frame.clone();
std::vector<cv::Mat> channels(3);
cv::split(filteredFrame, channels);
// Normalize each channel by a predefined filter value
channels[0] *= blueFilter / 255.0;
channels[1] *= greenFilter / 255.0;
channels[2] *= redFilter / 255.0;
// Merge channels back
cv::merge(channels, filteredFrame);This step reduces noise and enhances the contrast between the target and its surroundings.
Thresholding is a simple yet powerful technique for segmenting objects based on intensity. This step converts the filtered image into a binary image, separating the target (white pixels) from the background (black pixels).
// Convert to grayscale
cv::Mat grayFrame, thresholdedFrame;
cv::cvtColor(filteredFrame, grayFrame, cv::COLOR_BGR2GRAY);
// Apply binary thresholding
cv::threshold(grayFrame, thresholdedFrame, thresholdValue, 255, thresholdType);In Binary Thresholding, pixels above thresholdValue are set to 255 (white), and others to 0 (black).
Morphological operations refine binary images by removing noise and enhancing object boundaries. The opening operation (erosion followed by dilation) is used to clean up residual noise while maintaining the object's integrity.
// Define structuring element
cv::Mat element = cv::getStructuringElement(
morphShape(kernelShape),
cv::Size(2 * kernelSize + 1, 2 * kernelSize + 1),
cv::Point(kernelSize, kernelSize)
);
// Apply erosion and dilation
cv::Mat eroded, dilated;
cv::erode(thresholdedFrame, eroded, element);
cv::dilate(eroded, dilated, element);The structuring element's shape (morphShape) is chosen dynamically. Here, elliptical elements were found to be most effective for this application.
The centroid of the segmented object is calculated using image moments. The centroid's coordinates (cX, cY) are determined as:
// Compute image moments
cv::Moments m = cv::moments(dilated, true);
// Calculate centroid
int cX = (m.m00 != 0) ? static_cast<int>(m.m10 / m.m00) : -1;
int cY = (m.m00 != 0) ? static_cast<int>(m.m01 / m.m00) : -1;The centroid is used for feedback control and visualized on the processed frame.
To aid debugging and ensure correct functionality, the following elements are overlaid on the processed frames:
- A red circle marks the computed centroid.
- Blue lines represent the frame's center.
- A blue circle indicates the target area.
// Draw centroid
if (cX >= 0 && cY >= 0) {
cv::circle(dilated, cv::Point(cX, cY), 5, cv::Scalar(0, 0, 255), -1);
}
// Draw center of the frame
cv::line(dilated, cv::Point(320, 0), cv::Point(320, dilated.rows - 1), cv::Scalar(255, 0, 0), 2);
cv::line(dilated, cv::Point(0, 240), cv::Point(dilated.cols - 1, 240), cv::Scalar(255, 0, 0), 2);
// Draw circle representing the target area
cv::circle(dilated, cv::Point(320, 240), targetRadius, cv::Scalar(255, 0, 0), 2);For further details, check the Control System section, where the centroid is used to drive the servo actuators.