A simple project to control a IOT lightbulb by tracking a reflected infrared point. This project includes the base python file and some sample data files for training.
- Raspberry Pi 3 Model B Plus (Rev 1.3)
- AdruCAM NOIR 5MP OV5647
- Point-Based IR Reflective Device
- Raspbian
- Python 3 with the following modules installed:
- TP-Link Software
- ArduCAM Software: Helps provide a more-stable camera stream by disabling IR cutover. Place CameraLED.py in the same directory as the wandcv.py file.
Camera input is converted to grayscale and thresholded in order to extract only the more-prevelant features from the IR camera. We use a simple KNN background subtraction in order to remove stable background noise. Dialation and eroding are used to help connect neighboring points.
We use a simple accumulator-type object to create basic 'motion masks' in order to track the historical path of the IR points. Historical value decay has been set at .9 based on trial-and-error.
Once we have a history, we create a bounding box that surrounds all detected motion in the image. The portion of the input denoted by the bounding box is extracted and then scaled to 30x30 for inference (described below).
The bounding box is succeptable any noise that remains after thresholding and background substraction. More robust image processing capabilites would improve the stability of the extracted movement patterns.
We use a simple K-Nearest Neighbors algorithm to predict the input, given a set of training files.
Included in the repository is a set of training images. There are six training examples across five different types of classes and three examples for the base class. Each file is 100x100. The files are named _#.jpg and should be successfully parsed and trained by the python file.
Classes:
- s: Turns the light green; looks for 's'
- h: Turns the light yellow; looks for 'h'
- g: Turns the light red; looks for 'G', also works with 'O'
- r: Turns the light blue; looks for an inverted 'V'; the 'R' pattern was too close to the 'h' pattern
- v: Resets the light to a normal color
- Z: null (completely blank input). Makes no changes.
The normalized and scaled foreground motion mask is compared to the training files by using five (5) nearest neighbors and the jaccard distance metric. These were chosen using trial-and-error methodology. More data and a more-rigorous approach could improved inference stability.