Lucas-Kanade-tracking-and-Correlation-Filters

This repository contains implementation of Lucas-Kanade algorithm proposed by Lucas and Kanade. Lucas-Kanade algorithm can be used for sparse optical flow (associate feature points across frames) and tracking (associate image patch cross frames). This repo implements the algorithm for tracking a single template across 400 frames video.
Please unzip data.zip and then follow the instructions below.

Lucas Kanade Tracking with one single template

The "vanilla" algorithm for tracking. Detailed derivation can be referred to Lucas-Kanade 20 Years On: A Unifying Framework
Files included:
/data/carseq.npy
/src/LucasKanade.py
/src/testCarSequence.py
Run

python testCarSequence.py

Sample results

/src/carseqrects.npy stores the vertices coordinates of bounding box in each frame.

Lucas Kanade Tracking with template correction

The upgraded version of the first algorithm, which mitigates template drifting problem. Detailed derivation can be referred to The Template Update Problem
FIles included:
/data/carseq.npy
/src/LucasKanade.py
/src/TemplateCorrection.py
/src/testCarSequenceWithTemplateCorrection.py
Run

python testCarSequenceWithTemplateCorrection.py

Sample results
Blue bbox: with template correction | Red bbox: without template correction

Lucas Kanade Tracking with appearance basis

The former 2 algorithms may not suffice for real life challenges with drastic appearance variance. Through analyzing historical data collected, we can use an eigen-space approach to produce a principal template at each frame.
Files included:
/data/sylvseq.npy
/data/sylvbbases.npy
/src/LucasKanade.py /src/TemplateCorrection.py
/src/LucasKanadeBasis.py
/src/testSylvSequence.py
Run

python testSylvSequence.py

Sample results
Blue bbox: with appearance bases | Red bbox: without appearance bases

Lucas Kanade Tracking with dominant affine motion

This algorithm works on non-stationary camera video
Files included
/data/aerialseq.npy
/src/LucasKanadeAffine.py
/src/SubtractDominantMotion.py
/src/InverseCompositionAffine.py
/src/testAerialSequence.py
Run

python testAerialSequence.py

Sample results

Correlation Filters

The paper of correlation filters can be found hereVisual Object Tracking using Adaptive Correlation Filters
Files included
/src/Corr-Filters/lena.npy
/src/Corr-FIlters/example.py
Run

python example.py

Sample results