drishti 1.0 goals

[headupinclouds]

Goal: Support real time face detection and landmark estimation tuned for mobile devices w/ an initial simple model based relative eye gaze estimation feature. The target is an SDK size of 1 MB, and the total asset size should be < 4MB (int at least one configuration). OpenGL ES shader processing and NEON optimization will be used as much as possible to support frame rate performance. This will provide an initial framework for further improvements. Appearance based CNN regression approaches (i.e., training head pose normalized eye crops with ground truth gaze positions) that are required for state of the art gaze performance will be added later.

Adopt a high level guiding principle of minimizing build times for the full architecture set builds in each release: 5 iOS (Universal) + 4 android (armv7, arm64, x86, x86-64) + OS X (x86-64) + Linux (x86-64) + MSVC (x86-64). This can be time consuming. Dependencies should be eliminated where possible, and hunter package management w/ binary cache will be used to reduce build times for compiled libraries as needed. The two main culprits are: OpenCV, and Boost (and Qt is a 3rd for full development application builds).

Boost will be phased out with a goal of replacing current boost functionality with compact header only libraries: cereal for serialization, thread-pool-cpp for thread pools, nlohmann-json for json parsing, etc.

Support for OpenCV will continue, but release builds will build only the required functions needed to support the final SDK (this is current a very small list) . Having some notion of ABI toolchain compatibility would seem to minimize rebuilds, with Xcode being a main culprit. ~~~Frequent updates and small windows of compatible { OS X , Xcode , iOS } versions can force frequent upgrades. Upgrading an iOS version can force an upgrade to Xcode and in some cases that in turn can force an update to OS X. In practice it seems a polly iOS toolchain upgrade is needed fairly often. For pure C++ libraries it is unlikely that this has any impact on ABI compatibility. In cases where this is known (or can be detected (hard?)), it should be possible to use ios-10-0 libraries with an ios-10-1 toolchain. This most likely introduces a great deal of complexity, but I'd like to understand feasibility of this model a little better. If this end-to-end build time is fast enough with the above mentioned chnages, then there may not be a need for this (if it is is even possible).~~~. [Update: <- Out of scope for 1.0. Upgraded HW has improved end-to-end build times and keeping CI toolchains in sync w/ supported build types should leverage the cache better].

#88 and #63 would go a long way towards simplifying project structure (clunky alternate submodule and hunter_add_package() branching), which is most needed in cases where the host project and dependency are both under active development.

• consider hard C++14 requirement for consistent platform behavior: thread_local, lambda initialization, full libc++, etc

• android ndk 14 w/ -c++14 (may include -flto improvements)
• xcode avaialability? (get exact version)
• gcc version ? (get exact version)
• llvm clang ? (need exact version)

• hard requirement for >= opengl es 2.0 graphics
• drop boost support (functionality will be migrated to header only libraries)
• build minimal set of function required for OpenCV
• investigate polly toolchain ABI compatibility for more cache hits
• 3rd parties enhancement #88, Option for switching between submodules/hunter packages #63 : use hunter git submodule support to simplify concurrent package development: project(drishti) has hunter_add_package(ogles_gpgpu) and both packages are under active development
• Cascaded pose regression approaches (ellipse + point model) should leverage the richer ACF channel features, which may result in more compact regression trees, currently normalized pixel differences are used
• Cascaded pose regression models (2d point) should be updated to use variable rate lossy compression for sequential "time series" point data
• A consistent multi-channel cascaded pose regression framework should be added to unify the ellipse + 2d point shape approaches