Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize CPU usage - rotate image only if there is a standing up pose in the frame buffer #7

Open
ivelin opened this issue Feb 4, 2021 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@ivelin
Copy link

ivelin commented Feb 4, 2021

Looking at the ambianic edge logs in a real world usage, there is a constant stream of attempts to detect a pose in the original image and +/-90' rotations. This happens because most of the time there is no person in the camera view at all.

This is a suboptimal 3x use of CPU. Normally a single posenet pass on Raspberry Pi takes about 300ms (3fps). However after 2 rotations the total inference time goes up to 1-1.2sec (0.8-1fps). See log excerpt below:

ambianic-edge    | 2021-02-04 01:17:45 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1015.96 ms, 0.98 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:46 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1094.21 ms, 0.91 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:47 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1069.70 ms, 0.93 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:48 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1163.64 ms, 0.86 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:49 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1024.56 ms, 0.97 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:50 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1090.12 ms, 0.92 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:51 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1089.76 ms, 0.92 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:52 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1223.01 ms, 0.82 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:54 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1116.52 ms, 0.89 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:55 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1173.54 ms, 0.85 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:56 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1366.06 ms, 0.73 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:57 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1145.92 ms, 0.87 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:58 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1139.79 ms, 0.87 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:59 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1051.62 ms, 0.95 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:18:01 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1107.74 ms, 0.90 fps in pipeline area_watch
@ivelin
Copy link
Author

ivelin commented Feb 4, 2021

Thinking through this a bit, it seems like it only makes sense to try rotations if there is a buffered image frame with a standing pose.

The rotations are essentially there to look for people on the ground. It's a workaround to overcome the posenet+mobilnetv1 weakness in detecting non-vertical human poses.

However detecting a horizontal pose is only helpful if there is a previously saved vertical pose to compare it to.

Therefore I suggest implementing the following optimization that should bring the CPU usage down significantly.

# use to determine if there is a standing pose in a given frame
def standing_pose(sping_vector):
   return true if abs(angle_betwen(vertical_axis, spine_vector)) <= 90-fall_threshold_angle

# in the find_keypoints function replace the rotation pre-condition
# https://github.com/ambianic/ambianic-edge/blob/b55b4474ea718945970efb5e5da48587cc1f12d4/src/ambianic/pipeline/ai/fall_detect.py#L153

if pose_score < min_score:
  standing_pose_in_buffer = filter (lambda prev_frame: prev_frame.is_standing_pose, prev_frames)
  if standing_pose_in_buffer: 
    while pose_score < min_score and rotations:
  ...


This would drop CPU usage significantly (almost 60%), because rotations will be only attempted if there is a person detected in front of the camera and shortly after it is not detected which means that they are either not standing up or they are not visible.

Thoughts?

@ivelin
Copy link
Author

ivelin commented Feb 4, 2021

@bhavikapanara please take a look and share your comments on this optimization idea.

@ivelin ivelin added the enhancement New feature or request label Feb 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants