Performance Issue with VideoCaptureClient in SDK when Capturing 1920x1080 Video from Axis P3255-LVE

[ugrulas]

I'm encountering a significant performance issue when capturing video frames from an Axis P3255-LVE camera using the newer version of the SDK at a resolution of 1920x1080. When using OpenCV for frame capture, I am only able to achieve around 20 frames per second, which is significantly lower than the expected 30 frames per second that I obtain using an older opencv-image-capture-cpp example. Furthermore, with the new version of SDK's VideoCaptureClient, it takes approximately 150ms to read a single frame. For my application I need to process all the frames.

[github-actions[bot]]

This automatically generated reply acts as a friendly reminder.

Answers to your questions will most often come from the community, from developers like yourself. You will, from time to time, find that Axis employees answers some of the questions, but this is not a guarantee. Think of the discussion forum as a complement to other support channels, not a replacement to any of them.

If your question remains unanswered for a period of time, please revisit it to see whether it can be improved by following the guidelines listed in Axis support guidelines.

[vivekatoffice]

Hi @ugrulas,
The vdostream example only have values for 640x360:
format-h264 h264, 640x360, 30 fps

I tested and found this for 1940x1080, around 20 fps.

My Input:

The decoding of the video depends on hardware and resources available in Axis devices. The scene also plays an important role here. Which two examples you are comparing here?

[ugrulas]

For c++ example, I used opencv-image-capture-cpp which is available on previous release computer-vision-sdk-examples (1.12). That example is able to capture 30 frames per second. When I run something similar on python using sdk version 1.12. I could only get 20 fps. In the latest version of the sdk which uses GPRC/ACAP Runtime to capture video frames. Capturing a single frames takes 150ms . For python the below code should reproduce the results

import os
import time
import cv2
from tf_proto_utils import InferenceClient

class Detector:
    def __init__(self):
        self.first_frame = True
        self.grpc_host = os.environ['INFERENCE_HOST']
        self.setup_camera()
        self.read_environment()
    
    def setup_camera(self):
        self.camera = cv2.VideoCapture(1)
        self.camera.set(cv2.CAP_PROP_FPS, 30)
        self.camera.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
        self.camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
        self.camera.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"RGB3"))


    def read_environment(self):
        self.detection_threshold = float(os.environ.get('DETECTION_THRESHOLD', 0.3))
        self.inference_client = InferenceClient(self.grpc_host)
        self.model_path = os.environ['MODEL_PATH']
        print('Model Path:', self.model_path)

    def get_frame(self):
        start_time = time.time()
        frame_count = 0
        
        for _ in range(360):
            success, frame = self.camera.read()
            if success:
                frame_count += 1
            else:
                print('Error reading frame')

        end_time = time.time()
        print('Frame count:', frame_count)
        print('Frame dimensions:', frame.shape)
        print('Total time:', end_time - start_time)
        
        return True, frame

if __name__ == '__main__':
    detector = Detector()
    while True:
        frame_available, current_frame = detector.get_frame()

For the latest version you can reproduce by replacing get_frame function (https://github.com/AxisCommunications/acap-computer-vision-sdk-examples/blob/main/pose-estimator-with-flask/app/detector_with_flask.py#L145 with

def get_frame(self):
        tic = time.time()
        for i in range(360):
            frame = self.capture_client.get_frame()

        toc = time.time()
        print("Time to get 360 frame: ", toc - tic)
        
        # Cut the central area of the frame of a 1920x1080 to make it square
        frame = frame[:, 420:-420, :]
        small_frame = cv2.resize(frame, (192, 192), interpolation=cv2.INTER_AREA)
        success, key_points, confidences, = self.detect(small_frame)
        if not success:
            return False, 0

        image = self.draw_prediction(frame, key_points, confidences)
        return True, image

and set stream_framerate to 30

[ugrulas]

@vivekatoffice Do you have any suggestions ?

[vivekatoffice]

Hi @ugrulas ,
I have not tested the Python example recently but maybe this is because of the Python programming language itself?

Reference

[ugrulas]

Another problem is in the latest version that utilizes GRPC/ACAP Runtime to capture video frames, the process is significantly slower than the earlier implementation using OpenCV.