-
Notifications
You must be signed in to change notification settings - Fork 300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add zero-copy NVMM support with nvbufsurface (JetPack 5) #204
base: master
Are you sure you want to change the base?
Conversation
merge head repo
merge upstream
…image layout (cudaEglFrameTypePitch)
Thank you @poett1! I appreciate this a lot! I will need some time to test this on JetPack 4, 5, and 6 first, or |
@poett1 thank you, cherry-picking this cut my CPU usage in half. (I'm on 5.1.2) |
@catch-twenty-two thanks for checking out. Yes, reducing CPU load is one of the main benefits. Another benefit is that it drastically increases the possible frame rate you can feed into your application. I'm using four cameras with 720p/120 FPS in my example code. Without these changes, I reached around 40 FPS output, after applying the changes I was able to achieve the full 120 FPS. |
Just had some time today/yesterday and updated to the new API changes for nvbufsurface on JP5.
I checked this with IMX219 (raw) using Argus and See3CAM_CU135 (raw/jpeg) on JP5.1.2 (Orin NX). I hope someone can check this again with Jetpack 4.6. Currently, I do not have any devices running Jetpack below 5.1.2.
There are two main changes:
1. nvv4l2camerasrc:
I updated the code to use
nvv4l2camerasrc
with v4l2 cameras (that do not support Argus) ifinput_codec=RAW
on Jetson Devices.This will output a surface with NVBUF_LAYOUT_PITCH, which is transformed to NVBUF_LAYOUT_BLOCK_LINEAR in
gstBufferManager::Enqueue
(thus we need to link against nvbufsurftransform).Also, it seems like
eglFrame.planeCount=1
for this source. I commented out the check for this ingstBufferManager::Dequeue
to disabled warnings. If you have some other ideas for this, feel free to change it, but it seems to work fine.Sometimes
nvv4l2camerasrc
outputs bad frames (mostly green with a bar of snow). This also happens with gst-launch-1.0 on my Jetson. Reconnecting the USB camera again seems to solve it.This can be checked with:
video-viewer /dev/video1 --input-codec=raw
which will create the pipeline:nvv4l2camerasrc device=/dev/video1 do-timestamp=true ! video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1280, height=(int)720, framerate=30/1 ! appsink name=mysink sync=false
2. nvvidconv:
If
ENABLE_NVMM=ON
we will always usenvvidconv
to rotate the video now, even if the source does not output NVMM.nvvidconv
will output NVMM to appsink.This can be checked with:
video-viewer /dev/video1 --input-flip=rotate-180
which will create the pipeline:
v4l2src device=/dev/video1 do-timestamp=true ! image/jpeg, width=(int)1280, height=(int)720, framerate=60/1 ! jpegdec name=decoder ! video/x-raw ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM) ! appsink name=mysink sync=false
And with CSI Cameras:
video-viewer csi://0 --input-flip=rotate-180
the pipeline will be:
nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, framerate=30/1, format=(string)NV12 ! nvvidconv flip-method=0 ! video/x-raw(memory:NVMM) ! appsink name=mysink
I will be happy to help If any additional changes are necessary to get this into main.