Failing to publish compressed image and depth topic data on Nvidia AGX Orin #3105
Comments
|
Hi @nwissner-bdai May I please confirm that you are using the echo command below to access image_raw/compressed.
To access the compressed topics, the image-transport plugin must have been installed for Humble with the command below.
|
|
Hello @MartyG-RealSense , yes I am using |
|
@nwissner-bdai I have the same issue and the same version packages launching |
|
The errors @nwissner-bdai and @annb3 Which RealSense firmware version are your cameras using, please? When using librealsense SDK 2.54.1 the camera firmware should be 5.15.0.2 and when using SDK 2.54.2 it should be 5.15.1.0. Using firmware versions other than the recommended one for a particular librealsense version can cause errors. |
|
@MartyG-RealSense I have SDK 2.55.1 and firmware version 5.16.01 but I have always the problem: |
|
@annb3 The current ROS wrapper 4.54.1 is not designed for use with SDK 2.55.1 and firmware 5.16.0.1. There is not currently a wrapper for 2.55.1. I note though that @nwissner-bdai has been using 2.54.2 and still also experienced 'OutOfMemoryError'. Do the compressed topic echoing errors still occur if you use a low resolution and FPS of 640x480 and 15 FPS?
|
|
@MartyG-RealSense So the fw for the realsense camera I was using was not updated to 5.15.1.0, so I updated it and tested again to echo the compressed image topic data. Unfortunately still receiving the same errors. I also added the launch arguments you described and same errors. Do you recommend any other troubleshooting steps before I dig further into the problem? |
|
@nwissner-bdai I have also 2.55.1 and 5.16.0.1 and ROS wrapper works perfectly-
This solution works in my Jetson Orin. Let me know if you will solve in this way or will you find some other solutions.. |
|
Thanks so much @annb3 for sharing the difference that enabling the pointcloud filter made for you in your particular situation! Yes, please do let us know @nwissner-bdai if the above method works for you. Rebuilding the wrapper may not be a necessary step, as the launch file is independent from the wrapper files and is not affected by a rebuild. Instead of editing and saving the launch file, the launch command below may produce the same outcome.
|
|
@MartyG-RealSense I use the file inside an enourmous repository of concatenate launchers and so on. that's why I can't launch the camera in this way and I have to rebuild (need to modify general settings of the overall project and it is super fast). |
|
Thanks very much @annb3 for the clarification about your situation. |
|
@MartyG-RealSense Okay after doing some more testing, I have found that adding that additional launch argument you provided still produces the same errors I have been getting. Like @annb3, I too am working with an enormous repository made up of various research projects and am tasked with debugging this specific issue of publishing high resolution compressed image and depth data. My original testing involved launching a node from the repo which also spawned and: I am now going to test this standalone node on non agx-orin hardware (thinkpad) and will update with those results when I can. I'll be sure to double check versioning for the sdk, wrapper and firmware on the camera |
|
Thanks very much @nwissner-bdai for the detailed update. I look forward to your next report after further testing. Good luck! |
|
Hello @MartyG-RealSense , I have tested Librealsense SDK versions paired with their appropriate realsense-ros wrapper versions on both the AGX Orin and Lenovo Thinkpad platforms extensively, documenting the topic statistics with Lenovo ThinkpadSDK Software Version and Firmware Version
Successful Image and Compression Topics
Unsuccessful Image and Compression Topics with Error Logs
SDK Software Version and Firmware Version
Successful Image and Compression Topics
Unsuccessful Image and Compression Topics with Error Logs
SDK Software Version and Firmware Version
Successful Image and Compression Topics
Unsuccessful Image and Compression Topics with Error Logs
AGX OrinSDK Software Version and Firmware Version
Successful Image and Compression Topics
Unsuccessful Image and Compression Topics with Error Logs
SDK Software Version and Firmware Version
Successful Image and Compression Topics
Unsuccessful Image and Compression Topics with Error Logs
SDK Software Version and Firmware Version
Successful Image and Compression Topics
Unsuccessful Image and Compression Topics with Error Logs
Apologies if this information doesn't prove to be as useful as I had hoped, but it would be nice if the the community could take a special interest in this issue as it seems to be repeatable and prevalent to other users such as @annb3. If there is any other data you would recommend I collect on this issue I am happy to further investigate! |
|
Thank you very much for the detailed test results. It appears that for both the Lenovo and AGX, There was a past case at #1672 where a RealSense user also experienced low performance from |
|
@MartyG-RealSense Thank you for taking such consideration to look into this! I would say |
|
@MartyG-RealSense Oh I am working with ROS2 and not ROS1 for context. I do have the |
|
Thank you @nwissner-bdai - I will highlight this issue with high bandwidth usage to my Intel colleagues on the RealSense ROS team. Edit: the ROS team will attempt to look at this issue at a future date when there is the opportunity to work on it. |
|
@nwissner-bdai I can confirm that actually somehow I again receive the error: |
|
Hello @MartyG-RealSense , I was just wondering if the ROS team had made any progress on this issue that I could report back to my team as this is a blocking issue for them |
|
Hi @nwissner-bdai I will follow up with the RealSense ROS team about this issue. |
|
Hi @nwissner-bdai My Intel RealSense colleagues have officially created an internal ticket so that they can look at this issue with compression bandwidth at a future date when there is the opportunity to work on it. |
|
Sounds good, thank you for getting back to me @MartyG-RealSense |
|
If you are familiar with building from source (with Let me know if you need help building the code Thanks, |
|
Hi @nwissner-bdai Have you tried building from the private repo of @SamerKhshiboun using the link in the comment above, please? Thanks! |
|
@MartyG-RealSense I'll be taking over this issue from @nwissner-bdai. |
|
Thanks very much for the information, @gbrooks-bdai - good luck! |
|
Hello @MartyG-RealSense I wonder if there is some progress on this issue. I'm working with two platforms:
I'm manually building from the source code (to be able to enable the CUDA and GPU flags in both I'm not using the last version because the documentation is not clear about the versions I should use (see #3204 and #3205). I'm getting the error: in both platforms, as soon as I subscribe to the depth compressed topic. Is my combination of versions correct? Or can I install via apt with the GPU flags enabled for both the librealsense and realsense-ros libraries? Btw @SamerKhshiboun which versions of |
|
Hi @edgarcamilocamacho If you are using camera firmware driver 5.15.0.2 then librealsense 2.54.1 and ROS wrapper 4.54.1 will be the appropriate versions to use, and you are indeed using those versions. If you install from packages using the instructions on the installation_jetson.md page then CUDA support will be included in the packages and so will be automatically enabled without you having to set a flag. https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md When installing from packages with the distribution_linux.md instructions, CUDA support is not included in the packages. These packages should only be used on x86/x64 computers anyway, whilst the installation_jetson.md packages should be used for Nvidia Jetson. https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md If intalling the librealsense SDK from source code, a method that works well for both x86/x64 computers and Arm-based boards such as Jetson is to use the libuvc backend installation method at the link below, which has a pre-made build script called 'libuvc_installation.sh'. https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md When using the script with Jetson, edit line 46 to add the flag -DBUILD_WITH_CUDA=ON https://github.com/IntelRealSense/librealsense/blob/master/scripts/libuvc_installation.sh#L46 In regard to your question about the @SamerKhshiboun repo, I will defer to Samer about that question. |
|
Hi @edgarcamilocamacho Do you require further assistance with your problem, please? Thanks! |
|
Hi, Please ignore my last proposal for testing the private branch. I tested the latest RealsSense ROS Wrapper on my Ubuntu 24.04 with ROS2 Jazzy and with my private dynamic_subscriber_node that calculates the HZ (you can use it from here: https://github.com/SamerKhshiboun/dynamic_subscriber_node, please follow the readme with the examples) I tested these topics:
The results on all of them are 30.0 FPS. I think using a custom C++ subscriber is better than using the Please retry again with my custom C++ subscriber, or create your own, and update me. |
|
Amazing, we'll be testing in the next days and comment how it goes. |
|
Hi @edgarcamilocamacho Do you have an update about this case that you can provide, please? Thanks! |
Hello @MartyG-RealSense, the proposed method above doesn't seem to work. The steps i followed are as follows:
enable_color: true
enable_depth: true
enable_sync: true
rgb_camera.color_profile: 1920x1080x30 # results are the same with 640x480x30
rgb_camera.color_format: RGB8
publish_tf: true
initial_reset: true
pointcloud.enable: trueMoreover, after launching the realsense2_camera node, I got the following warning: My realsense environment is as follows:
I appreciate your assistance in this matter as well as tips I could try. Thank you. |
|
Hi @tiwaojo Which ROS2 branch are you using, please? The ROS2 repository of ffmpeg_image_transport says that it has been tested with ROS2 Humble and Iron. https://github.com/ros-misc-utilities/ffmpeg_image_transport Have you confirmed that |
|
My apologies, I was using ROS2 Humble and yes, the |
|
Thank you for the information. Are ffmpeg messages able to be published if the pointcloud filter is not enabled? |
|
Unfortunately no, disabling the pointcloud does not result in ffmpeg messages being published. |
|
@tiwaojo I note that you have used the flag -DBUILD_WITH_CUDA=true when installing librealsense. Can you confirm if you are using an Nvidia Jetson board as your computing device, please? AIf you are using a Jetson, because you get an OutOfMemory error when subscribing to the topic, have you tried reducing the load on the CPU by encoding in hevc_nvenc format using the |
|
Pardon the delay, we ran into some issues with setting up ffmpeg on the AGX and were troubleshooting.
We followed steps as documented here with supporting documents from nvidia and online to solve the problem. We had some success while using the ros2_v4l2_camera package with the ffmpeg transport by manually specifying the Moreover, we tried to play back a bag file to compress and decompress the messages using the ffmpeg_image_transport and image transport, but could not replicate the The opencv version i have on my jetson device is |
|
AS you are using a Jetson, can you confirm if you followed the instructions to install hardware accelerated encoding in ffmpeg_image transport, please? |
Yes, we followed the instructions in the link you shared to install the |
|
And reducing the resolution of the color stream makes no difference?
|
|
Hello, I can confirm that we can reproduce these issues under Jetpack 6.0 on a Jetson Orin AGX board with ROS 2 Humble. I have
This setup worked previously, properly receiving compressed images from the Which produces All of this is consistent with what other users have posted here. We haven't been using the This issue is consistent regardless of the profile selected for the color channel in our case. |
Thats Correct. |
|
#2858 is a previously reported case from 2023 of problems with using ffmpeg with RealSense ROS. At #2858 (comment) I suggested to the RealSense user in that case to try using the -map function of ffmpeg. Later in that case at #2858 (comment) my Intel RealSense colleagues affirmed that the advice to use the map function was a suitable answer and also suggested an alternative solution. |
|
Thank you. We will consider both options and report back with our findings once we I get the chance to. |
|
I believe to have found a workaround for:
This seems to point to some issue with either TL;DR; Set the RGB stream's encoding to BGR8 ( |
|
Thanks so much @agonzat for sharing your knowledge of what worked for you! |
|
I think the "BGR8" solution does not apply for the original issue, because it was about depth. I made the change just in clase and I still get: When I try to subscribe to the compressed depth |
|
Hey @agonzat, thanks for sharing your solution. I can confirm it also resolves my ffmpeg issue. As for the following error @edgarcamilocamacho
I was able to find a solution by setting the following image transport parameters in my config and subscribing to the .camera.depth.image_rect_raw.format: png
.camera.depth.image_rect_raw.compressed.format: png # can also be jpeg
.camera.depth.image_rect_raw.compressedDepth.format: pngI did discover that setting the Furthermore, I abandoned the |
|
Does anyone require further assistance with this case, please? Thanks! |
Issue Description
When running the realsense2_camera, the
camera/camera/color/image_rawandcamera/camera/depth/image_rect_rawtopics are echoing data just fine, but when trying to echo any data on the compressed topics, I am getting errors such as:When echoing
image_raw/compresseddata, and:When echoing
image_rect_raw/compresseddata.The text was updated successfully, but these errors were encountered: