Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development branch not building with Cuda #3281

Open
mewescott opened this issue Dec 17, 2024 · 12 comments
Open

Development branch not building with Cuda #3281

mewescott opened this issue Dec 17, 2024 · 12 comments
Labels

Comments

@mewescott
Copy link

Required Info
Camera Model D455
Firmware Version 5.16.0.1
Operating System & Version Ubuntu 20
Kernel Version (Linux Only) 5.15.136-tegra
Platform NVIDIA Jetson ORIN NX
Librealsense SDK Version 2.56.0 on development branch
Language python
Segment PC
ROS Distro Humble
RealSense ROS Wrapper Version development branch

Issue Description

I just reinstalled the development branch of both the SDK and wrapper and used this command to build:
colcon build --cmake-args -DBUILD_WITH_CUDA=ON

I am trying to optimize everything to run with GPU support. It failed to build and here is what I saw:

--- stderr: realsense2_camera_msgs
CMake Warning:
Manually-specified variables were not used by the project:

BUILD_WITH_CUDA

Finished <<< realsense2_camera_msgs [12.8s]
Starting >>> realsense2_camera
Starting >>> realsense2_description
--- stderr: realsense2_description
CMake Warning:
Manually-specified variables were not used by the project:

BUILD_WITH_CUDA

Finished <<< realsense2_description [1.11s]
[Processing: realsense2_camera]
--- stderr: realsense2_camera
CMake Warning (dev) at /usr/share/cmake-3.22/Modules/FindCUDA.cmake:723 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.

For compatibility with older versions of CMake, option is clearing the
normal variable 'CUDA_PROPAGATE_HOST_FLAGS'.
Call Stack (most recent call first):
/home/sentry3/opencv-4.10.1/lib/cmake/opencv4/OpenCVConfig.cmake:86 (find_package)
/home/sentry3/opencv-4.10.1/lib/cmake/opencv4/OpenCVConfig.cmake:108 (find_host_package)
/opt/ros/humble/share/cv_bridge/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
/opt/ros/humble/share/cv_bridge/cmake/cv_bridgeConfig.cmake:41 (include)
CMakeLists.txt:107 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning:
Manually-specified variables were not used by the project:

BUILD_WITH_CUDA

/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/rs_node_setup.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::CalibConfigReadService(realsense2_camera_msgs::srv::CalibConfigRead_Request_<std::allocator >::SharedPtr, realsense2_camera_msgs::srv::CalibConfigRead_Response_<std::allocator >::SharedPtr)’:
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/rs_node_setup.cpp:601:68: error: ‘class rs2::auto_calibrated_device’ has no member named ‘get_calibration_config’; did you mean ‘get_calibration_table’?
601 | res->calib_config = dev.asrs2::auto_calibrated_device().get_calibration_config();
| ^~~~~~~~~~~~~~~~~~~~~~
| get_calibration_table
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/rs_node_setup.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::CalibConfigWriteService(realsense2_camera_msgs::srv::CalibConfigWrite_Request
<std::allocator >::SharedPtr, realsense2_camera_msgs::srv::CalibConfigWrite_Response_<std::allocator >::SharedPtr)’:
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/rs_node_setup.cpp:615:48: error: ‘class rs2::auto_calibrated_device’ has no member named ‘set_calibration_config’; did you mean ‘set_calibration_table’?
615 | _dev.asrs2::auto_calibrated_device().set_calibration_config(req->calib_config);
| ^~~~~~~~~~~~~~~~~~~~~~
| set_calibration_table
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::setupFilters()’:
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:230:76: error: ‘rotation_filter’ is not a member of ‘rs2’; did you mean ‘spatial_filter’?
230 | _filters.push_back(std::make_shared(std::make_sharedrs2::rotation_filter(), _parameters, _logger));
| ^~~~~~~~~~~~~~~
| spatial_filter
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:230:92: error: no matching function for call to ‘make_shared< >()’
230 | _filters.push_back(std::make_shared(std::make_sharedrs2::rotation_filter(), _parameters, _logger));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/c++/11/memory:77,
from /opt/ros/humble/include/librealsense2/hpp/rs_types.hpp:18,
from /opt/ros/humble/include/librealsense2/rs.hpp:8,
from /home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/../include/base_realsense_node.h:17,
from /home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:15:
/usr/include/c++/11/bits/shared_ptr.h:875:5: note: candidate: ‘template<class _Tp, class ... _Args> std::shared_ptr<_Tp> std::make_shared(_Args&& ...)’
875 | make_shared(_Args&&... __args)
| ^~~~~~~~~~~
/usr/include/c++/11/bits/shared_ptr.h:875:5: note: template argument deduction/substitution failed:
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:230:92: error: template argument 1 is invalid
230 | _filters.push_back(std::make_shared(std::make_sharedrs2::rotation_filter(), _parameters, _logger));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::imu_callback(rs2::frame)’:
/home/sentry3/ros2_ws/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:520:71: error: ‘class rs2::motion_frame’ has no member named ‘get_combined_motion_data’
520 | auto combined_motion_data = frame.asrs2::motion_frame().get_combined_motion_data();
| ^~~~~~~~~~~~~~~~~~~~~~~~
gmake[2]: *** [CMakeFiles/realsense2_camera.dir/build.make:90: CMakeFiles/realsense2_camera.dir/src/base_realsense_node.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [CMakeFiles/realsense2_camera.dir/build.make:118: CMakeFiles/realsense2_camera.dir/src/rs_node_setup.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:159: CMakeFiles/realsense2_camera.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

Failed <<< realsense2_camera [41.6s, exited with code 2]

Summary: 5 packages finished [55.0s]
1 package failed: realsense2_camera
4 packages had stderr output: realsense2_camera realsense2_camera_msgs realsense2_description ros2_trt_pose

I have had this combination of development branches build before. Since last weeks update they don't work anymore.

@mewescott
Copy link
Author

install_realense_on_jetson.txt

Here is the script I use to install your SDK onto my Jetson ORIN NX. Something is a little off. Now my rs-enumerate-devices is not working either. This is the same script I have reinstalled with for a few months now. The only change is I added a few flags onto the cmake command while troubleshooting this issue at ChatGPT's recommendation. Let me know if you see anything that needs changed to make this work. I have the development branch of both the SDK and wrapper installed now but still have the issue above when running the colcon build command on the ROS workspace.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 18, 2024

Hi @mewescott -DBUILD_WITH_CUDA=ON is not a parameter that is used when building the ROS wrapper from source code with colcon build. It is used when building the librealsense SDK from source code with CMake. For example:

cmake ../ -DFORCE_RSUSB_BACKEND=true -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true -DBUILD_WITH_CUDA=ON

The parameter for enabling GPU acceleration when building the ROS wrapper with colcon build is -DBUILD_ACCELERATE_GPU_WITH_GLSL=ON

colcon build --cmake-args '-DBUILD_ACCELERATE_GPU_WITH_GLSL=ON'

I would recommend only using AI help tools such as ChatGPT as a rough guide and confirming its instructions with other references, as these tools sometimes generate incorrect instructions.

@mewescott
Copy link
Author

Thanks for the advice. I tried it and got a different error in the ROS build instead:

CMake Error at /usr/local/lib/cmake/realsense2-gl/realsense2-glTargets.cmake:74 (message):
The imported target "realsense2-gl::realsense2-gl" references the file

 "/usr/local/lib/librealsense2-gl.so.2.55.1"

but this file does not exist. Possible reasons include:

  • The file was deleted, renamed, or moved to another location.

  • An install or uninstall procedure did not complete successfully.

  • The installation package was faulty and contained

    "/usr/local/lib/cmake/realsense2-gl/realsense2-glTargets.cmake"

but not all the files it references.

Call Stack (most recent call first):
/usr/local/lib/cmake/realsense2-gl/realsense2-glConfig.cmake:35 (include)
CMakeLists.txt:124 (find_package)

I replaced the cmake within my original SDK install file with the one in your post. I built the ROS workspace with the COLCON build you suggested. Both repo's are on the development branch. Is there a reason you switched from -DFORCE_LIBUVC=true to this -DFORCE_RSUSB_BACKEND=true in the cmake?

install_realense_on_jetson.txt

Looks like it should be referencing the 2.56.0 version instead in the cmake.
-- Installing: /usr/local/lib/librealsense2-gl.so.2.56.0

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 18, 2024

The DFORCE_RSUSB_BACKEND flag is the successor of the DFORCE_LIBUVC flag. There were originally a few different flags with LIBUVC in their name and so the RSUSB one was created to consolidate the three LIBUVC flags into one single flag for greater simplicity.

However, it usually does not matter if the LIBUVC or RSUSB one is used, as both methods are supported and work very similarly.

If you are building librealsense on Jetson from source code then it may be worth trying the JetsonHacks website's buildLibrealsense.sh build script instead of your install_realsense_on_jetson script if you have not tried it already.

https://github.com/JetsonHacksNano/installLibrealsense?tab=readme-ov-file#buildlibrealsensesh

https://github.com/JetsonHacksNano/installLibrealsense/blob/master/buildLibrealsense.sh

@mewescott
Copy link
Author

I am running the install again after cleaning everything realsense related off this machine. I am hoping it is a residual issue since I did have 2.55.1 installed before. I have installed this several times and had it running with my script switching back and forth between different repo branches. Something has changed that is keeping it from working now. I can't even get the master branches to work now. I am trying to get the development branches working to use the rotation filter they just introduced. I am all but convinced it is something they have changed recently in the repo but I have been working on getting trt_pose installed too so it may have interfered.

@mewescott
Copy link
Author

Cleaning up the environment and reinstalling got me back to the original issue. I used your colcon command:

colcon build --cmake-args '-DBUILD_ACCELERATE_GPU_WITH_GLSL=ON'

both branches are on the development branches:

SDK:
sentry3@ubuntu:~/librealsense_build/librealsense$ git status
On branch development
Your branch is up to date with 'origin/development'.

nothing to commit, working tree clean

ROS wrapper:
sentry3@ubuntu:~/ros2_ws/src/realsense-ros$ git status
On branch ros2-development
Your branch is up to date with 'origin/ros2-development'.

nothing to commit, working tree clean

I still got this feedback:

/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::setupFilters()’:
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:230:76: error: ‘rotation_filter’ is not a member of ‘rs2’; did you mean ‘spatial_filter’?
230 | _filters.push_back(std::make_shared(std::make_sharedrs2::rotation_filter(), _parameters, _logger));
| ^~~~~~~~~~~~~~~
| spatial_filter
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:230:92: error: no matching function for call to ‘make_shared< >()’
230 | _filters.push_back(std::make_shared(std::make_sharedrs2::rotation_filter(), _parameters, _logger));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/c++/11/memory:77,
from /opt/ros/humble/include/librealsense2/hpp/rs_types.hpp:18,
from /opt/ros/humble/include/librealsense2/rs.hpp:8,
from /home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/../include/base_realsense_node.h:17,
from /home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:15:
/usr/include/c++/11/bits/shared_ptr.h:875:5: note: candidate: ‘template<class _Tp, class ... _Args> std::shared_ptr<_Tp> std::make_shared(_Args&& ...)’
875 | make_shared(_Args&&... __args)
| ^~~~~~~~~~~
/usr/include/c++/11/bits/shared_ptr.h:875:5: note: template argument deduction/substitution failed:
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:230:92: error: template argument 1 is invalid
230 | filters.push_back(std::make_shared(std::make_sharedrs2::rotation_filter(), parameters, logger));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::imu_callback(rs2::frame)’:
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/base_realsense_node.cpp:520:71: error: ‘class rs2::motion_frame’ has no member named ‘get_combined_motion_data’
520 | auto combined_motion_data = frame.asrs2::motion_frame().get_combined_motion_data();
| ^~~~~~~~~~~~~~~~~~~~~~~~
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/rs_node_setup.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::CalibConfigReadService(realsense2_camera_msgs::srv::CalibConfigRead_Request
<std::allocator >::SharedPtr, realsense2_camera_msgs::srv::CalibConfigRead_Response
<std::allocator >::SharedPtr)’:
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/rs_node_setup.cpp:601:68: error: ‘class rs2::auto_calibrated_device’ has no member named ‘get_calibration_config’; did you mean ‘get_calibration_table’?
601 | res->calib_config = dev.asrs2::auto_calibrated_device().get_calibration_config();
| ^~~~~~~~~~~~~~~~~~~~~~
| get_calibration_table
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/rs_node_setup.cpp: In member function ‘void realsense2_camera::BaseRealSenseNode::CalibConfigWriteService(realsense2_camera_msgs::srv::CalibConfigWrite_Request
<std::allocator >::SharedPtr, realsense2_camera_msgs::srv::CalibConfigWrite_Response
<std::allocator >::SharedPtr)’:
/home/sentry3/ros2_ws/src/realsense-ros/realsense2_camera/src/rs_node_setup.cpp:615:48: error: ‘class rs2::auto_calibrated_device’ has no member named ‘set_calibration_config’; did you mean ‘set_calibration_table’?
615 | _dev.asrs2::auto_calibrated_device().set_calibration_config(req->calib_config);
| ^~~~~~~~~~~~~~~~~~~~~~
| set_calibration_table
gmake[2]: *** [CMakeFiles/realsense2_camera.dir/build.make:90: CMakeFiles/realsense2_camera.dir/src/base_realsense_node.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [CMakeFiles/realsense2_camera.dir/build.make:118: CMakeFiles/realsense2_camera.dir/src/rs_node_setup.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:159: CMakeFiles/realsense2_camera.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

Failed <<< realsense2_camera [47.3s, exited with code 2]

Summary: 5 packages finished [1min 2s]
1 package failed: realsense2_camera
4 packages had stderr output: realsense2_camera realsense2_camera_msgs realsense2_description ros2_trt_pose

@mewescott
Copy link
Author

I just noticed I had another install of the SDK installed by apt:

sentry3@ubuntu:~/ros2_ws$ dpkg -l | grep realsense
apt list --installed | grep ros
ii ros-humble-librealsense2 2.55.1-1jammy.20241126.021057 arm64 Library for controlling and capturing data from the Intel(R) RealSense(TM) D400 devices.

It finally built after removing that.

@MartyG-RealSense
Copy link
Collaborator

It's excellent to hear that you succeeded after removing the other apt install of the SDK. Thanks very much for the update!

@mewescott
Copy link
Author

When I run a rosdep update it reinstalls ros-humble-librealsense2 version 2.55.1. There has to be something in the repo for the wrapper doing this. Can you guys fix that so it pulls the right version on the dev branch?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 19, 2024

Are you using a pointcloud or depth-color alignment in ROS, please? Enabling librealsense's CUDA support only accelerates those two particular features, plus RGB color conversion. So if you are not using pointcloud or alignment then having CUDA enabled in the librealsense build will not make a beneficial difference.

It is my understanding that a major (non-beta) release of librealsense is planned by the end of 2024. So if you can wait til then, the rosdep update problem should resolve itself as the new version will contain the SDK updates currently on the development branch, and it should update to the next major version after 2.55.1.

@mewescott
Copy link
Author

My application is human pose detection in 3D. I am trying the google hri fullbody repo now which I believe uses the depth and color image. I also need to detect aruco tags.

@MartyG-RealSense
Copy link
Collaborator

Pose applications do use depth and color data, though I have not previously seen one that uses pointclouds or aligned images.

pal-robotics have an aruco_ros Aruco marker detecter at https://github.com/pal-robotics/aruco_ros that is compatible with ROS2 Humble.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants