Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#Launch the graph in the KV260 CPU failing # #52

Open
surender009 opened this issue Mar 22, 2022 · 12 comments
Open

#Launch the graph in the KV260 CPU failing # #52

surender009 opened this issue Mar 22, 2022 · 12 comments

Comments

@surender009
Copy link
Contributor

surender009 commented Mar 22, 2022

i have built acceleration kernels by following Accelerating a perception computational graph

i have seen below error while launching perception nodes on KV260

xilinx-k26-som-2021_2:~$ ros2 launch perception_2nodes trace_rectify_resize.launch.py
[INFO] [launch]: All log files can be found below /home/petalinux/.ros/log/2018-03-09-07-25-33-773976-xilinx-k26-som-2021_2-1312
[INFO] [launch]: Default logging verbosity is set to INFO
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<LaunchService._process_one_event() done, defined at /usr/lib/python3.8/site-packages/launch/launch_service.py:226> exception=InvalidLaunchFileError('py')>
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/launch/launch_description_sources/any_launch_file_utilities.py", line 54, in get_launch_description_from_any_launch_file
return loader(launch_file_path)
File "/usr/lib/python3.8/site-packages/launch/launch_description_sources/python_launch_file_utilities.py", line 62, in get_launch_description_from_python_launch_file
launch_file_module = load_python_launch_file_as_module(python_launch_file_path)
File "/usr/lib/python3.8/site-packages/launch/launch_description_sources/python_launch_file_utilities.py", line 37, in load_python_launch_file_as_module
loader.exec_module(mod)
File "", line 783, in exec_module
File "", line 219, in _call_with_frames_removed
File "/krs_beta/share/perception_2nodes/launch/trace_rectify_resize.launch.py", line 29, in
from tracetools_launch.action import Trace
ModuleNotFoundError: No module named 'tracetools_launch'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/launch/launch_service.py", line 228, in _process_one_event
await self.__process_event(next_event)
File "/usr/lib/python3.8/site-packages/launch/launch_service.py", line 248, in __process_event
visit_all_entities_and_collect_futures(entity, self.__context))
File "/usr/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
File "/usr/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
File "/usr/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 38, in visit_all_entities_and_collect_futures
sub_entities = entity.visit(context)
File "/usr/lib/python3.8/site-packages/launch/action.py", line 108, in visit
return self.execute(context)
File "/usr/lib/python3.8/site-packages/launch/actions/include_launch_description.py", line 127, in execute
launch_description = self.__launch_description_source.get_launch_description(context)
File "/usr/lib/python3.8/site-packages/launch/launch_description_source.py", line 84, in get_launch_description
self._get_launch_description(self.__expanded_location)
File "/usr/lib/python3.8/site-packages/launch/launch_description_sources/any_launch_description_source.py", line 53, in _get_launch_description
return get_launch_description_from_any_launch_file(location)
File "/usr/lib/python3.8/site-packages/launch/launch_description_sources/any_launch_file_utilities.py", line 57, in get_launch_description_from_any_launch_file
raise InvalidLaunchFileError(extension, likely_errors=exceptions)
launch.invalid_launch_file_error.InvalidLaunchFileError: Caught exception when trying to load file of format [py]: No module named 'tracetools_launch'
xilinx-k26-som-2021_2:~$

@vmayoral
Copy link
Contributor

 from tracetools_launch.action import Trace
ModuleNotFoundError: No module named 'tracetools_launch'

You're missing the tracetools_launch package in your workspace, and that's complaining while running in embedded.

That package is part of https://gitlab.com/ros-tracing/ros2_tracing, which you should be fetching from https://github.com/Xilinx/KRS/blob/main/krs_rolling.repos#L28-L31. Make sure it's cross-compiled and deployed to your overlay workspace.

@surender009
Copy link
Contributor Author

i have added required tracing packages and tried to run on KV260 i have seen lttng exeption

xilinx-k26-som-2021_2:/home/petalinux# ros2 launch perception_2nodes trace_rectify_resize.launch.py
[INFO] [launch]: All log files can be found below /home/root/.ros/log/2018-03-09-05-45-08-170801-xilinx-k26-som-2021_2-1878
[INFO] [launch]: Default logging verbosity is set to INFO
[ERROR] [launch]: Caught exception in launch (see debug for traceback): lttng module not found, but still tried to use it
xilinx-k26-som-2021_2:/home/petalinux#

@vmayoral
Copy link
Contributor

[ERROR] [launch]: Caught exception in launch (see debug for traceback): lttng module not found, but still tried to use it

It appears you're missing the LTTng kernel modules. They shipped in 0.9.0 so unless you've done something funny with the Linux kernel, you should be getting them.

I'd debug this by:

  1. ensuring kernel modules are in the rootfs
  2. ensure they can be loaded
  3. ensure kernel modules are in the rootfs used to build the raw image (they certainly are in the raw SD card image I facilitated to @kevinkeryk and @jasvinderkhurana)

If you're still unable to continue after determining the cause of your issue with the 3 steps above, I'd encourage you to join the next call so that we can look at this deeper and debug it together.

@surender009
Copy link
Contributor Author

@vmayoral i checked the kv260 firmware 0.9.0 doesn't have lttng kernel modules in rootfs.cpio.gz

@vmayoral
Copy link
Contributor

vmayoral commented Mar 24, 2022

@vmayoral i checked the kv260 firmware 0.9.0 doesn't have lttng kernel modules in rootfs.cpio.gz

If that's the case then you'll need to build them using the Xilinx official BSP. I used the default vanilla kernel for KRS beta pre-release, so it should be pretty straightforward.

As a shortcut, can you check the image facilitated to @kevinkeryk and @jasvinderkhurana?

@surender009
Copy link
Contributor Author

yes, that image has kernel modules.

@vmayoral
Copy link
Contributor

vmayoral commented Mar 24, 2022

yes, that image has kernel modules

Then either build them yourself or get those from the raw SD card image. Also, a contribution to https://github.com/ros-acceleration/acceleration_firmware_kv260 with those modules included (if not there before) would be appreciated.

@surender009
Copy link
Contributor Author

ok, i will try to build them myself and let you know status.

@vmayoral
Copy link
Contributor

Edited the answer above @surender009. Feel free to take the fast track.

Then either build them yourself or get those from the raw SD card image.

@surender009
Copy link
Contributor Author

@vmayoral i have built lttng-modules from petalinux bsp and added in root file system and tried to run perception. but am getting same exception.

@vmayoral
Copy link
Contributor

@surender009, you need to be more specific for me to support you and appropriately provide in here dumps of your system and issues (please use Markdown syntax so that it's clear).

Things you may want to provide:

  • are you lttng kernel modules loaded? if not, why? have you ensured that kernel and modules match?
  • if so, have you tried launching lttng through its CLI (instead of using the ROS 2 connectors in the launch system)?

@jasvinderkhurana
Copy link
Collaborator

@surender009 , I assume by adding lttng in kernel modules, you resolved the problem, shall I close the issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants