Skip to content

ROS 2 Hardware Acceleration Working Group community governance model & list of projects

Notifications You must be signed in to change notification settings

ros-acceleration/community

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

49 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ROS 2 Hardware Acceleration Working Group

This document defines the scope and governance of the Working Group (WG). The rationale behind some decisions is further justified in REP-2008: ROS 2 Hardware Acceleration Architecture and Conventions.

Year Objective
21 ✅ 1) Design tools and conventions to seamlessly integrate acceleration kernels and related embedded binaries into the ROS 2 computational graphs leveraging its existing build system (ament_acceleration extensions) 1, meta build tools (colcon-acceleration extension) and a new firmware layer (acceleration_firmware) 2.
21 ✅ 2) Provide reference examples and blueprints for acceleration architectures used in ROS 2 and Gazebo.
21 ✅ 3) REP-2008 (ROS 2 Hardware Acceleration Architecture and Conventions) first draft proposed PR
22 ✅ 4) Facilitate testing environments that allow to benchmark accelerators with special focus on power consumption and time spent on computations (see HAWG benchmarking approach, community#9, tracetools_acceleration, ros2_kria)
22 ✅ 5) Survey the community interests on acceleration for ROS 2 and Gazebo (see discourse announcement, survey, result).
22 ✅ 6) Produce demonstrators with robot components, real robots and fleets that include acceleration to meet their targets (see acceleration_examples).
22 ✅ 7) Acceleration of complete ROS 2 computational graphs #20
22 3 8) Merge first hardware accelerators (kernels) into upstream packages (candidate: image_pipeline, see image_pipeline instrumented at #717 )
22 ✅ 9) Documentation and a "methodology to hardware accelerate a ROS 2 package" (see #20)
22 4 10) Organize workshops on robotics and ROS 2 Hardware Acceleration
22 🚧 11) Robotic Processing Unit, first demonstrators #25
22 ✅ 12) REP-2008 (ROS 2 Hardware Acceleration Architecture and Conventions) accepted as an official standard
22 ✅ 13) REP-2014 (Benchmarking performance in ROS 2) first draft proposed PR
23 ✅ 11) REP-2014 (Benchmarking performance in ROS 2), PR finalized and submitted. Unfortunately it got rejected without technical argumentation ( 🔥 heated discussion)
23 ✅ 14) More vendor support. Intel Quartus and oneAPI ecosystems enabled with the ROS 2 Hardware Acceleration Stack 5
23 ❌ 15) Robotics MCU #31 milestone 1 6 Instead, worked on an RTL-native FPGA-based ROS 2 implementation for High-Speed Networking: ROBOTCORE ROS 2
23 ✅ 16) RobotPerf benchmarks
23 🚧 17) Robotic Processing Unit, first demonstrators #25
24 🚧 17) Robotic Processing Unit, first demonstrators #25
24 🚧 18) New ROS-related network accelerators that build upon the FPGA-native ROS 2 message-passing infrastructure
24 🚧 19) Evolve RobotPerf and expand it. Include new benchmark categories and other communities

The ROS 2 Hardware Acceleration Stack

The ROS 2 Hardware Acceleration Stack is a series of extensions to ROS 2 which allow to leverage hardware acceleration and create custom compute architectures providing a faster ROS 2 execution and a timing-safe event-driven programming interface. The stack is composed of 4 key elements and aims to be hardware and vendor-agnostic:

# Standard/convention Title Rationale
0 REP 2014 Benchmarking performance in ROS 2 Describes some principles and guidelines for benchmarking performance in ROS 2
1 REP 2009 Type Negotiation Feature Allow ROS 2 Nodes to dynamically negotiate the message types used by publishers and subscriptions, as well adaptively modifying the behavior of publisher and subscriptions to align with accelerators
2 REP 2008 ROS 2 Hardware Acceleration Architecture and Conventions Architectural pillars and conventions required to introduce hardware acceleration in ROS 2 in a scalable and technology-agnostic manner. Presents one interface for all hardware acceleration vendors.
3 REP 2007 Type Adaptation Feature An extension to rclcpp that will make it easier to convert between ROS types and custom, user-defined types for Topics, Services, and Actions.
4 REP 2000 ROS 2 Releases and Target Platforms Production-grade multi-platform ROS support with Yocto

The ROS 2 Hardware Acceleration Working Group contributes and maintains open source implementations of various components of the stack above. A complete implementation of the ROS 2 Hardware Acceleration Stack including professional support, documentation, examples as well as reference designs is available at Acceleration Robotics' ROBOTCORE™. Other solutions that implement part of the stack include AMD's KRS (the Kria Robotics Stack), or NVIDIA's NITROS (NVIDIA Isaac Transport for ROS).

Reference hardware platforms

Official

The following boards are recommended and actively used for development:

Board Picture Description Firmware
Robotic Processing Unit and ROBOTCORE® A robot-specific processing unit specialized in ROS 2 computations. Features 16x CPUs, a GPU and an FPGA. Maps ROS computational graphs to the various compute units efficiently obtaining an increased throughput, lower latency, less power consumption, and additional real-time capabilities. reach out for early access
Kria KR260 Robotics Starter Kit The Kria™ KR260 robotics starter kit is built for robotics and industrial applications, complete with high performance interfaces and native ROS 2 support for ease of development by roboticists and software developers. acceleration_firmware_kr260
Kria KV260 Vision AI Starter Kit The Kria™ KV260 starter kit is a development platform for the K26, the first adaptive Single Board Computer. KV260 offers a compact board for edge vision and robotics applications. acceleration_firmware_kv260
Intel Agilex® 7 FPGA F-Series The Intel Agilex® 7 FPGA and SoC FPGA F-Series are general-purpose FPGAs for a wide range of applications across many markets. For robotics and industrial applications, the Agilex® 7 FPGA F-Series provides a 2x better performance-per-watt vs 7 nm node competitors7, a 40% lower total power consumption when compared to Intel® Stratix® 10 FPGAs and 38 TFLOPS with FP16 configuration8.

Unofficial

The following list includes boards that have been validated and have unofficial community support. No guarantees provided since no CI jobs are run on on these boards.

Community supported boards
Board Picture Description Firmware
Xilinx Zynq UltraScale+ MPSoC ZCU102 Evaluation Kit The ZCU102 Evaluation Kit enables designers to jumpstart designs for automotive, industrial, video, and communications applications. This kit features a Zynq® UltraScale+™ MPSoC with a quad-core Arm® Cortex®-A53, dual-core Cortex-R5F real-time processors, and a Mali™-400 MP2 graphics processing unit acceleration_firmware_zcu102
AVNET Ultra96-V2 The Ultra96-V2 is an Arm-based, Xilinx Zynq UltraScale+™ MPSoC development board based on the Linaro 96Boards Consumer Edition (CE) specification and designed with a certified radio module from Microchip providing Wi-Fi and Bluetooth. All components are updated to allow industrial temperature grade options. Additional power control and monitoring will be possible with the included Infineon PMICs. acceleration_firmware_ultra96v2
Zynq UltraScale+ MPSoC ZCU104 Evaluation Kit The ZCU104 Evaluation Kit enables designers to jumpstart designs for embedded vision applications such as surveillance, Advanced Driver Assisted Systems (ADAS), machine vision, Augmented Reality (AR), drones and medical imaging. This kit features a Zynq® UltraScale+™ MPSoC EV device with video codec and supports many common peripherals and interfaces for embedded vision use case.
NVIDIA Jetson Nano NVIDIA Jetson Nano The Jetson Nano™ Developer Kit is a small, powerful computer that lets you run multiple neural networks in parallel for applications like image classification, object detection, segmentation, and speech processing. All in an easy-to-use platform that runs in as little as 5 watts. acceleration_firmware_jetson_nano
NVIDIA Jetson AGX Orin NVIDIA Jetson AGX Orin The Jetson AGX Orin Developer Kit is a powerful AI computer with up to 275 Trillion Operation per Second (TOPS). reach out

Adding your board

The recipe for adding a new board to the community is as follows (reach out 📧 if you need help):

  1. Check out the Initial draft of REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions
  2. Create your own firmware repository (e.g. acceleration_firmware_Ultra96V2) for the corresponding board (see acceleration_firmware_kv260 for an example)
  3. Once the firmware repo is finalized, assess the capabilities of the hardware according to REP-2008 and create a table in the README.md that argues about it (see example here)
  4. Submit a PR to ros-acceleration/community to add your board to the community, with the corresponding support level according to REP-2008

Subprojects

This Working Group owns and maintains the following Subprojects. Its meetings and membership are largely focused on the direction, design, and work on the projects.

Subproject List

The following subprojects are owned by the Working Group:

Core Architecture

  • ament_acceleration: CMake macros and utilities to include hardware acceleration in the ROS 2 build system (ament) and its development flows.
  • ament_vitis: CMake macros and utilities to include Vitis platform into the ROS 2 build system (ament) and its development flows.
  • colcon-hardware-acceleration: An extension for colcon-core to include Hardware Acceleration capabilities.
  • ros2acceleration: acceleration command for ROS 2 command line tools.
  • tracetools_acceleration: LTTng tracing provider wrapper for ROS 2 packages in the Hardware Acceleration Working Group.
  • acceleration_firmware: Base ROS 2 package for hardware acceleration firmware. Used to organize firmware dependencies across vendors.

Examples and utilities for hardware acceleration

  • acceleration_examples: ROS 2 package examples demonstrating the use of hardware acceleration.
  • adaptive_component: A composable container for Adaptive ROS 2 Node computations. Select between FPGA, CPU or GPU at run-time.
  • image_pipeline: A fork for the ros-perception image_pipeline package that includes hardware acceleration capabilities for demonstrations purposes.

Robotic Processing Unit (RPU)

RobotPerf benchmarks

Standards for subprojects

Subprojects must meet the following criteria (and the WG agrees to maintain them upon adoption).

  • Build passes against ROS 2 Humble
  • If packages are part of nightly builds on the ROS build farm, there are no reported warnings or test failures
  • Issues and pull requests receive prompt responses
  • Releases go out regularly when bugfixes or new features are introduced
  • The backlog is maintained, avoiding longstanding stale issues

Adding new subprojects

To request introduction of a new subproject, add a list item to the "Subprojects" section and open a Pull Request to this repository, following the default Pull Request Template to populate the text of the PR.

PR will be merged on unanimous approval from Approvers.

Subproject changes

Modify the relevant list item in the "Subprojects" section and open a Pull Request to this repository, following the default Pull Request Template to populate the text of the PR.

PR will be merged on unanimous approval from Approvers.

Deprecating subprojects

Projects cease to be useful, or the WG can decide it is no longer in their interest to maintain. We do not commit to maintaining every subproject in perpetuity.

To suggest removal of a subproject, remove the relevant list item in the "Subprojects" section and open a Pull Request in this repository, following instructions in the Pull Request Template to populate the text of the PR.

PR will be merged on unanimous approval from Approvers. If the repositories of the subproject are under the WG's GitHub organization, they will be transferred out of the organization or deleted at this time.

Governance

Meetings

Except for vacation periods and other exceptions, regular WG Meeting will generally happen once a month or more often. Meetings are announced in ROS Discourse and minutes are kept.

Communication Channels

Backlog Management

Backlog management is performed using Github Projects. So far:

Membership, Roles and Organization Management

Working Group members may act in one or more of the following roles:

  • Member
    • Prerequisite: Attend at least one out of the last three Working Group meetings
    • Responsible for triaging issues
  • Reviewer
    • All reviewers are members
    • Prerequisite: Proven track record of high-quality reviews to WG subprojects
    • Responsible for reviewing pull requests
  • Approver
    • All approvers are reviewers
    • Prerequisite: Proven track record of high-quality contributions and reviews to WG subprojects
    • Responsible for approving and merging pull requests
    • Responsible for vetting and accepting new projects into the Working Group
  • Lead
    • Responsible for organizing and moderating working group meetings
    • Responsible for posting meeting materials (minutes, recordings, etc.)
    • Responsible for breaking ties

To become a member or change role, create an issue in this repository using the appropriate issue template. Such applications are accepted upon unanimous agreement from Approvers, and are typically based on the applicant's history with the subprojects of the Working Group.

Modifying this governance document

Changes to this document will be made via Pull Request. The PR will be merged if a majority of the Approvers agree.

References

Footnotes

  1. See ament_vitis

  2. See acceleration_firmware_kv260 for an exemplary vendor extension of the acceleration_firmware package

  3. Did not get buy-in from maintainers. Stopped the effort.

  4. Discarded for ROSCon and IROS. Lots of work.

  5. Reach out for more information about Intel's enablement extensions of the ROS build system and build tools.

  6. After months of work, given preliminary results and expectations derived from those, we decided to pivot elsewhere and wrap the Robotics MCU project. Shortly, the performance obtained in RISC-V soft-core based ROS 2 implementations explored was not bringing much value on top of existing CPU implementations of ROS 2. Instead, we invested resources in exploring an FPGA-based ROS 2 implementation for High-Speed Networking.

  7. Performance varies by use, configuration and other factors. Learn more at www.intel.com/PerformanceIndex​. Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available ​updates. See backup for configuration details. No product or component can be absolutely secure. Your costs and results may vary.

  8. With FP16 configuration based on current estimates

About

ROS 2 Hardware Acceleration Working Group community governance model & list of projects

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published