CI status |
|
---|
CCWS
is a development environment for ROS, which integrates functionality of
traditional catkin
workspaces and CI pipelines in order to facilitate
(cross-)compilation, testing, linting, documetation and binary package
generation. It is intended to be used both as a CI/CD backbone and a working
environment for developers. Note that CCWS
is not intended to be a complete
solution, but rather a basis for development of a vendor-specific workflow.
CCWS
is meant to be ROS version agnostic, but is mainly tested with ROS1.
There are also some technical issues wth ROS2 packages that must be resolved,
e.g., ament/google_benchmark_vendor#17.
-
Build profiles -- sets of configurations for build process, e.g., cmake toolchain, colcon configuration, environment variables, etc. Profiles do not conflict with each other and can be used in parallel without using separate clones of the workspace and packages.
-
Execution profiles -- simple shell mixins that are intended to modify run time environment, e.g., execute nodes in
valgrind
, alter node crash handling, etc. -
A number of features implemented via build profiles:
-
Cross compilation to several common platforms.
-
Documentation generation for the whole workspace or selected packages using
doxygen
, similar to https://github.com/mikepurvis/catkin_tools_document. -
Linting with
clang-tidy
andscan_build
. -
Various static checks as in https://github.com/sscpac/statick, in particular:
cppcheck
catkin_lint
https://github.com/fkie/catkin_lintyamllint
shellcheck
-
Binary debian package generation.
-
-
Package template which demonstrates how to use some of the features.
-
The number of parallel jobs can be selected based on available RAM instead of CPU cores, since RAM is likely to be the limiting factor.
-
Based entirely on
make
and shell scripts. All scripts and configurations are kept in the workspace and easy to adjust for specific needs.
Profile configurations are located in ccws/profiles/build
, common
subdirectory contains default parameters, which can be overriden by specific
profiles:
- [default]
reldebug
-- default compiler, cmake build type isRelWithDebInfo
scan_build
-- static checks withscan_build
andclang-tidy
.clang-tidy
parameters are defined in cmake toolchain and must be enabled in packages as shown in package templateCMakeLists
. This profile also usesclang
compiler.thread_sanitizer
-- compilation with thread sanitizer.addr_undef_sanitizers
-- compilation with address and undefined behavior sanitizers.static_checkers
-- static checkers and their configuration.doxygen
-- doxygen and its configuration.cross_raspberry_pi
-- cross-compilation for Raspberry Pi.cross_jetson_xavier
-- cross-compilation for Jetson Xavier.cross_jetson_nano
-- cross-compilation for Jetson Nano.clangd
-- collects compilation commands for a givenBASE_BUILD_PROFILE
and generates clangd configuration file in the workspace root.
Execution profiles set environment variables that can be used in launch scripts
to alter run time behavior as demonstrated in
ccws/pkg_template/catkin/launch/bringup.launch
, currently available profiles
are:
common
-- a set of common ROS parameters, e.g.,ROS_HOME
, it is automatically included in binary packages.test
-- setsCCWS_NODE_CRASH_ACTION
variable so that nodes that respect it becomerequired
, i.e., termination of such nodes would result in crash of test scripts and can thus be easily detected.valgrind
-- setsCCWS_NODE_LAUNCH_PREFIX
tovalgrind
and some variables that control behavior ofvalgrind
.core_pattern
-- sets core pattern to save core files in the artifacts directory.address_sanitizer
-- helper foraddr_undef_sanitizers
profile.
Execution profiles have no effect on build process and are taken into account
in *test*
targets or debian packages. test
execution profile is always used
in tests and additional profiles can be provided with EXEC_PROFILE="<profile1> <profile2>"
. These targets load profiles using setup.bash
script located in
the root folder of CCWS
, which can also be used manually, e.g., source setup.bash [<build_profile> [<exec_profile1> ...]]
. Note that the setup script
always includes common
profile, and uses test
execution profile if no other
execution profiles are specified.
Dependencies can be installed using make bp_install_build BUILD_PROFILE=<profile>
, which is going to install the following tools and
profile specific dependencies:
colcon
yq
-- https://github.com/asherikov/wshandler dependencycmake
ccache
-- can be disabled in cmake toolchainswget
See .ccws/test_main.mk
for command usage hints.
- Override developer and vendor specific parameters by adding them to
make/config.mk
, available parameters can be found in the top section ofMakefile
. - Install dependencies using
make bp_install_build BUILD_PROFILE=<profile>
targets, cross compilation profiles would require some extra steps as described below. In some minimalistic environments you may need to run./ccws/scripts/bootstrap.sh
before usingbp_install_build
target in order to installmake
and other utils. - Clone packages in
src
subdirectory, or create new usingmake new PKG=<pkg>
.
make build PKG="<pkg>"
where<pkg>
is one or more space separated package names.make <pkg>
-- a shortcut formake build
, but<pkg>
can be a substring of package name. All packages matching the given substring will be built.- The number of jobs can be overriden with
JOBS=X
parameter. make build PKG=<pkg> BUILD_PROFILE=scan_build
overrides default profile.
- Source
setup.bash <profile>
to be able to use packages. Setup scripts generated bycolcon
can also be used directly, e.g.,install/<profile>/local_setup.sh
, but in this case some ofCCWS
functionality won't be available.
make test PKG=<pkg>
test withcolcon
, ormake wstest
to test all.make ctest PKG=<pkg>
bypasscolcon
and runctest
directly ormake wsctest
to test all.
make BUILD_PROFILE=doxygen
,firefox artifacts/doxygen/index.html
CCWS
takes a somewhat uncommon approach to binary package generation which is
a middle ground between traditional ROS (1 package = 1 deb) and docker
containers: all packages built in the workspace are packed together into a
single debian 'superpackage'. Unlike bloom
CCWS
generates binary packages
directly instead of generating source packages first.
Binary package generation is implemented as a build profile mixin that can be
overlayed over an arbitrary build profile: make <pkg> BUILD_PROFILE=deb BASE_BUILD_PROFILE=reldebug
.
CCWS
approach has a number of advantages:
-
Binary compatibility issues are minimized compared to traditional ROS approach:
-
no need to worry about compatibilities between multiple standalone binary packages and perform ABI checks;
-
if base ROS packages are included, it is also possible to avoid binary incompatibilities between syncs of the same ROS release (those actually happen).
-
-
Package repository management can be sloppier compared to ROS when it comes to tags, versions, git submodules, etc, e.g., there is no need to maintain release repos for all packages.
-
Debian 'superpackages' are easier to handle than both standalone packages and docker containers, e.g., they can be generated by developers from their working branches and easily copied and installed on the target.
-
Debian packages have some advantages over docker containers in general:
-
Zero overhead during execution.
-
Straightforward access to hardware.
-
Easy installation of system services, udev rules, configs, etc.
-
-
Different versions of binary packages can be installed simultaneously, if they are built using different
VERSION
parameters.
Generally, it is necessary to install packages to the filesystem root during
compilation in order to get all paths right in catkin
cmake
files and
properly install system files. CCWS
avoids this using proot
similarly to
cross-compilation profiles.
Here <profile>
stands for cross_raspberry_pi
, cross_jetson_xavier
,
cross_jetson_nano
. Cross-compilation make targets can be found in
ccws/make/cross.mk
and ccws/profiles/<profile>/targets.mk
Note on cross_jetson_xavier
and cross_jetson_nano
: these profiles require
Ubuntu 18.04 / ROS melodic and install nvcc
, you may want to do this in a
container.
The general workflow is documented below, for more technical details see
ccws/doc/cross-compilation.md
and CCWS
CI test in .ccws/test_cross.mk
:
- Install profile dependencies with
make bp_install_build BUILD_PROFILE=<profile>
- Obtain system image:
cross_raspberry_pi
--bp_install_build
target automatically downloads standard image;cross_jetson_xavier
,cross_jetson_nano
--CCWS
does not obtain these images automatically, you have to manualy copy system partition image toccws/profiles/cross_jetson_xavier/system.img
.
- Initialize source repositories:
make wsinit REPOS="https://github.com/asherikov/staticoma.git"
- [when building all ROS packages] add ROS dependencies of all your
packages to the workspace
make dep_to_repolist ROS_DISTRO=melodic
, or a specific packagemake dep_to_repolist PKG=<pkg> ROS_DISTRO=melodic
; - fetch all packages
make wsupdate
.
- Install system dependencies of packages in your workspace to the system
image:
make cross_install PKG=staticoma BUILD_PROFILE=<profile> ROS_DISTRO=<distro>
- Compile packages:
- mount sysroot with
make cross_mount BUILD_PROFILE=<profile>
- build packages, e.g.
make staticoma BUILD_PROFILE=<profile>
or build and generate deb packagemake deb PKG=staticoma BUILD_PROFILE=<profile>
- unmount sysroot when done with
make cross_umount BUILD_PROFILE=<profile>
- mount sysroot with
A docker image with preinstalled CCWS
and dependencies is available for
testing, but it is recommended to build a tailored image using
ccws/examples/Dockerfile
as an example.
The image can be used in the following way:
docker pull asherikov/ccws
mkdir tmp_ws
# sources, build, install, cache will go heredocker run --rm -ti -v ./tmp_ws:/ccws/workspace asherikov/ccws bash
make wsinit REPOS="https://github.com/asherikov/qpmad.git"
...
CCWS
functionality can be extended in multiple ways:
- by adding new build profiles, e.g.,
make bp_new BUILD_PROFILE=vendor_static_checks BASE_BUILD_PROFILE=static_checks
, all profiles starting withvendor
prefix are ignored by git; - by adding execution profiles;
make
targets can be added by creating accws/profiles/build/vendor/<filename>.mk
file;- common
cmake
toolchain suffix can be added toccws/profiles/build/vendor/toolchain_suffix.cmake
.
show_vendor_files
target can be used to list all vendor specific files.
- Segmentation fault during cross-compilation or debian package generation
indside docker containers (both require
proot
): presumably due toseccomp
Linux feature, which can be disabled with--security-opt seccomp:unconfined
docker parameter. Disablingseccomp
forproot
withPROOT_NO_SECCOMP=1
seems to be unnecessary. - Programs compiled with sanitizers (
addr_undef_sanitizers
orthread_sanitizer
build profiles) output2: AddressSanitizer:DEADLYSIGNAL
orFATAL: ThreadSanitizer: unexpected memory mapping
when executed: the reason is tightened memory security with ASLR (address space layout randomization) in modern Linux kernels, see google/sanitizers#1614. The issue can be alleviated by settingsudo sysctl vm.mmap_rnd_bits=28
.
- https://github.com/ros-industrial/industrial_ci -- ROS specific CI scripts, noninteractive, "one-shot" design, no sanitizers, emulated cross compilation.
- https://github.com/ros-tooling/cross_compile/ -- emulated cross compilation for ROS and ROS2.
- https://github.com/HesselM/rpicross_notes -- cross compilation for Raspberry Pi done in a different way.
- https://github.com/ros-tooling/action-ros-ci --
github
action that covers some ofCCWS
functionality.
- Replace
ccache
with https://github.com/mbitsnbites/buildcache. - Integrate https://github.com/oclint/oclint
- https://github.com/ejfitzgerald/clang-tidy-cache or
https://github.com/mbitsnbites/buildcache can be used to cache
clang-tidy
runs. - https://github.com/mrtazz/checkmake might be useful for makefile linting.
- Cache cmake checks with https://github.com/cristianadam/cmake-checks-cache, https://github.com/polysquare/cmake-forward-cache might be useful too.
- Shell formatter https://github.com/mvdan/sh.
- https://github.com/myint/cppclean might be useful for unnecessary header detection, but looks stale.
- https://github.com/include-what-you-use/include-what-you-use
- Build time analysis with clang https://github.com/aras-p/ClangBuildAnalyzer and / or https://github.com/jrmadsen/compile-time-perf.
- Potential replacement for
scan_build
https://github.com/Ericsson/codechecker with extra checks and caching. - https://github.com/sscpac/statick is not going to be used, but some of its linters can be integrated.
- Source code spellcheck https://github.com/myint/scspell.
- https://github.com/jordansissel/fpm -- generic binary package generator,
potential replacement for
dpkg-deb
. - https://github.com/git-afsantos/haros -- ROS-aware static analysis, might
have issues with non
catkin_make
build environments. - https://github.com/Tencent/TscanCode -- C++ static analysis tool.
- https://github.com/DLu/roscompile/tree/main/roscompile -- linter for catkin packages.
- Use https://libguestfs.org/ or https://github.com/alperakcan/fuse-ext2
instead of loop devices, to avoid using sudo. There are some issues in Ubuntu
though, bug 759725, see https://libguestfs.org/guestfs-faq.1.html.
guestfs
is too slow to be practical. - Distributed compilation support with https://github.com/distcc/distcc can be useful.
- Add memory sanitizer profile as an alternative to
valgrind
,gcc
doesn't support it currently. - Add code coverage profile.
- Execution profile with https://github.com/yugr/libdebugme to automatically start debugger on a signal.
- https://github.com/yugr/valgrind-preload as an alternative to
valgrind
execution profile -- an overkill in general case though. - Control symbol visibility and verify with https://github.com/yugr/ShlibVisibilityChecker.
- Add
CodeQL
profile (https://github.com/github/codeql).