Merge branch 'develop-ros-templates' into develop. Close #162.

**Description**

The ROS backend used a fixed template to generate the ROS package. That
template does not fit all use cases, and users are heavily modifying the
output (which is hard to keep up with when there are changes), and or
not using ogma altogether for that reason.

Allowing users to pick their own ROS template would make Ogma more
versatile.

**Type**

- Feature: Enable customizing output produced.

**Additional context**

None.

**Requester**

- Ivan Perez.

**Method to check presence of bug**

Not applicable (not a bug).

**Expected result**

Ogma allows users to pick the custom ROS package template they want to
use instead of relying on the one provided by default.

The following dockerfile generates the ROS package using the default
template and using a copy and the default template and checks that both
are the same. It then adds a file to the copy of the template and checks
that the file is copied to the target directory when the custom template
is used, after which it prints the message "Success".

```Dockerfile
FROM ubuntu:trusty

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update

RUN apt-get install --yes software-properties-common
RUN add-apt-repository ppa:hvr/ghc
RUN apt-get update

RUN apt-get install --yes ghc-8.6.5 cabal-install-2.4
RUN apt-get install --yes libz-dev

ENV PATH=/opt/ghc/8.6.5/bin:/opt/cabal/2.4/bin:$PWD/.cabal-sandbox/bin:$PATH

RUN cabal update
RUN cabal v1-sandbox init
RUN cabal v1-install alex happy
RUN apt-get install --yes git

CMD git clone $REPO && \
    cd $NAME && \
    git checkout $COMMIT && \
    cd .. && \
    cabal v1-sandbox init                                                                       && \
    cabal v1-install $NAME/$PAT**/ --constraint="aeson>=2.0.3.0"                                && \
    cp -r $NAME/ogma-core/templates/ros custom-template-ros                                     && \
    ./.cabal-sandbox/bin/ogma ros --app-target-dir original                                        \
                                  --variable-db $NAME/ogma-cli/examples/ros-copilot/vars-db        \
                                  --variable-file $NAME/ogma-cli/examples/ros-copilot/variables    \
                                  --handlers-file $NAME/ogma-cli/examples/ros-copilot/handlers  && \
    ./.cabal-sandbox/bin/ogma ros --app-target-dir new                                             \
                                  --app-template-dir custom-template-ros                           \
                                  --variable-db $NAME/ogma-cli/examples/ros-copilot/vars-db        \
                                  --variable-file $NAME/ogma-cli/examples/ros-copilot/variables    \
                                  --handlers-file $NAME/ogma-cli/examples/ros-copilot/handlers  && \
    diff -rq original new                                                                       && \
    rm -rf new                                                                                  && \
    echo "Success" >> custom-template-ros/test                                                  && \
    ./.cabal-sandbox/bin/ogma ros --app-target-dir new                                             \
                                  --app-template-dir custom-template-ros                           \
                                  --variable-db $NAME/ogma-cli/examples/ros-copilot/vars-db        \
                                  --variable-file $NAME/ogma-cli/examples/ros-copilot/variables    \
                                  --handlers-file $NAME/ogma-cli/examples/ros-copilot/handlers  && \
    cat new/test
```

Command (substitute variables based on new path after merge):

```sh
$ docker run -e "REPO=https://github.com/NASA/ogma" -e "NAME=ogma" -e PAT="ogma-" -e "COMMIT=<HASH>" -it ogma-verify-162
```

**Solution implemented**

Modify `ogma-core` and `ogma-extra` to move the mustache template
expansion feature to `ogma-extra`. Adjust the cFS backend accordingly.

Modify `ogma-core` to use variable expansion to generate the ROS
package.

Modify `ogma-core`'s template to use the variables used by the ROS
package generation module.

Modify `ogma-core` to give users the ability to pick a template
directory via an optional input argument.

Modify `ogma-cli` to give users the ability to pick a template directory
via an optional input argument (exposing the corresponding argument from
`ogma-core`).

Modify `README` to demonstrate new capability.

**Further notes**

None.

ogma-cli/CHANGELOG.md

@@ -1,8 +1,9 @@
 # Revision history for ogma-cli
 
-## [1.X.Y] - 2024-11-11
+## [1.X.Y] - 2024-11-20
 
 * Provide ability to customize template in cfs command (#157).
+* Provide ability to customize template in ros command (#162).
 
 ## [1.4.1] - 2024-09-21
 

ogma-cli/README.md

@@ -236,9 +236,10 @@ the data needed by the monitors and report any violations. At present, support
 for ROS app generation is considered preliminary.
 
 ROS applications are generated using the Ogma command `ros`, which receives
-four main arguments:
+five main arguments:
 - `--app-target-dir DIR`: location where the ROS application files must be
   stored.
+- `--app-template-dir DIR`: location of the ROS application template to use.
 - `--variable-file FILENAME`: a file containing a list of variables that must
 be made available to the monitor.
 - `--variable-db FILENAME`: a file containing a database of known variables,
@@ -299,6 +300,74 @@ and the last step of the script
 `.github/workflows/repo-ghc-8.6-cabal-2.4-ros.yml`, which generates a ROS
 monitor with multiple variables and compiles the resulting code.
 
+### Template Customization
+
+By default, Ogma uses a pre-defined template to generate the ROS monitoring
+package. It's possible to customize the output by providing a directory with a
+set of files with a ROS package template, which Ogma will use instead.
+
+To choose this feature, one must call Ogma's `ros` command with the argument
+`--app-template-dir DIR`, where `DIR` is the path to a directory containing a
+ROS 2 package template. For example, assuming that the directory `my_template`
+contains a custom ROS package template, one can execute:
+
+```
+$ ogma ros --app-template-dir my_template/ --handlers filename --variable-file variables --variable-db ros-variable-db --app-target-dir ros_demo
+```
+
+Ogma will copy the files in that directory to the target path, filling in
+several holes with specific information. For the monitoring node, the variables
+are:
+
+- `{{variablesS}}`: this will be replaced by a list of variable declarations,
+  one for each global variable that holds information read from the ROS
+software bus that must be made accessible to the monitoring code.
+
+- `{{msgSubscriptionsS}}`: this will be replaced by a list of calls to
+  `create_subscription`, subscribing to the necessary information coming in the
+software bus.
+
+- `{{msgPublisherS}}`: this will be replaced by a list of calls to
+  `create_publisher`, to create topics to report property violations on the
+software bus.
+
+- `{{msgHandlerInClassS}}`: this will be replaced by the functions that will be
+  called to report a property violation via one of the property violation
+topics (publishers).
+
+- `{{msgCallbacks}}`: this will be replaced by function definitions of the
+  functions that will be called to actually update the variables with
+information coming from the software bus, and re-evaluate the monitors.
+
+- `{{msgSubscriptionDeclrs}}`: this will be replaced by declarations of
+  subscriptions used in `{{msgSubscriptionsS}}`.
+
+- `{{msgPublisherDeclrs}}`: this will be replaced by declarations of publishers
+  used in `{{msgPublishersS}}`.
+
+- `{{msgHandlerGlobalS}}`: this will be replaced by top-level functions that
+  call the handlers from the single monitoring class instance (singleton).
+
+Ogma will also generate a logging node that can be used for debugging purposes,
+to print property violations to a log. This second node listens to the messages
+published by the monitoring node in the software bus. For that node, the
+variables used are:
+
+- `{{logMsgSubscriptionsS}}`: this will be replaced by a list of calls to
+  `create_subscription`, subscribing to the necessary information coming in the
+software bus.
+
+- `{{logMsgCallbacks}}`: this will be replaced by function definitions of the
+  functions called to report the violations in the log. These functions are
+used as handlers to incoming messages in the subscriptions.
+
+- `{{logMsgSubscriptionDeclrs}}`: this will be replaced by declarations of
+  subscriptions used in `{{logMsgSubscriptionsS}}`.
+
+We understand that this level of customization may be insufficient for your
+application. If that is the case, feel free to reach out to our team to discuss
+how we could make the template expansion system more versatile.
+
 ### Current limitations
 
 The user must place the code generated by Copilot monitors in two files,

ogma-cli/src/CLI/CommandROSApp.hs

@@ -56,11 +56,12 @@ import Command.ROSApp ( ErrorCode, rosApp )
 
 -- | Options needed to generate the ROS application.
 data CommandOpts = CommandOpts
-  { rosAppTarget   :: String
-  , rosAppFRETFile :: Maybe String
-  , rosAppVarNames :: Maybe String
-  , rosAppVarDB    :: Maybe String
-  , rosAppHandlers :: Maybe String
+  { rosAppTarget      :: String
+  , rosAppTemplateDir :: Maybe String
+  , rosAppFRETFile    :: Maybe String
+  , rosAppVarNames    :: Maybe String
+  , rosAppVarDB       :: Maybe String
+  , rosAppHandlers    :: Maybe String
   }
 
 -- | Create <https://www.ros.org/ Robot Operating System> (ROS) applications
@@ -72,6 +73,7 @@ command :: CommandOpts -> IO (Result ErrorCode)
 command c =
   rosApp
     (rosAppTarget c)
+    (rosAppTemplateDir c)
     (rosAppFRETFile c)
     (rosAppVarNames c)
     (rosAppVarDB c)
@@ -94,6 +96,13 @@ commandOptsParser = CommandOpts
         <> value "ros"
         <> help strROSAppDirArgDesc
         )
+  <*> optional
+        ( strOption
+            (  long "app-template-dir"
+            <> metavar "DIR"
+            <> help strROSAppTemplateDirArgDesc
+            )
+        )
   <*> optional
         ( strOption
             (  long "fret-file-name"
@@ -127,6 +136,11 @@ commandOptsParser = CommandOpts
 strROSAppDirArgDesc :: String
 strROSAppDirArgDesc = "Target directory"
 
+-- | Argument template directory to ROS app generation command
+strROSAppTemplateDirArgDesc :: String
+strROSAppTemplateDirArgDesc =
+  "Directory holding ROS application source template"
+
 -- | Argument FRET CS to ROS app generation command
 strROSAppFRETFileNameArgDesc :: String
 strROSAppFRETFileNameArgDesc =

ogma-core/CHANGELOG.md

@@ -1,9 +1,10 @@
 # Revision history for ogma-core
 
-## [1.X.Y] - 2024-11-13
+## [1.X.Y] - 2024-11-20
 
 * Fix incorrect path when using Space ROS humble-2024.10.0 (#158).
 * Use template expansion system to generate cFS monitoring application (#157).
+* Use template expansion system to generate ROS monitoring application (#162).
 
 ## [1.4.1] - 2024-09-21
 

ogma-core/ogma-core.cabal

@@ -62,7 +62,8 @@ data-files:          templates/copilot-cfs/CMakeLists.txt
                      templates/copilot-cfs/fsw/src/copilot_cfs_events.h
                      templates/ros/Dockerfile
                      templates/ros/copilot/CMakeLists.txt
-                     templates/ros/copilot/src/.keep
+                     templates/ros/copilot/src/copilot_logger.cpp
+                     templates/ros/copilot/src/copilot_monitor.cpp
                      templates/ros/copilot/package.xml
                      templates/fprime/CMakeLists.txt
                      templates/fprime/Dockerfile
@@ -106,10 +107,7 @@ library
       base                    >= 4.11.0.0 && < 5
     , aeson                   >= 2.0.0.0  && < 2.2
     , bytestring
-    , Cabal                   >= 2.4      && < 3.10
-    , directory               >= 1.3.1.0  && < 1.4
     , filepath
-    , microstache             >= 1.0      && < 1.1
     , mtl
     , text                    >= 1.2.3.1  && < 2.1
 

ogma-core/src/Command/CFSApp.hs

@@ -47,23 +47,16 @@ module Command.CFSApp
 
 -- External imports
 import qualified Control.Exception         as E
-import           Control.Monad             (filterM, forM_)
-import           Data.Aeson                (Value (..), decode, object, (.=))
-import qualified Data.ByteString.Lazy      as B
+import           Data.Aeson                (decode, object, (.=))
 import           Data.List                 (find)
 import           Data.Text                 (Text)
 import           Data.Text.Lazy            (pack, unpack)
-import           Data.Text.Lazy.Encoding   (encodeUtf8)
-import           Distribution.Simple.Utils (getDirectoryContentsRecursive)
-import           System.Directory          (createDirectoryIfMissing,
-                                            doesFileExist)
-import           System.FilePath           (makeRelative, splitFileName, (</>))
-import           Text.Microstache          (compileMustacheFile,
-                                            compileMustacheText, renderMustache)
+import           System.FilePath           ( (</>) )
 
 -- Internal imports: auxiliary
-import Command.Result ( Result (..) )
-import Data.Location  ( Location (..) )
+import Command.Result         ( Result (..) )
+import Data.Location          ( Location (..) )
+import System.Directory.Extra ( copyTemplate )
 
 -- Internal imports
 import Paths_ogma_core ( getDataDir )
@@ -303,48 +296,3 @@ ecCannotEmptyVarList = 1
 -- permissions or some I/O error.
 ecCannotCopyTemplate :: ErrorCode
 ecCannotCopyTemplate = 1
-
--- * Generic template handling
-
--- | Copy a template directory into a target location, expanding variables
--- provided in a map in a JSON value, both in the file contents and in the
--- filepaths themselves.
-copyTemplate :: FilePath -> Value -> FilePath -> IO ()
-copyTemplate templateDir subst targetDir = do
-
-  -- Get all files (not directories) in the template dir. To keep a directory,
-  -- create an empty file in it (e.g., .keep).
-  tmplContents <- map (templateDir </>) . filter (`notElem` ["..", "."])
-                    <$> getDirectoryContentsRecursive templateDir
-  tmplFiles <- filterM doesFileExist tmplContents
-
-  -- Copy files to new locations, expanding their name and contents as
-  -- mustache templates.
-  forM_ tmplFiles $ \fp -> do
-
-    -- New file name in target directory, treating file
-    -- name as mustache template.
-    let fullPath = targetDir </> newFP
-          where
-            -- If file name has mustache markers, expand, otherwise use
-            -- relative file path
-            newFP = either (const relFP)
-                           (unpack . (`renderMustache` subst))
-                           fpAsTemplateE
-
-            -- Local file name within template dir
-            relFP = makeRelative templateDir fp
-
-            -- Apply mustache substitutions to file name
-            fpAsTemplateE = compileMustacheText "fp" (pack relFP)
-
-    -- File contents, treated as a mustache template.
-    contents <- encodeUtf8 <$> (`renderMustache` subst)
-                           <$> compileMustacheFile fp
-
-    -- Create target directory if necessary
-    let dirName = fst $ splitFileName fullPath
-    createDirectoryIfMissing True dirName
-
-    -- Write expanded contents to expanded file path
-    B.writeFile fullPath contents