[8SP;6SP] Improve Nested/Sub-Graph Integration (formerly 'hier-blocks')

[RalphSteinhagen]

This issue tracks the integration of nested or sub-graphs, previously referred to as 'hier-blocks' in GNU Radio 3.x, into GNU Radio 4.0. The aim is to reduce cognitive complexity, streamline reuse, and enhance modularity through a cleaner design that supports both unmanaged and managed sub-graphs.

Motivation and Use Cases

1. Reduction of Cognitive Complexity
   Group recurring topologies (e.g., filter cascades) into reusable sub-graphs. These sub-graphs can be instantiated within a top-level graph, simplifying development by reducing duplication and promoting better organisation.

   Proposed Actions:

   • Unmanaged Nested Graph in YAML-definition
     Store nested graphs as template-style sub-graphs in the same YAML file as the top-level graph. This approach minimises changes to the grc-loader and uses the same syntax for the sub-graph as for the top-level graph, except for a templated name.

   • Managed Nested Graph with Scheduler Integration
     Implement a structured approach where a domain-specific scheduler is associated with each sub-graph. This allows for optimisation strategies based on latency vs. throughput or computational domains like FPGA and GPUs. The scheduler is instantiated with the nested graph and passed as a Block for higher-level usage.

2. Composing New Block Functionalities from Existing Basic Blocks
   Facilitate the creation of new block functionalities by combining existing blocks (typically C++-based) within the same or other top-level graphs.

   Proposed Actions:

   • Dedicated YAML-based Nested Graph Definitions
     Implement dedicated YAML files for nested graph definitions, stored in the block registry, to be reused across different graphs. The implementation will leverage the grc-loader and its YAML-parsing capabilities.

   • Plugin-like Extensions for User-Defined Nested Graphs
     Introduce a plugin-style extension that allows users to browse and load nested graph definitions from user-specified directories, enhancing flexibility and modularity.

   • Plugin-like Extensions for External Graph Sources
     Expand the plugin architecture to support sources beyond local files, including databases or community-driven repositories via HTTP(S). This allows nested graph YAML definitions to be queried dynamically from websites or other HTTPS-based repositories.

Deprecating yaml-cpp in Favour of pmt-lib

To streamline dependencies, we aim to remove the old yaml-cpp dependency and transition to a new pmt-lib-based YAML parser/writer implementation. This change ensures feature equivalence and backward compatibility with existing functionality. The migration to pmt-lib is a prerequisite for the new nested graph functionalities to minimise duplication of work

[ivan-cukic]

Additionally:

• Messages for grouping blocks into a managed or unmanaged group (by default, exported ports are what was previously connected to the out world)
• Messages for duplicating blocks / groups (for UI to define blocks)
• Messages for exporting ports from groups

[ivan-cukic]

Patches made as a part of this task:

• PmtYaml: Parser fixes gnuradio4#477
• Use new yaml parser for grc gnuradio4#478
• Add subgraphs with exported ports gnuradio4#479

[RalphSteinhagen]

There are follow-up item documented here:

• Managed Nested SubGraph with Scheduler Integration gnuradio4#487
• Sub-graph's YAML-Based definition and plugin-like extension gnuradio4#488