Constants Across Platforms

[LeStarch]

F´ Version	devel
Affected Component	F´

---

Feature Description

Constants (max lengths, framing tokens) etc should be available across the GDS, Autocoder and FSW such that missmatches are properly handles.

[timcanham]

We should put some thought into where these live with respect to the proposal in #405 if the GDS will be maintained separately from the F Prime source tree, since there would be GDS dependencies on these constants.

[billallen256]

@timcanham has a good point. With just a couple languages involved, it seems like an Interface Control Document (ICD) should be sufficient, especially since constants shouldn't be redefined often. The various languages/components could implement it, referring to the ICD, with distinct tests to verify that the values continue to line up with the ICD.

The other option is to put them in a language-independent config file, but then there's the question of where to put the config file so all languages have easy access at build time.

[billallen256]

On the Python side, I'd suggest creating a separate fprime-constants package that can be used as a dependency by the other Python components.

[LeStarch]

The constants are controlled by projects that adapt F´ and may be different per-deployment (at least some of them). Thus, although they change infrequently, a project may need to run with multiple definitions much like how configuration.

Thus I am wondering if we need a set of "default" constants and a way to override them from a configuration file.

1. Define a generic format. E.G. fp_constants.yml
2. Improve autocoder to autocode FpAutoConstants.hpp
3. Create constants_handler.py to load above fp_constants.yml such that it overrides hardcoded constants.
4. Deliver fp_constants.yml as part of the dictionary folder of an F´ build. Thus it can be loaded using the same relative path.

@timcanham @billallen256 thoughts?

[timcanham]

Excellent idea. The question is the the format. Should we introduce yaml at this point, or should we use FPP or XML?

[LeStarch]

@timcanham yaml was just a toy example to convey the use of a file that could be read in by many languages. I will take the action to talk to the F´´ team to see what confirmation formats they are comfortable with. Ideally we would choose a format that works with the three languages we use commonly:

1. fpp and use with Scala
2. Python
3. c++ via autocoding

[billallen256]

@LeStarch thanks for the explanation. It sounds like they are more like configs than constants (configurable constants?). I know it sounds boring, but .ini files are well supported across languages, and keep people from getting too creative with the structure. It also transitions well to .toml if there's ever need for a bit more reach.

[LeStarch]

As of v3.0.0 our constants are specified in FPP format. We could just use that as the format specification as it is already feeding constants in the C++ layer. All we'd need to do is make a Python variation of the FPP parser. I will look into how feasible that would be.

[timcanham]

If we are going to do this effort, we should try to generate some shared constants so we can use them in the software and in the GDS, like command packet descriptor byte size, etc.

i.e. it would be good to generate .h and python files from the FPP constants.

[Joshua-Anderson]

As part of this fix we should enforce that command dictionary string lengths are no longer than the system max string length. If a user sets a command argument longer than the max string length, it's currently possible to generate invalid command sequences by suppling a string longer than command dispatcher supports (FW_CMD_STRING_MAX_SIZE)

[matt392code]

Core shared constants needed across both the Ground Data System (GDS) and Flight Software (FSW).
Core Shared Constants Categories:

1. Command and Telemetry Framing

// Packet Structure
MAX_PACKET_SIZE          // Maximum size of any packet
HEADER_SIZE              // Size of packet headers
DESCRIPTOR_SIZE          // Command packet descriptor byte size
CHECKSUM_SIZE            // Size of packet checksums
FRAME_TOKENS            // Special bytes/tokens used for packet framing

// Command Specific
CMD_STRING_MAX_SIZE      // Maximum length for command strings (FW_CMD_STRING_MAX_SIZE)
MAX_CMD_ARGS            // Maximum number of arguments per command
CMD_SEQUENCE_LENGTH     // Maximum length of command sequences

2. Telemetry and Event Sizes

// Telemetry
CHANNEL_ID_SIZE         // Size of telemetry channel IDs
MAX_TELEMETRY_VALUE    // Maximum size of telemetry values
TIMESTAMP_SIZE         // Size of timestamp fields

// Events
EVENT_MSG_SIZE         // Maximum size of event messages
MAX_EVENT_ARGS         // Maximum number of event arguments

3. System Identifiers

// Component IDs
MAX_COMPONENT_ID       // Maximum component identifier value
SYSTEM_ID_SIZE        // Size of system identifiers
PORT_ID_SIZE          // Size of port identifiers

// Instance Management
MAX_INSTANCES         // Maximum number of component instances
MAX_DEPLOYMENT_IDS    // Maximum number of deployment identifiers

4. Buffer and Queue Management

// Buffer Sizes
BUFFER_SIZE           // Standard buffer size
MAX_BUFFER_SIZE      // Maximum allowed buffer size
QUEUE_SIZE           // Default queue size
MAX_QUEUE_DEPTH      // Maximum queue depth

// Memory Management
POOL_BLOCK_SIZE      // Memory pool block size
MAX_ALLOCATIONS      // Maximum number of allocations

5. Communication Parameters

// Protocol
MAX_RETRY_ATTEMPTS    // Maximum number of retry attempts
TIMEOUT_SECONDS      // Default timeout in seconds
HEARTBEAT_PERIOD     // Heartbeat period for health monitoring

// Transport
MAX_UART_BAUD        // Maximum UART baud rate
MAX_IP_PACKET        // Maximum IP packet size

6. File System Constants

// File Operations
MAX_FILE_PATH        // Maximum file path length
MAX_FILE_SIZE        // Maximum file size
MAX_FILENAME_LEN     // Maximum filename length
MAX_FILES_OPEN       // Maximum number of open files

7. Logging and Debug

// Debug Settings
MAX_LOG_MSG_SIZE     // Maximum log message size
DEBUG_BUFFER_SIZE    // Debug buffer size
MAX_STACK_TRACE      // Maximum stack trace depth

Implementation Recommendations:

1. Create an FPP constants file with categorized sections:

module SharedConstants {
    # Command and Telemetry Framing
    constant MAX_PACKET_SIZE: U32 = 4096
    constant HEADER_SIZE: U32 = 64
    ...
    # System Identifiers
    constant MAX_COMPONENT_ID: U32 = 256
    ...
}

2. Validation Rules:

• Ensure command string lengths ≤ FW_CMD_STRING_MAX_SIZE
• Buffer sizes must be power of 2
• IDs must fit within specified bit widths
• Time values must be within system capabilities

3. Generation Features:

• Generate C++ header with compile-time constants
• Generate Python module with constant definitions
• Include validation checks in generated code
• Add documentation comments for each constant

4. Usage Guidelines:

• Constants should be used instead of magic numbers
• Changes require regression testing
• Document rationale for size-related constants
• Include units in constant names where applicable

[matt392code]

A more few additions to make it more robust:

1. Add Version Control:

module SharedConstants {
    # Version tracking
    constant CONSTANTS_VERSION: string = "1.0.0"
    constant CONSTANTS_HASH: string = "AUTO_GENERATED_AT_BUILD"  # Git hash

2. Add Range Validation:

# In the FPP file
module SharedConstants {
    @ Range validation for packet size
    @ Minimum: 512 (required header + minimum payload)
    @ Maximum: 65536 (maximum system memory constraint)
    constant MAX_PACKET_SIZE: U32 = 4096

3. Add Relationship Constraints:

# Ensure these relationships are enforced during generation
BUFFER_SIZE <= MAX_BUFFER_SIZE
QUEUE_SIZE <= MAX_QUEUE_DEPTH
MAX_FILE_PATH >= MAX_FILENAME_LEN

4. Add System-Specific Overrides:

module ProjectConstants {
    # Project can override defaults while maintaining minimums
    constant PROJECT_MAX_PACKET_SIZE: U32 = 
        max(SharedConstants.MAX_PACKET_SIZE, 8192)
}

Implementation Suggestions:

1. Add Build-Time Validation:

def validate_constants():
    assert is_power_of_two(BUFFER_SIZE), "Buffer size must be power of 2"
    assert MAX_PACKET_SIZE <= MAX_BUFFER_SIZE, "Packet cannot exceed buffer"
    assert MAX_FILE_PATH > MAX_FILENAME_LEN, "File path must exceed filename"

2. Add Constant Usage Tracking:

// Generated header
#define CONSTANT_USED(x) _mark_constant_used(#x)
void _mark_constant_used(const char* constant_name);

These enhancements would:

• Ensure version consistency across builds
• Prevent invalid constant combinations
• Allow project-specific customization
• Maintain system-wide constraints
• Track constant usage for optimization

[billallen256]

A lot has happened in the configuration world in the past few years with so much interest (and disgust) around templating for things like Helm. For example Pkl has templating and some of the validation features already, and can output to other formats like JSON. It also allows for defaults that can be overridden.

https://pkl-lang.org/blog/introducing-pkl.html

Jsonnet is also worth taking a look at.

[matt392code]

fprime-jsonnet-config.txt
Jsonnet might be more stable way to go since it is under the Apache 2.0 License.
Given that Pkl is controlled by Apple, and they might start asking for licensing fees when a probe using it is just about to enter Jupiter's orbit.
The attached solution offers several advantages for F Prime:

1. Single Source of Truth

• Constants defined once in Jsonnet
• Automatically generates both C++ and Python code
• Prevents inconsistencies between flight and ground software

2. Type Safety & Validation

• Jsonnet can validate values at generation time
• Ensures constants meet requirements before compilation
• Can add custom validation rules

3. Flexibility

• Easy to override values for different deployments
• Can extend for new constant types
• Maintains backward compatibility

4. Documentation

• Self-documenting configuration
• Clear organization of constants
• Easy to see relationships between values

[billallen256]

Pkl is Apache 2 as well: https://github.com/apple/pkl/blob/main/LICENSE.txt

[matt392code]

Understood.
However, the whole project is controlled by Apple, so there is nothing to prevent their corporate attorneys from attempting to extort JPL for millions of dollars when a probe is in orbit around Jupiter or Saturn. Patent attorneys can be quite creative and tenacious when attempting to extort large research institutions if they feel it would add to the bottom line to their corporation for that quarter.
Jsonnet is independent, so there is no chance of that happening. When a probe is billions of miles away, the project managers want to be sure that they will not have patent attorneys sniffing around for opportunities to extort JPL's budget.

[thomas-bc]

This ticket will be planned for post v3.7.0 as we are dependent on upcoming FPP work