Target Format Transpiler

[nigelht]

This is related to #50 as a different type of output vs target language. I'd like to generate output to other description languages similar to how #50 would generate Lua for Wireshark.

Our use case is definition of spacecraft telemetry and command packets in CCSDS format (https://public.ccsds.org/Pubs/133x0b1c2.pdf) in ksy and then exporting that into the definition languages of ground systems like COSMOS (https://cosmosrb.com) as well as structs for C and parsers for the existing target languages (in our case C++ and Java).

We would also want to add fields to define the constraint checking and conversion information (e.g. integer count to analog value via poly conversions) used in these system that would be ignored by the target languages except to include them as documentation. Sort of a half step to #81.

Is this something that kaitai would be suitable for?

[GreyCat]

Kaitai Struct is just a formal language, so I don't see any strong argument why it can't be used to do that. At the first glance, CCSDS format looks pretty much like a traditional ISO/OSI stack, which its own transport layer, application layer, etc.

Scanning COSMOS site for several minutes, I couldn't find any "definiltion languages" description to see if it's viable idea to convert .ksy to COSMOS or not. Could you point me to documentation on that matter, or better yet, some examples of these definitions? If they are special in some way, it might be actually a good idea to it the other way around, i.e. start with COSMOS descriptions, then convert them to .ksy, and then enjoy generated parsers / visualization tools benefits.

We would also want to add fields to define the constraint checking and conversion information (e.g. integer count to analog value via poly conversions) used in these system that would be ignored by the target languages except to include them as documentation.

You're free to define any implementation-specific tags in the format using - as the first symbol of the tag name, i.e. something like that will work:

seq:
  - id: foo
    type: u4
    -nigelht-special-tag-constraint: something
    -nigelht-special-tag-conversion: convert_one_to_another

These things would be completely ignored by ksc, but could be used by other tools (i.e. ksy -> COSMOS converters, doc generators, etc).

[nigelht]

Awesome regarding the tags. I suspected that was possible but missed it while skimming the docs.

The Cosmos definition files are documented here with examples near the bottom:

https://cosmosrb.com/docs/cmdtlm/

Is the wire shark code a good place to start in creating a converter? I haven’t looked at it yet but is that the general way you would want to structure this kind of functionality?

[GreyCat]

The Cosmos definition files are documented here with examples near the bottom:

Yeah, it looks pretty much like a subset of .ksy. So, unless you have to do any complex shenannigans with parse instances / value instances, out-of-stream parsing, etc, that should be relatively simple to convert.

Is the wire shark code a good place to start in creating a converter?

If we're talking about conversion of a simple subset, then actually just about any solution would be really simple, like literally 20-30 lines of code (given that you use existing YAML parser, of course).

[nigelht]

Looking at the various specs, one of the things that popped out at me was unions are allowed for telemetry definitions and I didn't see that in kaitai. Did I miss how to do that?

[KOLANICH]

I didn't see that in kaitai.

There are no built-in unions, but I have achieved the similar multiple times with ugly and inefficient hacks with _io.

[GreyCat]

There is no built-in support of "unions" per se, but the following is generally an equivalent:

seq:
  - id: foo
    type: u4
  - id: my_union
    type: union_type
    size: 8 # this one reserves exactly 8 bytes as a substream
types:
  union_type:
    instances:
      as_integer:
        pos: 0 # position in 8-byte substream, i.e. from the very beginning
        type: u8
      as_double:
        pos: 0
        type: f8

[webbnh]

@nigelht, we've had reasonable success using type-switching.

[GreyCat]

Looks like this discussion faded out, closing the issue. Please feel free to reopen if anything still remains.

[generalmimon]

Looks like this discussion faded out, closing the issue.

I think I can really close it now ;)