split_by_apid not matching variable length decoding APID

[jmbhughes]

I have a file containing only one APID. To confirm that, I ran split_by_apid and get only one dictionary entry for APID 32. Note that this packet definition has a variable expanding field at the end of each packet. Most of them will be the same size but sometimes they're the leftover bits of an image and can be smaller. So I've used

PacketArray(
           name="data",
           data_type="uint",
           bit_length=8,
           array_shape="expand",  
      ))

when I define the last field. However, when I look at the APID in the parsed output, it's something other than 32, and it's not even static. I'm not sure if there's something wrong with my data, something wrong with my code, or potentially something off in the VariableLength decoding. I'm not sure how much you can help with the vague information I've provided. I need to ask if I'm allowed to share the TLM and definition for this packet first.

One question I did have was why am I required to define bit_length for an expanding PacketArray? It seems like it is used to define the data type used in the output array and then discarded for the rest of the calculations maybe. I've used 8 here even though this field can expand to a very large size.

Has VariableLength been successfully used on other data?

Thanks for any advice!

[ddasilva]

You're right, it's hard to help without data! But here's what I can say:

The bit_length is the bit length of each array element. If you're not sure what the bit length is going to be, but you're sure the expanding field is going to be a clean number of bytes, then just set bit_length=8 and each array element will be one byte. The reason it's this way is because some people have expanding arrays of e.g. 10 bit unsigned integers and want them organized.

VariableLength.load() and split_by_apid() use different code to iterate through the packets, so it's possible something is up, Are you specifying the primary header fields manually? The docs tell you not to do this for VariableLength, but I think FixedLength is more forgiving and lets it slide. If that's not it, send your packet definition and I can come up with other ideas.

[jmbhughes]

I should read more carefully it seems! I didn't see the part of this page where it says not to define the primary header and was including everything. Thanks so much for pointing that out, especially with my vague request!