Test to distinguish equality validation of bytes vs. strings. #93
Conversation
…kes sure that a textual error message is thrown instead of HEX-bytes. kaitai-io/kaitai_struct_ruby_runtime#7
| }.to raise_error( | ||
| # Make extra sure that handling of "byte[]" is not used by looking at the message. | ||
| Kaitai::Struct::ValidationNotEqualError, | ||
| a_string_including('"BACK"', '"PACK"') |
There was a problem hiding this comment.
I would definitely compare the full message. This mere testing whether it contains specific strings constitutes too much creativity, and it's not very well convertible to the KST language-agnostic approach (when we decide to implement it in the KST translator).
BTW, for KSC errors we have ErrorMessagesSpec, which tries to compile malformed KSY specs in the kaitai_struct_tests / formats_err/ directory and compares them to the expected error message (for example, kaitai_struct_tests / formats_err/attr_bad_id.ksy). It also asserts the full message.
Anyway, it's definitely a good idea to extend the tests in the current KS test suite to also ensure that the error messages are consistent between all target languages (because as it will surely turn out, they're not).
There was a problem hiding this comment.
I would definitely compare the full message.[...]
I've decided against this explicitly because error messages are pretty inconsistent currently and might get normalized at some point. Most people don't care too much about things like lower vs. upper case of phrases, spelling, if it's complete sentences or not etc. What we DO care heavily instead is IF textual strings are printed human readable or using HEX-bytes. And the two phrases currently checked for focus exactly on that.
Why should the current test fail if wrong textual content is still printed human readable, but with some slightly different error message? Makes it unnecessary difficult to maintain the test in my opinion. Especially because the current approach of using a_string_including is a PoC only anyway and not supported by KST. Before checking concrete error messages across languages, one needs to decide how to make that concept available in KST in the long term.
Up until that point I suggest keeping things as easy as they are currently and focus on the important HEX vs. not-thing.
This makes sure that a textual error message is thrown instead of HEX-bytes.
valid_fail_eq_str_spec.rbis created manually to be able to check for concrete error messages, no idea if this is possible using KST already.kaitai-io/kaitai_struct_ruby_runtime#7