unicsv fixes #1064
unicsv fixes #1064
Conversation
robertlipe
commented
Apr 11, 2023
•
robertlipe
commented
- Changes to unicsv to improve robustness, allow read of Sidewalk 360 Eval files.
- Remove debug chatter.
- Allow DateTime to work in a unicsv column. All the hard work was there. The name "datetime" just wasn't hooked up to set the bits for the scanner.
…val files. A few small, but intense changes needed: human_to_dec(): we were accumulating *outlat and *outlon output arguments , provided as pointers to doubles, without ever actually initializing them. In an error case, if we pointed to wpots that happened to have doodoo in those memory addresses, but an invalid unk[0] deduced from the parse, we then partially accumulated the output values. This lead to la/longs of 1e25 or NaN or such, invariably crashing waypt_add. unicsv_parse_one_line(): Detect points without location (null island) and just count them instead of addding to internal structures. Points without location are now possible because of the above parser change. Tread them as noise and toss them, but notify user when it happens. Add informat, column counts when parsing header to see what we have ahd have not parsed. Unify debugging and error prose.for above.
and not just on the meridian. Fixes failing track test .
|
While your in here can you fix the extra QString/char* roundtrip in these two places (human_to_dec takes a QString): Lines 544 to 549 in cad16e1 I dislike using the null island as flag, it exists. Ok, I grant you it rarely occurs. |
|
Good points. I hated the calling convention while I was looking at it, but that function is just terrifying enough that I didn't want to get too radical with it. I thought there were more callers of this than there really are, so I really should just go make a better signature and fix them all. This function has the really weird convention of returning either or both, but we can work with that. Importantly (I couldn't find it last night...) the code coverage of that function is better than I thought: https://app.codacy.com/gh/GPSBabel/gpsbabel/file/86982252082/coverage?bid=26148649&fileBranchId=26148649 The reason for 6aa2d36 is exactly to handle points on the prime meridian OR the equator. We have some of these in our testsuite where we hand-created some polygons with coordinates of (3, 2) and (1, 0) degrees or something. With an actual return value on h_to_d we can at least issue a warning at the point of failure within the line (though we don't have line number or column number) but it's only really at the end of the line that we can tell if both set_lat and set_long have been called and we have a fully formed point. Maybe we can stuff sentinels in the newly created waypt_tmp when we created it and look to confirm they've been both changed before we proceed. I'll think more on this point. |
Coverage summary from CodacyMerging #1064 (7050664) into
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferences |
|
Oh. Now we have an actual problem. My code that oh-so-cleverly detects malformed unicsv that results in 0 lat long works perfectly ... and tosses hundreds of 0,0 data that appears in our test suite, notably as test cases for mkshort. Maybe we skootch those off 0,0 a bit to 0.1, 0.1 or something? I'd have to see if any of the other problems are "real". Rats! |
|
Can we add a Sidewalk 360 Eval file to test? |
