IATI CoVE: spaces in "blank" cells are converted

[stevieflow]

When a cell only contains a space, this is converted by CoVE, and so creates a value in the XML. This is then rendered as either invalid or an invalid code.

Action - IATI CoVE should strip out these characteristics , on conversion

Action - better guidance for uses / in file validation

[stevieflow]

@robredpath is this similar to the issue you identified, when a cell is empty (or an expected value), and not then identified in test results?

In this ticket, I refer to instances where the value is not blank but a typed " "

[robredpath]

@stevieflow - this is quite a different case, because in this case, there is data (even if it's 'just' whitespace) - and so the fact that it gets converted means that there's a value in the source map that we can use to locate the error.

My instinct is to say that we should get IATI CoVE to look for, and alert on, this specific case rather than stripping out the whitespace, because that should act as a prompt to users to look into where the whitespace is coming from. If it's being stored in a database as a single space character rather than a NULL value, for example - that's worthy of investigation, because WHERE $field IS NOT NULL is a very common pattern in SQL query design, which could lead to incorrect results elsewhere.

In the spirit of IATI CoVE being a bit more of a "toolbox", we could also add a "whitespace stripper" tool, or checkbox, or something. This could be a good middle ground - it makes users recognise that there is a data quality issue that should be looked into, but doesn't get in the way of the job at hand.

[stevieflow]

Understood / agreed

That sounds great, thanks @robredpath

I know we discussed the NULL value issue in context of 360 CoVE, so would assume this could be a generic action?

[robredpath]

@stevieflow yeah, anything we do for IATI could also be relevant to 360 CoVE, although this has the feel of something that might be a bit tricky to do in a universal way. Let's do it for IATI and see!

I'm starting to feel like we should have a to-do/wishlist for IATI CoVE work from April onwards. Do you want to start collecting things like this, maybe into the IATI reporting doc? Or somewhere else?

[stevieflow]

Thanks @robredpath

I think I mean: we saw this week an issue with some 360 data, whereby a NULL field was reported, but not identified in the test results

There seems to be the exact same issue with IATI

Could we therefore get this addressed across the board

(and yes, appreciate this ticket was originally about there being whitespace

[robredpath]

@stevieflow the challenge might be if it's something that we need to do at the XML layer - in which case we'll need to re-do it for the JSON-based standards. Hopefully, we can fix it at a layer of abstraction up - but I don't want to promise that!