As a user, I want a WARNING to be thrown when a delimited or character table value does not match the expected field_format

[jordanpadams]

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

Data User, Data Archivist

💪 Motivation

...so that I can be warned when a field's value does not meet the expectations of the field_format, but should not be an error.

Treat precision as "maximum precision" instead of "exact precision".

Add additional flag to disable this check.

📖 Additional Details

From SR:

4B.1.2 Field Formats

Attributes <field_format> and <validation_format> give the magnitude and precision of the data value.

<field_format> describes the general format of a value in a field. For binary tables, this is used to format the value for output, particularly to specify the appropriate number of significant digits when converting real values.

For character tables, <field_format> is used to describe the maximum length and alignment of the data. <field_format> also gives an indication of the maximum precision of real numbers, but does not require all values to have this precision.

<validation_format> is used with character tables to further constrain a field’s values. When a <validation_format> is present, the data in a table must match the format exactly in order to be valid. Additionally, if <validation_format> is present, it must not conflict with the field format. <validation_format> is not used for binary or delimited tables.

The specifier used in <field_format> and <validation_format> must be of an integer type if the field data type is a form of integer, or a floating point type if the field data type is a form of float, and of a string type for all other field data types.

field value should match the exact format defined by the validation_format, unlike the unlike validation_format, field_format is just a "guide", so we should just throw a WARNING when something doesn't match, and not for precision, this is just a max precision, not an exact precision.

Acceptance Criteria

Given a character table product with format defined for a field, and a value in a table that does not match that format
When I perform validate on that product
Then I expect validate to throw a WARNING the format is not being met
(automation ID: validate_integration.validate#816)

⚙️ Engineering Details

See test data from #681

[rchenatjpl]

@jordanpadams @al-niessner Do you want a separate ticket for this? In SR 4B.1.2 Field Formats, for the specifier "e,E", exactly one digit is always to the left of the decimal point, so 333.3e33 is illegal, i.e. that ought to be 3.333e35. The 3.6.0-snapshot version of validate allows either.

[jordanpadams]

@rchenatjpl sounds like we need a ticket for that. thanks for catching this!

[jordanpadams]

#992