You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Dec 12, 2024. It is now read-only.
From reading the code, I see that each metadata field is written out as a key:value pair where the value may either be an atomic value, or a list, depending on whether the column name(s) in the CSV is numeric or not.
The result of this is variability in the structure of metadata.json based on the structure of the CSV. This can be dealt with in three ways that I can think of:
Make the ingest task (on the repository app side) adaptable, where it expects either single values or lists.
Have the batch loader always write values as lists, even if they may be 1-item lists (i.e. key: [value] as opposed to key: value
Always use numbers in the CSV column names (e.g. title1 even when there is no title2). This puts a burden on the creator of the CSV to understand and remember.
The intended approach was #3. This way the batch loader doesn't have to have any knowledge about the cardinality of the field -- it would all be driven by the CSV. Yes, this does require that the CSV creator understand which fields GWSS expects to have what cardinality.
Of course, happy to defer to your judgement as I was making assumptions about GWSS.
Thanks. I suppose as long as we have and re-use a CSV template with column names meant for use with GWSS, then that should work. Creating metadata.json for some other system would just use a different template.
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
From reading the code, I see that each metadata field is written out as a key:value pair where the value may either be an atomic value, or a list, depending on whether the column name(s) in the CSV is numeric or not.
The result of this is variability in the structure of
metadata.json
based on the structure of the CSV. This can be dealt with in three ways that I can think of:key: [value]
as opposed tokey: value
title1
even when there is notitle2
). This puts a burden on the creator of the CSV to understand and remember.@justinlittman your thoughts?
The text was updated successfully, but these errors were encountered: