Skip to content
This repository has been archived by the owner on Dec 12, 2024. It is now read-only.

Consider always making metadata fields a list #1

Open
kerchner opened this issue Mar 21, 2018 · 2 comments
Open

Consider always making metadata fields a list #1

kerchner opened this issue Mar 21, 2018 · 2 comments

Comments

@kerchner
Copy link
Member

From reading the code, I see that each metadata field is written out as a key:value pair where the value may either be an atomic value, or a list, depending on whether the column name(s) in the CSV is numeric or not.

The result of this is variability in the structure of metadata.json based on the structure of the CSV. This can be dealt with in three ways that I can think of:

  1. Make the ingest task (on the repository app side) adaptable, where it expects either single values or lists.
  2. Have the batch loader always write values as lists, even if they may be 1-item lists (i.e. key: [value] as opposed to key: value
  3. Always use numbers in the CSV column names (e.g. title1 even when there is no title2). This puts a burden on the creator of the CSV to understand and remember.

@justinlittman your thoughts?

@justinlittman
Copy link
Contributor

The intended approach was #3. This way the batch loader doesn't have to have any knowledge about the cardinality of the field -- it would all be driven by the CSV. Yes, this does require that the CSV creator understand which fields GWSS expects to have what cardinality.

Of course, happy to defer to your judgement as I was making assumptions about GWSS.

@kerchner
Copy link
Member Author

Thanks. I suppose as long as we have and re-use a CSV template with column names meant for use with GWSS, then that should work. Creating metadata.json for some other system would just use a different template.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants