Validation Errors - Metadata Required Fields Break File Upload

[mheppler]

File upload fails silently in a preexisting dataset, in a dataverse that I had recently changed the general information on to now include the GSD metadata block, which has required fields.

Validation failed: Student's Program of Study is required. Faculty Name is required. Site Type is required. Course Name is required. Program Brief is required. GSD Tags is required. Faculty Recommendation is required. Student Name is required. Semester / Year is required.

[scolapasta]

This is tricky, since by adding a block to a dataverse you could make existing datasets "invalid." How do we want to handle this?

[posixeleni]

@scolapasta I am thinking that all new datasets would need to use these newly applied "required" fields however all older existing datasets would not fail (unless someone goes to Edit and it fails when they try to save). Would this be a trivial feature to support?

[eaquigley]

assigning to @scolapasta to answer @posixeleni 's question

[scolapasta]

That is effectively what is happening - it is failing on an edit. But the edit is a file edit and not a metadata edit. We could consider using the N/A values somehow. But we don't really want to modify the older versions, so maybe when the draft gets created, it creates these N/A values.

Like I said, tricky. Will discuss with @posixeleni next week and figure out something for beta 14 or 15.

[eaquigley]

Related to #2062 as that is the API error happening for this

[dliburd]

This may be related to this existing issue. I’m getting a “Dataset Save Failed” error when attempting to add data to a study with preexisting data files.

Error Message:
Dataset Save Failed - edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException: Validation failed: Subject is required.

Also the preexisting files are labeled with this message:
A file with this MD5 already exists in the dataset.

The dataset and the files with the MD5 error message were migrated from 3.6.
http://hdl.handle.net/1902.1/00565

[posixeleni]

A user requested that until we can resolve this issue some help text (or pop-up) like what I send to users would be helpful since we ( @suenjedt @sbarbosadataverse ) have received multiple support tickets where the user had no idea where to go and edit the "Subject"

The error message is occurring because since we migrated to 4.0 we introduced a new required metadata field called "Subject" for Datasets so before you can make any changes to your files you will have to go into your dataset and Edit > Metadata and select a Subject. Once this is saved then you can go in and modify the restrictions on your files.

To edit the Subject of your Dataset please follow these steps:

1. In your Dataverse select the Dataset you wish to update files in.
2. Once in your dataset click on the Edit button on the right hand side of the page and select "Metadata" in the dropdown. This is where you can add or edit any descriptions related to your dataset.
3. Once you are in the Edit > Metadata page for your Dataset scroll down till you see "Subject" and select any Subject(s) (checkboxes) that are applicable for this dataset. This will help others discover datasets that have a similar Subject.
4. Once you have selected a Subject(s) click on the "Save Changes" button located on the left at the top of the page.
5. Now you should be able to go and update your files.

[mheppler]

Related to #2190.

[eaquigley]

Need to have a meeting ASAP to finalize ideas for both this and #2062

[eaquigley]

Since we have caused this usability issue by requiring subject and contact email now, we need to make it as easy as possible for users to be able to upload + edit files and terms without losing their work and making them go to the Edit Metadata section of their dataset. I am proposing we figure out how to have a pop up that appears when a user clicks Save on the Files or Terms page that has subject field and the contact email field for them to fill in.

Workflow:
User clicks Edit buttons->pop up appears saying something like, "This dataset requires the following metadata fields to be filled in before any further editing or file uploads can be completed."->user does this, clicks save button->boom, no more usability issues and a user can edit files or edit terms.

Will create mockups in Balsamiq and link here to show how this should be done in the UI.

cc: @mcrosas @posixeleni @scolapasta

[mheppler]

FRD: https://docs.google.com/document/d/1fb3dmslWg15zNLH7DMIOaUza2ykZqvpFL9aOz0VeNok/edit

Mockups: https://iqssharvard.mybalsamiq.com/projects/newlyrequiredmetadataprompt/grid

[scolapasta]

After discussing with @sbarbosadataverse @kcondon @eaquigley we decided to modify the workflow to allow save on these pages and use N/A. This was opened as issue #2448.

[kcondon]

This essentially works with one exception: You cannot create a draft by uploading files when there is a required field not filled in. I will open this as a separate issue. Closing