Pre-existing datasets cannot be uploaded to through the API

[rliebz]

Some datasets exist on dataverse.harvard.edu that were migrated from thedata.harvard.edu, but do not meet the new standards for metadata. This includes fields such as "Subject" and "Contact Email" that are now required.

Expected:

Because uploading a file does not involve updating dataset metadata, a file upload to a dataset with incomplete metadata should be allowed.

Actual:

For datasets with incomplete metadata, the SWORD API will return the following exception when attempting to upload files:

<?xml version="1.0" encoding="ISO-8859-1"?>
<sword:error href="Couldn\'t update dataset edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException: Validation failed: Subject is required." xmlns:sword="http://purl.org/net/sword/terms/"><atom:title xmlns:atom="http://www.w3.org/2005/Atom">ERROR</atom:title><atom:updated xmlns:atom="http://www.w3.org/2005/Atom">2015-04-21T11:05:43Z</atom:updated><atom:generator uri="http://www.swordapp.org/" version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"/><sword:treatment>Processing failed</sword:treatment><atom:summary xmlns:atom="http://www.w3.org/2005/Atom">Couldn\'t update dataset edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException: Validation failed: Subject is required.</atom:summary></sword:error>

[pdurbin]

Similar problems have been reported from people using the GUI as well:

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=196320
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=196481
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=196609
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=196752

It seems like we've been recommending to people that they edit metadata and add the required fields but let me check with @posixeleni and @sbarbosadataverse

[pdurbin]

Related: #2051

[posixeleni]

@pdurbin migrated or new datasets, going forward, if someone in the UI goes into edit a dataset they will need to add the subject and the error messaging needs to be clear (if not already). The API would need some way to let a user know what the controlled vocabulary for a subject is otherwise how would they be able to successfully fill in the value(s) that the system is expecting?

[pdurbin]

@posixeleni right, #1510 is about exposing the possible values for "Subject".

@rliebz at a high level, one thing OSF could do is download the metadata for a dataset in JSON, fill in the Subject field, and re-upload the new JSON to create a new draft version. (That's basically what we're telling people to do in the GUI.) Then you can upload files.

[scolapasta]

We have also discussed the solution being to allow the file upload but not allow to publish if not valid. We will be looking at these types of issues in 4.01 or 4.0.2.

[scolapasta]

Assigning to 4.0.1 so we can decide what we want the solution to be.

[pdurbin]

I created an issue for the Python client with an example script of how to update metadata with JSON: IQSS/dataverse-client-python#15

[pdurbin]

This issue is quite old. I think an acceptable workaround is to edit the migrated dataset in the GUI to add whatever is being complained about such as a subject. Vote to close.