Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tsv metadata block import: add preview mode #2551

Closed
pdurbin opened this issue Sep 18, 2015 · 5 comments
Closed

tsv metadata block import: add preview mode #2551

pdurbin opened this issue Sep 18, 2015 · 5 comments
Labels
Component: Code Infrastructure formerly "Feature: Code Infrastructure" Type: Suggestion an idea

Comments

@pdurbin
Copy link
Member

pdurbin commented Sep 18, 2015

While working on #2310 I'm realizing it would be very handy to have a "preview" mode for loading tsv files as metadata blocks but not actually writing to the database. It would be a way to sanity check a tsv file. @sekmiller seems to like this idea as well.

Locally, I'm playing around with a "preview=true" query parameter that looks like this:

curl -s http://localhost:8080/api/admin/datasetfield/load?preview=true -X POST --data-binary @scripts/api/data/metadatablocks/customGSD.tsv -H "Content-type: text/tab-separated-values"

@scolapasta I'm not sure I should change this in 4.2 though. We can probably defer it so I'll push any change to a branch that we can merge in later.

(@bencomp I'm aware from #2357 that you're not a fan of the tsv import.) :)

@pdurbin
Copy link
Member Author

pdurbin commented Sep 18, 2015

In 948ff41 I have a proposal in a branch for adding the "preview" functionality I'm interested in.

(Mental note that http://localhost:8080/api/metadatablocks is handy way to see which blocks have been loaded.)

@pdurbin
Copy link
Member Author

pdurbin commented Sep 22, 2015

@sekmiller as we discussed, since you're looking into adding the ability to replace controlled vocabulary values per #2310 (comment) you might want to at least think about my proposal at 948ff41 to add a "preview" mode to the API endpoint for importing tsv files. I'll give this to you and move it to 4.2 but obviously you should feel free to move it out of 4.2 at any point.

This is how I was testing the preview mode I added in that branch (adding ?preview=true as a query parameter):

curl -s http://localhost:8080/api/admin/datasetfield/load?preview=true -X POST --data-binary @scripts/api/data/metadatablocks/customGSD.tsv -H "Content-type: text/tab-separated-values"

@pdurbin pdurbin assigned sekmiller and unassigned pdurbin Sep 22, 2015
@pdurbin pdurbin added this to the 4.2 milestone Sep 22, 2015
@scolapasta scolapasta modified the milestones: Candidates for 4.3, 4.2 Sep 25, 2015
@mercecrosas mercecrosas modified the milestones: Candidates for 4.3, In Review Nov 30, 2015
@sekmiller sekmiller removed their assignment Jan 28, 2016
@scolapasta scolapasta removed this from the Not Assigned to a Release milestone Jan 28, 2016
@pdurbin
Copy link
Member Author

pdurbin commented Apr 8, 2016

In a thread called "documentation for metadata block import format" @tdilauro started a Google doc where he's trying to document what he's figured out about how to make new metadata block: https://groups.google.com/d/msg/dataverse-community/VniGWZKlgiA/lKxiYO4iBQAJ

This seems on topic for this issue since before you can import a tsv file you've created (in "preview" mode or otherwise) you have to know enough about about how Dataverse processes the tsv file for a successful import. Basically, you need docs.

On a related note, the experiment by @raprasad to convert tsv files to JSON Schema at https://github.com/IQSS/json-schema-test is very interesting.

@pdurbin
Copy link
Member Author

pdurbin commented Jun 28, 2017

Does anyone else think this is a good idea? If so, please get in touch. Closing.

@pdurbin
Copy link
Member Author

pdurbin commented May 30, 2023

This new feature request sounds really useful:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Code Infrastructure formerly "Feature: Code Infrastructure" Type: Suggestion an idea
Projects
None yet
Development

No branches or pull requests

4 participants