Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Metadata for data files #1603

Merged
merged 3 commits into from
Oct 14, 2016
Merged

[ENH] Metadata for data files #1603

merged 3 commits into from
Oct 14, 2016

Conversation

janezd
Copy link
Contributor

@janezd janezd commented Sep 23, 2016

@astaric, you meant something like this?

screen shot 2016-09-24 at 00 51 54

Orange already read and wrote .metadata files, but in pickle format. This is highly inconvenient since the user can't edit it. I changed the format to a (very) simplified YAML: the file is expected to contain a single block mapping. Data is read as string. If we wish, we can either try to evaluate it (with fallback to string) or use PyYAML, although I don't think we need it.

@astaric
Copy link
Member

astaric commented Sep 24, 2016

@BlazZupan, @markotoplak does this change break your use of metadata?

@kernc
Copy link
Contributor

kernc commented Sep 27, 2016

It does break reading any existing metadata files. Might be avoided excepting to old unpickling.

@codecov-io
Copy link

codecov-io commented Sep 27, 2016

Current coverage is 88.67% (diff: 100%)

Merging #1603 into master will increase coverage by <.01%

@@             master      #1603   diff @@
==========================================
  Files            78         78          
  Lines          8124       8131     +7   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           7203       7210     +7   
  Misses          921        921          
  Partials          0          0          

Sunburst

Powered by Codecov. Last update f021cfd...ecc839d

@janezd
Copy link
Contributor Author

janezd commented Sep 27, 2016

I had it implemented (the other way around, I first checked whether it's a pickle). Then I "discovered" (don't ask me how) that nobody uses .metadata and decided to switch to a "better" format while it's still time.

I re-added it. For writing, pickle is used if there already exists a metadata file that is a pickle, or if the current attributes are not an OrderedDict with strings as keys and values.

This is imperfect, but I think that having a plain text format is better than a binary - when possible. I wouldn't like to have two metadata files. Any better ideas?

@janezd janezd force-pushed the file-metadata branch 3 times, most recently from ca75dee to 7e86f80 Compare September 30, 2016 12:38
@janezd
Copy link
Contributor Author

janezd commented Sep 30, 2016

Fixed as discussed:

  • when loading, it tries to unpickle, then to read as "YAML"
  • when saving, it uses "YAML" if the attributes are a dict with string keys and values

@astaric astaric merged commit 42c9910 into biolab:master Oct 14, 2016
@janezd janezd deleted the file-metadata branch October 14, 2016 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants