jabref-meta storage in bib file should be improved (by switching to embedded JSON)

[koppor]

Context

While seeing that diff, I thought, something is really wrong:

The semicolon on position 1 indicates that multiple meta data items are can be written into @Comment. This was clear to me today (and not in 2016 #960). This would be great as this would minimize the number of @Comment entries. However, the saveActions also use ; as delimiter (position 2).

The "feature" of non-merging the meta fields is long time present. See e.g., an old issue report #250.

Thus, a straight-forward merge is most probably not possible.

Code hint: Separation according to ; is done at org.jabref.logic.importer.util.MetaDataParser#getNextUnit

---

Call for new metadata storage

Single JSON in @comment field

Example:

@Comment{jabref-meta-0.1.0
{
  "saveActions" :
  {
    "state": true,
    "date": ["normalize_date", "action2"],
    "pages" : ["normalize_page_numbers"],
    "month" : ["normalize_month"]
  }
}
}

Content:

{
  "saveActions" :
  {
     "state": true,
    "date": ["normalize_date", "action2"],
    "pages" : ["normalize_page_numbers"],
    "month" : ["normalize_month"]
  }
}

Decision outcome: Use "Single JSON in @comment field"

---

Migration path:

• v6.0 can read and write both setting formats
  • when reading, the new format "wins" (if both exists)
• v7.0 can read both settings, but writes only new setting format

---

After this is implemented, we can work on #8701

---

ADR

Single JSON in @comment field

• Good, because a single @Comment element is enough
• Good, because JSON parser can directly be used
• Good, because we can nest elements in the json without the need of a custom format
• Neutral, because JSON is nested in BibTeX
• Bad, because syntax highlighting won't work
• Bad, because the meta format changes
• Bad, because looks "hacky"

Multiple JSON

Each preference could have a separate JSON nesting.

• Bad, because lookup would be done using BibTeX data and second lookup using JSON. The preferences should be in a consistent format.

BibTeX

Example (From koppor#232)

old:

@Comment{jabref-meta: saveActions:enabled;
date[normalize_date]
pages[normalize_page_numbers]
month[normalize_month]
;}

new:

@JabRef{saveActions,
  state = {enabled},
  date = {normalize_date, action2}
  pages = {normalize_page_numbers}
  month = {normalize_month}
}

• Good, because feels natural
• Good, because no additional parsing logic needs to be implemented
• Good, because we currently have only one level of key/value pairs for the meta data (to be checked)
• Bad, because even nested list (e.g., normalize_date, action2) is a custom format.
• Bad, because multiple elements have to be used: One for each meta data key
• Bad, because does not allow for nesting of properties
• Bad, because other tools might treat these entries special
• Bad, because "old" JabRef versions will treat these entries as "normal" entries

@comment and then nested

JabRef v5.9 (and before) used that format.

• Good, because arbitrary content can be used
• Bad, because the parsing logic needs to be written for the content inside

JSON at the end of the file

New entries always start with @. Anything outside the “argument” of a “command” starting
with an @ is considered as a comment. This gives an easy way to comment a given entry: just
remove the initial @. As usual when a language allows comments, don’t hesitate to use them so
that you have a clean, ordered, and easy-to-maintain database. Conversely, anything starting
with an @ is considered as being a new entry

@Article{demo,
   note={just an example article to illustrate the **previous** entry}
}

// jabref-meta-0.1.0
{
  "saveActions" :  {
   "state": true,
   "date": ["normalize_date", "action2"],
   "pages" : ["normalize_page_numbers"],
   "month" : ["normalize_month"]
  }
}

[Siedlerchr]

BibDesk on mac stores its groups into apple plist xml format:

[koppor]

Follow-up issues:

• Generic way to store preferences in .bib file / self-contained library settings #8701
• Enhance customization of entry types #9840 (comment)