Define optimal serialization format and variations (dropping or reviving the 2.9.2 format)

[koppor]

In JabRef 2.10, we introduced following new feature:

    Changed serialization of BibTeX entries:
        First, the required, then the optional and then all other fields are written.
        Thereby, fields are now ordered by name. Except the title, which is written first.
        The second word in of the BibTeX type is capitalized. E.g., Inproceedings got InProceedings
        Configurable: Start field contents in same column. Enabled by default.
        Configurable: Use camel case for field names (e.g., "HowPublished" instead of "howpublished"). Enabled by default.
        If no field name is given, then "UNKNOWN" is used. For instance, " = {X}" gets " UNKNOWN = {X}".

When a user collaborates with a person using an older version of JabRef, they cannot use version control properly as the serialization always changes. In version 2.11 beta 2, we offer a quick hack to go back to the old 2.9.2 behavior, which is somehow incomplete. See #10 (comment)

We can just focus on other issues and let time solve the issue or invest time to implement the old serialization again.

Related issue: #115

Request at sourceforge: https://sourceforge.net/p/jabref/feature-requests/864/

To track the progress of implementation, the consensus described below is added here.

Rationale: People are very emotional about formatting -> Modify as little as possible in the bib file

• Unmodified entries will not be formatted and written back exactly as read in (including all formatting issues, etc.
• Modifying any part of an entry results in a reformatting of the complete entry
• No more global configuration forsorting of entries on load / save. (sorting information is now stored in the file)
• New entries are added to the bottom of the file
• Fixed and non-configurable format for new or modified entries. Dropping all field saving options (Preferences -> File -> Field saving options)
• Fields are sorted as stated in the Bibtex / Biblatex manuals.
• The = is appended directly to the field key. The value part is indented to one space past the longest field key name + = (so that all values are aligned).
• Bibtex entry keys are written in camel case, starting upper case
• Field keys are formatted in lower case only
• No longer perform any changes of field content (no more space/tab/newline elimination, etc.)
• Provide formatter (Example: Remove tabs / newlines / duplicate spaces) that can be enabled by a user for specific fields onLoad / onSave. These are not enabled by default.
• If in doubt, do as biber does

[stefan-kolb]

1. We should NOT add compatibility patches. We want people to switch to JabRef 3.
2. We should probably revert some changes to be more BibTeX conform. We want to encourage usage of JabRef not plain editing and readability of bib files.
   • Remove the blank line (see Blank lines between required and optional fields #115)
   • Revert capitalization of second word of BibTex type and just use the standard tag (to investigate)
   • Revert camel casing of field names

Dunno what Configurable: Start field contents in same column. Enabled by default means, yet.

[koppor]

JabRef is BibTeX conform. There are habits only :)

I think, we should enable readability as the .bib files can be put under version control, compared, quickly edited, etc. Otherwise, we produce a kind of a binary format, which is hard to handle in version control systems (see versioned .docx documents).

I tend to like the new format. We can discuss whether we remove the casing again. I got used to it, but no one else offers such a format.

Maybe we should start a wiki page with all possible formats and vote on the preferred format and implement that.

Configuration options

See Options -> Preferences -> File -> "Field saving options"

• Use camel case for field names
• Start field contents in same column
• Include empty fields
• Wrap fields as ver 2.9.2
• Field sorting options: alphabetic, unsorted, user-defined order

Furthermore, one can configure the ordering of the entries: See "Database properties": Save as configured globally, original order, order as specified.

Should we drop all these options and define "the one and only" serialization? What about being as minimal intrusive as possible? For instance, when a group of researchers uses different tools to manage their bibliographies?

[mlep]

Sharing a file while using a revision control software causes trouble if the Field saving options are not identical among users.
Hence, to facilitate sharing, I suggest to add in the See Options -> Preferences -> File -> "Field saving options" a "master switch" named something like "Default sharing format". That way, to make sharing smooth, users will simply have to ensure this single switch is selected.
I do not know which of the 5 above options should be selected/unselected by the "Default sharing format" switch, but I am sure that ensuring that everybody have the same settings among 5 options is going to be a pain.

[stefan-kolb]

Considering the configuration options you mentioned, Olly, i would suggest to remove (most of) them and just define one format for JabRef that suits our ideas and concept. This removes the problems that mlep described.

[mlep]

Well, "one format [...] that suits our ideas and concept", may not suit users...
These could be because of personal preferences or because of compatibility with other software. So, I would suggest to keep the possibility of various field saving options.

[koppor]

I think, this is the discussion GNOMEv2 vs. GNOMEv3, where they removed various configuration options. I see the following options:

A) Remove all configuration options and introduce a "one size fits all" format
B) Keep everything as is
C) Add support for the old 2.9.2 format

A) This increases testability (as less configuration combinations have to be tested) and eases sharing of BibTeX files across JabRef users.
B) We still have to add test cases. - Side comment: At "database properties", one can add the file sorting options. We could add a button "make database ready for sharing" or, even better, set a suitable default JabRef configuration. As soon as the old version vanish (and the users have reset their preferences), additional configuration is only required for special cases
C) This causes much effort (at least 20h) and is only because there are users not updating their software. Even debian stable has JabRef v2.10.

In all cases, we should check whether other BibTeX-based managers or compatible. We can also neglect them and assume that other BibTeX managers are used seldomly. The Microsoft Office way ^^.

[stefan-kolb]

I agree with you all, there is no wrong or right here. My main argument is that

• configuration makes usage difficult as you have to know how to configure and that there is a configuration option for certain features
• makes portability problematic as there might be a lot of different user configurations
• is a threat to testing and stability because of various possible states of configuration

Therefore my personal principles

• should only be used if there is a real benefit by this configuration option
• the good old convention over configuration concept really condenses my personal favors.

[mlep]

To me, C) should not be implemented. When a new version (of a freely available software) is released, it is the responsibility of the user if he/she want to use the old version.
Between A and B, I favor B: if the default settings are the best for sharing, it is the responsibility of the user to use another configuration. And to report a bug if he/she find one (so, the user does the testing ;-) )

[koppor]

Regarding C), I get more bug reports than usual, which is an indicator that it is important for our users. See sf feature request #864 and related bugs. I can be the bad guy and just say "We don't have resources for that. If someone volunteers, we will include the patch.". Or we could collect donations for it. Or we should put it in the survey as option for next development (what we currently have)?

FLOSS COACH says: "some people are direct, critical or seem to be rude." 😎

[mlep]

Well, if C) is a feature requested by user, I guess it should be kept...
And if the main contributors to JabRef do not have time to do it, I do not see anything bad about asking the community for a volunteer.

[lenhard]

I'll throw in my hat for option A.

From my point of view, JabRef has gotten so complex and ill-structured that there's a considerable risk that it will die because no one can maintain or extend it anymore. Following this logic, anything that reduces complexity, even if we lose features, is desirable.