Separating imgCIF development from CBFlib?

[jamesrhester]

I'm wondering if anybody sees an advantage in separating the development of imgCIF, the dictionary, from CBFlib the library? They are, after all, logically different. imgCIF could become a separate repository under yayahjb, or the COMCIFS organisation could be used, or indeed some other organisation. There is an imgCIF repository under COMCIFS at the moment that directs people here and has an older version of the dictionary.

It is possible to split a Git repository so that it retains the history of particular files, so for example it would be possible to create a new imgCIF repository that included all the history of the imgCIF.dic files (and only those files).

[yayahjb]

Sounds like a guarantee of divergence between the specifications and the prototype implementation. Is that really a good idea?

…

On Tue, Jan 18, 2022 at 12:21 AM James Hester ***@***.***> wrote: I'm wondering if anybody sees an advantage in separating the development of imgCIF, the dictionary, from CBFlib the library? They are, after all, logically different. imgCIF could become a separate repository under yayahjb, or the COMCIFS organisation could be used, or indeed some other organisation. There is an imgCIF repository under COMCIFS at the moment that directs people here and has an older version of the dictionary. It is possible to split a Git repository so that it retains the history of particular files, so for example it would be possible to create a new imgCIF repository that included all the history of the imgCIF.dic files (and only those files). — Reply to this email directly, view it on GitHub <#34>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABB6EAKFH2VWRF5HXJGYVSTUWT2F7ANCNFSM5MGG7PNA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jamesrhester]

I don't see that there is anything magical about having the reference implementation and the standard in the same repository that ensures that they match. Indeed, where is the guarantee at the moment that a CIF written according to the latest version of imgCIF.dic will be correctly interpreted by CBFlib? And if there is such a check, couldn't that exact same mechanism work if the implementation and standard are in separate repositories? Github CI would allow us to implement all sorts of checks before anything is committed, including requiring an example CIF and running a CBFLib test suite on it.

Meanwhile, it sounds like the ideal at the moment is that all changes to imgCIF must be reflected in CBFLib. For example, before _diffrn_data_frame.detector_id can be added (as it logically must be) to imgCIF.dic, all necessary changes to the Fortran code must also be included in the changeset. That is, even if the change to imgCIF is transparently correct, someone has to change Fortran code before it can be accepted into imgCIF. This has the disadvantage of requiring contributors to be conversant with Fortran and leaving imgCIF at the mercy of the CBFLib implementation.

I'm not arguing against the principle of requiring a reference implementation for standards. However, the "everything in one repository" is not the only model that can be adopted. Imagine, for a moment, that imgCIF and CBFlib are in separate repositories: imgCIF could have a "development" tag, a "master" tag, and a "tested" tag, where everything in the "tested" branch has been implemented by at least one freely-available software library, and everything in the "master" tag has been accepted as theoretically correct, but may not have been implemented anywhere. Depending on the preferred policy, either "master" or "testing" versions are the released versions. Any commit to the "testing" versions is auto-checked by CI tools to make sure that e.g. CBFLib tests pass. In what way is this worse than the current situation?

I think also that we can have more confidence in our changes than most due to the relational nature of imgCIF: this gives us modularity, so that we know that adding non-key columns to one category limits the effect to that category and in general interactions between categories are explicitly controlled. So the recent additions of the _array_data.external_* items enhance only the behaviour of accessing array_data items, they cannot affect e.g. descriptions of axes.