On the backwards-compatibility or future-proofing of ekos

[scarlehoff]

As you know, I'm trying now to get rid of all the legacy code from nnpdf for which we no longer have a maintainer. For that one of the things we need to do is to replace apfel with eko (i.e., remove the _new modifier from evolven3fit_new). By now I think eko has been time-tested more than enough for this change... and I really would like to be able to kick evolven3fit out of the default installation.

however....

There is the issue of backwards compatibility, while eko is faster and more developed than apfel the ekos often cannot be used from one version to the next. This is a big problem since nnpdf releases usually contain O(20) theories (and with N3LO and MHOU we are moving to O(100)).
I understand the rapid development associated with the qed/n3lo/mhou releases meant compromising on some other things but now that that's finished... would it be possible to offer some kind of guarantee that an eko.tar for the evolution of the PDFs computed with a version of eko will be compatible with future versions?

Of course, there might come a time when it is necessary to break backwards compatibility for whatever the reason. Right now NNPDF releases happen more or less every ~ 4 years, so I would like that the theories that are prepared for a release are utilizazble until the next one (without having to update all ekos in all theories).

I'm not asking for bug-by-bug compatibility, but to include a compatibility layer every time the format of the eko.tar would change. I would prefer a compatibility-layer in eko that is able to recognize "oh, this is an eko using the 0.15 format, let's use the 0.15 compatibility layer" rather than a conversion mechanism but I would accept a conversion mechanism as well if it is well tested.

In practical terms (I talked about guarantees before, but what does that mean) I would like a test in the CI in this repository (I can write that) that goes through a set of representative theories and checks that the ekos are still usable...

[alecandido]

#106

[scarlehoff]

#214

However, if you think that the current version of the ekos/conversor is already robust enough I can write the test starting from 0.14.

I just want to make sure that people can, at the same time, use the latest version of nnpdf and eko.

[alecandido]

#214 is definitely outdated, a different solution has been applied in the meanwhile (i.e. recomputing when actually needed).

The only possible instability is now related to the runcards content, since it keeps being modified.
However, this should be easy enough to translate every time, by a combination of applying defaults and removing fields.

@scarlehoff write a test for version 2, after bumping this:

eko/src/eko/version.py

Line 8 in 8372ec3

__data_version__ = 1

current EKOs will version 1 set, but we can manually label with version 2 the moment we know they are working.
At least, we will be able to classify the various EKOs.

cc @felixhekhorn v0.15 should then update it to version 3.

[alecandido]

cc @felixhekhorn v0.15 should then update it to version 3.

Or better, let's make it 3dev. At least people should be warned that using master to compute EKOs have trade-offs.

[scarlehoff]

I'm happy with waiting for 0.15 with doing this. I just want to know there will be a point from which if I compute the ekos for the theories used in 4.0 I won't have to worry about them not working.

[alecandido]

Even fine, then version 2 will be the one produced by v0.15. But this will leave behind all the EKOs that have been currently produced.

[scarlehoff]

Then beter if I wait for 0.15 before computing O(20-50) ekos xd

[alecandido]

Then beter if I wait for 0.15 before computing O(20-50) ekos xd

This could happen in weeks. Consider that we are about to release next version of EKO since months (the big PR is #295).

[felixhekhorn]

The relevant issue is #145

[alecandido]

The relevant issue is #145

Nope, that is concerning the API in general, but the problem here is the backward support for computed data. We will need the converters.

[felixhekhorn]

Nope, that is concerning the API in general, but the problem here is the backward support for computed data. We will need the converters.

okay, let's say the two are related - the converter itself would surely fall under the public API

More on practical terms: would we add the converters in ekobox? (i.e. outside the library) this would make ekobox part of the public API (just something to keep in mind)

[scarlehoff]

If you go the converter route I'd need to be able to install it as a dependency to nnpdf to fallback to it whenever evolven3fit_new gets a too-old-to-read eko

[alecandido]

The converter will part of .io, and it will convert one version up.

So, we will have something like:

eko/
  io/
    convert/
      __init__.py
      v2.py
      v3.py
      ...

and they will be used to up-convert (downgrade will never be supported).

So, they will ship with the main eko package. ekobox can wrap in the CLI, but let's focus the support and stability on the core package.

[scarlehoff]

The converter will part of .io, and it will convert one version up.

Meaning you can always chain conversions, right?

[alecandido]

Meaning you can always chain conversions, right?

Exactly. We could even have a function to chain automatically, that is going to be simple :)