Fix Q2grid in evolven3fit_new

[giacomomagni]

Unfortunately Q2 values from eko.evolgrid (eko=0.13.2) are not sorted anymore, so we need this temporary fix to produce correct lhapdf files, otherwise the interpolation is just crazy.

We should consider to fix this in ekobox directly in the newer version ( @all_eko_autors ).

cc @niclaurenti, @andreab1997

[scarlehoff]

Is the error still there for 0.13.3?

Just so I know what the problem is (in order to add it to the tests of #1746), the problem is that the values are not sorted only in the info file? Or does this problem also occur in the grids themselves?

The reason I asked is that I'm worried that we are sorting it in the info file, so everything looks ok, but they are wrong in the grid files.

[giacomomagni]

Is the error still there for 0.13.3?

Yes between 0.13.2 and 0.13.3 only a bug in QED was fixed.
This problem was not spotted yet.

Just so I know what the problem is (in order to add it to the tests of #1746), the problem is that the values are not sorted only in the info file? Or does this problem also occur in the grids themselves?

The reason I asked is that I'm worried that we are sorting it in the info file, so everything looks ok, but they are wrong in the grid files.

Indeed this problem occurs inside the grids:Q2 blocks are not sorted so somehow lhapdf is not able to interpolate them.
Everything works fine, except that the pdfs values are just a nonsense.

[scarlehoff]

Indeed this problem occurs inside the grids:Q2 blocks are not sorted so somehow lhapdf is not able to interpolate them.
Everything works fine, except that the pdfs values are just a nonsense.

I see. So the grid itself is sorted (the values are correctly sorted per Q2) but the Q2 "header" is not? And your fix deals only with the header, right?

[giacomomagni]

I see. So the grid itself is sorted (the values are correctly sorted per Q2) but the Q2 "header" is not? And your fix deals only with the header, right?

Wait I might have misunderstood. This fix adjust both the grid values and the header in each replica file.

I'm now opening a PR on eko to do the proper fix there.

[scarlehoff]

Ah, nono, I misunderstood. I thought by "Q2 block" you meant the part of the grid where the Q2 values are written. Now it's clear. If this can be fixed & released in eko relatively quickly it would be better (and then we put here the ">0.13.4" in the conda file)

[alecandido]

Just to comment about the origin: before EKO was sorted because it was filled sorted, and then the sorting in the runcard was propagated to the output (had the input been sorted different, even the output would have).

However, at the moment this input/output connection is completely lost, because the EKO is possibly computed in a distributed way, and it was unreasonable to keep track of it.
In principle, we could sort the iterator itself, manually imposing a given ordering. Currently, it is not done: the iteration sequence is UB, you only know that iterating until the end you will get it all (in practice, it is iterating over the files in the tar, following the hash of the headers - so it is random more or less in the same sense of pRNG).

For PDF grids, it is relevant to keep them sorted, so NNPDF/eko#280 is the proper fix. For the iterator itself, I'd suggest that the user should invoke sorted() by himself, passing explicitly the key = lambda ep: ... (I'd suggest lambda ep: (ep[1], ep[0]) for most applications, since the evolution point is a pair (mu, nf), but you mostly want to block by nf, and sort within each block).

[scarlehoff]

Thanks! I just merged the new tag of eko into conda forge so I'll rerun the tests in a bit (to give condaf time to index it)

[scarlehoff]

I think I 3found another bug/convention difference: the actual qmax that one gets is defined by the operator but the info file will be set by the info file. This should also be fixed. I think both this and the order can (should?) be fixed from the evolven3fit side so I'll do the changes myself in this PR.

[scarlehoff]

(By "this PR" i actually meant #1750, this one I meant to merge but just realized I still had both tabs opened)