Replace `Grid::axes` with `Grid::evolve_info` about pineko

[scarlehoff]

axes and evolve_info do not produce exactly the same xgrid in all scenarios.

In order to test #84, I'm regenerating fktables for the whole of theory 400. Doing that I've discovered that for some grids (more explicitly, for ATLASDY2D8TEV-aMCfast_obs_1.pineappl.lz4, note that obs_0 is ok!) evolve_info and axes return grids of different sizes.

This is independent of the orders selected when asking for evolve_info.

Inspecting the two x-grids returned, axes and evolve_info differ because the grid returned by axes repeats two values:
0.00094688 and 0.00335233. I'm assuming evolve_info is rightly sanitising the results while axes is not. All checks of pineko pass with flying colors (because the content of xs in the grid and the operator is indeed the same). However, during the evolution, pineappl says "no way Jose".

I'm copying the error below although now I think that maybe it is a pienappl bug (why are evolve_info and evolve using a different grid?) or a grid problem (why does the grid repeat one value of x?) or both... @cschwan @alecandido @felixhekhorn

The grid is here. The error occurs independently of whether the eko was computed using main or #84 (so, it occurs independently of how the eko was computed, since the incompatibility seems to be introduced here, so using the xgrid from axes works but from evolve_info doesn't).

Python error when using pineko:

  File "/media/juacrumar/pinekoing/pineko/src/pineko/evolve.py", line 209, in evolve_grid                                                                                                                             
    fktable = grid.evolve(                           
  File "/media/juacrumar/pinekoing/pinevortex/lib/python3.10/site-packages/pineappl/grid.py", line 338, in evolve                                                                                                     
    self.raw.evolve(                                 
pyo3_runtime.PanicException: Nothing returned from evolution.: EvolutionFailure("operator information does not match grid's x-grid values")  

I've tried using the cli but I get the following error independently on which branch I used to compute the eko.

Error: missing field `Q2grid`

[alecandido]

Inspecting the two x-grids returned, axes and evolve_info differ because the grid returned by axes repeats two values:
0.00094688 and 0.00335233. I'm assuming evolve_info is rightly sanitising the results while axes is not.

In principle, the deduplication is almost the same:
https://github.com/NNPDF/pineappl/blob/f2152da2a6b0c387b6b4c999b9025489dc02c580/pineappl/src/grid.rs#L1808-L1809
https://github.com/NNPDF/pineappl/blob/f2152da2a6b0c387b6b4c999b9025489dc02c580/pineappl/src/grid.rs#L1353-L1354

The only differences are in the sorting step:

# .axes
x_grid.sort_by(|a, b| a.partial_cmp(b).unwrap());
# .evolve_info
x1.sort_by(f64::total_cmp);

and that axes is deduplicated only just before returning, while evolve_info is doing it for each iterated subgrid.

The other major difference is related to your original issue about Grid::axes: that is checking all x-grids to be the same for one (order, bin), and concatenating across different (order, bin), while Grid::evolve_info is just concatenating them all.

[felixhekhorn]

I've tried using the cli but I get the following error independently on which branch I used to compute the eko.

Error: missing field `Q2grid`

which CLI do you mean? eko? do you have more of the environment (e.g. exact call and full error)?

[scarlehoff]

pineappl evolve grid.lz4 eko.tar nnpdf40

[cschwan]

It probably can't cope with the new/changed (?) EKO format.

[alecandido]

It probably can't cope with the new/changed (?) EKO format.

https://github.com/NNPDF/pineappl/blob/f2152da2a6b0c387b6b4c999b9025489dc02c580/pineappl_cli/src/evolve.rs#L31

Now called mu2grid, because Q2 was inconsistent...

[cschwan]

• @scarlehoff where can I get the EKO?
• @alecandido is the EKO format version documented somewhere?

[alecandido]

• @alecandido is the EKO format version documented somewhere?

Properly documented, no, I'm sorry.
We still need to make some small changes to account for the numerical FONLL project, and I'm currently updating the runcard format NNPDF/eko#220

However, in principle it is actually pretty simple, and those changes are going to have a minor impact, so I can write it here and actually add to the docs, even if it will be updated and improved:

• an EKO file is a tar file, e.g. eko.tar, with a specific internal structure
• the internal structure is currently dictated by a dedicated class in eko: eko.io.struct.InternalPaths, and it is:

  metadata.yaml
  theory.yaml
  operator.yaml
  recipes/
  parts/
  operators/
  

  this is pretty stable, and there is no foreseen change
• the user facing part is mostly metadata.yaml and operators/, both have an interface in EKO, specified by the dedicated classes (eko.io.struct.Metadata, and directly eko.io.struct.EKO for the second) - you might be interested in the original runcards, but they are there mainly for reference (usage required information is duplicated and possibly updated in metadata.yaml)
• the actual operators are stored in operators/ in .npy format (or .npz, in case they include integration error) as rank 4 arrays
• the exact name reflects the value of their final scale, but this is going to change (because numerical FONLL will require to specify both the scale and the number of flavors, so the scale alone will be ambiguous)

[scarlehoff]

In the atached .tar.gz you can find one grid, one eko and one script that does the evolve that pineko does. If inside the script you use the xgrid form axes instead of evolveinfo it works.

You can run it as python evolve_this.py ATLASDY2D8TEV-aMCfast_obs_1.pineappl.lz4 ATLASDY2D8TEV-aMCfast_obs_1_mainbranch.tar if you have eko and pineappl installed.

tmp.tar.gz

[cschwan]

I'm not sure what's going on here, but It seems the EKO is wrong. How can I see the process-level x-grid values of the operator? Is there an easy way to extract them from the tarball?

[scarlehoff]

I think you can obtain the grid you want doing:

np.exp(operators.rotations.targetgrid.grid)

At the end of the script in the tarball (note the np.exp since the grid you get already had a log applied to it). This grid corresponds to the grid that pineappl returns when you doevol_info.

I would argue that even if the eko is wrong, pineappl should not return two different grids with axes and evolve_info.

[alecandido]

Another undocumented feature: Xgrid.raw is a property showing exactly the raw grid, independently on being log or not.

Thus

operators.rotations.targetgrid.raw

is what you want.

[cschwan]

I need the x-grid values at process level, not a fit level. What you told me gives me points at fit level, I think.

[alecandido]

You're always thinking the opposite way of EKO: in principle, evolution is meant to evolve a fit at all scales, so the process-level grid is called targetgrid.
The inputgrid is for the fit-level input.

Of course, for FK-tables it works just the opposite.

Unfortunately, EKOs are quite "covariant", so there is not a unique way to identify it: they might go up in scale or down, and evolve a grid or a PDF.
In practice, there is almost no safe way to call the various ends, so we picked the PDF point of view to give names. Thus, input is input-PDF, and target is target-PDF. We could have called processgrid and fitgrid, but it would a bit more counterintuitive for post-fit evolution.

However, if you have a better proposal is the perfect time to tell it. Later on, we'll try to freeze the names (they are almost already frozen, but we know we have to make a few breaking changes).

[cschwan]

Technically speaking this is a bug in Grid::evolve, see here: NNPDF/pineappl#223.

[cschwan]

This should be fixed in master, let me know if you need a prerelease.

[scarlehoff]

Thanks! I justs tested it and seems to work ok.

Then #84 can be merged to master, but if you prefer @andreab1997 I can rerun theory 400 to ensure that fixing this grid didn't break any others.

A pre-release will be needed I guess in order to make a release of pineko (I would put a minimum version of pineappl == 0.6.0 in #84 in any case... does poetry accept pre-releases @alecandido -since I would need at least alpha.5)

[andreab1997]

Thanks! I justs tested it and seems to work ok.

Then #84 can be merged to master, but if you prefer @andreab1997 I can rerun theory 400 to ensure that fixing this grid didn't break any others.

A pre-release will be needed I guess in order to make a release of pineko (I would put a minimum version of pineappl == 0.6.0 in #84 in any case... does poetry accept pre-releases @alecandido -since I would need at least alpha.5)

If you can rerun 400 please do it

[scarlehoff]

Ok, @cschwan just to make sure. What you did should've changed only the behaviour of evolve, right? evolve_info hasn't changed (so I can use the same ekos).

[cschwan]

@scarlehoff exactly, I only changed the numerical tolerance in functions called by Grid::evolve, and that leads to it finding the grid values it previously didn't find. Grid::evolve_info is unchanged.

I made another prelease here: https://pypi.org/project/pineappl/0.6.0a5/.

[alecandido]

A pre-release will be needed I guess in order to make a release of pineko (I would put a minimum version of pineappl == 0.6.0 in #84 in any case... does poetry accept pre-releases @alecandido -since I would need at least alpha.5)

Yes, with the allow-prereleases flag:
https://python-poetry.org/docs/dependency-specification/#expanded-dependency-specification-syntax

[scarlehoff]

Ok, I've rerun all fktables, that worked, so this doesn't break any of them. Yey!

However, since we have the fktables from @andreab1997 I also did a 1-1 comparison between them using convolute_with_one... and those results are not that good...

For some datasets, such as HERACOMB_SIGMARED_B.yaml (HERA_NC_318GEV_EAVG_SIGMARED_BOTTOM) I get differences beyond the % level.

>>> new_results
array([0.00119094, 0.0023079 , 0.00400382, 0.00271623, 0.00514309,
       0.00322998, 0.00218821, 0.00542587, 0.01225607, 0.01073866,
       0.00809805, 0.0059266 , 0.004914  , 0.0032805 , 0.01587305,
       0.01319804, 0.01007652, 0.00752765, 0.02447162, 0.01361705,
       0.00881446, 0.02106204, 0.01058296, 0.00814256, 0.02137009,
       0.00944453, 0.00827178])
>>> good_results
array([0.00119224, 0.0023125 , 0.00401096, 0.00272189, 0.00515239,
       0.00323755, 0.00219423, 0.00543734, 0.01221178, 0.01070744,
       0.00808576, 0.00592559, 0.0049167 , 0.00328678, 0.01583355,
       0.01317095, 0.01006146, 0.00752021, 0.02440836, 0.01358561,
       0.00879478, 0.02100241, 0.01055086, 0.00811102, 0.02013338,
       0.00871796, 0.00777171])

And after a few checks I can confirm that these differences are due to the EKO. I get the same result, it doesn't matter whether I use main or remove_axes_call (so I would say #84 is ok, in the sense that it doesn't generate a regression)

but if I use the ekos that @andreab1997 has in his folder for 4244 I get his fktables (even if I evolve them with my branch)

---

Then I tested against the fktables that we have in the server and those are again different but I'm not sure whether they are supposed to be different anyway...

Not sure what to make of this... if you think this is enough to merge #84 please do. If you want to investigate this further... let me know.

[andreab1997]

Ok, I've rerun all fktables, that worked, so this doesn't break any of them. Yey!

However, since we have the fktables from @andreab1997 I also did a 1-1 comparison between them using convolute_with_one... and those results are not that good...

For some datasets, such as HERACOMB_SIGMARED_B.yaml (HERA_NC_318GEV_EAVG_SIGMARED_BOTTOM) I get differences beyond the % level.

>>> new_results
array([0.00119094, 0.0023079 , 0.00400382, 0.00271623, 0.00514309,
       0.00322998, 0.00218821, 0.00542587, 0.01225607, 0.01073866,
       0.00809805, 0.0059266 , 0.004914  , 0.0032805 , 0.01587305,
       0.01319804, 0.01007652, 0.00752765, 0.02447162, 0.01361705,
       0.00881446, 0.02106204, 0.01058296, 0.00814256, 0.02137009,
       0.00944453, 0.00827178])
>>> good_results
array([0.00119224, 0.0023125 , 0.00401096, 0.00272189, 0.00515239,
       0.00323755, 0.00219423, 0.00543734, 0.01221178, 0.01070744,
       0.00808576, 0.00592559, 0.0049167 , 0.00328678, 0.01583355,
       0.01317095, 0.01006146, 0.00752021, 0.02440836, 0.01358561,
       0.00879478, 0.02100241, 0.01055086, 0.00811102, 0.02013338,
       0.00871796, 0.00777171])

And after a few checks I can confirm that these differences are due to the EKO. I get the same result, it doesn't matter whether I use main or remove_axes_call (so I would say #84 is ok, in the sense that it doesn't generate a regression)

but if I use the ekos that @andreab1997 has in his folder for 4244 I get his fktables (even if I evolve them with my branch)

Then I tested against the fktables that we have in the server and those are again different but I'm not sure whether they are supposed to be different anyway...

Not sure what to make of this... if you think this is enough to merge #84 please do. If you want to investigate this further... let me know.

So I would say that this is normal because those ekos I produced are produced with a branch of eko (that will be merged soon) because there the factorization scale variation was fixed. So if you produced the ekos with master of course they do not agree. In the server instead I should have exactly the same fks I have in dom.

[scarlehoff]

If I use your ekos I get your fktables (meaning, the ones that I can find in dom for theory 4244). Is that correct?
If I use mine I don't (but mine agree with what one would get with master).

[andreab1997]

If I use your ekos I get your fktables (meaning, the ones that I can find in dom for theory 4244). Is that correct? If I use mine I don't (but mine agree with what one would get with master).

yes exactly but you should also get exactly the same if you compare with the ones in the server. I mean, the fktables in 4244 are exactly the same (for DIS) of the ones in theory_424 in the server

[andreab1997]

Anyway if the central eko (meaning the one without fact scale variations) changes between master and the branch I am using that is something that @felixhekhorn and @alecandido should know.

[scarlehoff]

Ah, ok. No, I'm not looking at the sever. I'm really looking at the ones you have in your folder.

So then what do you think? Should we merge #84? It seems to be compatible with master and in any case it will be overwritten by your PR.

Anyway if the central eko (meaning the one without fact scale variations) changes between master and the branch I am using that is something that @felixhekhorn and @alecandido should know.

Note that I'm not creating ekos with your branch but using the ones you already had there.

[andreab1997]

Ah, ok. No, I'm not looking at the sever. I'm really looking at the ones you have in your folder.

So then what do you think? Should we merge #84? It seems to be compatible with master and in any case it will be overwritten by your PR.

Anyway if the central eko (meaning the one without fact scale variations) changes between master and the branch I am using that is something that @felixhekhorn and @alecandido should know.

Note that I'm not creating ekos with your branch but using the ones you already had there.

Yes but those ekos should be ok, I computed with that branch. I guess we can merge #84.

[scarlehoff]

ok. Btw, the fact that theory 400 is different (the one in the server) is expected right? (and, accordingly, theory 400 should not be used for "real" fits yet)?

[andreab1997]

ok. Btw, the fact that theory 400 is different (the one in the server) is expected right? (and, accordingly, theory 400 should not be used for "real" fits yet)?

No it is strange now because I updated it yesterday I believe. Only for DIS of course

[scarlehoff]

I was using theory 400 to check the hadronic fktables (and some did change)

[andreab1997]

I was using theory 400 to check the hadronic fktables (and some did change)

Hadronic make sense. Perfect, they were not updated