FIX: Sneaky bugs reported by Patrick McGuire

[nocollier]

No description provided.

[mcguirepatr]

Hi Nathan:
Thanks for trying to fix this.
It modifies the behavior, but not exactly how I expected.
The overall scores for the relationships are now missing for all the models, whereas I would only expect them to be missing for the relationships for the models that have missing data.
See this link:
https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output/GCP_290824a/
Or this screenshot:

I didn't rerun the whole ILAMB, I just reran ILAMB with these two lines in the Confrontation changed and I added a new model (SDGVMv9_24) prior to the rerun.
Can you take a 2nd look at it?
Thanks
Patrick McGuire

[nocollier]

Hey Patrick, Thanks for the report. I need to try to create a small example where I can reproduce the behavior. I have to finish a few reviews but will get bcak on this as soon as I can. Nate

…

On Thu, Aug 29, 2024 at 9:14 AM Patrick McGuire ***@***.***> wrote: Hi Nathan: Thanks for trying to fix this. It modifies the behavior, but not exactly how I expected. The overall scores for the relationships are now missing for all the models, whereas I would only expect them to be missing for the relationships for the models that have missing data. See this link: https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output/GCP_290824a/ Or this screenshot: Screenshot.2024-08-29.at.14.07.27.png (view on web) <https://github.com/user-attachments/assets/706bebf3-7bfa-477e-beb0-69c3e52fe6fe> I didn't rerun the whole ILAMB, I just reran ILAMB with these two lines in the Confrontation changed and I added a new model (SDGVMv9_24) prior to the rerun. Can you take a 2nd look at it? Thanks Patrick McGuire — Reply to this email directly, view it on GitHub <#102 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKFCBZWOQFUM7PU724IABTZT4NCJAVCNFSM6AAAAABNI2HJISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJXGYZDMNJVGM> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[minxu74]

@nocollier I tried to look into the problem by open the UD (https://gws-access.jasmin.ac.uk/public/ncas_climate/pmcguire/ILAMB_output/GCP_290824a/dashboard.html). I found the results from UD and index.html were different, but they are supposed to be identical. The reason is that the contents of scalars.json are different to scalar_database.json used by UD. The same problem also reported by #100 .

[nocollier]

This is a problem of me having bolted on things to ILAMB. Thanks Min, this gives me a place to start looking. I think the issue is that there are multiple places where we merge results. There is the OverallScore function of Confrontations, but then I also (later) wrote a harvest of all scalars and compute aggregations using pandas. I will track them all down and let's see if we can unify them so it is a single way.

[minxu74]

@nocollier Thanks a lot. I found the code and could fix the inconsistent problem if you are not available. But we need to discuss a way to compute composite scores and write the two json files consistently. We also could let index.html read the CMEC json file directly as the CMEC json file includes all the results and meta information.

[nocollier]

I believe the issue is that we need aggregation in 2 places:

1. Before generating the Dataset html pages. The score data is written into the html directly and so we need it at this level.
2. When leaving an artifact for use in the top level page.

I finished my reviews, will look at this today/tomorrow.

[mcguirepatr]

I ran out of space on my web server, so I am currently moving my ILAMB pages to another web server. (Just in case you're looking for any of them.)

[mcguirepatr]

Here are recent ILAMB links at my new web server:
https://gws-access.jasmin.ac.uk/public/landsurf_rdg/pmcguire/ILAMB_output/GCP_290824a/index.html
https://gws-access.jasmin.ac.uk/public/landsurf_rdg/pmcguire/ILAMB_output/GCP_290824a/dashboard.html