ESMValTool processor uses the "latest" version link in cp4cds data archive

[cehbrecht]

Description

The esmvaltool processes do not find data is the cmip5 archive. It makes use of the “latest” version link which is only available at ceda … this is not created on the replica. Can the CP4CDS config be updated to search the latest version when the latest tag is not available?

Environment

• c3s-magic-wps version used, if any:
• Python version, if any:
• Operating System:

Steps to Reproduce

Additional Information

[cehbrecht]

We check also on our CP4CDS site if we can create the "latest" link in the relica archive.

[nielsdrost]

He @cehbrecht, any update on using latest on your end?

[cehbrecht]

@nielsdrost Not yet. It is not decided yet if we will create the "latest" links ... people are on vacation. Safest for testing would be if we can use the esmvaltool config which figures out the "latest" version by itself ... even if it would just pick the first version found.

[cehbrecht]

It looks like esmvaltool can already handle different versions using [latestversion] in the configuration. Could you set this for CP4CDS?
https://github.com/ESMValGroup/ESMValCore/blob/fafc42bd7e66b0057736561f6e44672c39977609/esmvalcore/config-developer.yml#L145

There is an intention to update the ESGF scripts to update the "latest" links in the data repos ... but after vacation in August.

[nielsdrost]

Ah, yes, that should be easy enough to fix in ESMValTool. For the WPS we will have to use a bit of a trick since we are pinned to the 2.0a2 release. Luckily we can pass a custom config-developer.yml with the updated settings.