Too many unnessary settings in config-user.yml

[bouweandela]

At the moment there are many settings in config-user.yml that users will very rarely change. Therefore I propose to reduce the number of options exposed to users in the following way:

Proposed for removal (can be moved to the recipe for those diagnostics that support these options)

• write_plots: true
• write_netcdf: true
• output_file_type: ps # [ps]/pdf/png/eps/epsi

These are options that should be implemented by diagnostic developers, so this puts a burden on diagnostic developers to implement it and in many cases it does not work, because they are either not aware of these options, or for the last option, the library they use for plotting does not support a particular plot type. The amount of time that is saved by not creating plots or netcdf files or all possible plot types is probably only a small fraction of the total runtime, so the advantage of having these settings is rather unclear, but having them creates a bad user experience because they do not work in many cases and a bad diagnostic developer experience because they have to implement and test additional requirements.

Proposed for moving to an advanced users file, e.g config-developer.yml

• profile_diagnostic: false -> only interesting for Python diagnostic developers
• save_intermediary_cubes: false -> for debugging purposes, probably only interesting for diagnostic developers
• exit_on_warning: false -> specific to NCL diagnostics, probably only interesting for diagnostic developers

[bouweandela]

Setting max_data_filesize: 100 has been removed since this issue was opened.

[jvegreg]

With the new CLI, all config-user.yml options can be set as flags in the run command, maybe we should think in some command line options to be moved to it

[bouweandela]

Using these options in the configuration file instead of the diagnostic script section of the recipe would need to be deprecated in the upcoming release (2.1, planned for October 2020, feature freeze October 5th) and removed two releases later (2.3, planned June 2021), see also the release schedule.

[bouweandela]

We would need to make sure that the write_plots, write_netcdf, and output_file_type options are documented here for those diagnostic scripts/recipes that support them.

[bouweandela]

Setting write_plots, write_netcdf and output_file_type in config-user instead of in the recipe will now be deprecated in v2.2 and removed in v2.4, because #808 didn't make it into the 2.1 release.

[bouweandela]

After looking through the code of ESMValTool to see what it would take to remove output_file_type from config-user.yml, I think that may be a lot of work, especially for the NCL and R diagnostics. I think it's probably not worth the effort at this stage, so I removed that from pull request #808.

[bouweandela]

Superseded by #2371