load data files: update hints in error msgs and ignore the download by default

[pirat89]

As the leapp upgrade data files are nowadays part of the install rpm, there is no need to download them anymore. Also, we do not have any plans to provide the updated data files anywhere out of the rpm.

For that reason, update all texts and related error messages so people do not try to visit obsoleted article, trying to apply invalid (obsoleted) data files anymore.

I kept the original fetch functionality in case someone would like to use it for additional custom data files, but the load_data_asset function invoke the fetch without allowed downloads, so the fetch fails with the proper error msg if files are not stored locally.

@oamg/developers postponed for the May release. Mostly covered by pr #1121 (changed err msgs). Another changes in this one PR are expected later after the release.

jira: RHEL-18298

[github-actions]

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

• review please @oamg/developers to notify leapp developers of the review request
• /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build. If you need a different version of leapp from PR#42, use /packit test oamg/leapp#42

To launch regression testing public members of oamg organization can leave the following comment:

• /rerun to schedule basic regression tests using this pr build and latest upstream leapp build as artifacts
• /rerun 42 to schedule basic regression tests using this pr build and leapp*PR42* as artifacts
• /rerun-sst to schedule sst tests using this pr build and latest upstream leapp build as artifacts
• /rerun-sst 42 to schedule sst tests using this pr build and leapp*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

[pirat89]

Reviving the work on this.

[pirat89]

Outputs from manual testing (note: required data stream is now (in time of testing) 2.x)

• set repomap.json data stream to 1.0, pes-events.json to 4.0, set empty data stream in "DDDD".json file

Risk Factor: high (inhibitor)
Title: Incompatible Leapp data assets are present
Summary: The currently installed Leapp consumes data stream 2.0, but the following assets provide different streams:
The following assets are outdated
 - The asset repomap.json provides the data stream 1.0
The following assets are intended for a newer leapp
 - The asset pes-events.json provides the data stream 4.0
Remediation: [hint] All official assets (data files) are nowadays part of the installed rpms. This issue is usually encountered when the data files are incorrectly customized, replaced, or removed.  In case you want to recover original files, remove them (if still exist) and reinstall following rpms: leapp-upgrade-el7toel8.
The listed assets (data files) are usually inside the /etc/leapp/files/ directory.
Key: a064e45f1d80af13c7c41f28cf1cf2a2e0c50359
----------------------------------------
Risk Factor: high (inhibitor)
Title: Detected malformed Leapp data assets
Summary: The following assets are malformed:
 - The asset file device_driver_deprecation_data.json contains no values in its "provided_data_streams" field, or the field does not contain a list
Remediation: [hint] All official assets (data files) are nowadays part of the installed rpms. This issue is usually encountered when the data files are incorrectly customized, replaced, or removed.  In case you want to recover original files, remove them (if still exist) and reinstall following rpms: leapp-upgrade-el7toel8.
The listed assets (data files) are usually inside the /etc/leapp/files/ directory.
Key: c5f9381e4725700584c5b45cb3cfa04d84e8be63
----------------------------------------

tests with various other malformations & removal:

# echo "" > device_driver_deprecation_data.json  # invalid json format 
# rm pes-events.json  # missing file
#  # remove provided_data_streams from repomap.json

## ..... in report ....:

Risk Factor: high (error)
Title: The Device driver deprecation data file (at device_driver_deprecation_data.json) does not contain a valid JSON object.
Summary: {"hint": "All official data files are nowadays part of the installed rpms. That is the only official resource of actual official data files for in-place upgrades. This issue is usually encountered when the data files are incorrectly customized, replaced, or removed (e.g. by custom scripts). In case you want to recover the original /etc/leapp/files/device_driver_deprecation_data.json file, remove the current one (if still exists) and reinstall following packages: leapp-upgrade-el7toel8."}
Key: 6ab1ae455d68a7bce41d2f6f4017712262b32516
----------------------------------------
Risk Factor: high (error)
Title: Data file /etc/leapp/files/pes-events.json is missing or invalid.
Summary: {"details": "File /etc/leapp/files/pes-events.json does not exist.", "hint": "All official data files are nowadays part of the installed rpms. That is the only official resource of actual official data files for in-place upgrades. This issue is usually encountered when the data files are incorrectly customized, replaced, or removed (e.g. by custom scripts). In case you want to recover the original /etc/leapp/files/pes-events.json file, remove the current one (if still exists) and reinstall following packages: leapp-upgrade-el7toel8."}
Key: c305148ff23ccf107bb5fe7acde505db4dcff88a
----------------------------------------

(Update: ignore text below)
btw, the error representation in txt format seems little bit weird to me as I would expect from the past something like:

Risk:
Title:
Summary:
  Hint:
  Details:
  ...

maybe I remember it wrong. I will check that yet. Most likely this is thing of the framework and it is not related to changes in this PR at all.

Update: the current error report is correct. It's like that all the time as it has defined structure. The representation I spoke about is different in case of stdout:

============================================================
                           ERRORS                           
============================================================

2024-01-18 13:30:14.370016 [ERROR] Actor: load_device_driver_deprecation_data
Message: The Device driver deprecation data file (at device_driver_deprecation_data.json) does not contain a valid JSON object.
Summary:
    Hint: All official data files are nowadays part of the installed rpms. That is the only official resource of actual official data files for in-place upgrades. This issue is usually encountered when the data files are incorrectly customized, replaced, or removed (e.g. by custom scripts). In case you want to recover the original /etc/leapp/files/device_driver_deprecation_data.json file, remove the current one (if still exists) and reinstall following packages: leapp-upgrade-el7toel8.
2024-01-18 13:30:37.843962 [ERROR] Actor: pes_events_scanner
Message: Data file /etc/leapp/files/pes-events.json is missing or invalid.
Summary:
    Details: File /etc/leapp/files/pes-events.json does not exist.
    Hint: All official data files are nowadays part of the installed rpms. That is the only official resource of actual official data files for in-place upgrades. This issue is usually encountered when the data files are incorrectly customized, replaced, or removed (e.g. by custom scripts). In case you want to recover the original /etc/leapp/files/pes-events.json file, remove the current one (if still exists) and reinstall following packages: leapp-upgrade-el7toel8.

============================================================
                       END OF ERRORS                        
============================================================

[pirat89]

@fernflower rebased to cover changes in testing: #1162

[abadger]

Looks good! I made some suggestions but none of them are blockers.

Approved.

[abadger]

Approved!