[BUG] gitfs on fedora 38 gives _pygit2.GitError: error loading known_hosts:

[dukhat2259]

Description
After updating from fedora 37 to 38 and thus salt 3005.x to 3006.1 and attempting to use gitfs pillars it fails with infamous the "_pygit2.GitError: error loading known_hosts:" ( #64121 )

I'm creating this issue since fedora 38 seems to run the salt-master as root - there is no user salt to be found and the referenced bug is only talking about problems related to the salt user.

The master's log is full of

2023-06-20 08:14:13,480 [salt.utils.gitfs :284 ][ERROR   ][4725] Error occurred fetching git_pillar remote '__env__ [email protected]:z/salt.git': error loading known_hosts:
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/salt/utils/gitfs.py", line 1870, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/site-packages/pygit2/remote.py", line 146, in fetch
    payload.check_error(err)
  File "/usr/lib64/python3.11/site-packages/pygit2/callbacks.py", line 98, in check_error
    check_error(error_code)
  File "/usr/lib64/python3.11/site-packages/pygit2/errors.py", line 65, in check_error
    raise GitError(message)
_pygit2.GitError: error loading known_hosts:

Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
Please be as specific as possible and give set-up details.

/etc/salt/master

state_auto_order: True
show_jid: True
cli_summary: True
file_roots:
  __env__:
    - /srv/salt/__env__/states
top_file_merging_strategy: same
default_top: crash_and_burn
pillar_roots:
  __env__:
    - /srv/salt/__env__/pillars
pillarenv_from_saltenv: True
log_level: info
jinja_env:
  trim_blocks: True
  lstrip_blocks: True

/etc/salt/master.d/gitfs.conf

gitfs_provider: pygit2
gitfs_privkey: {{ ssh_key }}
gitfs_pubkey: {{ ssh_key }}.pub
gitfs_root: states
gitfs_base: crash_and_burn
gitfs_global_lock: False
fileserver_backend:
  - roots
  - gitfs
gitfs_remotes:
  - {{ git_url }}:
    - update_interval: 60
gitfs_saltenv:
  - master:
    - ref: master
  - __env__:
    - ref: __env__

git_pillar_update_interval: 60
git_pillar_provider: pygit2
git_pillar_privkey: {{ ssh_key }}
git_pillar_pubkey: {{ ssh_key }}.pub
ext_pillar:
  - git:
    - __env__ {{ git_url }}:
      - root: pillars
      - env: __env__

• on-prem machine
• VM (Proxmox KVM)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD
• classic packaging
• onedir packaging
• used bootstrap to install

Steps to Reproduce the behavior
(Include debug logs if possible and relevant)
Working configuration on fedora 37 upgraded into a non-working gitfs on fedora 38.
Enhanced logging did not provide any additional relevant messages

Expected behavior
A clear and concise description of what you expected to happen.

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
          Salt: 3006.1

Python Version:
        Python: 3.11.3 (main, May 24 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)]

Dependency Versions:
          cffi: 1.15.1
      cherrypy: Not Installed
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.0.3
       libgit2: 1.6.3
  looseversion: 1.2.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.4
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.0
     pycparser: 2.20
      pycrypto: Not Installed
  pycryptodome: 3.17.0
        pygit2: 1.12.0
  python-gnupg: Not Installed
        PyYAML: 6.0
         PyZMQ: 24.0.1
        relenv: Not Installed
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: fedora 38
        locale: utf-8
       machine: x86_64
       release: 6.3.8-200.fc38.x86_64
        system: Linux
       version: Fedora Linux 38

[OrangeDog]

Duplicates #64121