does not even list plugins without sudo access

[yarikoptic]

I am new to SOS. Was considering it for a replacement of our "ad-hoc" datalad wtf (sample output could be found under expandable WTF section within e.g. datalad/datalad#7691). But was surprised that it seems sos is not usable without sudo access, even cannot list plugins?

(base) yoh@dhcp-10-31-167-64 sos % git describe 
3.9-1694-gace5715f
(base) yoh@dhcp-10-31-167-64 sos % git describe --tags
4.8.2-4-gace5715f
(base) yoh@dhcp-10-31-167-64 sos % sos report --list-plugins
/Users/yoh/miniconda/bin/sos:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('sos==4.8.2')

WARNING: Failed to load 'magic' module version >= 0.4.20 which sos aims
to use for detecting binary files. A less effective method will be used.
It is recommended to install proper python3-magic package with the
module.

Could not initialize 'report': Component must be run with root privileges
(base) yoh@dhcp-10-31-167-64 sos % echo $?
1

which is IMHO heavily limits the applicability for use of sos in 3rd party software which most often would not be used by users with sudo access.

NB above is on a loaner OSX laptop... yet to see what happens under linux

For this particular example -- I think it should be able to list plugins without requiring sudo access.

[TurboTurtle]

At one point in the past we did support non-root executions, but what we found from feedback from both users and support engineers, as well as some analytics on collected sos reports was that

1. Most people (over 95%, I forget the exact number at this point) ran it as root regardless and
2. There were numerous complaints about non-root reports due to them missing a lot of support-relevant data since a lot of the collections we try to perform require root privileges themselves.

The current root requirement within sos report (as opposed to sos collect which does not need local root, but does need remote root to run reports on target hosts) is set very early on. We could ease that a bit to at least allow listing plugins, but I'd be cautious about supporting non-root report generation again.

Side note about OSX: we don't support Mac OS generally, and certainly not for report. It might work for a very limited set of collections, but we don't test that OS at all. collect might have better luck, but again we don't develop for or test against Mac OS. It has kicked around in my head over the last year or so about changing that (mainly because I'm forced to use a MBP for work, so it's always something staring me in the face), but there's a lot of work we'd need to do in order to properly support sos there.

[TurboTurtle]

but I'd be cautious about supporting non-root report generation again.

Just to clarify on this point.

I'd be cautious from the standpoint of "is there a use case that warrants the maintenance of supporting non-root reports", rather than anything else. I think we'd still want everything to default to the assumption of root being required, but there's a not-awful implementation that could tie plugin enablement to root-ness and such.

If you could expand a bit on datalad in terms of where it gets deployed and what kind of diagnostic data you like to see when troubleshooting, we can definitely talk about opening up sos to non-root again.

[yarikoptic]

Sorry for the delay. I might have actually misunderstood the main idea behind sos.

DataLad (http://datalad.org provides some more information) is a tool, based on git and git-annex, so typically ran by non-privileged users to manage their code and data. Quite often users would run into issues. To collect and provide back to us all potentially relevant data about environment (OS, filesystems, env vars, etc) and versions of dependencies we hacked up datalad wtf command (also available as Python module), which users can simply call and provide such details.

Here is an example of such output from the issue I have mentioned in the original description

WTF

configuration <SENSITIVE, report disabled by configuration>

credentials

• keyring:
  • active_backends:
    • SecretService Keyring
    • EncryptedKeyring with [PBKDF2] AES256.CFB v.1.0 at /home/heberto/.local/share/python_keyring/crypted_pass.cfg
    • PlaintextKeyring with no encyption v.1.0 at /home/heberto/.local/share/python_keyring/keyring_pass.cfg
  • config_file: /home/heberto/.config/python_keyring/keyringrc.cfg
  • data_root: /home/heberto/.local/share/python_keyring

datalad

• version: 1.1.4

dataset

• branches:
  • git-annex@cc8ef36
  • master@532c98e
• id: ea7231ea-bb02-4859-8b95-9a8c5d6017fa
• path: /home/heberto/data/yaric_spikeglx/raw
• repo: AnnexRepo

dependencies

• annexremote: 1.6.5
• boto3: 1.34.130
• cmd:7z: 23.01
• cmd:annex: 10.20240129
• cmd:bundled-git: UNKNOWN
• cmd:git: 2.43.0
• cmd:ssh: 9.6p1
• cmd:system-git: 2.43.0
• cmd:system-ssh: 9.6p1
• humanize: 4.9.0
• iso8601: 2.1.0
• keyring: 25.2.1
• keyrings.alt: 5.0.1
• msgpack: 1.0.8
• platformdirs: 4.2.2
• requests: 2.32.3

environment

• LANG: en_US.UTF-8
• LC_ADDRESS: sv_SE.UTF-8
• LC_IDENTIFICATION: sv_SE.UTF-8
• LC_MEASUREMENT: sv_SE.UTF-8
• LC_MONETARY: sv_SE.UTF-8
• LC_NAME: sv_SE.UTF-8
• LC_NUMERIC: sv_SE.UTF-8
• LC_PAPER: sv_SE.UTF-8
• LC_TELEPHONE: sv_SE.UTF-8
• LC_TIME: sv_SE.UTF-8
• PATH: /home/heberto/miniconda3/envs/work/bin:/home/heberto/.local/bin:/home/heberto/.nvm/versions/node/v18.15.0/bin:/usr/local/go/bin:/home/heberto/miniconda3/condabin:/home/heberto/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/home/heberto/.cargo/bin:/home/heberto/.local/bin:/home/heberto/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/heberto/go/bin:/opt/mssql-tools18/bin

extensions

git-annex

• build flags:
  • Assistant
  • Webapp
  • Pairing
  • Inotify
  • DBus
  • DesktopNotify
  • TorrentParser
  • MagicMime
  • Benchmark
  • Feeds
  • Testsuite
  • S3
  • WebDAV
• dependency versions:
  • aws-0.24.1
  • bloomfilter-2.0.1.2
  • crypton-0.33
  • DAV-1.3.4
  • feed-1.3.2.1
  • ghc-9.4.7
  • http-client-0.7.14
  • persistent-sqlite-2.13.2.0
  • torrent-10000.1.3
  • uuid-1.3.15
  • yesod-1.6.2.1
• key/value backends:
  • SHA256E
  • SHA256
  • SHA512E
  • SHA512
  • SHA224E
  • SHA224
  • SHA384E
  • SHA384
  • SHA3_256E
  • SHA3_256
  • SHA3_512E
  • SHA3_512
  • SHA3_224E
  • SHA3_224
  • SHA3_384E
  • SHA3_384
  • SKEIN256E
  • SKEIN256
  • SKEIN512E
  • SKEIN512
  • BLAKE2B256E
  • BLAKE2B256
  • BLAKE2B512E
  • BLAKE2B512
  • BLAKE2B160E
  • BLAKE2B160
  • BLAKE2B224E
  • BLAKE2B224
  • BLAKE2B384E
  • BLAKE2B384
  • BLAKE2BP512E
  • BLAKE2BP512
  • BLAKE2S256E
  • BLAKE2S256
  • BLAKE2S160E
  • BLAKE2S160
  • BLAKE2S224E
  • BLAKE2S224
  • BLAKE2SP256E
  • BLAKE2SP256
  • BLAKE2SP224E
  • BLAKE2SP224
  • SHA1E
  • SHA1
  • MD5E
  • MD5
  • WORM
  • URL
  • X*
• local repository version: 10
• operating system: linux x86_64
• remote types:
  • git
  • gcrypt
  • p2p
  • S3
  • bup
  • directory
  • rsync
  • web
  • bittorrent
  • webdav
  • adb
  • tahoe
  • glacier
  • ddar
  • git-lfs
  • httpalso
  • borg
  • hook
  • external
• supported repository versions:
  • 8
  • 9
  • 10
• upgrade supported from repository versions:
  • 0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
• version: 10.20240129

location

• path: /home/heberto/data/yaric_spikeglx/raw
• type: dataset

metadata.extractors

metadata.filters

metadata.indexers

python

• implementation: CPython
• version: 3.11.9

system

• distribution: ubuntu/24.04/noble
• encoding:
  • default: utf-8
  • filesystem: utf-8
  • locale.prefered: UTF-8
• filesystem:
  • CWD:
    • max_pathlength: 4096
    • mount_opts: rw,relatime,errors=remount-ro
    • path: /home/heberto/data/yaric_spikeglx/raw/M541/M541-2024-08-31_g0/M541-2024-08-31_g0_imec0
    • type: ext4
  • HOME:
    • max_pathlength: 4096
    • mount_opts: rw,relatime,errors=remount-ro
    • path: /home/heberto
    • type: ext4
  • TMP:
    • max_pathlength: 4096
    • mount_opts: rw,relatime,errors=remount-ro
    • path: /tmp
    • type: ext4
• max_path_length: 342
• name: Linux
• release: 6.8.0-49-generic
• type: posix
• version: doRegexSub doesn't replace the content in the archive properly #49-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 4 02:06:24 UTC 2024

We find such output often very informative and useful. More often now I want to add similar information to other user tools we develop. So logical would be e.g. to extract datalad wtf command into an independent package, but we never got time for that , and then I ran into sos and I thought it could be such a platform we could "instrument" easily in our tools and avoid reimplementing. E.g. it could have collected versions of relevant loaded (or installed) python packages, filesystem, etc.