[Linux] ppid_map get PermissionError if /proc is mounted with hidepid=1

[ggrzeczkowicz]

Summary

• OS: Linux
• Architecture: 64bit
• Psutil version: 5.8.0
• Python version: 3.8.12
• Type: core

Description

The pslinux/ppid_map function can raise an uncaught PermissionError if /proc is mounted with hidepid=1 (see proc man).

Obtain Traceback:

Exception in thread Thread-10:
Traceback (most recent call last):
  File "~/conda/envs/python-env/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "~/conda/envs/python-env/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "~/conda/envs/python-env/lib/python3.8/site-packages/wandb/sdk/internal/stats.py", line 132, in _thread_body
    stats = self.stats()
  File "~/conda/envs/python-env/lib/python3.8/site-packages/wandb/sdk/internal/stats.py", line 185, in stats
    in_use_by_us = gpu_in_use_by_this_process(handle)
  File "~/conda/envs/python-env/lib/python3.8/site-packages/wandb/sdk/internal/stats.py", line 43, in gpu_in_use_by_this_process
    our_processes = base_process.children(recursive=True)
  File "~/conda/envs/python-env/lib/python3.8/site-packages/psutil/__init__.py", line 272, in wrapper
    return fun(self, *args, **kwargs)
  File "~/conda/envs/python-env/lib/python3.8/site-packages/psutil/__init__.py", line 899, in children
    ppid_map = _ppid_map()
  File "~/conda/envs/python-env/lib/python3.8/site-packages/psutil/_pslinux.py", line 1555, in ppid_map
    with open_binary("%s/%s/stat" % (procfs_path, pid)) as f:
  File "~/conda/envs/python-env/lib/python3.8/site-packages/psutil/_common.py", line 711, in open_binary
    return open(fname, "rb", **kwargs)
PermissionError: [Errno 1] Operation not permitted: '/proc/1/stat'

So the statement:

# Note: we should be able to access /stat for all processes
# aka it's unlikely we'll bump into EPERM, which is good.

is not true.

I'm not sure if adding PermissionError to the except statement is enough.

[giampaolo]

Mmm. Interesting. I didn't know about hidepid=1. The code comment is indeed incorrect then, but I don't think we can do anything to avoid this error. It basically means that Process.children() cannot be used if hidepid=1.

[smuzaffar]

I also have hit the same issue on shared nodes. hidepid=1 is set on these nodes so that users can not read other users process info. With psutils, we are mostly interested in our own processes. So I think ignoring PermissionError should still work and report the utilization of process owned by the user his/her self.