Erratic denies by "deny_audit perm=any pattern=ld_so : all"

[Grummel7]

I am running fapolicyd (fapolicyd-1.0.2-6.el8.x86_64) on Red Hat Enterprise Linux release 8.5 with the following rules:

# while ensuring that only trusted libraries are used. This provides good
# performance while ensuring that there is not much interference by
# the daemon.

%languages=application/x-bytecode.ocaml,application/x-bytecode.python,application/java-archive,text/x-java,application/x-java-applet,text/javascript,text/x-awk,text/x-gawk,text/x-lisp,application/x-elc,text/x-lua,text/x-m4,text/x-perl,text/x-php,text/x-python,text/x-R,text/x-ruby,text/x-script.guile,text/x-tcl,text/x-luatex,text/x-systemtap

# Carve out an exception for dracut initramfs building
allow perm=any uid=0 : dir=/var/tmp/
allow perm=any uid=0 trust=1 : all

# Prevent execution by ld.so
deny_audit perm=any pattern=ld_so : all

# We have to carve out an exception for the system updaters
# or things go very bad (deadlock).
allow perm=open exe=/usr/bin/rpm : all
allow perm=open exe=/usr/libexec/platform-python3.6 comm=dnf : all

# Only allow known ELF libs - this is ahead of executable because typical
# executable is linked with a dozen or more libraries.
allow perm=open all : ftype=application/x-sharedlib trust=1
deny_audit perm=open all : ftype=application/x-sharedlib

# Allow trusted programs to execute
allow perm=execute all : trust=1

# Need to carve out an exception for ansible, which uses python
allow perm=any uid=0 : dir=/tmp/ansible
allow perm=any uid=0 : dir=/root/.ansible/tmp/
allow perm=any uid=991 : dir=/home/ccms/.ansible/tmp/

# Allow any program to open trusted language files
allow perm=open all : ftype=%languages trust=1
deny_audit perm=any all : ftype=%languages

# Allow all shell script execution and sourcing
allow perm=any all : ftype=text/x-shellscript

# Deny execution for anything untrusted
deny_audit perm=execute all : all

# Allow everything else to open any file
allow perm=open all : all

Occasionally it blocks seemingly random accesses.

Heres a list of denies from fapolicyd --debug:

rule=3 dec=deny_audit perm=open auid=1201 pid=281219 exe=/usr/bin/bash : path=/usr/share/locale/locale.alias ftype=text/plain
rule=3 dec=deny_audit perm=execute auid=1201 pid=281260 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=open auid=1201 pid=281260 exe=/usr/bin/bash : path=/usr/bin/timeout ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=281260 exe=/usr/bin/bash : path=/usr/bin/timeout ftype=application/x-executable
rule=3 dec=deny_audit perm=execute auid=-1 pid=281505 exe=/usr/bin/timeout : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=execute auid=1201 pid=281665 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=open auid=1201 pid=281665 exe=/usr/bin/bash : path=/usr/bin/sort ftype=application/x-executable
rule=3 dec=deny_audit perm=opuid=1201 pid=281665 exe=/usr/bin/bash : path=/usr/bin/sort ftype=application/x-executable
rule=3 dec=deny_audit perm=execute auid=-1 pid=281794 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=open auid=-1 pid=281794 exe=/usr/bin/bash : path=/usr/bin/timeout ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=313018 exe=/usr/bin/bash : path=/usr/bin/sed ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=313018 exe=/usr/bin/bash : path=/usr/share/locale/locale.alias ftype=text/plain
rule=3 dec=deny_audit perm=execute auid=1201 pid=350490 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=open auid=1201 pid=350490 exe=/usr/bin/bash : path=/usr/bin/ps ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=350492 exe=/usr/bin/bash : path=/usr/bin/grep ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=350492 exe=/usr/bin/bash : path=/usr/bin/grep ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=350490 exe=/usr/bin/bash : path=/usr/bin/ps ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=350490 exe=/usr/bin/bash : path=/usr/share/locale/locale.alias ftype=text/plain
rule=3 dec=deny_audit perm=open auid=1201 pid=350492 exe=/usr/bin/bash : path=/usr/share/locale/locale.alias ftype=text/plain
rule=3 dec=deny_audit perm=execute auid=1201 pid=350496 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=open auid=1201 pid=350496 exe=/usr/bin/bash : path=/usr/bin/grep ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=350496 exe=/usr/bin/bash : path=/usr/bin/grep ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=1201 pid=350496 exe=/usr/bin/bash : path=/usr/share/locale/locale.alias ftype=text/plain
rule=3 dec=deny_audit perm=execute auid=-1 pid=376966 exe=/usr/lib/polkit-1/polkitd : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=13 dec=deny_audit perm=open auid=991 pid=495696 exe=/usr/bin/cat : path=/usr/share/tracker/stop-words/stopwords.en ftype=text/x-python

As far as i understand it, the lines with perm=execute and path=/usr/lib64/ld-2.28.so match the definition of rule 3. I can also reproduce it by using ldd as non-root.
All the other lines are not reproducible and I cannot see how they match rule 3.
Most seemed to be triggered by conky, but other application also encounter random denies and some of them cause real trouble.

The last line is also reproducible, but doesn't bother me too much.

[stevegrubb]

This is not an obvious place to look, but this describes how it is supposed to work:
https://github.com/linux-application-whitelisting/fapolicyd/blob/main/src/library/rules.c#L968
I would need to see the adjacent approved accesses to diagnose what is happening. If you can detail what is happening around just one of the denials, I might have a better chance of determining what is happening.

[Grummel7]

Okay, it seems the first deny in each case is always one with execute and ld_so, the others seem to follow.
But nevertheless, the same accesses work most of the time and only occationally one is denied.
Here's an example. Cdsy.x is one of my applications, executing df via popen.
First an example like many other, that works, then one that was denied.

rule=8 dec=allow perm=execute auid=-1 pid=1712641 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/bin/bash ftype=application/x-executable
rule=16 dec=allow perm=open auid=-1 pid=1712641 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/bin/bash ftype=application/x-executable
rule=8 dec=allow perm=execute auid=-1 pid=1712641 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712641 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=1712641 exe=/usr/bin/bash : path=/etc/ld.so.cache ftype=application/octet-stream
rule=6 dec=allow perm=open auid=-1 pid=1712641 exe=/usr/bin/bash : path=/usr/lib64/libtinfo.so.6.1 ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712641 exe=/usr/bin/bash : path=/usr/lib64/libdl-2.28.so ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712641 exe=/usr/bin/bash : path=/usr/lib64/libc-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=1712641 exe=/usr/bin/bash : path=/usr/lib/locale/locale-archive ftype=application/octet-stream
rule=16 dec=allow perm=open auid=-1 pid=1712641 exe=/usr/bin/bash : path=/usr/lib64/gconv/gconv-modules.cache ftype=application/octet-stream
rule=8 dec=allow perm=execute auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=16 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=8 dec=allow perm=execute auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/df : path=/etc/ld.so.cache ftype=application/octet-stream
rule=6 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/df : path=/usr/lib64/libc-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/df : path=/usr/lib/locale/locale-archive ftype=application/octet-stream
rule=16 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/df : path=/usr/share/locale/locale.alias ftype=text/plain
rule=16 dec=allow perm=open auid=-1 pid=1712642 exe=/usr/bin/df : path=/usr/lib64/gconv/gconv-modules.cache ftype=application/octet-stream

rule=8 dec=allow perm=execute auid=-1 pid=1712830 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/bin/bash ftype=application/x-executable
rule=16 dec=allow perm=open auid=-1 pid=1712830 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/bin/bash ftype=application/x-executable
rule=8 dec=allow perm=execute auid=-1 pid=1712830 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712830 exe=/opt/aida/21.2.0.trunk.43448/Css/Cdsy.x : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=1712830 exe=/usr/bin/bash : path=/etc/ld.so.cache ftype=application/octet-stream
rule=6 dec=allow perm=open auid=-1 pid=1712830 exe=/usr/bin/bash : path=/usr/lib64/libtinfo.so.6.1 ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712830 exe=/usr/bin/bash : path=/usr/lib64/libdl-2.28.so ftype=application/x-sharedlib
rule=6 dec=allow perm=open auid=-1 pid=1712830 exe=/usr/bin/bash : path=/usr/lib64/libc-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=1712830 exe=/usr/bin/bash : path=/usr/lib/locale/locale-archive ftype=application/octet-stream
rule=16 dec=allow perm=open auid=-1 pid=1712830 exe=/usr/bin/bash : path=/usr/lib64/gconv/gconv-modules.cache ftype=application/octet-stream
rule=8 dec=allow perm=execute auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=16 dec=allow perm=open auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=3 dec=deny_audit perm=execute auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=3 dec=deny_audit perm=open auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=3 dec=deny_audit perm=open auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/share/locale/locale.alias ftype=text/plain

For some reason these random denies are rare, but sometime I get a couple of them within a short time period.
I've attach a file containing such a "denial burst".
Here's an extreme case from the attached file. It looks like two completely unrelated processes (sudo and polkitd) got a deny at the same time.

rule=8 dec=allow perm=execute auid=-1 pid=3606746 exe=/usr/bin/sudo : path=/usr/bin/chronyc ftype=application/x-executable
rule=8 dec=allow perm=execute auid=-1 pid=3606745 exe=/usr/lib/polkit-1/polkitd : path=/usr/bin/pkla-check-authorization ftype=application/x-executable
rule=16 dec=allow perm=open auid=-1 pid=3606746 exe=/usr/bin/sudo : path=/usr/bin/chronyc ftype=application/x-executable
rule=3 dec=deny_audit perm=execute auid=-1 pid=3606746 exe=/usr/bin/sudo : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib
rule=16 dec=allow perm=open auid=-1 pid=3606745 exe=/usr/lib/polkit-1/polkitd : path=/usr/bin/pkla-check-authorization ftype=application/x-executable
rule=3 dec=deny_audit perm=execute auid=-1 pid=3606745 exe=/usr/lib/polkit-1/polkitd : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib

fapolicyd.debug.log.4.gz

[stevegrubb]

Out of curiosity, which kernel are you using?

[Grummel7]

Out of curiosity, which kernel are you using?

# uname -a
Linux fm-ccms8-2.comsoft.de 4.18.0-348.12.2.el8_5.x86_64 #1 SMP Mon Jan 17 07:06:06 EST 2022 x86_64 x86_64 x86_64 GNU/Linux

I think it all comes down to these lines:

# Good case: bash does fork & exec /usr/bin/df
rule=8  dec=allow      perm=execute auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=16 dec=allow      perm=open    auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=8  dec=allow      perm=execute auid=-1 pid=1712642 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib

# Bad case:  bash does fork & exec /usr/bin/df
rule=8  dec=allow      perm=execute auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=16 dec=allow      perm=open    auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/bin/df ftype=application/x-executable
rule=3  dec=deny_audit perm=execute auid=-1 pid=1712831 exe=/usr/bin/bash : path=/usr/lib64/ld-2.28.so ftype=application/x-sharedlib

Whatever makes the difference, is apparently not logged by --debug.

Btw.: I think it would also be a good idea to add a timestamp to the debug output.output is a timestamp.

[stevegrubb]

You can change the syslog_format in the conf file to log anything needed for troubleshooting. One thing important that's missing is the trust field. The default currently is:

syslog_format = rule,dec,perm,auid,pid,exe,:,path,ftype,trust

I have also run across 3rd party security kernel modules that cause problems. I don't know if that is relevant.

[scarneysc]

We are seeing similar issues and we don't have a third party security kernel module installed. Any other ideas?