add support for nvenc and accel_assist

[jsorg71]

No description provided.

[matt335672]

Fix some build issues with:-

--- a/scripts/install_xrdp_build_dependencies_with_apt.sh
+++ b/scripts/install_xrdp_build_dependencies_with_apt.sh
@@ -74,6 +74,7 @@ case "$ARCH"
 in
     amd64)
         PACKAGES_AMD64_MIN=" \
+            libepoxy-dev \
             libpam0g-dev \
             libssl-dev \
             libx11-dev \
@@ -118,6 +119,7 @@ in
         PACKAGES="$PACKAGES \
             g++-multilib \
             gcc-multilib \
+            libepoxy-dev:i386 \
             $LIBFREETYPE_DEV:i386 \
             libgl1-mesa-dev:i386 \
             libglu1-mesa-dev:i386 \

Also:-

1. some files are missing from the xrdp_accel_assist directory; xrdp_accel_assist_yami.c xrdp_accel_assist_yami.h encoder_headers/yami_inf.h
2. need to run astyle

[matt335672]

Happy to merge this as-is, when CI is fixed.

[matt335672]

Suggest adding error logging if this doesn't work, for triaging user issues.

[matt335672]

I don't want to hold up merging this, but I'd like to ask if we think the hack to call cuInit() is in the right place. Wouldn't it be better done as part of the session if NVidia is being used?

[jsorg71]

I don't want to hold up merging this, but I'd like to ask if we think the hack to call cuInit() is in the right place. Wouldn't it be better done as part of the session if NVidia is being used?

I tried to just call cuInit() in the session but as the user it fails. Once the call is made as root, something gets 'driver init' and then it will work as the user. I guess xrdp might not be running as root, is that right? Also, is this a security risk?
In order to reproduce this problem, you have to boot a server up without X GUI running as Xorg on console also does the 'driver init'.
I tried to use nvida_persistence or something like that as it claimed to fix this but it didn't work.
I've been meaning to ask NVidia about this but the problem is hard to describe.

[jsorg71]

I rebased this branch on current devel

[jsorg71]

Should we make xrdp_accel_assit a compile time flag because with it, epoxy and gbm are required to build xrdp. We didn't need that before? Xorg, or at least the modesetting driver requires the run time libraries but now we require the development packages and the user may never plan to use xrdp_accel_assist.

[matt335672]

As far as the cuInit() goes, either sesman or sesexec runs as root. Since #2974, xrdp may not be running as root, and in fact it won't be on Debian/Ubuntu anyway - they patched it to run as non-root a long time ago.

I think a compile-time option is the best way to go, for platforms that may not support NVidia acceleration (are there any?).