[FEA] Limit Host Memory Usage

[revans2]

Is your feature request related to a problem? Please describe.
Currently the Spark RAPIDs plugin uses as much host memory as it feels it needs to. We have put some pools in place with the idea that a user wouldn’t really need to go beyond that, but in practice it can happen and then YARN, Kubernetes, docker, etc kills the executors because they tried to use more memory than the container is configured to use.

This epic is to put in place all of the changes needed so we effectively have a hard limit on the amount of host memory that the plugin uses and that amount is easy to configure.

Framework Changes:

• Remove the pageable pool #8857 This is to make things simpler to work with and reason about.
• [FEA] Add maximum host memory config #8875
• [FEA] Add reserved host memory config #8876
• [FEA] Add config for host memory limiting mode #8877
• [FEA] Provide CUDF java APIs for memory allocation #8884
• [FEA] CUDF API to get allocation size needed for copy to host. #8885
• [FEA] Add API to make HostMemoryBuffers/HostColumnVectors Spillable #8882
• [FEA] Add host memory allocation APIs #8879
• [FEA] Trigger host memory spilling when more host memory is needed #8881
• [FEA] Add host memory task metrics and explore a host memory allocation watch dog. #8880
• [FEA] Update CUDF host memory allocation APIs so allocation is pluggable. #9862
• [FEA] Stress test host memory allocation/spill APIs #8883
• [FEA] clean up host memory limit configs #8878
• [FEA] support unspill for RapidsHostColumnarBatch #9107
• [FEA] Run Scale tests regularly under a memory constrained situation #9965
• [FEA] Move kudo code from jni repo to plugin repo #12169
• [BUG] split should not happen immediately after fallback on memory contention (OOM state machine bug)  #12158
• [FEA] Support unspill for SpillableHostBuffer #12184

Operator Changes:

• [FEA] Add Host Memory Retry Columnar To Row Conversion #8886
• [FEA] Add Host Memory Retry for Row to Columnar Conversion #8887
• [FEA] Update JSON, CSV, and HiveText to retry host memory allocations #8888
• [FEA] Add retry for host memory allocations for output file formats #8889
• [FEA] Add retry for Host Memory Usage in Parquet, ORC, and AVRO reads #8890
• [FEA] Remove support for GpuRowBasedUserDefinedFunction #8891
• [FEA] Add retry for host memory allocation of GpuRandom #8898
• [FEA] Update All usages of JCudfSerialization to retry host memory allocations #8899
• [FEA] Add retry to Multi-threaded shuffle reader for host memory allocations #8900
• [FEA] Stress test UCX shuffle after Host memory limits are in place #8901
• [FEA] Limit host memory usage on python UDF execs. #8903
• [FEA] Add retry for off heap host memory allocations for parquet cached batch serializer #8904
• [FEA] Add retry to GPU timezone database caching operation spark-rapids-jni#1570
• [FEA] Kudo shuffle read should be retryable and spillable on Host Memory #12215

Debugability:

• [FEA] Add Host Memory Logging similar to RMM logging #10102
• [FEA] Add unified logging for host/device/spill #10103
• [FEA] Dump GPU/CPU memory footprint when GPU/CPU OOM happens #12173