cleanup & update docs

PurdueAF · Mar 27, 2024 · 1974107 · 1974107
1 parent 7d35a85
commit 1974107
Show file tree

Hide file tree

Showing 8 changed files with 76 additions and 118 deletions.
diff --git a/README.rst b/README.rst
@@ -32,13 +32,14 @@ This benchmark is designed for generic but comprehensive performance tests of th
   * Parallelized via ``Dask`` using local cluster
   * Parallelized via ``Dask`` using Gateway cluster
 
-* Loading and reading columns from NanoAOD ROOT files is done using ``uproot``.
+* Multiple data access options:
+
+  * Explicit list of files or directories in local or mounted filesystem
+  * List of files, blocks, or datasets at CMS DBS accessed via network, e.g. XRootD or XCache
 
-* Generic operations applied to data in columns:
+* Loading and reading columns from NanoAOD ROOT files is done using ``uproot``.
 
-  * Nothing (just open the ROOT file with a given method)
-  * Load column data into memory
-  * Perform a simple operation on a column, e.g. ``mean()``
+* Abstract "operation" with a given timeout executed by workers to emulate data processing during analysis. 
 
 * Time profiling
 * Measuring size of columns in bytes

diff --git a/docs/source/modules/af_benchmark.data_access.rst b/docs/source/modules/af_benchmark.data_access.rst
diff --git a/docs/source/modules/af_benchmark.executor.rst b/docs/source/modules/af_benchmark.executor.rst
diff --git a/docs/source/modules/af_benchmark.executors.rst b/docs/source/modules/af_benchmark.executors.rst
@@ -0,0 +1,45 @@
+af\_benchmark.executors package
+===============================
+
+Submodules
+----------
+
+af\_benchmark.executors.base module
+-----------------------------------
+
+.. automodule:: af_benchmark.executors.base
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+af\_benchmark.executors.dask module
+-----------------------------------
+
+.. automodule:: af_benchmark.executors.dask
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+af\_benchmark.executors.futures module
+--------------------------------------
+
+.. automodule:: af_benchmark.executors.futures
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+af\_benchmark.executors.sequential module
+-----------------------------------------
+
+.. automodule:: af_benchmark.executors.sequential
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: af_benchmark.executors
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/source/modules/af_benchmark.processing.rst b/docs/source/modules/af_benchmark.processing.rst
diff --git a/docs/source/modules/af_benchmark.profiling.rst b/docs/source/modules/af_benchmark.profiling.rst
diff --git a/docs/source/modules/af_benchmark.rst b/docs/source/modules/af_benchmark.rst
@@ -7,10 +7,7 @@ Subpackages
 .. toctree::
    :maxdepth: 4
 
-   af_benchmark.data_access
-   af_benchmark.executor
-   af_benchmark.processing
-   af_benchmark.profiling
+   af_benchmark.executors
 
 Submodules
 ----------
@@ -23,6 +20,30 @@ af\_benchmark.benchmark module
    :undoc-members:
    :show-inheritance:
 
+af\_benchmark.data\_loader module
+---------------------------------
+
+.. automodule:: af_benchmark.data_loader
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+af\_benchmark.time\_profiler module
+-----------------------------------
+
+.. automodule:: af_benchmark.time_profiler
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+af\_benchmark.uproot\_processor module
+--------------------------------------
+
+.. automodule:: af_benchmark.uproot_processor
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 Module contents
 ---------------
 

diff --git a/tests/config-default.yaml b/tests/config-default.yaml
@@ -7,7 +7,6 @@ executor:
 processor:
   parallelize_over: files
   columns: 5
-  operation: sum
   load_columns_into_memory: True
   worker_operation_time: 0