Release 3.0.1 (PR #170)

Release 3.0.1

.travis.yml

@@ -6,6 +6,7 @@ branches:
   only:
     - develop
     - master
+    - version-3
 
 cache:
   - pip: true
@@ -33,10 +34,6 @@ jobs:
       language: python
       python: "3.6"
       dist: xenial
-    - name: "Python 3.5"
-      python: "3.5"
-    - name: "Python 2.7"
-      python: "2.7"
 
     - stage: "Linting and formatting"
       name: "Formatting with black"

AUTHORS

@@ -18,4 +18,7 @@ Chronological list of authors
   - Michael Gecht
 
 2018
-  - Marc Siggel
+  - Marc Siggel
+
+2020
+  - Sebastian Kehl

CHANGELOG.rst

@@ -1,3 +1,18 @@
+3.0.1 (2020-10-09)
+==================
+
+Features
+--------
+
+- Add ability to scan number of MPI ranks and OpenMP threads. (`#165 <https://github.com/bio-phys/MDBenchmark/issues/165>`_)
+- Add support to run multiple simulations on single nodes (GROMACS-only). (`#168 <https://github.com/bio-phys/MDBenchmark/issues/168>`_)
+
+3.0.0
+=====
+
+This version was skipped intentionally. Version 3.0.1 is the first release of the major version 3.
+
+
 2.0.1 (2020-03-04)
 ==================
 

DEVELOPER.rst

@@ -25,6 +25,14 @@ Using ``poetry`` you can simply run ``poetry install`` to
 install all dependencies. ``poetry`` will take care of creating a
 virtual environment for you.
 
+Working on the documentation
+----------------------------
+
+You will need to install extra packages to work on the documentation. Run
+``poetry install --extra docs`` to install all necessary dependencies. When in
+the ``docs`` folder, you can run ``poetry run make livehtml`` to start a local
+preview of the documentation, that will rebuild when you update a file.
+
 Running commands in the virtual environment
 -------------------------------------------
 
@@ -106,7 +114,7 @@ contain the file ``.invisible``. Also the file ``CHANGELOG.rst`` should have
 been updated.
 
 **Important:** Make sure that the version numbers inside
- ``mdbenchmark/__init__.py`` and ``CHANGELOG.rst`` match.
+``mdbenchmark/__init__.py`` and ``CHANGELOG.rst`` match.
 
 3. Generate dist files
 ----------------------
@@ -120,7 +128,7 @@ First make sure that your ``wheel`` package is up-to-date::
 
 Next we can generate a source distribution package and universal wheel::
 
-   $ python setup.py sdist bdist_wheel --universal
+    $ python setup.py sdist bdist_wheel --universal
 
 Check that the tarball inside ``./dist/`` includes all needed files (source
 code, ``README.rst``, ``CHANGELOG.rst``, ``LICENSE``), !

LICENSE

@@ -677,36 +677,6 @@ the library.  If this is what you want to do, use the GNU Lesser General
 Public License instead of this License.  But first, please read
 <https://www.gnu.org/licenses/why-not-lgpl.html>.
 
-==========================================================================
-
-cadishi.py (mdbenchmark/ext/cadishi) is released under the MIT license:
-
---------------------------------------------------------------------------
-
-MIT License
-
-Copyright (c) 2015-2017 Klaus Reuter, Max Planck Computing and Data Facility,
-                        Giessenbachstraße 2, 85748 Garching, Germany.
-Copyright (c) 2015-2017 Juergen Koefinger, Max Planck Institute of Biophysics,
-                        Max-von-Laue-Straße 3, 60438 Frankfurt am Main, Germany.
-
-Permission is hereby granted, free of charge, to any person obtaining a copy of
-this software and associated documentation files (the "Software"), to deal in
-the Software without restriction, including without limitation the rights to
-use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
-the Software, and to permit persons to whom the Software is furnished to do so,
-subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
-FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
-COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
-IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
-CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
 ============================================================================
 
 click_test.py (mdbenchmark/ext/click_test) is released under the MIT license
@@ -733,4 +703,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
+THE SOFTWARE.

README.rst

@@ -1,6 +1,6 @@
-============================================
-  Benchmark molecular dynamics simulations
-============================================
+========================================
+Benchmark molecular dynamics simulations
+========================================
 
 .. image:: https://img.shields.io/pypi/v/mdbenchmark.svg
     :target: https://pypi.python.org/pypi/mdbenchmark
@@ -51,21 +51,28 @@ pip
 
 .. code::
 
-   pip install mdbenchmark
+    pip install mdbenchmark
 
 conda
 -----
 
 .. code::
 
-   conda install -c conda-forge mdbenchmark
+    conda install -c conda-forge mdbenchmark
+
+pipx
+----
+
+.. code::
+
+    pipx install mdbenchmark
 
 pipenv
 ------
 
 .. code::
 
-   pipenv install mdbenchmark
+    pipenv install mdbenchmark
 
 After installation MDBenchmark is accessible on your command-line via ``mdbenchmark``::
 
@@ -91,6 +98,8 @@ Features
 - Automatically detects the queuing system on your high-performance cluster and submits jobs accordingly.
 - Grabs performance from the output logs and creates a fancy plot.
 - Benchmarks systems on CPUs and/or GPUs.
+- Find the best parameters by scanning different numbers of MPI ranks and OpenMP threads.
+- Run multiple instances of the same simulation on a single node using GROMACS' ``--multidir`` option.
 
 Short usage reference
 =====================
@@ -104,31 +113,41 @@ Benchmark generation
 Assuming you want to benchmark GROMACS version 2018.3 and your TPR file is
 called ``protein.tpr``, run the following command::
 
-  mdbenchmark generate --name protein --module gromacs/2018.3
+    mdbenchmark generate --name protein --module gromacs/2018.3
 
 To run benchmarks on GPUs simply add the ``--gpu`` flag::
 
-  mdbenchmark generate --name protein --module gromacs/2018.3 --gpu
+    mdbenchmark generate --name protein --module gromacs/2018.3 --gpu
 
 Benchmark submission
 --------------------
 
 After you generated your benchmarks, you can submit them at once::
 
-  mdbenchmark submit
+    mdbenchmark submit
 
 Benchmark analysis
 ------------------
 
 As soon as the benchmarks have been submitted you can run the analysis via
 ``mdbenchmark analyze``. Systems that have not finished yet will be marked with a question mark (``?``). You can save the performance results to a CSV file and subsequently create a plot from the data::
 
-    # Print performance results to console and save them to a file called results.csv 
+    # Print performance results to console and save them to a file called results.csv
     mdbenchmark analyze --save-csv results.csv
-    
+
     # Create a plot from the results present in the file results.csv
     mdbenchmark plot --csv results.csv
 
+Literature
+==========
+
+Please cite the latest MDBenchmark publication if you use the tool to benchmark
+your simulations. This will help raise awareness of benchmarking and help people
+improve their simulation performance, as well as reduce overall resource
+wastage.
+
+M\. Gecht, M. Siggel, M. Linke, G. Hummer, J. Köfinger MDBenchmark: A toolkit to optimize the performance of molecular dynamics simulations. J. Chem. Phys. 153, 144105 (2020); https://doi.org/10.1063/5.0019045
+
 Contributing
 ============
 

docs/analyze.rst

@@ -47,19 +47,6 @@ single benchmark::
 
   mdbenchmark analyze --directory draco_gromacs/2018.3
 
-Plotting of benchmark results
------------------------------
-
-  .. warning::
-
-   The function |mdbenchmark.analyze.plot|_ was deprecated with version 2.0. You should migrate to the newer ``mdbenchmark plot``, as it provides more functionality and the former version will be removed in the future.
-
-MDBenchmark provides a quick and simple way to plot the results of the
-benchmarks, giving you a ``.pdf`` file as output. To generate a plot simply use
-the ``--plot`` option::
-
-  mdbenchmark analyze --plot
-
 Plot the number of cores
 ~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/conf.py

@@ -51,17 +51,17 @@
 
 # General information about the project.
 project = "MDBenchmark"
-copyright = "2017-2018, The MDBenchmark development team"
+copyright = "2017-2020, The MDBenchmark development team"
 author = "Written by the MDBenchmark development team"
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = "2.0"
+version = "3.0"
 # The full version, including alpha/beta/rc tags.
-release = "2.0.1"
+release = "3.0.1"
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/general.rst

@@ -46,17 +46,17 @@ NAMD
 .. note::
 
   **NAMD support is experimental.** If you encounter any problems or bugs, we
-  would appreciate to `hear from you`_.
+    would appreciate to `hear from you`_.
 
 Generating benchmarks for NAMD follows a similar process to GROMACS. Assuming
 the NAMD configuration file is called ``protein.namd``, you will also need the
 corresponding ``protein.pdb`` and ``protein.psf`` inside the same folder.
 
 .. warning::
 
-  Please be aware that all paths given in the ``protein.namd`` file
-  must be absolute paths. This ensures that MDBenchmark does not destroy paths
-  when copying files around during benchmark generation.
+    Please be aware that all paths given in the ``protein.namd`` file must be
+    absolute paths. This ensures that MDBenchmark does not destroy paths when
+    copying files around during benchmark generation.
 
 In analogy to the GROMACS setup, you can execute the following command to
 generate benchmarks for a module named ``namd/2.12``:

docs/generate.rst

@@ -94,18 +94,74 @@ simply use the ``--host`` option::
 Running on CPUs or GPUs
 -----------------------
 
-Depending on your setup you might want to run your simulations only on GPUs 
-or CPUs. You can do so with the ``--cpu/--no-cpu`` and ``--gpu/--no-gpu`` flags, ``-c/-nc` and ``-g/-ng` respectively.
-If neither of both options is given, benchmarks will be generated for CPUs only.
-The default template for the MPCDF cluster ``draco`` showcases the ability to
-run benchmarks on GPUs::
+Depending on your setup you might want to run your simulations only on GPUs or
+CPUs. You can do so with the ``--cpu/--no-cpu`` and ``--gpu/--no-gpu`` flags,
+``-c/-nc` and ``-g/-ng`` respectively. If neither of both options is given,
+benchmarks will be generated for CPUs only. The default template for the MPCDF
+cluster ``draco`` showcases the ability to run benchmarks on GPUs::
 
   mdbenchmark generate --gpu
 
 This generates benchmarks for both GPU and CPU partitions. If you only want to run on
 GPUs this is easily achieved with::
 
-   mdbenchmark generate --gpu --no-cpu
+  mdbenchmark generate --gpu --no-cpu
+
+
+Using a different number of ranks or threads
+--------------------------------------------
+
+The correct choice on the number of MPI ranks and OpenMP threads and
+hyperthreading depends on your available hardware and software resources, your
+simulation system and used MD engine. MDBenchmark can help you scan different
+numbers of ranks and threads.
+
+.. note::
+
+  The following was only tested with GROMACS.
+
+To use this feature, you first need to know the number of physical cores on your
+compute nodes. MDBenchmark will try to guess the number of physical cores. The
+guess is only correct if the machine from which you submit the jobs, i.e., a
+login node on a supercomputer, has the same number of cores as the actual
+compute nodes. You can override the number of physical cores with the
+``--physical-cores`` options.
+
+In addition, Intel CPUs are able run two calculations on the same core at the
+same time. This feature is called "hyperthreading". If your CPU supports
+hyperthreading, then it also has logical cores, which is twice the number of
+physical cores. Assuming the CPUs of your compute node have 40 physical cores
+and supports hyperthreading, you need to specify the following settings::
+
+  mdbenchmark generate --physical-cores 40 --logical-cores 80
+
+The above example would generate benchmarks without hyperthreading. To enable
+hyperthreading you need to specify the ``--hyperthreading`` option::
+
+  mdbenchmark generate --hyperthreading
+
+Now that you have defined the number of available cores and whether you want to
+toggle hyperthreading, you can define the number of MPI ranks that MDBenchmark
+should use for the job::
+
+  mdbenchmark generate --ranks 2 --ranks 10 --ranks 40
+
+The above command will generate jobs using 2, 10 and 40 MPI ranks. MDBenchmark
+will calculate the number of OpenMP threads by itself. As a general rule:
+`number_of_cores = number_of_ranks * number_of_threads`. If your CPU does not
+support hyperthreading, then the number of cores equals the number of physical
+cores. If it does support hyperthreading, then it equals the number of logical
+cores.
+
+Combining all options you can run benchmarks on 1-10 with and without GPUs using
+either 4, 8 or 20 MPI ranks with hyperthreading with the following command::
+
+  mdbenchmark generate --max-nodes 10 --cpu --gpu --ranks 4 --ranks 8 --ranks 20 --physical-cores 40 --logical-cores 80 --hyperthreading
+
+In the above case, MDBenchmark will generate jobs with 4 MPI ranks/20 OpenMP
+threads; 8 MPI ranks/10 OpenMP threads and 20 MPI ranks/4 OpenMP threads to
+fulfill the constraint from above. A total of 60 benchmarks will be generated
+(``10 (nodes) * 2 (gpu/cpu) * 3 (ranks)``).
 
 
 Limiting the run time of benchmarks
@@ -128,6 +184,29 @@ If you want your benchmark jobs to have specific names, use the ``--job-name`` o
 
   mdbenchmark generate --job-name my_benchmark
 
+Multiple jobs per node
+----------------------
+
+.. note::
+
+  Multiple simulations per node are currently only supported with GROMACS. The
+  developers of MDBenchmark welcome all support to implement further MD engines.
+
+It is possible to run multiple simulations on a single node to use the available
+resources more efficiently. For example, when a node is equipped with two GPUs
+it is possible to run either 1, 2 or 4 simulations on this single node. Each
+instance of the simulation will generate shorter trajectories, but the overall
+performance (the sum of all instances) will most likely be bigger than running a
+single simulation on one node. This is especially useful when one is interested
+in running many short simulations, instead of a single long simulation.
+
+To use this feature, users can specify the ``--multidir`` option. This will make
+use of the built-in functionality availabe in GROMACS, which itself will take
+care of running multiple independent instances of the same system. The following
+command will run four benchmarks of a single system on the same node::
+
+  mdbenchmark generate --multidir 4
+
 .. _modules: https://linux.die.net/man/1/module
 .. _draco: https://www.mpcdf.mpg.de/services/computing/draco
 .. _hydra: https://www.mpcdf.mpg.de/services/computing/hydra