From 1a000bef46d76cbb173c28811d1bb613490c5c96 Mon Sep 17 00:00:00 2001 From: Fabian Senf Date: Fri, 1 Sep 2023 14:24:37 +0200 Subject: [PATCH] docs: Revised dev guide - mainly minor spelling and formatting corrections - added exclude pattern to conf.py to limit rendering to the actual content - changed toctree: excl. sphinx-based doc -> it is now linked in contributing.rst --- doc/code_structure.rst | 10 ++++- doc/conf.py | 4 ++ doc/contributing.rst | 35 +++++++++------- doc/index.rst | 1 - doc/mentoring.rst | 2 +- doc/testing_sphinx-based_rendering.rst | 55 ++++++++++++++++---------- 6 files changed, 69 insertions(+), 38 deletions(-) diff --git a/doc/code_structure.rst b/doc/code_structure.rst index cedf1ed7..cea93fdd 100644 --- a/doc/code_structure.rst +++ b/doc/code_structure.rst @@ -9,7 +9,7 @@ Modules The **tobac** package currently consists of three **main modules**: -1. The :py:mod:`tobac.feature_detection` contains methods to identify objects (*features*) in 2D or 3D (3D or 4D when including the time dimensions) gridded data. This is done by identifying contiguous regions above or below one or multiple user-defined thresholds. The module makes use of :py:mod:`scipy.ndimage.label`, a generic image processing method that labels features in an array. The methods in :py:mod:`tobac.feature_detection` are high-level functions that enable a fast and effective feature detection and create easy-to-use output in form of a :py:mod:`pandas.DataFrame` that contains the coordinates and some basic information on each detected feature. The most high-level methods that is commonly used by users is `tobac.feature_detection_multithreshold`. +1. The :py:mod:`tobac.feature_detection` contains methods to identify objects (*features*) in 2D or 3D (3D or 4D when including the time dimensions) gridded data. This is done by identifying contiguous regions above or below one or multiple user-defined thresholds. The module makes use of :py:mod:`scipy.ndimage.label`, a generic image processing method that labels features in an array. The methods in :py:mod:`tobac.feature_detection` are high-level functions that enable a fast and effective feature detection and create easy-to-use output in form of a :py:mod:`pandas.DataFrame` that contains the coordinates and some basic information on each detected feature. The most high-level methods that is commonly used by users is :py:func:`tobac.feature_detection_multithreshold`. 2. The :py:mod:`tobac.segmentation` module contains methods to define the extent of the identified feature areas or volumes. This step is needed to create a mask of the identified features because the feature detection currently only saves the center points of the features. The segmentation procedure is performed by using the watershedding method, but more methods are to be implemented in the future. Just as the feature detection, this module can handle both 2D and 3D data. @@ -34,6 +34,14 @@ For more information on each submodule, refer to the respective source code docu One thing to note is that **tobac** as of now is purely functional. The plan is, however, to move towards a more object-oriented design with base classes for the main operations such as feature detection and tracking. + +======== +Examples +======== + +To help users get started with **tobac** and to demonstrate the various functionalities, **tobac** hosts several detailed and **illustrated examples** in the form of Jupyter notebooks. They are hosted under the directory `examples/` and be executed by the user. + + ============================ Migrating to xarray and dask ============================ diff --git a/doc/conf.py b/doc/conf.py index 2b8c717a..16b21aa5 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -26,6 +26,9 @@ html_static_path = ["_static"] +exclude_patterns = [ + "_build", +] project = "tobac" @@ -34,6 +37,7 @@ # allow dropdowns collapse_navigation = False + # Include our custom CSS (currently for special table config) def setup(app): app.add_css_file("theme_overrides.css") diff --git a/doc/contributing.rst b/doc/contributing.rst index 3ab327df..7cd5eddb 100644 --- a/doc/contributing.rst +++ b/doc/contributing.rst @@ -4,13 +4,16 @@ How to contribute ------------------------- +Step-by-step overview: https://github.com/tobac-project/tobac/blob/main/CONTRIBUTING.md + + ========================= Code of conduct ========================= -We are a multi-institutional and international community that aims to maintain and increase our diversity. We acknowledge that we all come with different experiences and capacities. Therefore, we strive to foster an inclusive and respectful environmentwhere we help and support each other. We welcome any types of contributions and believe that we together can create accessible, reusable, and maintanable code that empowers researchers and enables groundbreaking science. +We are a multi-institutional and international community that aims to maintain and increase our diversity. We acknowledge that we all come with different experiences and capacities. Therefore, we strive to foster an inclusive and respectful environment where we help and support each other. We welcome any types of contributions and believe that we together can create accessible, reusable, and maintanable code that empowers researchers and enables groundbreaking science. -We would like to refer to the `Python code of conduct `_ as we follow the same principlesfor communication and working with each other! +We would like to refer to the `Python code of conduct `_ as we follow the same principles for communication and working with each other! ========================= git basics @@ -25,12 +28,12 @@ git basics * Learn about `pull requests `_ * Learn about `how to commit and push changes from your local repository `_ -* **Create an issue**: If you have an idea for a new feature or a suggestion for any kind of code changes, please create an issue for this. We use `our issues `_ to milestones `Link text `_, i.e. the different versions of **tobac** to come. -The issues act, therefore, not only as a place for reporting bugs, but also as a collection of *to do* points. +* **Create an issue**: If you have an idea for a new feature or a suggestion for any kind of code changes, please create an issue for this. We sort `our issues `_ into `milestones `_ to priorized work and manage our workflow, i.e. the different versions of **tobac** to come. + + The issues act, therefore, not only as a place for reporting bugs, but also as a collection of *to do* points. -You can also work on any issue that was created by somebody else and is already out there. A tip is to look for the **good first issue** label, if you are a new developer. These issues are usually fairly easy to address and can be good to practice our GitHub workflow. +* **Work on an issue**: You can also work on any issue that was created by somebody else and is already out there. A tip is to look for the **good first issue** label, if you are a new developer. These issues are usually fairly easy to address and can be good to practice our GitHub workflow. -* https://github.com/tobac-project/tobac/blob/main/CONTRIBUTING.md * **Create a pull request from your fork:** We use our personal forks of the tobac repository to create pull requests. This means that you have to first commit and push your local changes to your personal fork and then create a pull request from that fork: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork @@ -40,27 +43,31 @@ You can also work on any issue that was created by somebody else and is already Tips on working on your local code =================================== -* Install tobac package with :code:`pip install -e` . This allows you to directly test your local code changes as you run tobac. Instead of using the **tobac** version of the latest release, your local version of tobac will be used when you import **tobac** in a python script or IDE. Note that this way of installing a local package will use the code of the checked in branch, so this allows you also to test code while switching between branches. +* Install tobac package with :code:`pip install -e` + * This allows you to directly test your local code changes as you run tobac. Instead of using the **tobac** version of the latest release, your local version of tobac will be used when you import **tobac** in a python script or IDE. + * *Note that* this way of installing a local package will use the code of the checked in branch, so this allows you also to test code while switching between branches. -* You can locally build the documentation page: +* You can locally **build the documentation page**: + * see :doc:`testing_sphinx-based_rendering` * Writing `meaningful commit messages `_ can be very helpful for you and people who review your code to better understand the code changes. + ========================= Our branching strategy ========================= While you can use any type of branching strategy and naming as you work in your personal fork, we have three branches in the tobac repository: -* :code:``RC_*` -* :code:``dev_*` -* :code:``hotfix` +* :code:`RC_*` +* :code:`dev_*` +* :code:`hotfix` -:code:``RC_*` is the release candidate of the next tobac version. The asterisk stands here for the specific tobac version: RC_vx.x.x (e.g. RC_v1.5.0). Pull requests to this branch need two reviews to be accepted before it can be merged into main. +:code:`RC_*` is the release candidate of the next tobac version. The asterisk stands here for the specific tobac version: RC_vx.x.x (e.g. RC_v1.5.0). Pull requests to this branch need two reviews to be accepted before it can be merged into main. -:code:``dev_*` is the development branch where we experiment with new features. This branch is perfectly suited to collaboratively work on a feature together with other **tobac** developers (see :doc:`mentoring`). In general, this branch is used for long-term, comprehensive code changes that might not be covered by a single pull request and where it might not be conceivable in which future **tobac** version to include the changes. There are no branch protection rules for this branch, which means that collaborators of our GitHub organization can directly push changes to this branch. Note that **dev_** can never directly merged into main, it has be merged into the release candidate branch RC_* first! There can be more than one `dev_*` branch, therefore it we recommand to describe the feature to work on in the respective branch (e.g. :code:`dev_xarray_transition`). +:code:`dev_*` is the development branch where we experiment with new features. This branch is perfectly suited to collaboratively work on a feature together with other **tobac** developers (see :doc:`mentoring`). In general, this branch is used for long-term, comprehensive code changes that might not be covered by a single pull request and where it might not be conceivable in which future **tobac** version to include the changes. There are no branch protection rules for this branch, which means that collaborators of our GitHub organization can directly push changes to this branch. Note that **dev_** can never directly merged into main, it has be merged into the release candidate branch :code:`RC_*` first! There can be more than one `dev_*` branch, therefore it we recommand to describe the feature to work on in the respective branch (e.g. :code:`dev_xarray_transition`). -:code:``hotfix` is the branch we use for hotfixes, i.e. bug fixes that need to be released as fast as possible because it influences people's code. This branch needs only one review before it can directly merged into :code:``main`. +:code:`hotfix` is the branch we use for hotfixes, i.e. bug fixes that need to be released as fast as possible because it influences people's code. This branch needs only one review before it can directly merged into :code:`main`. In brief: **Unless you are collaboratively working on a comprehensive feature or on a hotfix, the branch to submit your pull request to is the next release candidate RC_v.x.x.x** diff --git a/doc/index.rst b/doc/index.rst index f04f4e4c..042f6628 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -66,7 +66,6 @@ The project is currently being extended by several contributors to include addit code_structure contributing - testing_sphinx-based_rendering code_reviews mentoring diff --git a/doc/mentoring.rst b/doc/mentoring.rst index 98f1f984..c956af5e 100644 --- a/doc/mentoring.rst +++ b/doc/mentoring.rst @@ -8,7 +8,7 @@ Writing code collaboratively We firmly believe that code can only get better if more than two eyes and one brain work on it. Therefore, we aim to write code collaboratively, in particular, when comprehensive refactoring or enhancements of the code are done. In practice, this can be done by creating a **draft pull request**. This makes it really easy to iteratively improve a pull request with the feedback from others until the pull request is ready for review. -When you work on a comprehensive feature with multiple developers, it is recommended to create a draft pull request on the :code:`dev_*` branch. As explained in our :doc:`our branching strategy `, this branch does not undergo any protection rules. It is meant to experiment with new code and all collaborators of the `tobac-project organization `_ can directly push to this branch. Creating a draft pull request has the advantage of facilitating the communication with other developers who contribute to the same new feature. You can directly see which changes they make, comment these and discuss ways to go forward. +When you work on a comprehensive feature with multiple developers, it is recommended to create a draft pull request on the :code:`dev_*` branch. As explained in :doc:`our branching strategy `, this branch does not undergo any protection rules. It is meant to experiment with new code and all collaborators of the `tobac-project organization `_ can directly push to this branch. Creating a draft pull request has the advantage of facilitating the communication with other developers who contribute to the same new feature. You can directly see which changes they make, comment these and discuss ways to go forward. ============== Get a mentor diff --git a/doc/testing_sphinx-based_rendering.rst b/doc/testing_sphinx-based_rendering.rst index a6e0659c..f0a7443e 100644 --- a/doc/testing_sphinx-based_rendering.rst +++ b/doc/testing_sphinx-based_rendering.rst @@ -5,8 +5,9 @@ How to check the Sphinx-based rendering The workflow has been tested in a linux system. We aim to build a static website out of the documentation material present in ``tobac``. +================================== 1. Preparing the Local Environment ----------------------------------- +================================== - **choose a separate place for your testing** @@ -31,34 +32,44 @@ website out of the documentation material present in ``tobac``. .. code:: bash - git fetch --all - git checkout -t origin/ + /tmp/website-testing/tobac> git fetch --all + /tmp/website-testing/tobac> git checkout -t origin/ - **Python environment** - create a python virtual env - ``bash /tmp/website-testing> python -m venv .python3-venv`` - - and install requirements \```bash # deactivation conda is only - necessary if your loaded conda before … /tmp/website-testing> - conda deactivate + .. code:: bash + + /tmp/website-testing> python -m venv .python3-venv + + + - and install requirements + + .. code:: bash + + # deactivation conda is only necessary if your loaded conda before … + /tmp/website-testing> conda deactivate + + # activate the new env and upgrade ``pip`` + /tmp/website-testing> source .python3-venv/bin/activate + /tmp/website-testing> pip install –upgrade pip + + # now everything is installed into the local python env! + /tmp/website-testing> pip install -r tobac/doc/requirements.txt - # activate the new env and upgrade ``pip`` /tmp/website-testing> - source .python3-venv/bin/activate /tmp/website-testing> pip - install –upgrade pip + # and also install RTD scheme + /tmp/website-testing> pip install sphinx_rtd_theme - # now everything is installed into the local python env! - /tmp/website-testing> pip install -r tobac/doc/requirements.txt + `pip`-based installation takes a bit of time, but is much faster than `conda`. + - # and also install RTD scheme /tmp/website-testing> pip install - sphinx_rtd_theme +If the installation runs without problems, you are ready to build the website. - \`\`\ ````\ pip\ ``-based installation takes a bit of time, but is much faster than``\ conda`. - If the installation runs without problems, you are ready to build - the website. -2. Building the Website ------------------------ +================================== +1. Building the Website +================================== Actually, only few steps are needed to build the website, i.e. @@ -78,8 +89,9 @@ Actually, only few steps are needed to build the website, i.e. /tmp/website-testing/tobac> firefox doc/_build/html/index.html +================================== 3. Parsing Your Local Changes ------------------------------ +================================== Now, we connect to your locally hosted ``tobac`` repository and your development branch. @@ -115,8 +127,9 @@ development branch. /tmp/website-testing/tobac> firefox _build/html/index.html +========================================== Option: Check Rendering of a Pull requests ------------------------------------------- +========================================== - **check the pull request out**: Now, assume the PR has the ID ```` and you define the branch name ``BRANCH_NAME`` as you like