Skip to content

Commit

Permalink
docs: Revised dev guide
Browse files Browse the repository at this point in the history
- mainly minor spelling and formatting corrections
- added exclude pattern to conf.py to limit rendering to the actual content
- changed toctree: excl. sphinx-based doc -> it is now linked in contributing.rst
  • Loading branch information
fsenf committed Sep 1, 2023
1 parent 15e8129 commit 1a000be
Show file tree
Hide file tree
Showing 6 changed files with 69 additions and 38 deletions.
10 changes: 9 additions & 1 deletion doc/code_structure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Modules

The **tobac** package currently consists of three **main modules**:

1. The :py:mod:`tobac.feature_detection` contains methods to identify objects (*features*) in 2D or 3D (3D or 4D when including the time dimensions) gridded data. This is done by identifying contiguous regions above or below one or multiple user-defined thresholds. The module makes use of :py:mod:`scipy.ndimage.label`, a generic image processing method that labels features in an array. The methods in :py:mod:`tobac.feature_detection` are high-level functions that enable a fast and effective feature detection and create easy-to-use output in form of a :py:mod:`pandas.DataFrame` that contains the coordinates and some basic information on each detected feature. The most high-level methods that is commonly used by users is `tobac.feature_detection_multithreshold`.
1. The :py:mod:`tobac.feature_detection` contains methods to identify objects (*features*) in 2D or 3D (3D or 4D when including the time dimensions) gridded data. This is done by identifying contiguous regions above or below one or multiple user-defined thresholds. The module makes use of :py:mod:`scipy.ndimage.label`, a generic image processing method that labels features in an array. The methods in :py:mod:`tobac.feature_detection` are high-level functions that enable a fast and effective feature detection and create easy-to-use output in form of a :py:mod:`pandas.DataFrame` that contains the coordinates and some basic information on each detected feature. The most high-level methods that is commonly used by users is :py:func:`tobac.feature_detection_multithreshold`.

2. The :py:mod:`tobac.segmentation` module contains methods to define the extent of the identified feature areas or volumes. This step is needed to create a mask of the identified features because the feature detection currently only saves the center points of the features. The segmentation procedure is performed by using the watershedding method, but more methods are to be implemented in the future. Just as the feature detection, this module can handle both 2D and 3D data.

Expand All @@ -34,6 +34,14 @@ For more information on each submodule, refer to the respective source code docu

One thing to note is that **tobac** as of now is purely functional. The plan is, however, to move towards a more object-oriented design with base classes for the main operations such as feature detection and tracking.


========
Examples
========

To help users get started with **tobac** and to demonstrate the various functionalities, **tobac** hosts several detailed and **illustrated examples** in the form of Jupyter notebooks. They are hosted under the directory `examples/` and be executed by the user.


============================
Migrating to xarray and dask
============================
Expand Down
4 changes: 4 additions & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@

html_static_path = ["_static"]

exclude_patterns = [
"_build",
]

project = "tobac"

Expand All @@ -34,6 +37,7 @@
# allow dropdowns
collapse_navigation = False


# Include our custom CSS (currently for special table config)
def setup(app):
app.add_css_file("theme_overrides.css")
Expand Down
35 changes: 21 additions & 14 deletions doc/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,16 @@
How to contribute
-------------------------

Step-by-step overview: https://github.com/tobac-project/tobac/blob/main/CONTRIBUTING.md


=========================
Code of conduct
=========================

We are a multi-institutional and international community that aims to maintain and increase our diversity. We acknowledge that we all come with different experiences and capacities. Therefore, we strive to foster an inclusive and respectful environmentwhere we help and support each other. We welcome any types of contributions and believe that we together can create accessible, reusable, and maintanable code that empowers researchers and enables groundbreaking science.
We are a multi-institutional and international community that aims to maintain and increase our diversity. We acknowledge that we all come with different experiences and capacities. Therefore, we strive to foster an inclusive and respectful environment where we help and support each other. We welcome any types of contributions and believe that we together can create accessible, reusable, and maintanable code that empowers researchers and enables groundbreaking science.

We would like to refer to the `Python code of conduct <https://www.python.org/psf/conduct/>`_ as we follow the same principlesfor communication and working with each other!
We would like to refer to the `Python code of conduct <https://www.python.org/psf/conduct/>`_ as we follow the same principles for communication and working with each other!

=========================
git basics
Expand All @@ -25,12 +28,12 @@ git basics
* Learn about `pull requests <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>`_
* Learn about `how to commit and push changes from your local repository <https://docs.github.com/en/migrations/importing-source-code/using-the-command-line-to-import-source-code/adding-locally-hosted-code-to-github>`_

* **Create an issue**: If you have an idea for a new feature or a suggestion for any kind of code changes, please create an issue for this. We use `our issues <https://github.com/tobac-project/tobac/issues>`_ to milestones `Link text <https://github.com/tobac-project/tobac/milestones>`_, i.e. the different versions of **tobac** to come.
The issues act, therefore, not only as a place for reporting bugs, but also as a collection of *to do* points.
* **Create an issue**: If you have an idea for a new feature or a suggestion for any kind of code changes, please create an issue for this. We sort `our issues <https://github.com/tobac-project/tobac/issues>`_ into `milestones <https://github.com/tobac-project/tobac/milestones>`_ to priorized work and manage our workflow, i.e. the different versions of **tobac** to come.

The issues act, therefore, not only as a place for reporting bugs, but also as a collection of *to do* points.

You can also work on any issue that was created by somebody else and is already out there. A tip is to look for the **good first issue** label, if you are a new developer. These issues are usually fairly easy to address and can be good to practice our GitHub workflow.
* **Work on an issue**: You can also work on any issue that was created by somebody else and is already out there. A tip is to look for the **good first issue** label, if you are a new developer. These issues are usually fairly easy to address and can be good to practice our GitHub workflow.

* https://github.com/tobac-project/tobac/blob/main/CONTRIBUTING.md

* **Create a pull request from your fork:** We use our personal forks of the tobac repository to create pull requests. This means that you have to first commit and push your local changes to your personal fork and then create a pull request from that fork: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork

Expand All @@ -40,27 +43,31 @@ You can also work on any issue that was created by somebody else and is already
Tips on working on your local code
===================================

* Install tobac package with :code:`pip install -e` . This allows you to directly test your local code changes as you run tobac. Instead of using the **tobac** version of the latest release, your local version of tobac will be used when you import **tobac** in a python script or IDE. Note that this way of installing a local package will use the code of the checked in branch, so this allows you also to test code while switching between branches.
* Install tobac package with :code:`pip install -e`
* This allows you to directly test your local code changes as you run tobac. Instead of using the **tobac** version of the latest release, your local version of tobac will be used when you import **tobac** in a python script or IDE.
* *Note that* this way of installing a local package will use the code of the checked in branch, so this allows you also to test code while switching between branches.

* You can locally build the documentation page:
* You can locally **build the documentation page**:
* see :doc:`testing_sphinx-based_rendering`

* Writing `meaningful commit messages <https://www.conventionalcommits.org/en/v1.0.0/>`_ can be very helpful for you and people who review your code to better understand the code changes.


=========================
Our branching strategy
=========================

While you can use any type of branching strategy and naming as you work in your personal fork, we have three branches in the tobac repository:

* :code:``RC_*`
* :code:``dev_*`
* :code:``hotfix`
* :code:`RC_*`
* :code:`dev_*`
* :code:`hotfix`

:code:``RC_*` is the release candidate of the next tobac version. The asterisk stands here for the specific tobac version: RC_vx.x.x (e.g. RC_v1.5.0). Pull requests to this branch need two reviews to be accepted before it can be merged into main.
:code:`RC_*` is the release candidate of the next tobac version. The asterisk stands here for the specific tobac version: RC_vx.x.x (e.g. RC_v1.5.0). Pull requests to this branch need two reviews to be accepted before it can be merged into main.

:code:``dev_*` is the development branch where we experiment with new features. This branch is perfectly suited to collaboratively work on a feature together with other **tobac** developers (see :doc:`mentoring`). In general, this branch is used for long-term, comprehensive code changes that might not be covered by a single pull request and where it might not be conceivable in which future **tobac** version to include the changes. There are no branch protection rules for this branch, which means that collaborators of our GitHub organization can directly push changes to this branch. Note that **dev_** can never directly merged into main, it has be merged into the release candidate branch RC_* first! There can be more than one `dev_*` branch, therefore it we recommand to describe the feature to work on in the respective branch (e.g. :code:`dev_xarray_transition`).
:code:`dev_*` is the development branch where we experiment with new features. This branch is perfectly suited to collaboratively work on a feature together with other **tobac** developers (see :doc:`mentoring`). In general, this branch is used for long-term, comprehensive code changes that might not be covered by a single pull request and where it might not be conceivable in which future **tobac** version to include the changes. There are no branch protection rules for this branch, which means that collaborators of our GitHub organization can directly push changes to this branch. Note that **dev_** can never directly merged into main, it has be merged into the release candidate branch :code:`RC_*` first! There can be more than one `dev_*` branch, therefore it we recommand to describe the feature to work on in the respective branch (e.g. :code:`dev_xarray_transition`).

:code:``hotfix` is the branch we use for hotfixes, i.e. bug fixes that need to be released as fast as possible because it influences people's code. This branch needs only one review before it can directly merged into :code:``main`.
:code:`hotfix` is the branch we use for hotfixes, i.e. bug fixes that need to be released as fast as possible because it influences people's code. This branch needs only one review before it can directly merged into :code:`main`.

In brief: **Unless you are collaboratively working on a comprehensive feature or on a hotfix, the branch to submit your pull request to is the next release candidate RC_v.x.x.x**

Expand Down
1 change: 0 additions & 1 deletion doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ The project is currently being extended by several contributors to include addit

code_structure
contributing
testing_sphinx-based_rendering
code_reviews
mentoring

Expand Down
2 changes: 1 addition & 1 deletion doc/mentoring.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Writing code collaboratively
We firmly believe that code can only get better if more than two eyes and one brain work on it. Therefore, we aim to write code collaboratively, in particular, when comprehensive refactoring or enhancements of the code are done. In practice, this can be done by creating a **draft pull request**. This makes it really easy to iteratively improve a pull request with the feedback from others until the pull request is ready for review.


When you work on a comprehensive feature with multiple developers, it is recommended to create a draft pull request on the :code:`dev_*` branch. As explained in our :doc:`our branching strategy <contributing>`, this branch does not undergo any protection rules. It is meant to experiment with new code and all collaborators of the `tobac-project organization <https://github.com/tobac-project>`_ can directly push to this branch. Creating a draft pull request has the advantage of facilitating the communication with other developers who contribute to the same new feature. You can directly see which changes they make, comment these and discuss ways to go forward.
When you work on a comprehensive feature with multiple developers, it is recommended to create a draft pull request on the :code:`dev_*` branch. As explained in :doc:`our branching strategy <contributing>`, this branch does not undergo any protection rules. It is meant to experiment with new code and all collaborators of the `tobac-project organization <https://github.com/tobac-project>`_ can directly push to this branch. Creating a draft pull request has the advantage of facilitating the communication with other developers who contribute to the same new feature. You can directly see which changes they make, comment these and discuss ways to go forward.

==============
Get a mentor
Expand Down
55 changes: 34 additions & 21 deletions doc/testing_sphinx-based_rendering.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ How to check the Sphinx-based rendering
The workflow has been tested in a linux system. We aim to build a static
website out of the documentation material present in ``tobac``.

==================================
1. Preparing the Local Environment
----------------------------------
==================================

- **choose a separate place for your testing**

Expand All @@ -31,34 +32,44 @@ website out of the documentation material present in ``tobac``.

.. code:: bash
git fetch --all
git checkout -t origin/<BRANCH>
/tmp/website-testing/tobac> git fetch --all
/tmp/website-testing/tobac> git checkout -t origin/<BRANCH>
- **Python environment**

- create a python virtual env
``bash /tmp/website-testing> python -m venv .python3-venv``

- and install requirements \```bash # deactivation conda is only
necessary if your loaded conda before … /tmp/website-testing>
conda deactivate
.. code:: bash
/tmp/website-testing> python -m venv .python3-venv
- and install requirements

.. code:: bash
# deactivation conda is only necessary if your loaded conda before …
/tmp/website-testing> conda deactivate
# activate the new env and upgrade ``pip``
/tmp/website-testing> source .python3-venv/bin/activate
/tmp/website-testing> pip install –upgrade pip
# now everything is installed into the local python env!
/tmp/website-testing> pip install -r tobac/doc/requirements.txt
# activate the new env and upgrade ``pip`` /tmp/website-testing>
source .python3-venv/bin/activate /tmp/website-testing> pip
install –upgrade pip
# and also install RTD scheme
/tmp/website-testing> pip install sphinx_rtd_theme
# now everything is installed into the local python env!
/tmp/website-testing> pip install -r tobac/doc/requirements.txt
`pip`-based installation takes a bit of time, but is much faster than `conda`.


# and also install RTD scheme /tmp/website-testing> pip install
sphinx_rtd_theme
If the installation runs without problems, you are ready to build the website.

\`\`\ ````\ pip\ ``-based installation takes a bit of time, but is much faster than``\ conda`.
If the installation runs without problems, you are ready to build
the website.

2. Building the Website
-----------------------
==================================
1. Building the Website
==================================

Actually, only few steps are needed to build the website, i.e.

Expand All @@ -78,8 +89,9 @@ Actually, only few steps are needed to build the website, i.e.
/tmp/website-testing/tobac> firefox doc/_build/html/index.html
==================================
3. Parsing Your Local Changes
-----------------------------
==================================

Now, we connect to your locally hosted ``tobac`` repository and your
development branch.
Expand Down Expand Up @@ -115,8 +127,9 @@ development branch.
/tmp/website-testing/tobac> firefox _build/html/index.html
==========================================
Option: Check Rendering of a Pull requests
------------------------------------------
==========================================

- **check the pull request out**: Now, assume the PR has the ID ``<ID>`` and you define the branch name ``BRANCH_NAME`` as you like

Expand Down

0 comments on commit 1a000be

Please sign in to comment.