Docs: fix broken links and add check on Github action (aiidateam#5003)

A significant number of links in the AiiDA documentation were broken.
Here we fix these links and add an automatic check on github actions that will fail if links break or broken links are introduced.

.github/workflows/ci-code.yml

@@ -1,4 +1,4 @@
-name: continuous-integration
+name: continuous-integration-code
 
 on:
   push:

.github/workflows/ci-style.yml

@@ -1,4 +1,4 @@
-name: continuous-integration
+name: continuous-integration-style
 
 on:
   push:

.github/workflows/docs-build.yml

@@ -0,0 +1,43 @@
+name: docs-build
+
+on:
+  push:
+    branches-ignore: [gh-pages]
+  pull_request:
+    branches-ignore: [gh-pages]
+    paths: ['docs/**']
+
+jobs:
+
+  docs-linkcheck:
+
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python 3.8
+      uses: actions/setup-python@v2
+      with:
+        python-version: 3.8
+    - name: Install python dependencies
+      run: |
+        pip install -e .[docs,tests]
+    - name: Build HTML docs
+      id: linkcheck
+      run: |
+        make -C docs html linkcheck 2>&1 | tee check.log
+        echo "::set-output name=broken::$(grep '(line\s*[0-9]*)\(\s\)broken\(\s\)' check.log)"
+      env:
+        SPHINXOPTS: -nW --keep-going
+
+    - name: Show docs build check results
+      run: |
+        if [ -z "${{ steps.linkcheck.outputs.broken }}" ]; then
+            echo "No broken links found."
+            exit 0
+        else
+            echo "Broken links found:"
+            echo "${{ steps.linkcheck.outputs.broken }}"
+            exit 1
+        fi

aiida/orm/nodes/data/cif.py

@@ -50,7 +50,7 @@ def cif_from_ase(ase, full_occupancies=False, add_fake_biso=False):
     """
     Construct a CIF datablock from the ASE structure. The code is taken
     from
-    https://wiki.fysik.dtu.dk/ase/epydoc/ase.io.cif-pysrc.html#write_cif,
+    https://wiki.fysik.dtu.dk/ase/ase/io/formatoptions.html#ase.io.cif.write_cif,
     as the original ASE code contains a bug in printing the
     Hermann-Mauguin symmetry space group symbol.
 

aiida/orm/querybuilder.py

@@ -2168,7 +2168,7 @@ def iterall(self, batch_size=100):
         Same as :meth:`.all`, but returns a generator.
         Be aware that this is only safe if no commit will take place during this
         transaction. You might also want to read the SQLAlchemy documentation on
-        http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.yield_per
+        https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.yield_per
 
 
         :param int batch_size:
@@ -2191,7 +2191,7 @@ def iterdict(self, batch_size=100):
         Same as :meth:`.dict`, but returns a generator.
         Be aware that this is only safe if no commit will take place during this
         transaction. You might also want to read the SQLAlchemy documentation on
-        http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.yield_per
+        https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.yield_per
 
 
         :param int batch_size:

aiida/tools/data/orbital/realhydrogen.py

@@ -258,7 +258,7 @@ class RealhydrogenOrbital(Orbital):
     OrbitalData class.
 
     Following the notation of table 3.1, 3.2 of Wannier90 user guide
-    http://www.wannier.org/doc/user_guide.pdf
+    (which can be downloaded from http://www.wannier.org/support/)
     A brief description of what is meant by each of these labels:
 
     :param radial_nodes: the number of radial nodes (or inflections) if no

docs/source/conf.py

@@ -408,3 +408,8 @@ def setup(app):
     nitpick_ignore = [
         tuple(line.strip().split(None, 1)) for line in handle.readlines() if line.strip() and not line.startswith('#')
     ]
+
+linkcheck_ignore = [r'http://localhost:\d+/',
+                    r'http://127.0.0.1:\d+/',
+                    r'http://www.wannier.org/support/',
+                    r'https://github.com/aiidateam/aiida-diff/blob/92c61bdcc2db201d69da4d8b83a2b3f5dd529bf1/aiida_diff/data/__init__.py#L14-L20']

docs/source/howto/installation.rst

@@ -89,7 +89,7 @@ Place this command in your shell or virtual environment activation script to aut
 This file is shell specific, but likely one of the following:
 
     * the startup file of your shell (``.bashrc``, ``.zsh``, ...), if aiida is installed system-wide
-    * the `activation script <https://virtualenv.pypa.io/en/latest/userguide/#activate-script>`_ of your virtual environment
+    * the `activators <https://virtualenv.pypa.io/en/latest/user_guide.html#activators>`_ of your virtual environment
     * a `startup file <https://conda.io/docs/user-guide/tasks/manage-environments.html#saving-environment-variables>`_ for your conda environment
 
 

docs/source/howto/plugins_develop.rst

@@ -18,22 +18,22 @@ Creating a plugin package
 =========================
 
 
-AiiDA plugins can be bundled and distributed in a `Python package <packages>`_ that provides a set of extensions to AiiDA.
+AiiDA plugins can be bundled and distributed in a `Python package <packages_>`_ that provides a set of extensions to AiiDA.
 
 .. note::
 
   The Python community uses the term 'package' rather loosely.
-  Depending on context, it may refer simply to a folder containing individual Python modules or it may include the files necessary for building and installing a package to be distributed via the `Python Package Index (PyPI) <pypi>`_.
+  Depending on context, it may refer simply to a folder containing individual Python modules or it may include the files necessary for building and installing a package to be distributed via the `Python Package Index (PyPI) <pypi_>`_.
 
-.. _packages: https://docs.python.org/2/tutorial/modules.html?highlight=package#packages
+.. _packages: https://docs.python.org/3/tutorial/modules.html#packages
 
 
 Quickstart
 ----------
 
-The fastest way to jumpstart an AiiDA plugin package is to use the `AiiDA plugin cutter <plugin-cutter>`_ in order to template the basic folder structure, already customized according to the desired name of your plugin, following AiiDA conventions.
+The fastest way to jumpstart an AiiDA plugin package is to use the `AiiDA plugin cutter <plugin-cutter_>`_ in order to template the basic folder structure, already customized according to the desired name of your plugin, following AiiDA conventions.
 
-Simply go to the `AiiDA plugin cutter <plugin-cutter>`_ and follow the usage instructions.
+Simply go to the `AiiDA plugin cutter <plugin-cutter_>`_ and follow the usage instructions.
 See also the `aiida-diff`_ demo plugin package for an in-depth explanation of the files & folders produced by the plugin cutter.
 
 In the following, we explain some of the conventions implemented by the AiiDA plugin cutter.
@@ -52,7 +52,7 @@ The naming convention for AiiDA plugin packages is ``aiida-mycode`` for the plug
 
    Python package names cannot contain dashes, thus the underscore.
 
-If you intend to eventually publish your plugin package, please go to the `AiiDA plugin registry <registry>`_  and choose a name that is not already taken.
+If you intend to eventually publish your plugin package, please go to the `AiiDA plugin registry <registry_>`_  and choose a name that is not already taken.
 You are also encouraged to pre-register your package (instructions provided on the registry), both to reserve your plugin name and to inform others of your ongoing development.
 
 
@@ -62,7 +62,7 @@ Folder structure
 ----------------
 
 The overall folder structure of your plugin is up to you, but it is useful to follow a set of basic conventions.
-Here is an example of a folder structure for an AiiDA plugin, illustrating different levels of nesting (see also the `aiida-diff demo plugin`_)::
+Here is an example of a folder structure for an AiiDA plugin, illustrating different levels of nesting (see also the `aiida-diff demo plugin <aiida-diff_>`_)::
 
    aiida-mycode/           - distribution folder
       aiida_mycode/        - top-level package (from aiida_mycode import ..)
@@ -137,7 +137,7 @@ Your new entry point should now show up in ``verdi plugin list aiida.calculation
     * If the ``-e`` option is given, a symbolic link from the python package search path to the ``aiida-diff`` directory is created and the ``.egg-info`` folder is put there instead.
       Changes to the **source code** will be picked up by python without reinstalling (when restarting the interpreter),  but changes to the **metadata** will not.
 
-    For further details, see the Python `packaging user guide <https://packaging.python.org/distributing/#configuring-your-project>`_.
+    For further details, see the Python `packaging user guide <packaging_>`_.
 
 
 
@@ -146,7 +146,7 @@ Your new entry point should now show up in ``verdi plugin list aiida.calculation
 Testing a plugin package
 =========================
 
-Writing tests for your AiiDA plugins and running continuous integration tests using free platforms like `GitHub Actions <ghactions>`_ is the best way to ensure that your plugin works and keeps working as it is being developed.
+Writing tests for your AiiDA plugins and running continuous integration tests using free platforms like `GitHub Actions <ghactions_>`_ is the best way to ensure that your plugin works and keeps working as it is being developed.
 We recommend using the `pytest`_ framework for testing AiiDA plugins.
 
 For an example of how to write tests and how to set up continuous integration, see the `aiida-diff`_ demo plugin package.
@@ -171,7 +171,7 @@ Many tests require a full AiiDA environment to be set up before the test starts,
 The pytest library has the concept of `fixtures`_ for encapsulating code you would like to run before a test starts.
 AiiDA ships with a number of fixtures in :py:mod:`aiida.manage.tests.pytest_fixtures` that take care of setting up the test environment for you (for more details, see :ref:`topics:plugins:testfixtures`).
 
-In order to make these fixtures available to your tests, create a ``conftest.py`` (see also `pytest docs <conftest>`_) at the root level of your plugin package as follows::
+In order to make these fixtures available to your tests, create a ``conftest.py`` (see also `pytest docs <conftest_>`_) at the root level of your plugin package as follows::
 
    import pytest
    pytest_plugins = ['aiida.manage.tests.pytest_fixtures']  # make AiiDA's fixtures available
@@ -212,8 +212,8 @@ You can now start writing tests e.g. in a ``tests/test_calculations.py`` file::
 In order to run your tests, simply type ``pytest`` at the root level or your package.
 pytest automatically discovers and executes files, classes and function names starting with the word ``test``.
 
-.. _conftest: https://docs.pytest.org/en/stable/fixture.html?highlight=conftest#conftest-py-sharing-fixture-functions
-.. _fixtures: https://docs.pytest.org/en/latest/fixture.html
+.. _conftest: https://docs.pytest.org/en/latest/how-to/fixtures.html#scope-sharing-fixtures-across-classes-modules-packages-or-session
+.. _fixtures: https://docs.pytest.org/en/latest/how-to/fixtures.html
 
 
 .. _how-to:plugins-develop:document:
@@ -252,7 +252,7 @@ For simple plugins, a well-written ``README.md`` can be a good start.
 Once the README grows out of proportion, you may want to consider creating a dedicated documentation website.
 
 The `Sphinx <http://www.sphinx-doc.org/en/master/>`_ tool makes it very easy to create documentation websites for python packages, and the `ReadTheDocs <http://readthedocs.org/>`_ service will host your sphinx documentation online for free.
-The `aiida-diff demo plugin <aiida-diff>`_ comes with a full template for a sphinx-based documentation, including a mix of manually written pages and an automatically generated documentation of your plugin's python API.
+The `aiida-diff demo plugin <aiida-diff_>`_ comes with a full template for a sphinx-based documentation, including a mix of manually written pages and an automatically generated documentation of your plugin's python API.
 See the `developer guide of aiida-diff <https://aiida-diff.readthedocs.io/en/latest/developer_guide/index.html>`_ for instructions on how to build it.
 
 AiiDA provides a sphinx extension for inserting automatically generated documentations of ``Process`` classes (calculations and workflows) into your sphinx documentation (analogous to the information displayed by ``verdi plugin list``).
@@ -280,25 +280,25 @@ Here,
 Publishing a plugin package
 ===========================
 
-AiiDA plugin packages are published on the `AiiDA plugin registry <registry>`_ and the `python package index (PyPI) <pypi>`_.
+AiiDA plugin packages are published on the `AiiDA plugin registry <registry_>`_ and the `python package index (PyPI) <pypi_>`_.
 
 Before publishing your plugin, make sure your plugin comes with:
 
  * a ``setup.json`` file with the plugin metadata
  * a ``setup.py`` file for installing your plugin via ``pip``
  * a license
 
-For examples of these files, see the `aiida-diff demo plugin <aiida-diff>`_.
+For examples of these files, see the `aiida-diff demo plugin <aiida-diff_>`_.
 
 .. _how-to:plugins-develop:publish:plugin-registry:
 
 Publishing on the plugin registry
 ---------------------------------
 
-The `AiiDA plugin registry <registry>`_ aims to be the home for all publicly available AiiDA plugins.
+The `AiiDA plugin registry <registry_>`_ aims to be the home for all publicly available AiiDA plugins.
 It collects information on the type of plugins provided by your package, which AiiDA versions it is compatible with, etc.
 
-In order to register your plugin package, simply go to the `plugin registry <registry>`_ and follow the instructions in the README.
+In order to register your plugin package, simply go to the `plugin registry <registry_>`_ and follow the instructions in the README.
 
 .. note::
 
@@ -310,7 +310,7 @@ We encourage you to **get your plugin package listed as soon as possible**, both
 Publishing on PyPI
 ------------------
 
-For distributing AiiDA plugin packages, we recommend to follow the `guidelines for packaging python projects <packaging>`_, which include making the plugin available on the `python package index <PyPI>`_.
+For distributing AiiDA plugin packages, we recommend to follow the `guidelines for packaging python projects <packaging_>`_, which include making the plugin available on the `python package index <PyPI_>`_.
 This makes it possible for users to simply ``pip install aiida-myplugin``.
 
 .. note::
@@ -323,4 +323,4 @@ This makes it possible for users to simply ``pip install aiida-myplugin``.
 .. _ghactions: https://github.com/features/actions
 .. _registry: https://github.com/aiidateam/aiida-registry
 .. _pypi: https://pypi.python.org
-.. _packaging: https://packaging.python.org/distributing/#configuring-your-project
+.. _packaging: https://packaging.python.org/tutorials/packaging-projects/

docs/source/howto/share_data.rst

@@ -215,7 +215,7 @@ Deploying a REST API server
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The ``verdi restapi`` command runs the REST API through the ``werkzeug`` python-based HTTP server.
-In order to deploy production instances of the REST API for serving your data to others, we recommend using a fully fledged web server, such as `Apache <https://httpd.apache.org/>`_ or `NGINX <https://www.nginx.com/>`_, which then runs the REST API python application through the `web server gateway interface (WSGI) <wsgi.readthedocs.io/>`_.
+In order to deploy production instances of the REST API for serving your data to others, we recommend using a fully fledged web server, such as `Apache <https://httpd.apache.org/>`_ or `NGINX <https://www.nginx.com/>`_, which then runs the REST API python application through the `web server gateway interface (WSGI) <https://wsgi.readthedocs.io/>`_.
 
 .. note::
     One Apache/NGINX server can host multiple instances of the REST APIs, e.g. serving data from different AiiDA profiles.
@@ -228,7 +228,7 @@ A ``myprofile-rest.wsgi`` script for an AiiDA profile ``myprofile`` would look l
 
 In the following, we explain how to run this wsgi application using Apache on Ubuntu.
 
-    #. Install and enable the ``mod_wsgi`` `WSGI module <modwsgi.readthedocs.io/>`_ module:
+    #. Install and enable the ``mod_wsgi`` `WSGI module <https://modwsgi.readthedocs.io/>`_ module:
 
     .. code-block:: console
 

docs/source/intro/about.rst

@@ -15,6 +15,6 @@ AiiDA is built to support and streamline the four core pillars of the ADES model
 *  **Plugin interface:** AiiDA can support via plugins any computational code and data analytics tool, data type, scheduler, connection mode, etc. (see `public plugin repository <https://aiidateam.github.io/aiida-registry/>`__).
 *  **HPC interface:** AiiDA can seamlessly deal with heterogeneous and remote computing resources; it works with many schedulers out of the box (`SLURM <https://slurm.schedmd.com>`__, `PBS Pro <https://www.pbspro.org/>`__, `torque <http://www.adaptivecomputing.com/products/torque/>`__, `SGE <http://gridscheduler.sourceforge.net/>`__ or `LSF <https://www.ibm.com/support/knowledgecenter/SSETD4/product_welcome_platform_lsf.html>`__).
 *  **Open science:** AiiDA allows to export both full databases and selected subsets, to be shared with collaborators or made available and browsable online on the `Archive <https://archive.materialscloud.org/>`__ and `Explore <https://www.materialscloud.org/explore>`__ sections of `Materials Cloud <https://www.materialscloud.org>`__.
-*  **Open source:** AiiDA is released under the `MIT open-source license <LICENSE.txt>`__.
+*  **Open source:** AiiDA is released under the `MIT open-source license <https://github.com/aiidateam/aiida-core/blob/develop/LICENSE.txt>`__.
 
 See also the `list of AiiDA-powered scientific publications <http://www.aiida.net/science/>`__ and `testimonials from AiiDA users <http://www.aiida.net/testimonials/>`__.

docs/source/topics/plugins.rst

@@ -26,9 +26,9 @@ An AiiDA plugin should not:
 * Use protected functions, methods or classes of AiiDA (those starting with an underscore ``_``)
 * Monkey patch anything within the ``aiida`` namespace (or the namespace itself)
 
-Failure to comply will likely prevent your plugin from being listed on the official `AiiDA plugin registry <registry>`_.
+Failure to comply will likely prevent your plugin from being listed on the official `AiiDA plugin registry <registry_>`_.
 
-If you find yourself in a situation where you feel like you need to do any of the above, please open an issue on the `AiiDA repository <core>`_ and we can try to advise on how to proceed.
+If you find yourself in a situation where you feel like you need to do any of the above, please open an issue on the `AiiDA repository <core_>`_ and we can try to advise on how to proceed.
 
 
 .. _core: https://github.com/aiidateam/aiida-core
@@ -399,4 +399,4 @@ In particular:
 .. _unittest: https://docs.python.org/library/unittest.html
 .. _fixture: https://docs.pytest.org/en/latest/fixture.html
 .. _click: https://click.palletsprojects.com/
-.. _Entry points: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins
+.. _Entry points: https://packaging.python.org/guides/creating-and-discovering-plugins/