Merge pull request #278 from Living-with-machines/pre-commit

Add pre-commit

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,22 +1,22 @@
 ### Summary
 
-Describe the problem you're trying to fix in this pull request. 
+Describe the problem you're trying to fix in this pull request.
 Please reference any related issue and use fixes/close to automatically close them, if pertinent. For example: "Fixes #58", or "Addresses (but does not close) #238". -->
 
 Fixes #<NUM>
 
 ### Describe your changes
-  
+
 Describe changes/updates you have made to the code.
 
 ### Checklist before assigning a reviewer (update as needed)
-  
+
 - [ ] Self-review code
 - [ ] Ensure submission passes current tests
 - [ ] Add tests
-  
+
 ### Reviewer checklist
-  
+
 Please add anything you want reviewers to specifically focus/comment on.
 
 - [ ] Everything looks ok?

.github/workflows/mr_ci.yml

@@ -41,4 +41,4 @@ jobs:
 
       - name: Test with pytest
         run: |
-          python -m pytest ./tests
+          python -m pytest ./tests

.github/workflows/mr_pip_ci.yml

@@ -44,4 +44,4 @@ jobs:
 
       - name: Test with pytest
         run: |
-          python -m pytest ./tests
+          python -m pytest ./tests

.github/workflows/publish-to-conda-forge.yml

@@ -6,7 +6,7 @@ on:
   push:
     tags:
       - v*
-      
+
 jobs:
 
   build_conda:

.github/workflows/publish-to-pypi.yml

@@ -31,4 +31,4 @@ jobs:
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
         user: __token__
-        password: ${{ secrets.PYPI_API_TOKEN }}
+        password: ${{ secrets.PYPI_API_TOKEN }}

.pre-commit-config.yaml

@@ -0,0 +1,36 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: "v4.4.0"
+    hooks:
+      - id: check-added-large-files
+      - id: check-case-conflict
+      - id: check-merge-conflict
+      - id: check-symlinks
+      - id: check-yaml
+      - id: debug-statements
+      - id: end-of-file-fixer
+      - id: mixed-line-ending
+      - id: name-tests-test
+        args: ["--pytest-test-first"]
+      - id: requirements-txt-fixer
+      - id: trailing-whitespace
+
+  - repo: https://github.com/psf/black-pre-commit-mirror
+    rev: "23.9.1"
+    hooks:
+      - id: black-jupyter
+        language_version: python3.8
+
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: "v0.0.290"
+    hooks:
+      - id: ruff
+        args: ["--fix", "--show-fixes"]
+
+exclude: |
+  (?x)^(
+    conda/.*|
+    worked_examples/.*|
+    .*/__init__.py|
+    tests/test_import.py
+  )$

.readthedocs.yaml

@@ -29,4 +29,3 @@ python:
    - requirements: docs/requirements.txt
    - method: pip
      path: .
-

.ruff.toml

@@ -0,0 +1,20 @@
+select = [
+  "E", "F", "W", # flake8
+  "B",           # flake8-bugbear
+  "I",           # isort
+  "UP"           # pyupgrade
+#  "N",           # pep8 naming
+]
+ignore = [
+  "E501",   # Line too long
+  "E722",   # Do not use bare 'except'
+  "B904",   # Within an 'except' clause ...
+]
+unfixable = [
+  "F841", # Removes unused variables
+]
+flake8-unused-arguments.ignore-variadic-names = true
+isort.required-imports = ["from __future__ import annotations"]
+
+# Assume Python 3.8
+target-version = "py38"

Contributors.md

@@ -25,4 +25,3 @@ You can request contact information through the project members above, or tag th
 | Andy Smith<br>([@andrewphilipsmith](https://github.com/andrewphilipsmith)) | Research Data Scientist (Turing) | 2022 - 2023 |
 | Daniel van Strien<br>([@davanstrien ](https://github.com/davanstrien)) | Data Librarian (British Library) | 2019-2021 |
 | Olivia Vane<br>([@ov212 ](https://github.com/ov212)) | Research Software Engineer (British Library) | 2019-2021 |
-

docs/requirements.txt

@@ -1,8 +1,8 @@
-sphinx_rtd_theme
 myst-parser
-sphinx-autoapi
-sphinx-copybutton
 nbsphinx
 pandoc
+sphinx-autoapi
+sphinx-copybutton
+sphinx-togglebutton
+sphinx_rtd_theme
 sphinxcontrib-napoleon
-sphinx-togglebutton

docs/source/About.rst

@@ -36,13 +36,13 @@ MapReader becomes useful when the number of maps you wish to analyze exceeds the
 
 This exact number will vary depending on:
 
-- the size of your maps, 
+- the size of your maps,
 - the features you want to find,
 - the skills you (or your team) have,
 - the amount of time at your disposal.
 
-Deciding to use MapReader, which uses deep learning computer vision (CV) models to predict the class of content on patches across many sheets, means weighing the pros and cons of working with the data output that is inferred by the model. 
-Inferred data can be evaluated against expert-annotated data to understand its general quality (are all instances of a feature of interest identified by the model? does the model apply the correct label to that feature?), but in the full dataset there *will necessarily be* some percentage of error. 
+Deciding to use MapReader, which uses deep learning computer vision (CV) models to predict the class of content on patches across many sheets, means weighing the pros and cons of working with the data output that is inferred by the model.
+Inferred data can be evaluated against expert-annotated data to understand its general quality (are all instances of a feature of interest identified by the model? does the model apply the correct label to that feature?), but in the full dataset there *will necessarily be* some percentage of error.
 
 MapReader creates output that you can link and analyze in relation to other geospatial datasets (e.g. census, gazetteers, toponyms in text corpora).
 

docs/source/Beginners-info.rst

@@ -4,11 +4,11 @@ Coding Basics
 Using the terminal
 -------------------
 
-A terminal is a command-line interface where you can type in commands and interact with your computer's operating system. 
-It can be a powerful tool for programmers, allowing them to execute complex tasks with just a few keystrokes. 
-However, it can also be intimidating for beginners who are not familiar with using it. 
+A terminal is a command-line interface where you can type in commands and interact with your computer's operating system.
+It can be a powerful tool for programmers, allowing them to execute complex tasks with just a few keystrokes.
+However, it can also be intimidating for beginners who are not familiar with using it.
 
-Here are some resources to help you get started with using the terminal on your computer: 
+Here are some resources to help you get started with using the terminal on your computer:
 
 - `Introduction to the command line <https://curriculum.dhinstitutes.org/workshops/command-line/>`__
 - `Introduction to the bash command line  <https://programminghistorian.org/en/lessons/intro-to-bash>`__
@@ -26,11 +26,11 @@ i.e. not a program/app that you can click and find in your start page/ launch pa
 Conda and python virtual environments
 --------------------------------------
 
-A virtual environment is a way to create an isolated environment in which you can install specific versions of packages and dependencies for your Python projects. 
-This is useful because it allows you to avoid conflicts between different projects that may require different versions of the same package. 
-In the installation instructions provided, there are two methods for setting up a virtual environment for MapReader: using Anaconda (also known as conda) or using venv, which is Python's native way of handling virtual environments. 
+A virtual environment is a way to create an isolated environment in which you can install specific versions of packages and dependencies for your Python projects.
+This is useful because it allows you to avoid conflicts between different projects that may require different versions of the same package.
+In the installation instructions provided, there are two methods for setting up a virtual environment for MapReader: using Anaconda (also known as conda) or using venv, which is Python's native way of handling virtual environments.
 
-If you're new to virtual environments in Python, this tutorial provides a good introduction: 
+If you're new to virtual environments in Python, this tutorial provides a good introduction:
 
 - `Getting started with python environments (using conda) <https://towardsdatascience.com/getting-started-with-python-environments-using-conda-32e9f2779307>`__
 - `Why you need python environments and how to manage them with conda <https://www.freecodecamp.org/news/why-you-need-python-environments-and-how-to-manage-them-with-conda-85f155f4353c/>`__
@@ -40,10 +40,10 @@ If you're new to virtual environments in Python, this tutorial provides a good i
 Jupyter notebooks
 ------------------
 
-A Jupyter notebook is an interactive computational environment that allows you to write and run code, visualize data, and write narrative text all in the same place. 
-It's a popular tool among data scientists and is commonly used for data analysis, machine learning, and scientific computing. 
+A Jupyter notebook is an interactive computational environment that allows you to write and run code, visualize data, and write narrative text all in the same place.
+It's a popular tool among data scientists and is commonly used for data analysis, machine learning, and scientific computing.
 
-If you're new to Jupyter notebooks, here's a great place to start: 
+If you're new to Jupyter notebooks, here's a great place to start:
 
 - `Introduction to Jupyter Notebooks <https://programminghistorian.org/en/lessons/jupyter-notebooks>`__
 - `How to use Jupyter Notebooks: A beginner's tutorial <https://www.dataquest.io/blog/jupyter-notebook-tutorial/>`__

docs/source/Coc.rst

@@ -1,15 +1,15 @@
 Code of Conduct and Inclusivity
 ================================
 
-We are currently in the process of developing a Code of Conduct. 
+We are currently in the process of developing a Code of Conduct.
 In the meantime, we look to the `Code of Conduct <https://github.com/alan-turing-institute/the-turing-way/blob/main/CODE_OF_CONDUCT.md>`_ from The Turing Way as a model.
 
-We aim to be inclusive of people from all walks of life and all research fields. 
+We aim to be inclusive of people from all walks of life and all research fields.
 These intentions must be reflected in the contributions that we make.
 
-We therefore encourage intentional, inclusive actions from contributors to MapReader. 
+We therefore encourage intentional, inclusive actions from contributors to MapReader.
 Here are a few examples of such actions:
 
 - Use respectful, gender-neutral and inclusive language.
 - Aim to include perspectives of researchers from different research backgrounds such as science, humanities and social sciences by not limiting the scope to only scientific domains.
-- Make sure that color palettes used throughout figures are accessible to color-blind readers and contributors.
+- Make sure that color palettes used throughout figures are accessible to color-blind readers and contributors.

docs/source/Contribution-guide/Code.rst

@@ -1,7 +1,7 @@
 How to add to or update the MapReader code
 ===========================================
 
-MapReader's code is written in `Python <https://www.python.org/>`_. 
+MapReader's code is written in `Python <https://www.python.org/>`_.
 There are a number of ways in which you can contribute to our code, these include:
 
 - Reviewing part of the code.
@@ -12,20 +12,20 @@ There are a number of ways in which you can contribute to our code, these includ
 Before you begin
 -----------------
 
-If you already have an idea for changes to the MapReader code, please ensure your idea has a corresponding issue on the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ (or create one if needed). 
-If, on the other hand, you would like to contribute to the code but don't currently have an idea of what to work on, head to our `open issues <https://github.com/Living-with-machines/MapReader/issues>`_ tab and find something that interests you. 
+If you already have an idea for changes to the MapReader code, please ensure your idea has a corresponding issue on the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ (or create one if needed).
+If, on the other hand, you would like to contribute to the code but don't currently have an idea of what to work on, head to our `open issues <https://github.com/Living-with-machines/MapReader/issues>`_ tab and find something that interests you.
 
-In either case, before you begin working, either assign yourself to the issue or comment on it saying you will be working on making the required changes. 
+In either case, before you begin working, either assign yourself to the issue or comment on it saying you will be working on making the required changes.
 
-You will then need to fork the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ in order to make your changes. 
+You will then need to fork the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ in order to make your changes.
 
 Style guide
 -----------
 
-When making your changes, please: 
+When making your changes, please:
 
 - Try to align to the `PEP 8 style guide for python code <https://peps.python.org/pep-0008/>`.
-- Try to use the numpy-style docstrings (as per `this link <https://numpydoc.readthedocs.io/en/latest/format.html#>_`). 
+- Try to use the numpy-style docstrings (as per `this link <https://numpydoc.readthedocs.io/en/latest/format.html#>_`).
 - Ensure all docstrings are kept up to date and reflect any changes to code functionality you have made.
 - Add and run tests for your code.
 - If you add new dependencies, add these to our ``setup.py``.
@@ -39,7 +39,6 @@ When you are done making changes, please:
 When you are finished
 ----------------------
 
-Once you are happy with the changes you have made, please create a new `pull request <https://github.com/Living-with-machines/MapReader/pulls>`_ to let us know you'd like us to review your code. 
+Once you are happy with the changes you have made, please create a new `pull request <https://github.com/Living-with-machines/MapReader/pulls>`_ to let us know you'd like us to review your code.
 
 If possible, please link your pull request to any issue(s) your changes fix/address and write a thorough description of the changes you have made.
-

docs/source/Contribution-guide/Contribution-guide.rst

@@ -4,7 +4,7 @@ Contribution guide
 Welcome! We are pleased to know that you're interested in contributing to MapReader!
 
 MapReader is a collaborative project, now expanding its community beyond the initial group in the `Living with Machines <https://livingwithmachines.ac.uk/>`_ project (The Alan Turing Institute).
-We welcome all contributions but **please** follow these guidelines to make sure your contributions can be easily integrated into the project. 
+We welcome all contributions but **please** follow these guidelines to make sure your contributions can be easily integrated into the project.
 
 .. contents:: Table of Contents
     :local:
@@ -45,13 +45,13 @@ We welcome contributions from community members of all skill levels and so have
 
 ----------
 
-Guides 
+Guides
 ------
 
-.. toctree:: 
+.. toctree::
     :maxdepth: 1
 
     GitHub-guide
     Worked-examples
     Documentation
-    Code 
+    Code

docs/source/Contribution-guide/Documentation.rst

@@ -1,7 +1,7 @@
 How to add to or update the MapReader documentation
 ====================================================
 
-MapReader's documentation is generated using `Sphinx <https://www.sphinx-doc.org/en/master/index.html>`_ and hosted on `Read the docs <https://readthedocs.org/>`_. 
+MapReader's documentation is generated using `Sphinx <https://www.sphinx-doc.org/en/master/index.html>`_ and hosted on `Read the docs <https://readthedocs.org/>`_.
 There are a number of ways you can contribute to our documentation, these include:
 
 - Updating or adding clarity to existing documentation.
@@ -15,31 +15,31 @@ If you would like to edit or add to the MapReader documentation, you will need t
 
 To do this (assuming you have installed MapReader from source, as per our `Installation instructions <https://mapreader.readthedocs.io/en/latest/Install.html>`_), use:
 
-.. code-block:: bash 
+.. code-block:: bash
 
     conda activate mapreader
     pip install sphinx
     pip install -r MapReader/docs/requirements.txt
 
-.. note:: You may need to change the file path depending on where your MapReader directory is saved. 
+.. note:: You may need to change the file path depending on where your MapReader directory is saved.
 
 Writing in reStructuredText
 ---------------------------
 
 reStructuredText (rst) is the default plaintext markup language used by `Sphinx <https://www.sphinx-doc.org/en/master/index.html>`_ and is the primary language used throughout our documentation.
-If you have never used or written in rst, `this primer <https://docutils.sourceforge.io/rst.html>`_ is a great place to start. 
+If you have never used or written in rst, `this primer <https://docutils.sourceforge.io/rst.html>`_ is a great place to start.
 There are also numerous other rst 'cheatsheets' (e.g. `here <https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#rst-primer>`__ and `here <https://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html>`__) available online, so have a google.
 
-To help make your rst files easier to read and review, **please start each new sentence on a new line**. 
+To help make your rst files easier to read and review, **please start each new sentence on a new line**.
 This will make no difference to how the text is displayed, but will make it much easier to read when reviewing changes in a pull request.
 
 Before you begin
 ----------------
 
-Before you begin making your changes to the documentation, you should ensure there is a corresponding issue on the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ (or create one if needed). 
+Before you begin making your changes to the documentation, you should ensure there is a corresponding issue on the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ (or create one if needed).
 You should then either assign yourself to the issue or comment on it saying you will be working on making these changes.
 
-You will then need to fork the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ in order to make your changes. 
+You will then need to fork the `MapReader repository <https://github.com/Living-with-machines/MapReader>`_ in order to make your changes.
 
 Please also be aware that API documentation is auto-generated by the ``sphinx-autoapi`` extension.
 If you would like to change something in the API documentations, you will need to change the docstrings in the code itself.
@@ -52,11 +52,11 @@ Style guide
 
 .. ::
 
-    One 
+    One
     ===
     Two
     ---
-    Three 
+    Three
     ~~~~~
     Four
     ^^^^
@@ -66,7 +66,7 @@ Style guide
 - Use ``.. contents::`` to automatically generate a table of contents detailing sections within the current page. e.g.
 
 ::
-    
+
     .. contents:: Table of contents
         :depth: 1
 
@@ -103,7 +103,6 @@ Followed by specific example:
 When you are finished
 ----------------------
 
-Once you are happy with the changes you have made, please create a new `pull request <https://github.com/Living-with-machines/MapReader/pulls>`_ to let us know you'd like us to review your work. 
+Once you are happy with the changes you have made, please create a new `pull request <https://github.com/Living-with-machines/MapReader/pulls>`_ to let us know you'd like us to review your work.
 
 If possible, please link your pull request to any issue(s) your changes fix/address and write a thorough description of the changes you have made.
-