Merge pull request #521 from guerinclement/feature/code_quality

Feature/code quality
MAIF · Jan 29, 2024 · d4649b2 · d4649b2
2 parents f96b407 + cc6ab5c
commit d4649b2
Show file tree

Hide file tree

Showing 107 changed files with 8,415 additions and 8,568 deletions.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -45,4 +45,4 @@ Please describe the tests that you ran to verify your changes. Provide instructi
 - [ ] My changes generate no new warnings
 - [ ] I have added tests that prove my fix is effective or that my feature works
 - [ ] New and existing unit tests pass locally with my changes
-- [ ] Any dependent changes have been merged and published in downstream modules
+- [ ] Any dependent changes have been merged and published in downstream modules
diff --git a/.gitignore b/.gitignore
@@ -104,7 +104,7 @@ ENV/
 # Rope project settings
 .ropeproject
 
-# DS Store 
+# DS Store
 .DS_Store
 
 # for installation purposes
@@ -114,4 +114,4 @@ install-dev
 /catboost_info/
 
 # Data
-shapash/data/telco_customer_churn.csv
+shapash/data/telco_customer_churn.csv
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -0,0 +1,70 @@
+default_language_version:
+    python: python3
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+    -   id: check-ast
+    -   id: check-byte-order-marker
+    -   id: check-case-conflict
+    -   id: check-docstring-first
+    -   id: check-executables-have-shebangs
+    -   id: check-json
+    -   id: check-yaml
+        exclude: ^chart/
+    -   id: debug-statements
+    -   id: end-of-file-fixer
+        exclude: ^(docs/|gdocs/)
+    -   id: pretty-format-json
+        args: ['--autofix']
+        exclude: .ipynb
+    -   id: trailing-whitespace
+        args: ['--markdown-linebreak-ext=md']
+        exclude: ^(docs/|gdocs/)
+    -   id: mixed-line-ending
+        args: ['--fix=lf']
+        exclude: ^(docs/|gdocs/)
+    -   id: check-added-large-files
+        args: ['--maxkb=500']
+    - id: no-commit-to-branch
+      args: ['--branch', 'master', '--branch', 'develop']
+-   repo: https://github.com/psf/black
+    rev: 21.12b0
+    hooks:
+    -   id: black
+        args: [--line-length=120]
+        additional_dependencies: ['click==8.0.4']
+#-   repo: https://github.com/pre-commit/mirrors-mypy
+#    rev: 'v0.931'
+#    hooks:
+#    -   id: mypy
+#        args: [--ignore-missing-imports, --disallow-untyped-defs, --show-error-codes, --no-site-packages]
+#        files: src
+# -   repo: https://github.com/PyCQA/flake8
+#     rev: 6.0.0
+#     hooks:
+#     -   id: flake8
+#         exclude: ^tests/
+#         args: ['--ignore=E501,D2,D3,D4,D104,D100,D106,D107,W503,D105,E203']
+#         additional_dependencies: [ flake8-docstrings, "flake8-bugbear==22.8.23" ]
+-   repo: https://github.com/pre-commit/mirrors-isort
+    rev: v5.4.2
+    hooks:
+    -   id: isort
+        args: ["--profile", "black", "-l", "120"]
+-   repo: https://github.com/asottile/pyupgrade
+    rev: v2.7.2
+    hooks:
+    -   id: pyupgrade
+        args: [--py37-plus]
+-   repo: https://github.com/asottile/blacken-docs
+    rev: v1.8.0
+    hooks:
+    -   id: blacken-docs
+        additional_dependencies: [black==21.12b0]
+-   repo: https://github.com/compilerla/conventional-pre-commit
+    rev: v2.1.1
+    hooks:
+    -   id: conventional-pre-commit
+        stages: [commit-msg]
+        args: [] # optional: list of Conventional Commits types to allow e.g. [feat, fix, ci, chore, test]
diff --git a/.readthedocs.yml b/.readthedocs.yml
@@ -21,7 +21,7 @@ build:
   os: ubuntu-20.04
   tools:
     python: "3.10"
-    
+
 # Optionally set the version of Python and requirements required to build your docs
 python:
    install:

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -58,7 +58,7 @@ To contribute to Shapash, you will need to create a personal branch.
 ```
 git checkout -b feature/my-contribution-branch
 ```
-We recommand to use a convention of naming branch. 
+We recommand to use a convention of naming branch.
 - **feature/your_feature_name** if you are creating a feature
 - **hotfix/your_bug_fix** if you are fixing a bug
 
@@ -70,7 +70,7 @@ Before committing your modifications, we have some recommendations :
 ```
 pytest
 ```
-- Try to build Shapash 
+- Try to build Shapash
 ```
 python setup.py bdist_wheel
 ```
@@ -91,7 +91,7 @@ git commit -m ‘fixed a bug’
 git push origin feature/my-contribution-branch
 ```
 
-Your branch is now available on your remote forked repository, with your changes. 
+Your branch is now available on your remote forked repository, with your changes.
 
 Next step is now to create a Pull Request so the Shapash Team can add your changes to the official repository.
 
@@ -104,7 +104,7 @@ To create one, on the top of your forked repository, you will find a button "Com
 
 <img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/assets/images/contributing/shapash-compare-pr.png" alt="pull request" />
 
-As you can see, you can select on the right side which branch of your forked repository you want to associate to the pull request. 
+As you can see, you can select on the right side which branch of your forked repository you want to associate to the pull request.
 
 On the left side, you will find the official Shapash repository.
 
@@ -130,4 +130,4 @@ Your pull request is now ready to be submitted. A member of the Shapash team wil
 
 You have contributed to an Open source project, thank you and congratulations ! 🥳
 
-Show your contribution to Shapash in your curriculum, and share it on your social media. Be proud of yourself, you gave some code lines to the entire world !
+Show your contribution to Shapash in your curriculum, and share it on your social media. Be proud of yourself, you gave some code lines to the entire world !
diff --git a/LICENSE b/LICENSE
@@ -174,4 +174,4 @@
       incurred by, or claims asserted against, such Contributor by reason
       of your accepting any such warranty or additional liability.
 
-   END OF TERMS AND CONDITIONS
+   END OF TERMS AND CONDITIONS
diff --git a/Makefile b/Makefile
@@ -84,4 +84,4 @@ dist: clean ## builds source and wheel package
 	ls -l dist
 
 install: clean ## install the package to the active Python's site-packages
-	python setup.py install
+	python setup.py install
diff --git a/README.md b/README.md
@@ -38,7 +38,7 @@ With Shapash, you can generate a **Webapp** that simplifies the comprehension of
 
 Additionally, Shapash contributes to data science auditing by **presenting valuable information** about any model and data **in a comprehensive report**.
 
-Shapash is suitable for Regression, Binary Classification, and Multiclass problems. It is **compatible with numerous models**, including Catboost, Xgboost, LightGBM, Sklearn Ensemble, Linear models, and SVM. For other models, solutions to integrate Shapash are available; more details can be found [here](#how_shapash_works).
+Shapash is suitable for Regression, Binary Classification and Multiclass problems. It is **compatible with numerous models**, including Catboost, Xgboost, LightGBM, Sklearn Ensemble, Linear models, and SVM. For other models, solutions to integrate Shapash are available; more details can be found [here](#how_shapash_works).
 
 > [!NOTE]
 > If you want to give us feedback : [Feedback form](https://framaforms.org/shapash-collecting-your-feedback-and-use-cases-1687456776)
@@ -72,9 +72,9 @@ Shapash is suitable for Regression, Binary Classification, and Multiclass proble
 | 2.2.x         |  Dataset Filter <br>                                                              | New tab in the webapp to filter data. And several improvements in the webapp: subtitles, labels, screen adjustments                   |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/webapp.png" width="50" title="webapp">](https://github.com/MAIF/shapash/blob/master/tutorial/tutorial01-Shapash-Overview-Launch-WebApp.ipynb)
 | 2.0.x         |  Refactoring Shapash <br>                                                                   | Refactoring attributes of compile methods and init. Refactoring implementation for new backends                   |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/modular.png" width="50" title="modular">](https://github.com/MAIF/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb)
 | 1.7.x         |  Variabilize Colors <br>                                                                   | Giving possibility to have your own colour palette for outputs adapted to your design                   |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/variabilize-colors.png" width="50" title="variabilize-colors">](https://github.com/MAIF/shapash/blob/master/tutorial/common/tuto-common02-colors.ipynb)
-| 1.6.x         |  Explainability Quality Metrics <br> [Article](https://towardsdatascience.com/building-confidence-on-explainability-methods-66b9ee575514)                                                                   | To help increase confidence in explainability methods, you can evaluate the relevance of your explainability using 3 metrics: **Stability**, **Consistency** and **Compacity**                   |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/quality-metrics.png" width="50" title="quality-metrics">](https://github.com/MAIF/shapash/blob/master/tutorial/explainability_quality/tuto-quality01-Builing-confidence-explainability.ipynb) 
-| 1.4.x         |  Groups of features <br> [Demo](https://shapash-demo2.ossbymaif.fr/)                  | You can now regroup features that share common properties together. <br>This option can be useful if your model has a lot of features. |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/groups_features.gif" width="120" title="groups-features">](https://github.com/MAIF/shapash/blob/master/tutorial/common/tuto-common01-groups_of_features.ipynb)    | 
-| 1.3.x         |  Shapash Report <br> [Demo](https://shapash.readthedocs.io/en/latest/report.html)     | A standalone HTML report that constitutes a basis of an audit document.                                                                |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/report-icon.png" width="50" title="shapash-report">](https://github.com/MAIF/shapash/blob/master/tutorial/generate_report/tuto-shapash-report01.ipynb)    | 
+| 1.6.x         |  Explainability Quality Metrics <br> [Article](https://towardsdatascience.com/building-confidence-on-explainability-methods-66b9ee575514)                                                                   | To help increase confidence in explainability methods, you can evaluate the relevance of your explainability using 3 metrics: **Stability**, **Consistency** and **Compacity**                   |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/quality-metrics.png" width="50" title="quality-metrics">](https://github.com/MAIF/shapash/blob/master/tutorial/explainability_quality/tuto-quality01-Builing-confidence-explainability.ipynb)
+| 1.4.x         |  Groups of features <br> [Demo](https://shapash-demo2.ossbymaif.fr/)                  | You can now regroup features that share common properties together. <br>This option can be useful if your model has a lot of features. |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/groups_features.gif" width="120" title="groups-features">](https://github.com/MAIF/shapash/blob/master/tutorial/common/tuto-common01-groups_of_features.ipynb)    |
+| 1.3.x         |  Shapash Report <br> [Demo](https://shapash.readthedocs.io/en/latest/report.html)     | A standalone HTML report that constitutes a basis of an audit document.                                                                |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/report-icon.png" width="50" title="shapash-report">](https://github.com/MAIF/shapash/blob/master/tutorial/generate_report/tuto-shapash-report01.ipynb)    |
 
 ## 🔥 Features
 
@@ -83,19 +83,19 @@ Shapash is suitable for Regression, Binary Classification, and Multiclass proble
 <p align="center">
   <img align="left" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-02.png?raw=true" width="28%"/>
   <img src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-06.png?raw=true" width="28%" />
-  <img align="right" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-04.png?raw=true" width="28%" /> 
+  <img align="right" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-04.png?raw=true" width="28%" />
 </p>
 
 <p align="center">
   <img align="left" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-01.png?raw=true" width="28%" />
   <img src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-resize.png?raw=true" width="18%" />
-  <img align="right" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-13.png?raw=true" width="28%" /> 
+  <img align="right" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-13.png?raw=true" width="28%" />
 </p>
 
 <p align="center">
   <img align="left" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-12.png?raw=true" width="33%" />
   <img src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-03.png?raw=true" width="28%" />
-  <img align="right" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-10.png?raw=true" width="25%" /> 
+  <img align="right" src="https://github.com/MAIF/shapash/blob/master/docs/_static/shapash-grid-images-10.png?raw=true" width="25%" />
 </p>
 
 
@@ -145,13 +145,13 @@ Shapash can use category-encoders object, sklearn ColumnTransformer or simply fe
 
 Shapash is intended to work with Python versions 3.8 to 3.11. Installation can be done with pip:
 
-```python
+```bash
 pip install shapash
 ```
 
 In order to generate the Shapash Report some extra requirements are needed.
 You can install these using the following command :  
-```python
+```bash
 pip install shapash[report]
 ```
 
@@ -167,24 +167,25 @@ The 4 steps to display results:
 
 ```python
 from shapash import SmartExplainer
+
 xpl = SmartExplainer(
-  model=regressor,
-  features_dict=house_dict,  # Optional parameter
-  preprocessing=encoder, # Optional: compile step can use inverse_transform method
-  postprocessing=postprocess, # Optional: see tutorial postprocessing  
+    model=regressor,
+    features_dict=house_dict,  # Optional parameter
+    preprocessing=encoder,  # Optional: compile step can use inverse_transform method
+    postprocessing=postprocess,  # Optional: see tutorial postprocessing
 )
 ```
 
 - Step 2: Compile  Dataset, ...
   > There 1 mandatory parameter in compile method: Dataset
- 
+
 ```python
 xpl.compile(
-    x=xtest,    
-    y_pred=y_pred, # Optional: for your own prediction (by default: model.predict)
-    y_target=yTest, # Optional: allows to display True Values vs Predicted Values
-    additional_data=xadditional, # Optional: additional dataset of features for Webapp
-    additional_features_dict=features_dict_additional, # Optional: dict additional data    
+    x=xtest,
+    y_pred=y_pred,  # Optional: for your own prediction (by default: model.predict)
+    y_target=yTest,  # Optional: allows to display True Values vs Predicted Values
+    additional_data=xadditional,  # Optional: additional dataset of features for Webapp
+    additional_features_dict=features_dict_additional,  # Optional: dict additional data
 )
 ```  
 
@@ -193,7 +194,7 @@ xpl.compile(
 
 ```python
 app = xpl.run_app()
-``` 
+```
 
 [Live Demo Shapash-Monitor](https://shapash-demo.ossbymaif.fr/)
 
@@ -203,15 +204,15 @@ app = xpl.run_app()
 
 ```python
 xpl.generate_report(
-    output_file='path/to/output/report.html',
-    project_info_file='path/to/project_info.yml',
+    output_file="path/to/output/report.html",
+    project_info_file="path/to/project_info.yml",
     x_train=xtrain,
     y_train=ytrain,
     y_test=ytest,
     title_story="House prices report",
     title_description="""This document is a data science report of the kaggle house prices tutorial project.
         It was generated using the Shapash library.""",
-    metrics=[{'name': 'MSE', 'path': 'sklearn.metrics.mean_squared_error'}]
+    metrics=[{"name": "MSE", "path": "sklearn.metrics.mean_squared_error"}],
 )
 ```
 
@@ -220,9 +221,9 @@ xpl.generate_report(
 - Step 5: From training to deployment : SmartPredictor Object
   > Shapash provides a SmartPredictor object to deploy the summary of local explanation for the operational needs.
   It is an object dedicated to deployment, lighter than SmartExplainer with additional consistency checks.
-  SmartPredictor can be used with an API or in batch mode. It provides predictions, detailed or summarized local 
+  SmartPredictor can be used with an API or in batch mode. It provides predictions, detailed or summarized local
   explainability using appropriate wording.
-  
+
 ```python
 predictor = xpl.to_smartpredictor()
 ```