Merge pull request #247 from amosproj/improve-sphinx-doc

Improve Sphinx documentation

.gitignore

@@ -63,6 +63,8 @@ bin/
 !**/docs/make.bat
 !**/docs/Makefile
 !**/docs/readme_link.md
+!**/docs/documentation.rst
+!**/docs/_static
 
 # Env files
 *.env

Documentation/Build-Documentation.md

@@ -13,7 +13,7 @@ fill in all values with the corresponding secrets.
 To create the virtual environment in this project you must have `pipenv`
 installed on your machine. Then run the following commands:
 
-```[bash]
+```bash
 # for development environment
 pipenv install --dev
 # for production environment
@@ -22,7 +22,7 @@ pipenv install
 
 To work within the environment you can now run:
 
-```[bash]
+```bash
 # to activate the virtual environment
 pipenv shell
 # to run a single command
@@ -55,6 +55,6 @@ To run the application the `pipenv` environment must be installed and all needed
 environment variables must be set in the `.env` file. Then the application can
 be started via
 
-```[bash]
+```bash
 pipenv run python src/main.py
 ```

Documentation/Contribution.md

@@ -24,20 +24,20 @@ The stable branches `main` and `dev` are protected against direct pushes. To com
 
 Before contributing to this repository make sure that you are identifiable in your git user settings. This way commits and PRs created by you can be identified and easily traced back.
 
-```[bash]
+```bash
 git config --local user.name "Manu Musterperson"
 git config --local user.email "[email protected]"
 ```
 
 Any commit should always contain a commit message that references an issue created in the project. Also, always signoff on your commits for identification reasons.
 
-```[bash]
+```bash
 git commit -m "Fixed issue #123" --signoff
 ```
 
 When doing pair programming be sure to always have all SDs mentioned in the commit message. Each SD should be listed on a new line for clarity reasons.
 
-```[bash]
+```bash
 git commit -a -m "Fixed problem #123
 > Co-authored-by: Manu Musterperson <[email protected]>" --signoff
 ```
@@ -50,7 +50,7 @@ Here is a standard way to merge pull requests:
 
 1. Have all your local changes added, committed, and pushed on the remote **feature-1** branch
 
-   ```[bash]
+   ```bash
    git checkout feature-1
    git add .
    git commit -m "added a feature" --signoff  # don't forget the signoff ;)
@@ -59,7 +59,7 @@ Here is a standard way to merge pull requests:
 
 2. Make sure your local main branch up-to-date
 
-   ```[bash]
+   ```bash
    git checkout main
    git pull main
    ```
@@ -78,7 +78,7 @@ _**In case of merge conflict:**_
 Should we experience merge conflict after step 3, we should solve the merge conflicts manually, below the title of "This branch has conflicts that must be resolved" click on web editor (you can use vscode or any editor you want).
 The conflict should look like this:
 
-```[bash]
+```bash
 <<<<<<< HEAD
 // Your changes at **feature-1** branch
 =======

Documentation/Controller.md

@@ -21,12 +21,12 @@ Merchant Size Predictor is labelled as the (Estimated) Value Predictor here.
 
 ### Component Diagram
 
-![Component Diagram](Media/component-diagram-with-controller.svg)
+![Component Diagram](_static/component-diagram-with-controller.svg)
 
 ### Sequence Diagram
 
-![Sequence Diagram](Media/sequence-diagram.svg)
+![Sequence Diagram](_static/sequence-diagram.svg)
 
 ### Controller Workflow Diagram
 
-![Controller Workflow Diagram](Media/controller-workflow-diagram.jpg)
+![Controller Workflow Diagram](_static/controller-workflow-diagram.jpg)

Documentation/Design-Documentation.md

@@ -17,7 +17,7 @@ Merchant Size Predictor (MSP).
 
 ## Component Diagram
 
-![Component Diagram](Media/component-diagram.svg)
+![Component Diagram](_static/component-diagram.svg)
 
 ## External Software
 

Documentation/Google-Search-Strategy.md

@@ -32,4 +32,4 @@ we try to combine the search methods and derive the most probable result.
 4. Else: Return Phone-based search results
 5. Else: Return nothing
 
-![Search Strategy](./Media/google_places_strategy.drawio.png)
+![Search Strategy](./_static/google_places_strategy.drawio.png)

Documentation/Miscellaneous.md

@@ -47,7 +47,7 @@ This file contains content that was moved over from our [Wiki](https://github.co
 
 ## Pre-commit:
 
-```[bash]
+```bash
 # If not installed yet
 pip install pre-commit
 
@@ -147,7 +147,7 @@ AI models needed that solve a regression or probability problem
 
 #### install stuck
 
-```[bash]
+```bash
 pipenv install –dev
 ```
 
@@ -168,7 +168,7 @@ Terminal can't run docker image (on windows)
 don't analyze a certain part of the code with reuse
 **Solution**:
 
-```[bash]
+```bash
 # REUSE-IgnoreStart
   ...
 # REUSE-IgnoreEnd

Documentation/User-Documentation.md

@@ -25,7 +25,7 @@ To execute the final program, ensure the environment is installed (refer to
 build-documents.md) and run `python .\src\main.py` either locally or via the
 build process. The user will be presented with the following options:
 
-```[bash]
+```bash
 Choose demo:
 (0) : Base Data Collector
 (1) : Data preprocessing
@@ -41,7 +41,7 @@ Configuration options are presented:
 
 `Do you want to list all available pipeline configs? (y/N)` If `y`:
 
-```[bash]
+```bash
 Please enter the index of requested pipeline config:
 (0) : config_sprint09_release.json
 (1) : just_run_search_offeneregister.json
@@ -57,7 +57,7 @@ Please enter the index of requested pipeline config:
 If `n`: proceed to pipeline step selection for data enrichment. Subsequent
 questions arise:
 
-```[bash]
+```bash
 Run Scrape Address (will take a long time)(y/N)?
 Run Search OffeneRegister (will take a long time)(y/N)?
 Run Phone Number Validation (y/N)?
@@ -127,7 +127,7 @@ preprocessed_data is stored.
 
 Six machine learning models are available:
 
-```[bash]
+```bash
 (0) : Random Forest
 (1) : XGBoost
 (2) : Naive Bayes
@@ -147,7 +147,7 @@ prompted with a series of questions:
   instead of the 5 classes. It is worth noting that grouping the S, M and L
   classes alltogether as one class resulted in boosting the classification
   performance.
-- ```[bash]
+- ```bash
   Do you want to train on a subset of features?
   (0) : ['Include all features']
   (1) : ['google_places_rating', 'google_places_user_ratings_total', 'google_places_confidence', 'regional_atlas_regional_score']
@@ -159,7 +159,7 @@ learning models
 
 Then, the user would be given multiple options:
 
-```[bash]
+```bash
 (1) Train
 (2) Test
 (3) Predict on single lead

Documentation/data-fields.csv

@@ -6,8 +6,8 @@ Phone,string,Phone number of the lead,Lead data,-,49 1234 56789
 Email,string,Email of the lead,Lead data,-,[email protected]
 domain,string,"The domain of the email is the part that follows the ""@"" symbol, indicating the organization or service hosting the email address.",processing,Email,example.com
 email_valid,boolean,Checks if the email is valid.,email_validator package,Email,True/False
-first_name_in_account,boolean,Checks if first name is written in "Account" input,processing,First Name,True/False
-last_name_in_account,boolean,Checks if last name is written in "Account" input,processing,Last Name,True/False
+first_name_in_account,boolean,"Checks if first name is written in ""Account"" input",processing,First Name,True/False
+last_name_in_account,boolean,"Checks if last name is written in ""Account"" input",processing,Last Name,True/False
 number_formatted,string,Phone number (formatted),phonenumbers package,Phone,49123456789
 number_country,string,Country derived from phone number,phonenumbers package,Phone,Germany
 number_area,string,Area derived from phone number,phonenumbers package,Phone,Erlangen

README.md

@@ -20,7 +20,7 @@ The repository contains the file `.env.template`. This file is a template for th
 
 To create the virtual environment in this project you must have `pipenv` installed on your machine. Then run the following commands:
 
-```[bash]
+```bash
 # for development environment
 pipenv install --dev
 # for production environment
@@ -29,7 +29,7 @@ pipenv install
 
 To work within the environment you can now run:
 
-```[bash]
+```bash
 # to activate the virtual environment
 pipenv shell
 # to run a single command
@@ -38,7 +38,7 @@ pipenv run <COMMAND>
 
 To install new packages in the environment add them to the `Pipfile`. Always pin the exact package version to avoid package conflicts and unexpected side effects from package upgrades.
 
-```[bash]
+```bash
 # to add a package to the development environment
 [dev-packages]
 <PACKAGE_NAME> = "==<VERSION_NUMBER>"
@@ -57,27 +57,27 @@ This application is run using a Docker container. For this the `Dockerfile` at r
 
 To build the application run
 
-```[bash]
+```bash
 ./build_app.sh
 ```
 
 To run the application interactively run
 
-```[bash]
+```bash
 ./run_app.sh
 ```
 
 ## Database Connection
 
 To build the Docker containers
 
-```[bash]
+```bash
 docker-compose build
 ```
 
 To run the Docker containers
 
-```[bash]
+```bash
 docker-compose run sumup_app
 ```
 
@@ -87,7 +87,7 @@ This project is operated under an MIT license. Every file must contain the REUSE
 
 [REUSE documentation](https://reuse.software/faq/)
 
-```[bash]
+```bash
 # SPDX-License-Identifier: MIT
 # SPDX-FileCopyrightText: 2023
 ```
@@ -96,7 +96,7 @@ This project is operated under an MIT license. Every file must contain the REUSE
 
 This repository uses `pre-commit` hooks to ensure a consistent and clean file organization. Each registered hook will be executed when committing to the repository. To ensure that the hooks will be executed they need to be installed using the following command:
 
-```[bash]
+```bash
 pre-commit install
 ```
 

src/docs/_static

@@ -0,0 +1 @@
+../../Documentation/_static

src/docs/documentation.rst

@@ -0,0 +1,81 @@
+.. SPDX-License-Identifier: MIT
+.. SPDX-FileCopyrightText: 2024 Simon Zimmermann <[email protected]>
+
+Build Documentation
+===================
+.. include:: ../../Documentation/Build-Documentation.md
+   :parser: myst_parser.sphinx_
+
+User Documentation
+==================
+.. include:: ../../Documentation/User-Documentation.md
+   :parser: myst_parser.sphinx_
+
+Design Documentation
+====================
+.. include:: ../../Documentation/Design-Documentation.md
+   :parser: myst_parser.sphinx_
+
+Data Fields
+-----------
+.. include:: ../../Documentation/Data-Fields.md
+   :parser: myst_parser.sphinx_
+
+.. _\./data-fields\.csv:
+
+Data Fields CSV
+---------------------
+
+The following table highlights the data fields obtained for each lead.
+The acquisition of such data may derive from the Lead Form or may be extracted from external sources utilizing APIs.
+
+.. csv-table:: Data Field Definition
+   :file: ../../Documentation/data-fields.csv
+   :header-rows:  1
+   :widths: auto
+
+Google Search Strategy
+----------------------
+.. include:: ../../Documentation/Google-Search-Strategy.md
+   :parser: myst_parser.sphinx_
+
+OpenLLM Business Type Analysis
+------------------------------
+.. include:: ../../Documentation/OpenLLm-Business-Type-Analysis.md
+   :parser: myst_parser.sphinx_
+
+Classifier Comparison
+=====================
+.. include:: ../../Documentation/Classifier-Comparison.md
+   :parser: myst_parser.sphinx_
+
+Concepts, Unrealized Ideas & Miscellaneous
+==========================================
+
+.. include:: ../../Documentation/ideas.md
+   :parser: myst_parser.sphinx_
+
+Controller
+----------
+.. include:: ../../Documentation/Controller.md
+   :parser: myst_parser.sphinx_
+
+Twitter API Limitation
+----------------------
+.. include:: ../../Documentation/Twitter-API-Limitation.md
+   :parser: myst_parser.sphinx_
+
+Contribution
+------------
+.. include:: ../../Documentation/Contribution.md
+   :parser: myst_parser.sphinx_
+
+SBOM Generator
+--------------
+.. include:: ../../Documentation/SBOM_generator.md
+   :parser: myst_parser.sphinx_
+
+Miscellaneous
+-------------
+.. include:: ../../Documentation/Miscellaneous.md
+   :parser: myst_parser.sphinx_

src/docs/index.rst

@@ -14,6 +14,7 @@ Welcome to Sales Lead Qualifier's documentation!
    :caption: Contents:
 
    readme_link
+   documentation
    modules
 
 Indices and tables