Streamline our source roadmap, specifications, and update our deploym…

…ent workflow

    update deploy documentation workflow dependencies and add comments explaining repo setting requirements

    update source roadmap to help newcomers navigate the project monorepo, fix bug in runhaz.sh

    rename deployment chapter to operations

    streamline design documentation

    add figma iframe

    remove reference chapter from documentation

.github/workflows/docs.yaml

@@ -1,5 +1,11 @@
-name: 'Deploy Documentation'
-# when changes are made to the documentation, build and deploy it to GitHub Pages
+name: 'Build and Deploy Project Documentation'
+
+# This Workflow builds a static site using mdBook and deploys it to GitHub Pages.
+
+# The GitHub repo settings must be configured for pages deployed from this workflow to persist after
+# another push to the repository. Go to Repository Settings > Pages -> Build and Deployment,
+# set the source to the `GitHub Actions` branch.
+# For more information on mdbook, see https://rust-lang.github.io/mdBook/
 
 on:
   push:
@@ -17,32 +23,36 @@ concurrency:
   group: 'pages'
   cancel-in-progress: true
 
+env:
+  BOOK_DIRECTORY: ./docs/guide
+  MDBOOK_VERSION: v0.4.37
+
 jobs:
   build:
     name: 'Build Documentation'
     runs-on: ubuntu-latest
     defaults:
       run:
-        working-directory: ./docs/guide
+        working-directory: ${{ env.BOOK_DIRECTORY }}
     steps:
-      - name: 'Checkout Repo'
-        uses: actions/checkout@v2
+      - name: Checkout
+        uses: actions/checkout@v4
 
-      - name: 'Setup mdBook'
+      - name: Setup mdBook
         run: |
           mkdir bin
-          curl -sSL https://github.com/rust-lang/mdBook/releases/download/v0.4.25/mdbook-v0.4.25-x86_64-unknown-linux-gnu.tar.gz | tar -xz --directory=bin
+          curl -sSL https://github.com/rust-lang/mdBook/releases/download/${{env.MDBOOK_VERSION}}/mdbook-${{env.MDBOOK_VERSION}}-x86_64-unknown-linux-gnu.tar.gz | tar -xz --directory=bin
 
-      - name: 'Build Documentation'
+      - name: Build Documentation
         run: bin/mdbook build
 
-      - name: 'Upload Docs'
-        uses: actions/upload-pages-artifact@v1
+      - name: Upload Docs
+        uses: actions/upload-pages-artifact@v3
         with:
-          path: ./docs/guide/book
+          path: '${{ env.BOOK_DIRECTORY }}/book'
 
   deploy_pages:
-    name: 'Deploy Documentation'
+    name: Deploy Documentation
     needs: build
     environment:
       name: github-pages
@@ -51,4 +61,4 @@ jobs:
     steps:
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v1
+        uses: actions/deploy-pages@v4

docs/guide/src/SUMMARY.md

@@ -2,22 +2,19 @@
 
 - [Haztrak Introduction](introduction.md)
   - [About e-Manifest](e-Manifest.md)
-- [Project Design](design/index.md)
+- [Project Design](design/source-roadmap.md)
+  - [Repo Roadmap](design/source-roadmap.md)
   - [Source Design Document](design/sdd.md)
-    - [Http Server Design](design/http-server.md)
+    - [Http Server Design](design/server.md)
     - [Client Design](design/browser-client.md)
     - [Database Design](design/db-design.md)
     - [Async Task Design](design/task-queue.md)
-  - [Source Navigation](design/source-roadmap.md)
   - [Testing](design/testing.md)
 - [Development](development/local-development.md)
   - [Local Development](development/local-development.md)
   - [Continuous Integration (CI)](development/ci-cd.md)
   - [Continuous Delivery (CD)](development/ci-cd.md)
-- [Deployment](deployment/index.md)
-  - [Configuration](deployment/configuration.md)
-  - [Containerization](deployment/containerization.md)
-  - [Deploying](deployment/deployment.md)
-- [References](references/index.md)
-  - [Demos](references/demos.md)
-  - [Notes for Future Maintainers](references/future-maintainers.md)
+- [Operations](operations/configuration.md)
+  - [Configuration](operations/configuration.md)
+  - [Containerization](operations/containerization.md)
+  - [Deploying](operations/deployment.md)

docs/guide/src/design/db-design.md

@@ -11,28 +11,12 @@ the means to, for example, draft or update electronic manifests without submitti
 the changes to RCRAInfo immediately.
 
 The database schema is maintained in version control via a series of 'migration'
-scripts. This enables us to initiate a new database and scaffold the expected
+scripts. This also enables haztrak to initiate a new database and scaffold the expected
 schema quickly and consistently for local development, testing, and backup.
 
 The Haztrak project currently utilizes [PostgreSQL](https://www.postgresql.org/),
 a widely used open-source object-relational database system known for reliability and performance.
 
-## Django ORM
-
-Haztrak uses the Django Framework's ORM (Object-Relational Mapping), a powerful tool
-that allows Haztrak to define the schema for, and interact with, the databases using Python
-objects instead of writing raw SQL queries.
-See the [Django model layer documentation](https://docs.djangoproject.com/en/4.1/topics/db/models/) for a more comprehensive introduction.
-
-## DB Migrations
-
-In addition to defining the database schema through Django's ORM, Django also comes with built-in
-support for managing database changes through [migration scripts](https://cloud.google.com/architecture/database-migration-concepts-principles-part-1). These migrations are checked into
-source control, which can then be applied to a standing database or used to spin up a new database
-as the current database schema is simply the sum of its changes.
-See the [Django Migrations documentation](https://docs.djangoproject.com/en/4.1/topics/migrations/)
-for more information.
-
 ## Notes on generating the ERD
 
 We use the `pygraphviz` python library and the `django-extensions` reusable Django app to generate the above entity relationship diagram (ERD).
@@ -41,24 +25,18 @@ We use the `pygraphviz` python library and the `django-extensions` reusable Djan
 
 This library doesn't come with pre-compiled [wheels](https://realpython.com/python-wheels/), so you'll need to install the C header files.
 
-#### Fedora/RHEL
-
-On Fedora (37) the following packages can be installed to meet this requirement.
+Fedora/RHEL
 
 ```shell
 dnf install python3-devel graphviz-devel
 ```
 
-#### Ubuntu/Debian
-
-The following packages should accomplish the same task, however this is untested.
+Ubuntu/Debian
 
 ```shell
 apt-get install libpython3-dev libgraphviz-dev
 ```
 
-### Python Dependencies
-
 after installing the necessary header files, install the `pygraphviz` package from PyPI.
 
 ```shell

docs/guide/src/design/sdd.md

@@ -3,7 +3,7 @@
 This document's purpose is to outline Haztrak's system architecture and design. For more information on the project's specifications and requirements, see the [Software Requirements Specification](./srs.md).
 
 - [Architecture Overview](#architecture-overview)
-  - [Server](./http-server.md)
+  - [Server](./server.md)
   - [Database](./db-design.md)
   - [Task Queue](./task-queue.md)
   - [Task Scheduler](./task-queue.md#periodic-tasks)
@@ -14,49 +14,14 @@ This document's purpose is to outline Haztrak's system architecture and design.
 
 ## Architecture Overview
 
-As a reference implementation, Haztrak follows a service oriented design however it would not be classified as a microservice architecture. The project is partitioned into a select number of containerized components that are deployed separately but closely work together, including:
+Haztrak follows is consist of a number of containerized components. The components that are deployed separately but closely work together. The overall system would be classified as a monolithic architecture
+however, certain components could be scaled independently (e.g., the number servers instances is separate from the front end), and newer versions of components could be deployed without affecting the entire system, given they do not introduce breaking changes. Nevertheless, the system is not designed to be categorized as microservices.
 
-1. An [HTTP server](./http-server.md) that provides a RESTful API for the client and admin site.
-2. A [relational database](./db-design.md) for persisting user data and data synced with RCRAInfo.
-3. An in memory database (caching layer)
+1. A [Server](./http-server.md) that exposes a RESTful API and a serv side rendered admin UI.
+2. A [relational database](./db-design.md).
+3. An in memory datastore
 4. A [task queue](./task-queue.md)
 5. A [task scheduler](./task-queue.md#periodic-tasks)
-6. A [user interface for the browser](./browser-client.md)
+6. A client side rendered [user interface](./browser-client.md)
 
-## Admin Site
-
-The Admin site provides a quick, model-centric interface where trusted
-users can manage content. It's not intended to provide a process centric interface,
-admin user's should not be, for example, signing manifests through the admin site.
-
-The admin interface is an out-of-the-box feature of the [Django framework](https://docs.djangoproject.com/en/4.1/ref/contrib/admin/).
-It can be found by appending `/admin` to the URL of the host and port of HTTP server, for example `http://localhost:8000/admin`
-
-## In-memory Database
-
-The in-memory data store serves a couple purposes,
-
-- As a message broker for Haztrak's [task queue](./task-queue.md)
-- A cache for the [http server](./http-server.md)
-
-As a cache, the in-memory data store is utilized to increase performance by allowing Haztrak to cut down on latency for recently used resources including recent database queries, and computed values. As a message broker, the data store provides a reliable way for the back end service to communicate which each other (e.g., launch background tasks).
-
-The Haztrak project currently uses [Redis](https://redis.io/) as both the message broker and in-memory data store.
-
-## Versioning
-
-Haztrak uses [semantic versioning](https://semver.org/) to keep track
-of its software releases. Semantic versioning is a widely used versioning system that allows
-developers to convey the nature of the changes in the software using
-a version number.
-
-The Haztrak project stores versions are in
-[Git tags](https://git-scm.com/book/en/v2/Git-Basics-Tagging). When a new
-version of the software is released, a new Git tag is created to represent that version.
-these tags are then used for the container images that are built and
-released for that version. Since the Git tag and image tag correspond,
-the source for a given container tag can always be easily found. Containers built
-from non-release commits should use
-
-Haztrak is stored in a monorepo, the front-end and back-end
-containers are built and released simultaneously with the same version number.
+<iframe style="border: 1px solid rgba(0, 0, 0, 0.1);" width="800" height="450" src="https://www.figma.com/embed?embed_host=share&url=https%3A%2F%2Fwww.figma.com%2Ffile%2FFZmRRNHv7jvefDDY39KlKY%2FHaztrak%3Ftype%3Dwhiteboard%26node-id%3D402%253A818%26t%3D5Oo3kgbSqnCDRqF4-1" allowfullscreen></iframe>

docs/guide/src/design/server.md

@@ -0,0 +1,40 @@
+## Server and Rest API
+
+The REST API is the primary way users read and change data in the Haztrak system. It is implemented using Python, the [Django Framework](https://www.djangoproject.com/), and [Django Rest Framework](https://www.django-rest-framework.org/).
+
+The choice to use Python stemmed from a desire to use a language that the team was already familiar with,
+something that is easy to learn (regardless of whether Python is your primary language, it's easy to read).
+The choice to use Django was made because it is a mature, well-documented, and widely used framework that provides a lot of functionality out of the box.
+
+Many Django developer have turned to what they call the 'service layer', which
+usually encapsulated the use cases business logic. It's a contentious topic whether the policy/business logic should be
+here or somewhere else, those in favor typically cite the principles discussed in Uncle Bob's classic book
+[CLean Architecture](https://www.goodreads.com/en/book/show/18043011-clean-architecture).
+
+### Useful Links for Django architecture, the service layer, and more
+
+[DjangoCon 2021 talk by Paul Wolf](https://www.youtube.com/watch?v=l5AtMQbAsAk&t=75s).
+[Hack Soft Django Styleguide repo](https://github.com/HackSoftware/Django-Styleguide).
+[Martin Fowler article on BoundedContext](https://www.martinfowler.com/bliki/BoundedContext.html).
+[Article on service layer by Martin Fowler](https://martinfowler.com/eaaCatalog/serviceLayer.html).
+[Against Service Layers in Django (reddit.com)](https://www.reddit.com/r/django/comments/fjqvwc/against_service_layers_in_django/?sort=top).
+
+## Admin Site
+
+The Admin site provides a quick, model-centric interface where trusted
+users can manage content. It's not intended to provide a process centric interface,
+admin user's should not be, for example, signing manifests through the admin site.
+
+The admin interface is an out-of-the-box feature of the [Django framework](https://docs.djangoproject.com/en/4.1/ref/contrib/admin/).
+It can be found by appending `/admin` to the URL of the host and port of HTTP server, for example `http://localhost:8000/admin`
+
+## In-memory Database
+
+The in-memory data store serves a couple purposes,
+
+- As a message broker for Haztrak's [task queue](./task-queue.md)
+- A cache for the [http server](./http-server.md)
+
+As a cache, the in-memory data store is utilized to increase performance by allowing Haztrak to cut down on latency for recently used resources including recent database queries, computed values, and external service calls. As a message broker, the data store provides a reliable way for the back end service to communicate which each other (e.g., launch background tasks).
+
+The Haztrak project currently uses [Redis](https://redis.io/) as both the message broker and in-memory data store.