Merge pull request #18 from sassoftware/validation

New regular task frequency table, and task title and content enhancements
sassoftware · Dec 19, 2024 · 7dda30b · 7dda30b
2 parents e585aa7 + a2863b4
commit 7dda30b
Show file tree

Hide file tree

Showing 9 changed files with 278 additions and 18 deletions.
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,3 @@
+scripts/__pycache__/*
+scripts/__pycache__/build_from_template.cpython-38.pyc
+scripts/__pycache__/checklistsharedfunctions.cpython-38.pyc
diff --git a/checklist.md b/checklist.md
@@ -4,7 +4,7 @@
 
 > Please read the **[Project README](./README.md)** if you have not done so already.
 
-This checklist lists both [initial](#initial-task-checklist) tasks, and [regular](#regular-task-checklist) tasks.
+This checklist lists both [initial](#initial-task-checklist) tasks, and [regular](#regular-task-checklist) tasks. The regular tasks are then also presented in a [task schedule](#regular-task-schedule), indicating at a glance a suggested frequency at which each regular task might be performed.
 
 Alternatively, view the same [tasks grouped by topic](./tasks_by_topic.md).
 
@@ -31,7 +31,7 @@ regular scheduled basis.
 | 0020 | [Install the SAS Viya CLI](tasks/install_sas_viya_cli.md) | Ensure you have installed the sas-viya cli and its plugins. | Kubernetes & IT Admin | - |
 | 0030 | [Develop an update plan](tasks/develop_update_plan.md) | Develop an update plan: outline tasks required before, during and after an update | Kubernetes & IT Admin | Yes |
 | 0040 | [Maintain a Secure Password Database](tasks/maintain_password_database.md) | Maintain a secure and encrypted password-protected password database using an appropriate software tool or service | Kubernetes & IT Admin | - |
-| 0050 | [Use customer-provided certificates](tasks/encryption_with_customer_provided_certificates.md) | Generate new or renew expiring TLS certificates used for encryption of data in transit. | Kubernetes & IT Admin | - |
+| 0050 | [Update TLS certificates](tasks/encryption_with_public_ca_signed_tls_certificates.md) | Generate new or renew expiring TLS certificates used for encryption of data in transit. | Kubernetes & IT Admin | - |
 | 0060 | [Load POSIX attributes](tasks/load_posix_attributes.md) | Load POSIX attributes for identities when attributes are not returned from the authentication provider | Kubernetes & IT Admin | - |
 | 0070 | [Create and Configure User home-directories](tasks/createandconfigure_user_homedirs.md) | Create and configure user-home directories | Kubernetes & IT Admin | - |
 | 0080 | [Ensure You Have Provided Sufficient Storage for Path-Based Caslibs](tasks/ensure_storage_for_caslibs.md) | Ensure you have provided sufficient filesystem storage of an appropriate type for path-based caslibs. | Kubernetes & IT Admin | - |
@@ -75,7 +75,7 @@ regular scheduled basis.
 
 ## Regular Task Checklist
 
-This table lists smaller tasks that should be repeated on a regular basis. See the suggested frequency table for an example of when you might run each of these tasks.
+This table lists smaller tasks that should be repeated on a regular basis. See the [Regular Task Schedule](#regular-task-schedule) which follows this table for an example of when you might run each of these tasks.
 
 > *23 tasks tagged as Regular;Done*
 
@@ -105,4 +105,36 @@ This table lists smaller tasks that should be repeated on a regular basis. See t
 | 0670 | [Manage content stored in PostgreSQL](tasks/manage_postgresql_content.md) | Manage content stored in PostgresQL | Monthly | PostgreSQL | - |
 | 0680 | [Maintain SAS Infrastructure Data Server](tasks/maintain_postgresql_server.md) | Perform routine maintenance on the SAS Infrastructure Data Server | Monthly | PostgreSQL | - |
 
-</br>Generated by build_from_template.py on: 06 Sep 2023 17:04:07.</br>
+## Regular Task Schedule
+
+This table shows the same regular tasks as the [Regular Task Checklist](#regular-task-checklist) table above, but is focussed more on how often we suggest each regular task might be run.
+
+> *23 tasks tagged as Regular;Done*
+
+| # ▴ | Title | Annually | Quarterly | Monthly | Weekly | Daily | Other |
+|---|---|:-:|:-:|:-:|:-:|:-:|---|
+| 0460 | [Renew your SAS Viya License](tasks/update_licenses.md) | ◎ |  |  |  |  |  |
+| 0470 | [Update the SAS Viya CLI](tasks/update_sas_viya_cli.md) |  | ◎ |  |  |  |  |
+| 0480 | [Renew your OIDC client secret](tasks/renew_oidc_client_secret.md) |  |  |  |  |  | When secret changes |
+| 0490 | [Update External Credentials](tasks/update_external_credentials.md) |  |  |  |  |  | When credentials change |
+| 0500 | [Check the Status of SAS services](tasks/check_service_status.md) |  |  |  |  | ◎ |  |
+| 0510 | [Monitor Memory, CPU, Network, and Disk Throughput Usage](tasks/monitor_usage.md) |  |  |  |  | ◎ |  |
+| 0520 | [Monitor Log Messages](tasks/monitor_logs.md) |  |  |  |  | ◎ |  |
+| 0530 | [Examine the User Activity Report](tasks/examine_user_activity_report.md) |  |  |  | ◎ |  |  |
+| 0540 | [Change log levels](tasks/change_log_levels.md) |  |  |  |  |  | When troubleshooting |
+| 0550 | [Monitor Storage Space](tasks/monitor_storage_space.md) |  |  |  | ◎ |  |  |
+| 0560 | [Monitor Observability Storage](tasks/monitor_observabilty_storage.md) |  |  |  | ◎ |  |  |
+| 0570 | [Stop and Start SAS Viya's Monitoring and Logging components](tasks/stop_and_start_sas_viya_monitoring_for_kubernetes.md) |  |  |  |  |  | When not in use |
+| 0580 | [Onboard and Offboard Users](tasks/onboard_and_offboard_users.md) |  |  |  | ◎ |  |  |
+| 0590 | [Keep your Software Current](tasks/keep_software_current.md) |  |  | ◎ |  |  |  |
+| 0600 | [Regularly Re-Validate your SAS Viya Deployment](tasks/validate_deployment_regularly.md) |  |  |  | ◎ |  |  |
+| 0610 | [Periodically Run an Inventory Scan on the Viya Environment](tasks/inventory_scan.md) |  |  | ◎ |  |  |  |
+| 0620 | [Stop and Start SAS Viya software](tasks/stop_and_start_viya.md) |  |  |  |  |  | When not in use |
+| 0630 | [Inspect the Status of Scheduled Jobs](tasks/inspect_job_status.md) |  |  |  |  | ◎ |  |
+| 0640 | [Test the Process to Restore From Backups](tasks/test_restore_process.md) |  |  | ◎ |  |  |  |
+| 0650 | [Configure CAS server startup to load data](tasks/cas_server_startup.md) |  |  | ◎ |  |  |  |
+| 0660 | [Monitor Compute Sessions](tasks/monitor_compute_sessions.md) |  |  |  |  |  | Daily or as often as necessary |
+| 0670 | [Manage content stored in PostgreSQL](tasks/manage_postgresql_content.md) |  |  | ◎ |  |  |  |
+| 0680 | [Maintain SAS Infrastructure Data Server](tasks/maintain_postgresql_server.md) |  |  | ◎ |  |  |  |
+
+</br>Generated by build_from_template.py on: 19 Dec 2024 13:58:00.</br>
diff --git a/scripts/build_from_template.py b/scripts/build_from_template.py
@@ -37,7 +37,7 @@
 import contextlib
 import os
 import datetime
-from checklistsharedfunctions import toc
+from checklistsharedfunctions import toc, rtfreq
 from distutils import util
 
 debug=True
@@ -138,7 +138,42 @@ def pushd(strNewDir):
                         # Call the toc function defined in checklistsharedfunctions to actually generate a TOC from the tasks files
                         # print('strTagFilter:'+strTagFilter+', intSortColumn:'+str(intSortColumn)+', boolReverse:'+str(boolReverse)+' and type of boolReverse is '+str(type(boolReverse)))
                         toc(fileOut, strProjectRootPath, strTaskSubdirectory, strTagFilter, intSortColumn, boolReverse, boolShowTags,boolShowFreq, boolShowTopic, boolShowEssential,strTopicFilter)
+                    elif line.startswith('@_RTFREQ_'):
+                        # Regular task frequency table
 
+                        # Default sort and filter
+                        strTagFilter="Regular"
+                        intSortColumn=0 # 0:SortString, 1:Title
+                        boolReverse=False
+                        boolShowEssential=False
+                        strTopicFilter=""
+
+                        # Parameters passed after the '@_RTFREQ_' can override the default sort and filter
+                        strRTFREQParameters=line[len('@_RTFREQ_ '):].rstrip()
+                        lstRTFREQParameters=strRTFREQParameters.split(',')
+                        # print('Found a RTFREQ placeholder with '+str(len(lstRTFREQParameters))+' parameters: '+strRTFREQParameters)
+                        for strRTFREQParameter in lstRTFREQParameters:
+                            lstRTFREQParameterNameAndValue=strRTFREQParameter.split('=')
+                            strParameterName=lstRTFREQParameterNameAndValue[0]
+                            anyParameterValue=lstRTFREQParameterNameAndValue[1]
+                            # print('Parameter name: '+strParameterName+', Value: '+anyParameterValue)
+                            # Validate parameter name and value, set sort and filter if valid
+                            if strParameterName == 'strTagFilter':
+                                strTagFilter=anyParameterValue # May be several tags separated by semicolons
+                                # print('strTagFilter: '+strTagFilter)
+                            elif strParameterName == 'intSortColumn' and anyParameterValue.isnumeric():
+                                intSortColumn=int(anyParameterValue)
+                            elif strParameterName == 'boolReverse' and (anyParameterValue=='True' or anyParameterValue=='False'):
+                                boolReverse=bool(util.strtobool(anyParameterValue))
+                            elif strParameterName == 'boolShowEssential' and (anyParameterValue=='True' or anyParameterValue=='False'):
+                                boolShowEssential=bool(util.strtobool(anyParameterValue))
+                            elif strParameterName == 'strTopicFilter':
+                                strTopicFilter=anyParameterValue # May be several tags separated by semicolons
+                                # print('strTopicFilter: '+strTopicFilter)
+
+                        # Call the rtfreq function defined in checklistsharedfunctions to actually generate a Regular Task frequency table from the tasks files
+                        # print('strTagFilter:'+strTagFilter+', intSortColumn:'+str(intSortColumn)+', boolReverse:'+str(boolReverse)+' and type of boolReverse is '+str(type(boolReverse)))
+                        rtfreq(fileOut,strProjectRootPath,strTaskSubdirectory,strTagFilter,intSortColumn,boolReverse,boolShowEssential,strTopicFilter)
                     elif line.startswith('@_#_'):
                         # This is a comment in the template - do nothing
                         pass