From 05e87f5d1be97755d2cc25502d851367f0a7a02c Mon Sep 17 00:00:00 2001 From: "Shahid N. Shah" Date: Thu, 4 Jan 2024 13:48:01 -0500 Subject: [PATCH] docs: elaborate architecture options --- README.md | 24 +- .../docs/cloud-only-architecture.drawio.svg | 301 ++++++++++++++++++ ...> self-sufficient-architecture.drawio.svg} | 159 ++++----- 3 files changed, 403 insertions(+), 81 deletions(-) create mode 100644 support/docs/cloud-only-architecture.drawio.svg rename support/docs/{architecture.drawio.svg => self-sufficient-architecture.drawio.svg} (85%) diff --git a/README.md b/README.md index 9c0a8f98..5f9c2500 100644 --- a/README.md +++ b/README.md @@ -103,6 +103,26 @@ $ deno task ahc-hrsn-screening-doc # generate documentation for the libra ## Architecture and Approach -TODO: narratives and explanations +This code allows multiple operating models, but these two are the most likely +use cases: -![Architecture](support/docs/architecture.drawio.svg) +- Cloud-only model where a QE does not operate their own code instances to + validate or manage data and relies on QCS infrastructure only. + - Benefits: No local infrastructure required + - Detriments: No ability to validate or analyze aggregated data before sending + to QCS environment +- Self-sufficient model where a QE can operate their own code instances for + validation and data management locally and then forwards processed data to the + QCS infrastructure. + - Benefits: Ability to validate and analyze aggregated data before sending to + QCS environment + - Detriments: Needs small amount of local infrastructure to operate the same + code running at QCS + +### Cloud Only Model + +![Architecture](support/docs/cloud-only-architecture.drawio.svg) + +### Self-sufficient QE Model + +![Architecture](support/docs/self-sufficient-architecture.drawio.svg) diff --git a/support/docs/cloud-only-architecture.drawio.svg b/support/docs/cloud-only-architecture.drawio.svg new file mode 100644 index 00000000..b430afb8 --- /dev/null +++ b/support/docs/cloud-only-architecture.drawio.svg @@ -0,0 +1,301 @@ + + + + + + + + + + + +
+
+
+ QE acquires sources (any technique) +
+
+
+ + + QE acquires sources (any technique) + + + + + + + + + + + +
+
+
+ FHIR Observation Resources +
+
+
+ + + FHIR Observation Res... + + + + + + + + +
+
+
+ FHIR/ JSON +
+ ETL +
+
+
+ + + FHIR/ JSON... + + + + + + + + +
+
+
+ PostgreSQL RDS +
+
+
+ + + PostgreSQL RDS + + + + + + + + + +
+
+
+ Export from Source +
+
+
+ + + Export from... + + + + + + + +
+
+
+ Validate via +
+ REST API +
+
+
+ + + Validate via... + + + + + + + + + +
+
+
+ Prepare CSVs or Excel Workbooks(s) +
+
+
+ + + Prepare CSVs or Excel... + + + + + + + + + + +
+
+
+ Fix Issues +
+
+
+ + + Fix Issues + + + + + + + + + +
+
+
+ QE Network Participants (HDOs, HDPs) +
+
+
+ + + QE Network Participants (HDOs, HDPs) + + + + + + + +
+
+
+ Upload via +
+ REST API +
+
+
+ + + Upload via... + + + + + + + +
+
+
+ + Exception / error diagnostics must be SQL queryable and e-mail / notification friendly + +
+
+
+ + + Exception / error diagnos... + + + + + + + +
+
+
+ Ingestion Process +
+
+
+ + + Ingestion Pr... + + + + + + + +
+
+
+ Should include local community patient ID and demographics +
+
+
+ + + Should include lo... + + + + + + + +
+
+
+ + SHIN-NY Data Lake + +
+
+
+ + + SHIN-NY Data Lake + + + + + + + + + +
+
+
+ QEs Upload CSVs +
+ or Excel Workbooks (SFTP or S3) to QCS environment +
+
+
+ + + QEs Upload CSVs... + + + + + + + + + Text is not SVG - cannot display + + + + \ No newline at end of file diff --git a/support/docs/architecture.drawio.svg b/support/docs/self-sufficient-architecture.drawio.svg similarity index 85% rename from support/docs/architecture.drawio.svg rename to support/docs/self-sufficient-architecture.drawio.svg index f1d133a0..0299a883 100644 --- a/support/docs/architecture.drawio.svg +++ b/support/docs/self-sufficient-architecture.drawio.svg @@ -1,8 +1,6 @@ - + - - @@ -96,18 +94,18 @@
- Aggregated QEs SQL + PostgreSQL RDS
- Aggregated QEs... + PostgreSQL RDS - - + + @@ -144,27 +142,8 @@ - - - - - - -
-
-
- Upload SQL DB -
- (SFTP) -
-
-
- - - Upload SQL DB... - - - + + @@ -184,20 +163,20 @@ - + -
+
- Prepare Excel Workbooks(s) + Prepare CSVs or Excel Workbooks(s)
- - Prepare Excel Wor... + + Prepare CSVs or Exc... @@ -277,14 +256,14 @@ - - - - + + + + -
+
Content & @@ -294,7 +273,7 @@
- + Content &... @@ -372,35 +351,15 @@ - - - - - + + + - - - - - -
-
-
- SQLite File -
-
-
- - - SQLite File - - - - + -
+
exception reports (Excel) @@ -408,7 +367,7 @@
- + exception reports... @@ -485,11 +444,11 @@ - + -
+
QCS / TBD boundary @@ -497,7 +456,7 @@
- + QCS / TBD boundary @@ -525,11 +484,11 @@ - + -
+
All code should should run in AWS, Azure, ORACLE, or any other cloud infra. @@ -537,7 +496,7 @@
- + All code should should run... @@ -619,11 +578,11 @@ - + -
+
Aggregation SQL Script @@ -631,7 +590,7 @@
- + Aggregation... @@ -707,20 +666,22 @@ - + -
+
- Upload Excel Workbooks (SFTP) + Option 1: +
+ QEs Upload CSVs or Excel Workbooks (SFTP or S3)
- - Upload Excel Workboo... + + Option 1:... @@ -740,6 +701,46 @@ + + + + +
+
+
+ "Bundled" archive +
+ (instead of ZIP file) +
+
+
+ + + "Bundled" archive... + + + + + + + +
+
+
+ Option 2: +
+ QEs Upload Bundled +
+ SQLite File (SFTP or S3) +
+
+
+ + + Option 2:... + + +