Merge branch 'main' into setup-keycloak-multifactor-authentication

.github/pull_request_template.md

@@ -4,5 +4,5 @@ Fixes #[issue number] or Closes #[issue number]
 
 **Engineer Checklist**
 - [ ] I have run `./gradlew spotlessApply` to check my code follows the project's style guide
-- [ ] I have built and run the efsity jar to verify my change fixes the issue and/or does not break the application 
+- [ ] I have built using the command `./gradlew clean assemble` and run the efsity jar to verify my change fixes the issue and does not break the application 
 

.github/workflows/ci.yml

@@ -15,15 +15,15 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
-      - name: Set up JDK 11
+      - name: Set up JDK 17
         uses: actions/setup-java@v4
         with:
           distribution: 'temurin'
-          java-version: 11
+          java-version: 17
 
       - name: Run Unit Tests and Generate Coverage Report
         run: ./gradlew test jacocoTestReport
-        working-directory: efsity
+        working-directory: efsity-cli
 
   run-spotless-check:
     runs-on: ubuntu-latest
@@ -38,7 +38,7 @@ jobs:
 
       - name: Run spotless check
         run: ./gradlew spotlessCheck
-        working-directory: efsity
+        working-directory: efsity-cli
 
   run-importer-unit-tests:
     runs-on: ubuntu-latest

README.md

@@ -4,6 +4,7 @@ A repo to hold our FHIR content and configuration creation tools and scripts.
 
 - [cleaner](https://github.com/onaio/fhircore-tooling/tree/main/cleaner)
 - [efsity](https://github.com/onaio/fhircore-tooling/tree/main/efsity)
+- [efsity-ide](https://github.com/onaio/fhircore-tooling/tree/main/efsity-ide)
 - [importer](https://github.com/onaio/fhircore-tooling/tree/main/importer)
 - [sm-gen](https://github.com/onaio/fhircore-tooling/tree/main/sm-gen)
 - [uploader](https://github.com/onaio/fhircore-tooling/tree/main/uploader)

efsity/.gitignore → efsity-cli/.gitignore

@@ -9,3 +9,5 @@ fhircore-tooling.iml
 # build files
 bin
 target
+
+generatedResources

efsity/README.md → efsity-cli/README.md

@@ -65,12 +65,14 @@ The tool supports generation of a QuestionnaireResponse via [openai api](https:/
 Optionally, you can also provide the [model](https://platform.openai.com/docs/models) you would like to use, the number of [max-token](https://platform.openai.com/docs/api-reference/completions/create#completions/create-max_tokens) and an output file path
 
 ```console
-$ fct generateResponse -i /path/to/questionnaire.json -k your_api_key
+$ fct generateResponse -i /path/to/questionnaire.json -fs https://example.smartregister.org/fhir
 ```
 
 **Options**
 ```
 -i or --input - file path to the questionnaire
+-gm or --generation-mode - the generation mode to be used. Default is set to 'populate', but can also be set 'ai'
+-fs or --fhir-server - the FHIR server used to generate the questionnaire response
 -k or --apiKey - api key to authenticate
 -m or --model - (Optional) model you want to use. The default is `gpt-3.5-turbo-16k`
 -t or --tokens - (Optional) max number of tokens to be used for the request. The default is 9000
@@ -170,6 +172,38 @@ $ fct validateFileStructure -i ~/Workspace/fhir-resources/<project> -s ~/path/to
 If the file structure matches the schema then a positive result is printed to the terminal, otherwise an error 
 is thrown showing the issue.
 
+### Validating Questionnaires and  StructureMap 
+The tool supports two validation modes:
+
+**1.Single mode** - Validates a single StructureMap and questionnaire.
+**2.Project mode** - Validates all project files, including multiple questionnaires and StructureMaps.
+
+The tool maps each questionnaire to the specified StructureMap, then generates the corresponding `QuestionnaireResponse`. Once generated, a bundle is created with the related resources (e.g., Patient, Condition).
+
+To run the tool, ensure the following:
+
+1. Install h`api=fhir-jpaserver-starter`.
+2. In `src/main/resources/application.yaml`, set `hapi: fhir`: to `true`.
+3. Run the command: `mvn spring-boot:run`
+
+The server will be accessible at http://localhost:8080/fhir, and the `CapabilityStatement` can be found at http://localhost:8080/fhir/metadata.
+**Options**
+```
+-i or --input            Path to the project questionnaire folder to be validated.
+-q or --questionnaire     Path to a single questionnaire file to be validated.
+-v or --validate          (Optional) Boolean flag to validate FHIR resources before publishing. Default is `false`.
+-sm or --structure-map    Path to the folder containing StructureMaps. Nested directories are supported.
+
+```
+
+```console (single-mode)
+$  fct validateStructureMap -q ~/Workspace/fhir-resources/<project>/questionnaire-folder/resource.json -sm ~/Workspace/fhir-resources/coda/structure_map-folder/coda-child-structure-map.txt
+```
+
+```console (project-mode)
+$  fct validateStructureMap -i ~/Workspace/fhir-resources/<project>/questionnaire-folder/ -sm ~/Workspace/fhir-resources/coda/structure_map-folder/`
+```
+
 ### Localization
 Tool that supports localization by the use of the translation extension
 

efsity/build.gradle.kts → efsity-cli/build.gradle.kts

@@ -18,7 +18,7 @@ repositories {
 
 group = "org.smartregister"
 
-version = "2.3.8-SNAPSHOT"
+version = "2.3.11-DEV-SNAPSHOT"
 
 description = "fhircore-tooling (efsity)"
 
@@ -83,6 +83,9 @@ dependencies {
   implementation(deps.picocli)
   implementation(deps.xstream)
   implementation(deps.icu4j)
+  implementation(deps.javafaker) { exclude(group = "org.yaml") }
+  implementation(deps.snakeyaml)
+  implementation("ca.uhn.hapi.fhir:hapi-fhir-validation:6.8.0")
 
   testImplementation(kotlin("test"))
   testImplementation("junit:junit:4.13.2")

efsity-cli/docs/QuestionnaireResponseGenerator.md

@@ -0,0 +1,60 @@
+# QuestionnaireResponse Generator Command
+## Overview
+The QuestionnaireResponseGeneratorCommand is designed to generate FHIR QuestionnaireResponse resources from FHIR Questionnaire resources. This tool supports two modes of generation:
+
+1. **Populate Mode**: (Default) Uses the FHIR Clinical Reasoning Module's `$populate` operation to generate a questionnaireResponse from a questionnaire.
+2. **AI Mode**: Utilizes an AI model to generate a QuestionnaireResponses based on a Questionnaire.
+
+
+### Options
+```
+-i, --input: (Required) Path to the input file containing the FHIR Questionnaire resource.
+-gm, --generation-mode: (Optional) Generation mode. Options: populate (default), ai.
+-fs, --fhir-server: (Optional) Base URL of the FHIR server to use for generation (required for populate mode).
+-e, --extras: (Optional) Path to extra definitions for Faker value generation.
+-k, --apiKey: (Optional) API key for accessing AI services (required for ai mode).
+-m, --model: (Optional) AI model to use. Default: gpt-3.5-turbo-16k.
+-t, --tokens: (Optional) Max tokens for AI generation. Default: 9000.
+-o, --output: (Optional) Path to the output file or directory. Default: current directory.
+```
+
+### Example Usage
+Generate a QuestionnaireResponse using populate mode:
+```shell
+fct generateResponse -i /path/to/questionnaire.json -fs http://fhir-server.example.com/fhir -e /path/to/extras.json
+```
+Generate a QuestionnaireResponse using AI mode:
+
+```shell
+fct generateResponse -i /path/to/questionnaire.json -gm ai -k YOUR_API_KEY
+```
+
+#### FHIR Server
+You can quickly spin up the latest HAPI FHIR server to try this out by:
+- Cloning the repo https://github.com/hapifhir/hapi-fhir-jpaserver-starter
+- Open your `application.yaml` file in `src/main/resources` and update `cr-enabled` to `true`. This will enable the Clinical Reasoning module
+- Run the server: `mvn spring-boot:run`
+- _extra tip_: Ensure that the questionnaire you are using has a uuid as the identifier. Sometimes the resource creation fails if the id is all numbers
+
+### Answer generation
+Using the populate mode, once the QuestionnaireResponse is generated, we also need to generate answers to populate it. The tool by default uses Java's `Random` mainly for this. Generating integers, decimals, booleans and strings that are basically a concatenation of "FakeString" with a random integer.
+
+The tool optionally, also leverages the `Faker` library for more realistic fake data generation. To use this option you need to provide an _extras.json_ file that has a list of the questions _(their link_ids)_ that you want to generate andswers for, and the categories/methods from the Faker class that you'd like to use.
+
+Example extras.json file
+```json
+{
+  "445e224e-1b01-4003-95d3-429d0e6459c4": {"category": "name", "method": "firstName"},
+  "64554be3-acd2-4ea9-bcde-9bff728b22bc": {"category": "name", "method": "lastName"},
+  "dcef66ee-f162-4619-b7a0-0560fee4638c": {"category": "demographic", "method": "sex"},
+  "48b9e73a-04eb-4084-9566-76ff86ff25a2": {"category": "address", "method": "fullAddress"},
+  "a98e3117-3bbd-4b4e-8634-8a4cac2641de": {"category": "medical", "method": "diseaseName"},
+  "5a6477d2-431d-4711-9f41-3f43991f88e2": {"category": "number", "method": "randomDigit"},
+  "3c331069-7e95-4df1-bc0b-f383f1304c37": {"category": "lorem", "method": "sentence"}
+}
+```
+
+You can see a full list of options from Faker [here](https://dius.github.io/java-faker/apidocs/index.html)
+
+## AI Mode
+NOTE: Temporarily unsupported at the moment

efsity/libs.versions.toml → efsity-cli/libs.versions.toml

@@ -2,18 +2,20 @@
 buildConfig-version="4.1.2"
 caffeine-version="2.9.3"
 common-compress-version="1.22"
-cql-version="2.11.0"
+cql-version="3.3.2"
 gson-version="2.10.1"
-hapi-fhir-core-version="5.6.106"
+hapi-fhir-core-version="6.0.22"
 hapi-fhir-utilities-version="3.8.0"
-hapi-fhir-version="6.0.1"
+hapi-fhir-version="6.8.0"
 info-picocli-version="4.7.4"
 jackson-version="2.15.2"
+javafaker-version="1.0.2"
 json-version="20230618"
 jsonschemafriend-version="0.12.2"
 kotlin-stdlib="1.8.22"
 opencds-cql-version="2.4.0"
 project-build-sourceEncoding="UTF-8"
+snakeyaml-version="2.3"
 spotless-version ="6.20.0"
 xstream="1.4.20"
 icu4j-version = "75.1"
@@ -43,7 +45,9 @@ jsonschemafriend = { module = "net.jimblackler.jsonschemafriend:core", version.r
 jackson-annotations = { module = "com.fasterxml.jackson.core:jackson-annotations", version.ref = "jackson-version" }
 jackson-core = { module = "com.fasterxml.jackson.core:jackson-core", version.ref = "jackson-version" }
 jackson-databind = { module = "com.fasterxml.jackson.core:jackson-databind", version.ref = "jackson-version" }
+javafaker = { module = "com.github.javafaker:javafaker", version.ref = "javafaker-version"}
 picocli = { module = "info.picocli:picocli", version.ref = "info-picocli-version" }
+snakeyaml = { module="org.yaml:snakeyaml", version.ref ="snakeyaml-version"}
 xstream = { module = "com.thoughtworks.xstream:xstream", version.ref = "xstream" }
 icu4j = { module="com.ibm.icu:icu4j", version.ref = "icu4j-version" }
 

efsity/src/main/java/org/smartregister/Main.java → efsity-cli/src/main/java/org/smartregister/Main.java

@@ -21,7 +21,8 @@
       TranslateCommand.class,
       QuestionnaireResponseGeneratorCommand.class,
       ValidateFileStructureCommand.class,
-      PublishFhirResourcesCommand.class
+      PublishFhirResourcesCommand.class,
+      ValidateStructureMapCommand.class,
     })
 public class Main implements Runnable {
   public static void main(String[] args) {