Merge pull request #68 from singnet/develop

Sync Develop Into Master

.dockerignore

@@ -0,0 +1,7 @@
+env/
+.vscode
+docs
+build
+.github
+__pycache__
+.pytest_cache

.env.example

@@ -10,6 +10,8 @@ DAS_MONGODB_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
 DAS_MONGODB_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
 DAS_DATABASE_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
 DAS_DATABASE_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
-PYTHONPATH=/app
+
+ATOMDB_PACKAGE_PATH=
+QUERY_ENGINE_PACKAGE_PATH=
 
 DAS_KNOWLEDGE_BASE=/data/samples/animals.metta

CHANGELOG

@@ -0,0 +1,3 @@
+[#57] Enable Running Integration Tests Locally with AtomDB Source Code and Local Query Engine
+[#62] Create integration tests for get_incoming_links
+[#64] Shutdown containers before run tests

Makefile

@@ -18,4 +18,10 @@ unit-tests-coverage:
 integration-tests:
 	./scripts/run-tests.sh integration
 
-pre-commit: unit-tests-coverage lint
+pre-commit: unit-tests-coverage lint
+
+serve:
+	@docker compose up --build --force-recreate -d
+
+stop:
+	@docker compose down

README.md

@@ -17,6 +17,7 @@ OpenFaaS, or Open Functions as a Service, is an open-source platform simplifying
 ## Running an OpenFaaS Function Locally
 
 ### Requirements
+
 - Docker
 - Docker Compose
 
@@ -33,11 +34,10 @@ The project architecture consists of:
 - **das-query-engine**: Operates on the same network as the 'openfaas' container.
 - **Port 8080**: Exposed for connection to the host machine.
 
-
 ### Step-by-Step Guide
 
 1. **Cloning the Project**
-   
+
    Clone the project repository to your local environment.
 
    ```bash
@@ -58,7 +58,7 @@ The project architecture consists of:
    Execute the following command at the root of the project to start the required services:
 
    ```bash
-   docker compose up -d
+   make serve
    ```
 
    This will start containers for Redis, MongoDB, and a temporary container named 'canonical-load'. The latter is used to load initial data into the Redis and MongoDB databases. After its execution, a container named 'openfaas' will start, which includes the faas-cli.
@@ -72,20 +72,44 @@ The project architecture consists of:
    To shut down the containers and clean up the environment, execute the following command:
 
    ```bash
-   docker compose down
+   make stop
    ```
 
 ### Testing
 
 To run automated tests for the project, use the following script:
 
 ```bash
-./scripts/run-tests.sh
+make integration-tests
+```
+
+```bash
+make unit-tests
 ```
 
 This script will set up the necessary environment for testing.
 
+### Use the hyperon-das and hyperon-das-atomdb from the host machine
+
+This feature has been implemented to allow developers to test the integration of AtomDB and Query Engine packages locally, even before publishing them on PyPI. This facilitates efficient testing during the development phase.
 
+1. Open the `.env` file at the root of your project.
+
+2. Add the following environment variables, adjusting the paths as necessary:
+
+   ```dotenv
+   ATOMDB_PACKAGE_PATH=/path/to/your/atomdb/package
+   QUERY_ENGINE_PACKAGE_PATH=/path/to/your/query/engine/package
+   ```
+
+   Make sure to replace `/path/to/your/atomdb/package` and `/path/to/your/query/engine/package` with the actual paths of the AtomDB and Query Engine packages on your system.
+
+3. If you prefer to use the latest versions of the AtomDB and Query Engine packages published on PyPI, leave the variables empty:
+
+   ```dotenv
+   ATOMDB_PACKAGE_PATH=
+   QUERY_ENGINE_PACKAGE_PATH=
+   ```
 
 ## Obtaining Function Logs in faasd Using the `ctr` Command
 

das-query-engine/actions.py

@@ -107,7 +107,6 @@ def get_incoming_links(
             atom_handle,
             **kwargs,
         )
-
 
     @execution_time_tracker
     @remove_none_args

das-query-engine/tests/integration/handle/base_test_action.py

@@ -26,10 +26,33 @@ def malformed_event(self):
     def unknown_action_event(self):
         return {"body": {"action": "unknown_action", "input": {}}}
 
+    @staticmethod
+    def _sorted_nested(obj):
+        if isinstance(obj, list):
+            return sorted(str(BaseTestHandlerAction._sorted_nested(item)) for item in obj)
+        elif isinstance(obj, dict):
+            return sorted(
+                (key, BaseTestHandlerAction._sorted_nested(value)) for key, value in obj.items()
+            )
+        elif isinstance(obj, (int, float, str, bool, type(None))):
+            return obj
+        else:
+            return str(obj)
+
+    @staticmethod
+    def _compare_nested(obj1, obj2):
+        return BaseTestHandlerAction._sorted_nested(obj1) == BaseTestHandlerAction._sorted_nested(
+            obj2
+        )
+
     def assert_successful_execution(self, valid_event, expected_output):
         response = handle(valid_event, context={})
+        body = json.loads(response["body"])
         assert response["statusCode"] == 200
-        assert response["body"] == json.dumps(expected_output), f"Assertion failed:\nResponse Body: {response['body']}\nExpected: {json.dumps(expected_output)}"
+        assert BaseTestHandlerAction._compare_nested(
+            body,
+            expected_output,
+        ), f"Assertion failed:\nResponse Body: {response['body']}\nExpected: {json.dumps(expected_output)}"
 
     def assert_payload_malformed(self, malformed_event):
         response = handle(malformed_event, context={})