Feature/testing framework #41

micolspitale93 · 2021-04-28T08:51:49Z

Merging testing branch with develop

…o-express containers.

… start.

RMichaelSwan

Reviewed and tested. db.test worked well for me. Once you address the comments and add a little documentation, I think it'll be about ready after that (assuming the use case is to simply provide people a place to start--some discussion on use cases may be warranted).

docker-compose-full-db.yml

RMichaelSwan · 2021-05-06T09:05:50Z

harmoni_core/harmoni_pattern/test/sequential_speak.test

+    <!-- This is an integrative test that requires the speaker interaction -->
+    <include file="$(find harmoni_speaker)/launch/speaker_service.launch"></include>
+    <include file="$(find harmoni_tts)/launch/tts_service.launch"></include>
+    <include file="$(find harmoni_bot)/launch/bot_service.launch"></include>
+    <include file="$(find harmoni_face)/launch/face_service.launch"></include>
+    <include file="$(find harmoni_pattern)/launch/sequence_pattern.launch">
+        <arg name="pattern_name" default="speak_test"/>
+    </include>
+
+
+    <!-- <rosparam file="$(find harmoni_pattern)/config/configuration.yaml" subst_value="True"/>
+    <param name="instance_id" value="default"/>
+	<node pkg="harmoni_pattern" type="sequential_pattern.py" name="harmoni_pattern_default" output="screen"/> -->
+
+
+    <test test-name="test_sequential" pkg="harmoni_pattern" type="test_sequential_speak.py"/>
+</launch>


I really like this integration test =) Provides a good example of using almost the entire system and could probably be referenced as part of a tutorial.

Since it requires user interaction, documentation is needed somewhere that specifies what people are expected to do.

Agree. I have not checked Chris documentation yet.
@chrismbirmingham do you have any reference to this integration test?

RMichaelSwan · 2021-05-06T09:13:50Z

harmoni_core/harmoni_pattern/test/test_sequential_speak.py

+    def test_IO(self):
+        rospy.loginfo(
+            "TestSequential[TEST]: basic test to ensure pattern can start a service "
+        )
+        while not rospy.is_shutdown() and not self.result:
+            self.rate.sleep()
+        assert self.result == True


Need to add more test cases before we can say this is complete, right? At least some quantitative indication that the sequence is moved through correctly.

Note: the setUp and init functions will be re-run for each test case...this is usually okay, but if you don't like it, or need to put all nodes into a specific state afterwards, I recommend using setUpClass (uses class variables instead of class instance variables) and/or tearDown().

Agreed. Let me add some indicators about the progress of the testing.

As regards your note, I think that for now we can keep the setUp and init funcion that run every time since we have a single testing function.

Normally you would want to separate out your test functions so that you know which one passes and which one does not...I suppose since it's not automated and requires user interaction though, you can just do a single test function and print out statements based on progress through the test.

RMichaelSwan · 2021-05-06T09:14:59Z

harmoni_core/harmoni_pattern/test/sequential_speak.test

+	<node pkg="harmoni_pattern" type="sequential_pattern.py" name="harmoni_pattern_default" output="screen"/> -->
+
+
+    <test test-name="test_sequential" pkg="harmoni_pattern" type="test_sequential_speak.py"/>


Should specify time-limit="###.#" since you likely need more than 60 seconds to test this.

http://wiki.ros.org/roslaunch/XML/test#Attributes

Thank you! I'm adding it for this testing

harmoni_core/harmoni_pattern/test/test_sequential_speak.py

harmoni_core/harmoni_recorder/test/rostest_db.py

RMichaelSwan · 2021-05-06T09:40:28Z

harmoni_core/harmoni_recorder/test/unittest_db.py

@@ -0,0 +1,53 @@
+#!/usr/bin/env python3


Calling this a unit test would indicate it can be tested without roscore running (i.e. just like a regular python package)...is this true?

It was supposed to be an unittest, but it is not yet.

harmoni_core/harmoni_recorder/src/harmoni_recorder/template_storedata.py

RMichaelSwan · 2021-05-06T10:01:49Z

harmoni_core/harmoni_recorder/requirements.txt

@@ -0,0 +1 @@
+pymongo


I added this file, but we should update the relevant dockerfiles to add this dependency too.

…ction-lab/HARMONI into feature/testing-framework

Co-authored-by: Michael Swan <[email protected]>

…mments.

RMichaelSwan · 2021-05-10T16:03:40Z

harmoni_core/harmoni_pattern/test/test_sequential.py

@@ -30,12 +30,13 @@ class TestSequential(unittest.TestCase):

    def __init__(self, *args):
        super(TestSequential, self).__init__(*args)


If you don't define init at all, the super init call will automatically occur.

Do you suggest to remove the init in all the testing files for all the packages?

Sure. Worked fine for my testing, and consistency is good.

RMichaelSwan · 2021-05-10T16:05:36Z

harmoni_core/harmoni_pattern/test/test_sequential_speak.py

+    This is a sequence of:
+    - bot
+    - tts
+    - speak and lip-sync
+    If all the requests succeed, the test passes.


I suppose it's okay for now, but long term I would want to make sure it runs correctly for any possible sequential sequence, not just testing some specific sequence.

RMichaelSwan and others added 5 commits April 21, 2021 07:22

dlib face detector refactor, working test, and proper namespace setup.

4181b28

added mongodb client for harmoni + docker compose with mongo and mong…

f320917

…o-express containers.

adding testing, but not working. import fails.

ef6033e

fix typo in tts and bot. passed testing harmoni_recorder.

3bcf634

adding testing for multiple steps pattern, and fix sequential pattern…

bf6e264

… start.

micolspitale93 requested review from chrismbirmingham and RMichaelSwan April 28, 2021 08:51

micolspitale93 mentioned this pull request Apr 28, 2021

harmoni_bot roslaunch wants test args #42

Closed

chrismbirmingham and others added 2 commits May 5, 2021 21:07

Add Harmoni Readthedocs

465b3bc

mongodb python requirements file.

d1fe9ec

RMichaelSwan requested changes May 6, 2021

View reviewed changes

chrismbirmingham and others added 4 commits May 6, 2021 16:40

Working Docs

8f3fd5a

Merge branch 'feature/testing-framework' of https://github.com/intera…

0505563

…ction-lab/HARMONI into feature/testing-framework

remove useless libraries

be5f3e7

Co-authored-by: Michael Swan <[email protected]>

fix bugs. moving everything from init to setup. addressing michael co…

7fe7d63

…mments.

RMichaelSwan mentioned this pull request May 10, 2021

Proper Secret Handling for MongoDB #43

Open

RMichaelSwan reviewed May 10, 2021

View reviewed changes

addressing Michael comments.

47632b6

RMichaelSwan approved these changes May 10, 2021

View reviewed changes

removing init for rostest script.

b6427bf

chrismbirmingham merged commit 231c709 into develop May 11, 2021

chrismbirmingham deleted the feature/testing-framework branch May 11, 2021 14:13

RMichaelSwan mentioned this pull request May 12, 2021

Sequential Pattern Testing #44

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/testing framework #41

Feature/testing framework #41

micolspitale93 commented Apr 28, 2021

RMichaelSwan left a comment

RMichaelSwan May 6, 2021

micolspitale93 May 10, 2021

RMichaelSwan May 6, 2021

micolspitale93 May 10, 2021

RMichaelSwan May 10, 2021

RMichaelSwan May 6, 2021 •

edited

Loading

micolspitale93 May 10, 2021

RMichaelSwan May 6, 2021

micolspitale93 May 10, 2021

RMichaelSwan May 6, 2021

micolspitale93 May 10, 2021

RMichaelSwan May 10, 2021

micolspitale93 May 10, 2021

RMichaelSwan May 10, 2021

micolspitale93 May 10, 2021

RMichaelSwan May 10, 2021

		<node pkg="harmoni_pattern" type="sequential_pattern.py" name="harmoni_pattern_default" output="screen"/> -->


		<test test-name="test_sequential" pkg="harmoni_pattern" type="test_sequential_speak.py"/>

		@@ -30,12 +30,13 @@ class TestSequential(unittest.TestCase):

		def __init__(self, *args):
		super(TestSequential, self).__init__(*args)

		@@ -0,0 +1 @@
		pymongo

Feature/testing framework #41

Feature/testing framework #41

Conversation

micolspitale93 commented Apr 28, 2021

RMichaelSwan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RMichaelSwan May 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RMichaelSwan May 6, 2021 •

edited

Loading