Initial commit. #1
Conversation
This test is running xtest in the hikey.
| Library SSHLibrary | ||
| Library String | ||
|
|
||
| *** Variables *** |
There was a problem hiding this comment.
Config vars should be separate from the test cases. Store them in https://github.com/linaro-home/lhg-robot-config
There was a problem hiding this comment.
How would the test cases be run? It would relate to how should we manage the config variables.
We can put all the config variables (like target IP address, login credentials) into a single file, for example lhg-robot-config/resource.robot. Before running the test, user should edit the config variables manually.
Should we consider the situation running the same test cases with multiple targets simultaneously?
There was a problem hiding this comment.
See the input keywords here:
http://robotframework.org/robotframework/latest/libraries/BuiltIn.html#Import%20Library
I don't assume multiple targets on one Host machine. We could still use docker for that if it becomes a requirement.
| *** Test Cases *** | ||
| Execute xtest regression test | ||
| [Documentation] Execute "xtest -t regression" | ||
| Set Client Configuration timeout=40s prompt=${HIKEY_PROMPT} |
There was a problem hiding this comment.
Connection related keywords should be not part of the test steps. SSH related keywords should be abstracted away, so the test case doesn't know what protocol is used. The connection handling needs to be moved into a robot library to the lhg-robot-libs project.
|
|
||
| Execute xtest benchmark test | ||
| [Documentation] Execute "xtest -t benchmark" | ||
| Set Client Configuration timeout=10s prompt=${HIKEY_PROMPT} |
| Should Contain ${result} TEE test application done! | ||
|
|
||
| *** Keywords *** | ||
| Open Connection And Log In |
There was a problem hiding this comment.
This should be moved into a helper library.
| Login ${USERNAME} ${PASSWORD} | ||
|
|
||
| Run Tee Supplicant | ||
| Start Command /usr/bin/tee-supplicant |
There was a problem hiding this comment.
Don't use SSH library directly. Abstract it into a library.
| Start Command pkill tee-supplicant | ||
|
|
||
| Run Regression Test | ||
| write xtest -t regression |
| Start Command /usr/bin/tee-supplicant | ||
|
|
||
| Terminate Tee Supplicant | ||
| Start Command pkill tee-supplicant |
| Run Tee Supplicant | ||
| Start Command /usr/bin/tee-supplicant | ||
|
|
||
| Terminate Tee Supplicant |
There was a problem hiding this comment.
This is not a test case. It should be a keyword. Also the test cases should not depend on the order they are run. Test cases must not have run time dependencies on each other.
| Terminate Tee Supplicant | ||
| Start Command pkill tee-supplicant | ||
|
|
||
| Run Regression Test |
There was a problem hiding this comment.
This test case is depending on success of previous test cases. There should be no dependency between test cases, or how the test cases are ordered.
There was a problem hiding this comment.
"Run Regression Test" is a keyword which is called by the first test case "Execute xtest regression test"
There is a very good example (https://bitbucket.org/robotframework/webdemo) for reference. In this example, all config variables and new keywords are collected in resource.robot. The test cases, for example valid_login.robot, which include resource.robot are colloquial and easy to understand.
For our project, there are several files, except test case, lhg-robot-config and lhg-robot-libs
In my imagination
-
lhg-robot-config
Collect all config variables and, maybe, new keywords which make test cases clear and easy to understand. For example "Run Regression Test".
-
lhg-robot-libs
Python codes which is like fundamental building blocks. For example, connection handling function.
Is my understanding correct?
| Run Regression Test | ||
| write xtest -t regression | ||
|
|
||
| Run Benchmark Test |
There was a problem hiding this comment.
This test case is depending on success of previous test cases. There should be no dependency between test cases, or how the test cases are ordered.
This test is running xtest in the hikey.