From 856b5637a8652f3a4992ff5761c15c8c4623bde3 Mon Sep 17 00:00:00 2001 From: Maciej Pijanowski Date: Sun, 14 Jul 2024 18:07:16 +0200 Subject: [PATCH] osfv_cli/test: init Signed-off-by: Maciej Pijanowski --- osfv_cli/test/common/keywords.robot | 64 +++++++++++++++++ osfv_cli/test/rte.robot | 101 +++++++++++++++++++++++++++ osfv_cli/test/snipeit.robot | 102 ++++++++++++++++++++++++++++ 3 files changed, 267 insertions(+) create mode 100644 osfv_cli/test/common/keywords.robot create mode 100644 osfv_cli/test/rte.robot create mode 100644 osfv_cli/test/snipeit.robot diff --git a/osfv_cli/test/common/keywords.robot b/osfv_cli/test/common/keywords.robot new file mode 100644 index 0000000..1887589 --- /dev/null +++ b/osfv_cli/test/common/keywords.robot @@ -0,0 +1,64 @@ +*** Settings *** +Documentation Common keywords to be used during osfv_cli testing + +Library Process + + +*** Variables *** +${RTE_IP_APU2}= 192.168.10.172 +${ASSET_ID_APU2}= 31 + +${RTE_IP_FW4C}= 192.168.10.168 +${ASSET_ID_FW4C}= 42 + +${RTE_IP_V1410}= 192.168.10.199 + +${RTE_IP_VP4630}= 192.168.10.244 + +${RTE_IP_Z690}= 192.168.10.199 + + +*** Keywords *** +Run Flash Probe + [Arguments] ${ip} + ${result}= Run Process osfv_cli rte --rte_ip ${ip} flash probe + Should Not Match ${result.stdout} *No EEPROM/flash device found.* + Should Match ${result.stdout} *Found * flash chip * on * + Should Be Equal As Integers ${result.rc} 0 + Should Be Empty ${result.stderr} + RETURN ${result} + +Check In Test Devices + Check In ${RTE_IP_APU2} + Check In ${RTE_IP_V1410} + Check In ${RTE_IP_VP4630} + Check In ${RTE_IP_Z690} + Check In ${RTE_IP_FW4C} + +Check Out + [Documentation] Check out asset using osfv_cli as user defined in ~/.osfv.snipeit.yml + [Arguments] ${rte_ip} ${snipeit_config}=%{HOME}/.osfv/snipeit.yml + ${result}= Run Process + ... osfv_cli + ... snipeit + ... check_out + ... --rte_ip + ... ${rte_ip} + ... env:SNIPEIT_CONFIG_FILE_PATH=${snipeit_config} + Log ${result.stdout} + Log ${result.stderr} + RETURN ${result} + +Check In + [Documentation] Check in asset using osfv_cli as user defined in ~/.osfv.snipeit.yml + [Arguments] ${rte_ip} ${snipeit_config}=%{HOME}/.osfv/snipeit.yml + ${result}= Run Process + ... osfv_cli + ... snipeit + ... check_in + ... --rte_ip + ... ${rte_ip} + ... env:SNIPEIT_CONFIG_FILE_PATH=${snipeit_config} + Log ${result.stdout} + Log ${result.stderr} + RETURN ${result} diff --git a/osfv_cli/test/rte.robot b/osfv_cli/test/rte.robot new file mode 100644 index 0000000..f6524a5 --- /dev/null +++ b/osfv_cli/test/rte.robot @@ -0,0 +1,101 @@ +*** Settings *** +Documentation The goal of this suite is to evaluate the most important features +... of the "rte" command, and the "rte.py" library as a consequence as +... well. We have a wide variety of hardware platforms in the lab, +... which are connected in various ways, with various programmers. +... This suite ensures that at least one platform from each type +... of setup can be evaluated before proceeding with another release +... of the osfv-scripts. + +Library Process +Resource ../test/common/keywords.robot + +Suite Setup Run Keyword Check In Test Devices + + +*** Test Cases *** +Test Flash Probe On RTE v1.0.0 + relay + [Documentation] Check if flash on APU2 can be probed. This covers a case + ... with v1.0.0 revision of RTE hardware, where SPI lines + ... cannot be controlled. + Run Flash Probe ${RTE_IP_APU2} + +Test Flash Probe On RTE v1.1.0 + relay + [Documentation] Check if flash on APU2 can be probed. This covers a case + ... with v1.1.0 revision of RTE hardware, where SPI lines + ... can be controlled, and relay is used for power control. + Run Flash Probe ${RTE_IP_VP4630} + +Test Flash Probe On RTE v1.1.0 + Sonoff + [Documentation] Check if flash on MSI Z690 can be probed. This covers + ... a case with v1.1.0 revision of RTE hardware, where SPI + ... lines can be controlled, and Sonoff is used for power + ... control. + Run Flash Probe ${RTE_IP_Z690} + +Test Flash Probe On RTE with CH341A programmer + [Documentation] Check if flash on APU2 can be probed. This covers a case + ... with CH341 as a programmer, instead of RTE. + Run Flash Probe ${RTE_IP_V1410} + +Test Flash Probe - Asset Checked Out Manually + [Documentation] If we have checked out the device manually before using + ... osfv_cli, the device should remain checked out on us, + ... not checked in after the script finishes execution. + Check Out ${RTE_IP_APU2} + ${probe_result}= Run Flash Probe ${RTE_IP_APU2} + Should Match + ... ${probe_result.stdout} + ... *Since the asset ${ASSET_ID_APU2} has been checkout manually by you prior running this script, it will NOT be checked in automatically.* + Should Match + ... ${probe_result.stdout} + ... *Please return the device when work is finished.* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_APU2}* + +Test Flash Probe - Asset Checked Out Automatically + [Documentation] If we have not checked out the device manually before using + ... osfv_cli, the it should be automaticlaly checked out for + ... the duration of the script, and checked in once the script + ... finishes execution. + Check In ${RTE_IP_APU2} + ${probe_result}= Run Flash Probe ${RTE_IP_APU2} + Should Match + ... ${probe_result.stdout} + ... *Since the asset ${ASSET_ID_APU2} has been checkout automatically by this script, it is automatically checked in as well.* + Should Match ${probe_result.stdout} *Asset ${ASSET_ID_APU2} successfully checked in.* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Not Match ${list_result.stdout} *Asset ID: ${ASSET_ID_APU2}* + +Test Flash Probe - Asset Checked Out By Someone Else + [Documentation] If the assed has been checked out by someone else, the script + ... should stop execution to prevent hardware conflict. + Check In ${RTE_IP_APU2} + # Check out device to robot user + ${checkout_result}= Check Out ${RTE_IP_APU2} snipeit_config=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${checkout_result.stdout} Asset ${ASSET_ID_APU2} successfully checked out* + ${list_result}= Run Process + ... osfv_cli + ... snipeit + ... list_my + ... env:SNIPEIT_CONFIG_FILE_PATH=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_APU2}* + + # Probing should fail + ${probe_result}= Run Process osfv_cli rte --rte_ip ${RTE_IP_APU2} flash probe + Should Match ${probe_result.stdout} *Error checking out asset ${ASSET_ID_APU2}* + Should Match + ... ${probe_result.stderr} + ... *Exiting to avoid conflict. Check who is working on this device and contact them first.* + + # Device should not be checked out on me + ${list_result}= Run Process osfv_cli snipeit list_my + Should Not Match ${list_result.stdout} *Asset ID: ${ASSET_ID_APU2}* + + # Device should be checked out on the original owner + ${list_result}= Run Process + ... osfv_cli + ... snipeit + ... list_my + ... env:SNIPEIT_CONFIG_FILE_PATH=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_APU2}* diff --git a/osfv_cli/test/snipeit.robot b/osfv_cli/test/snipeit.robot new file mode 100644 index 0000000..09e80f6 --- /dev/null +++ b/osfv_cli/test/snipeit.robot @@ -0,0 +1,102 @@ +*** Settings *** +Documentation The goal of this suite is to evaluate the most important features +... of the "snipeit" command, and the "snipeit.py" library as a consequence as +... well. + +Library Process +Resource ../test/common/keywords.robot + +Test Setup Run Keyword Check In Test Device +Test Teardown Run Keyword Check In Test Device + + +*** Variables *** +${RTE_IP_TEST_DEVICE}= ${RTE_IP_FW4C} +${ASSET_ID_TEST_DEVICE}= ${ASSET_ID_FW4C} + + +*** Test Cases *** +Check Out Unused Device + [Documentation] If device was unused, check out should be successful. + ${checkout_result}= Check Out ${RTE_IP_TEST_DEVICE} + Should Match ${checkout_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked out* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + +Check Out Used Device (by me) + [Documentation] If device was already checked out by me, it should stay + ... checked out this way. No error should be returned. + ${checkout_result}= Check Out ${RTE_IP_TEST_DEVICE} + Should Match ${checkout_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked out* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + +Check Out Used Device (by someone else) + [Documentation] If device was already checked out by someone else, it + ... should stay checked out this way. An error should be + ... returned that devices is already used by someone else. + # Check out device to robot user + ${checkout_result}= Check Out ${RTE_IP_TEST_DEVICE} snipeit_config=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${checkout_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked out* + ${list_result}= Run Process + ... osfv_cli + ... snipeit + ... list_my + ... env:SNIPEIT_CONFIG_FILE_PATH=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + + # Try to check out already used device + ${checkout_result}= Check Out ${RTE_IP_TEST_DEVICE} + Should Match ${checkout_result.stdout} *Error checking out asset ${ASSET_ID_TEST_DEVICE}* + Should Match ${checkout_result.stdout} *That asset is not available for checkout!* + +Check In Unused Device + [Documentation] If device was unused, we should be informed that there is + ... no need to check in, as the device is not used. + ${checkin_result}= Check In ${RTE_IP_TEST_DEVICE} + Should Match ${checkin_result.stdout} *Error checking in asset ${ASSET_ID_TEST_DEVICE}* + Should Match ${checkin_result.stdout} *hat asset is already checked in.* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Not Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + +Check In Used Device (by me) + [Documentation] If device was used by me, check in should be successful. + # Check out device + ${checkout_result}= Check Out ${RTE_IP_TEST_DEVICE} + Should Match ${checkout_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked out* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + + # Check in device + ${checkin_result}= Check In ${RTE_IP_TEST_DEVICE} + Should Match ${checkin_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked in* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Not Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + +Check In Used Device (by someone else) + [Documentation] If device was used by someone else, check in should fail. + ... An error should be returned that the device is already + ... used bye someone else. + # Check out device to robot user + ${checkout_result}= Check Out ${RTE_IP_TEST_DEVICE} snipeit_config=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${checkout_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked out* + ${list_result}= Run Process + ... osfv_cli + ... snipeit + ... list_my + ... env:SNIPEIT_CONFIG_FILE_PATH=%{HOME}/.osfv-robot/snipeit.yml + Should Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + + # Check in device as me - should fail (?) + ${checkin_result}= Check In ${RTE_IP_TEST_DEVICE} + Should Match ${checkin_result.stdout} Asset ${ASSET_ID_TEST_DEVICE} successfully checked in* + ${list_result}= Run Process osfv_cli snipeit list_my + Should Not Match ${list_result.stdout} *Asset ID: ${ASSET_ID_TEST_DEVICE}* + + +*** Keywords *** +Check In Test Device + [Documentation] Make sure that at the start of each case, device is + ... checked in by both users. + Check In ${RTE_IP_TEST_DEVICE} + Check In ${RTE_IP_TEST_DEVICE} snipeit_config=%{HOME}/.osfv-robot/snipeit.yml