twister: pytest: Bluetooth: multi harness boards test against DuT #83641
Conversation
zephyrbot
added
area: Bluetooth
area: Bluetooth Host
zephyrbot
requested review from
alwa-nordic,
cvinayak,
gchwier,
golowanow,
hakehuang,
hermabe,
jhedberg,
kartben,
LukaszMrugala,
nashif,
PerMac,
rugeGerritsen,
sjanc,
Thalley and
theob-pro
Add one harness_devices pytest fixture, Which will build and flash different harness apps into multi harness boards, then init all harness devices as dut for tests. Add tests for central_ht and peripehral_ht apps with this feature. Signed-off-by: Jingsai Lu <[email protected]>
| from twister_harness.helpers.domains_helper import ZEPHYR_BASE | ||
| sys.path.insert(0, os.path.join(ZEPHYR_BASE, 'scripts')) # import zephyr_module in environment.py | ||
| sys.path.insert(0, os.path.join(ZEPHYR_BASE, 'scripts', 'pylib', 'twister')) | ||
| from twisterlib.hardwaremap import HardwareMap |
There was a problem hiding this comment.
this is not good, why fixture mixed with hardware map which supposed to be platform agnostic
There was a problem hiding this comment.
hi hake, thanks for these review and comments,
I put the harness devices configuration info in another yaml, and want to import and reuse twister HardwareMap class to load and parse this new yaml,
for the harness boards info, do you have any suggestion, where/which file to store/manage those info ?
all the boards in the current dut_map.yaml will be recognized as duts and scheduled for tests in parallels.
| harness_device_yml = request.config.getoption('--harness_device_map') | ||
| harness_apps = request.config.getoption('--harness_apps') | ||
| logger.info(f'harness_device_yml:{harness_device_yml}') | ||
| logger.info(f'harness_apps:{harness_apps}') |
There was a problem hiding this comment.
please update the twister doc to explain the usage of this harness
| extra_configs: | ||
| - CONFIG_NXP_MONOLITHIC_NBU=n | ||
| harness_config: | ||
| pytest_dut_scope: session | ||
| pytest_args: ['--harness_apps=zephyr/samples/bluetooth/central_ht'] |
There was a problem hiding this comment.
seeing from below code, the harness_apps is a list, so need define list like
pytest_args:
harness_apps:
- "zephyr/samples/bluetooth/central_ht"
by the way, why start from zephyr?
| extra_config = ['-' + config for config in extra_config] | ||
| # build harness_app image for harness device | ||
| build_dir = f'./harness_{harness_hw.platform}_{os.path.basename(appname)}' | ||
| cmd = ['west', 'build', appname, '-b', harness_hw.platform, '--build-dir', build_dir] + extra_config |
There was a problem hiding this comment.
this could be update in CMakefile as dependency. refer to samples/subsys/ipc/openamp/remote
There was a problem hiding this comment.
thanks hake, will check how to do this with CMakefile system for the dependency board settings
| # Copyright (c) 2025 NXP | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import logging | ||
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def test_os_boot(dut, harness_devices): | ||
| dut.readlines_until('Booting Zephyr OS build', timeout=5) | ||
| harness_devices[0].readlines_until('Booting Zephyr OS build', timeout=5) | ||
|
|
||
|
|
||
| def test_bluetooth_boot(dut, harness_devices): | ||
| dut.readlines_until('Advertising successfully started', timeout=5) | ||
| harness_devices[0].readlines_until('Scanning successfully started', timeout=5) | ||
|
|
||
|
|
||
| def test_bluetooth_connection(dut, harness_devices): | ||
| harness_devices[0].readlines_until('Connected', timeout=5) | ||
|
|
||
|
|
||
| def test_match_temp_value_over_ble(dut, harness_devices): | ||
| dut.readlines_until('Indication success', timeout=5) | ||
| harness_devices[0].readlines_until(r'Temperature \d{1,2}C', timeout=5) |
There was a problem hiding this comment.
Will the test case be running as part of PR CI, nightly or weekly on the upstream main branch? (if not, these test vectors can get stale when someone change the strings).
Add one harness_devices pytest fixture,
Which will build and flash different harness apps into multi harness boards, then init all harness devices as dut for tests.
Add tests for central_ht and peripehral_ht apps with this feature.
Test on frdm_mcxw71 with command:
west twister -vvv --platform frdm_mcxw71 -T zephyr/samples/bluetooth/central_ht --device-testing --hardware-map dut_map.yml --pytest-args=--harness_device_map=harness_device_map.yml
dut_map.yml
harness_device_map.yml