TIP Open WiFi Test Automation

Overview

This repository contains the test automation framework and scripts for TIP Open WiFi. Test Framework is implemented in pytest

The vision is to create an open test framework to ensure production grade Open Wifi Solution.

This test automation tied into our overall tooling at TIP as shown below:

General guidelines

This testing code adheres to generic pep8 style guidelines, most notably:

1. Documentation strings
2. Naming conventions
3. Sphynx docstring format

Using pytest as the test execution framework.

Using pylint for code quality monitoring.

We are using the pylint package to do the linting. Documentation for it can be found here. In general, the customizations are possible via the .pylintrc file:

1. Line length below 120 characters is fine (search for max-line-length)
2. No new line at the end of file is fine (search for missing-final-newline)
3. Multiple new lines at the end of file are fine (search for trailing-newlines)
4. Indent using 4 spaces (search for indent-string)

In future we should enforce a policy, where we cannot merge a code where the pylint scoe goes below 7:

pylint --fail-under=7 *py

the command above would produce a non-zero exit code if the score drops below 7.

TIP Controller Integration

Using Swagger Autogenerated CloudSDK Library pypi package (implemented with swagger codegen).

Reporting

Currently we use the pytest allure integration to create visual reports for the test outcomes.

Additionally we generate overview pages, e.g. like this one to consolidate multiple test reports of the same test type.

These overview pages are being deployed by this workflow and their configuration is set in this file.

Miscellaneous

1. Do not use old style string formatting: "Hello %s" % var; use f"Hello {var} instead
2. use """ in Docstrings

Useful links

https://docs.pytest.org/en/latest/example/markers.html
https://docs.pytest.org/en/latest/usage.html
http://pythontesting.net/framework/pytest/pytest-introduction/

Test status

Best Practice

1. Ensure to have a python version > 3
2. We strongly recommend to use virtualenv to ensure that your dev environment sandbox is created.

Code style

All code must be written in python 3 and conform to PEP 8 style guide. The test framework is built using pytest.

Directory Structure

├── lanforge              - /* to be migrated */
├── libs
│   ├── controller_tests  -/* Library Support for controller_tests part  */
    ├── apnos             -/* Library Support for Access Points (uses AP SSH)  */
│   ├── lanforge          -/* Library Support for LanForge Traffic Generator */
│   ├── perfecto          -/* Library Support for Perfecto Systems */
│   ├── testrails         -/* Result Visualization (will be depreciated ) */
├── tests                 -/* Pytest cases Directory */
│   ├── _basic_test_setup
│   ├── access_point_tests
│   ├── controller_tests
│   ├── e2e
      ├── advanced
      ├── basic
      ├── interOp
      ├── mdu
      |── mesh
      |── scale
    |── README.md         -/* Pytest framework and testcases information */

Test Automation Setup Instructions

# Let's assume '~/automation' directory for setting wlan-testing
cd ~/
mkdir automation
cd automation
git clone https://github.com/Telecominfraproject/wlan-testing
git clone https://github.com/Telecominfraproject/wlan-lanforge-scripts
cd wlan-testing
./sync_repos.bash
mkdir ~/.pip
echo "[global]" > ~/.pip/pip.conf
echo "index-url = https://pypi.org/simple" >> ~/.pip/pip.conf
echo "extra-index-url = https://tip-read:[email protected]/artifactory/api/pypi/tip-wlan-python-pypi-local/simple" >> ~/.pip/pip.conf
pip3 install -r requirements.txt

cd tests
vim configuration.py    # Setup your testbed data info here

# Check if everything is good 
pytest --collect-only 


# for visualizing allure reports, Install nodejs
# Set Java home

# install allure command
npm install -g allure-commandline --save-dev

Executing and Contributing new test cases

Follow instructions provided here to understand the Testing Framework, and How to run and write new tests

Issue Reporting

Please submit issues using our JIRA project here

Use the Test Automation backlog