Added automated tests with cypress for Schedule form under Applicatio…

[asirvadAbrahamVarghese]

…n-Settings

CP4AIOPS-14547
Pr that adds cypress tests for Schedule form(Settings > Application Settings > Settings > Schedules > Configuration > Add a new schedule)

• Validates conditional rendering with dropdown selection
• Validates adding a schedule
• Validates editing a schedule
• Validates enabling a schedule
• Validates disabling a schedule
• Validates Queueing a schedule
• Validates deleting a schedule

Tests executed across browsers

@miq-bot assign @GilbertCherrie
@miq-bot add-label cypress
@miq-bot add-label test

[GilbertCherrie]

@asirvadAbrahamVarghese Can you split the tests up into multiple tests. Try not to combine everything into 1 test. The different test cases can be:

1. Validate visibility of elements based on dropdown selections
2. Cancel button works on the form
3. Can add, edit and delete a schedule
4. Reset button works on the form
5. Can enable, disable and queue a schedule

[GilbertCherrie]

Can you change the file name to schedule.cy.js

[Fryguy]

cc @jrafanie

[jrafanie]

Note, we'll need to be careful of side effect test which modify the tables that are presented in the UI as a failure could fail to clean up after itself. This could cause other tests to fail, for example, if they are listing schedules and asserting the contents.

[asirvadAbrahamVarghese]

‘Test name’ and ‘Dummy name’ are the two possible schedules created during these tests. To handle cleanup, defined an afterEach block that checks for these names and deletes them if they exist, especially in cases where assertions might fail - 82d569885aa283acb65a0e51db59ae14aebe088f

During GHA, since these tests will run against a blank database, we shouldn’t expect any pre-existing records with those names. However, if that’s not guaranteed, we might have to consider to generate unique schedule names for each test using a combination of a base name and a timestamp or date string, with the help of Date utility functions.

[jrafanie]

It looks like some of the tests are sporadic in nature and are fairly slow. Sporadic failures are often timing issues waiting for ajax or pages to be rendered as the computer drives the browser much faster than a user would.

I don't know what's "too slow" but it would be good to note any steps in the tests that are slow and either comment on it so we can try to fix it later or see if we can make them faster.

How slow is it locally @asirvadAbrahamVarghese ?

[jrafanie]

I don't know what's "too slow" but it would be good to note any steps in the tests that are slow and either comment on it so we can try to fix it later or see if we can make them faster.

FYI, I started on some work to try to get each cypress test file to login only once and share that session with each test in the file. Unfortunately, there is other state in the session that leaks between tests in this way, but it might be something we can do if we can figure out how to share just the login and not have to do that in each test.

See: #9499

[asirvadAbrahamVarghese]

It looks like some of the tests are sporadic in nature and are fairly slow. Sporadic failures are often timing issues waiting for ajax or pages to be rendered as the computer drives the browser much faster than a user would.

I don't know what's "too slow" but it would be good to note any steps in the tests that are slow and either comment on it so we can try to fix it later or see if we can make them faster.

How slow is it locally @asirvadAbrahamVarghese ?

Yeah right @jrafanie, this can fail intermittently due to timing issues, I am looking further into that.
In local it takes 43s:

With session management implemented (as in PR #9499 ), the execution time should improve. I just ran the tests using session management.

[miq-bot]

Checked commits asirvadAbrahamVarghese/manageiq-ui-classic@cc52e2c~...82d5698 with ruby 3.1.5, rubocop 1.56.3, haml-lint 0.62.0, and yamllint
0 files checked, 0 offenses detected
Everything looks fine. 🍰

[jrafanie]

With session management implemented (as in PR #9499 ), the execution time should improve. I just ran the tests using session management.

By the way, we haven't really documented it but there are two improvements to set in your environment but they must be understood as they can cause issues

1. Disable code reloading in dev mode, by running the server with CYPRESS=true, CYPRESS=true bin/rails server or CYPRESS=true yarn cypress:run. Note, by default rails will reload changed ruby UI code and views between each request so you can make changes in files, save, and the next request will have those changes. This is expensive and not needed in cypress testing unless you're also changing UI code along with the tests. This means you need to restart your server if you change code with this env variable set.
   This is an easy way to speed up cypress by disabling code reloading manageiq#23461

2. Disable UI miq_debug notifications in cypress mode. These notifications cause some form fields to be covered leading us to use force: true on some cypress commands. Again, enable CYPRESS=true in the env
   Disable miq_debug for cypress for now #9451, fixed in 995315c

[jrafanie]

One concern to keep in mind is tests here can fail in any part of the test. You may need to ensure you're on the right page by visiting it first, before looking for the list group item.

Ultimately, I hope we can minimize the tests that make changes to what's presented on the various screens by saving or creating objects. The hope is we can "tag" these tests with "rollback" or "transaction" or something so we can transparently tell the database to begin a transaction and automatically roll it back afterwards.

[jrafanie]

cc @GilbertCherrie @elsamaryv

[jrafanie]

I know we're handling errors here. I'm curious if these are from web socket notifications. If so, I wonder if we can improve behavior and reliability by disabling web socket notifications in cypress EXCEPT in tests that are testing web socket notifications. I haven't looked at that yet cc @GilbertCherrie @elsamaryv

[jrafanie]

Along these same lines, I was noticing that we always load the 4 dashboard widgets (chart/reports) after login on each page. Part of the PR I started doing to try to share the same login between tests in the same file, #9499, I wanted to change login to load a different page after the login such as /utilization or /optimization... I wonder if we can leverage the "my settings" start page to pick a page that is a faster start page for all users:

Another option would be to customize our default dashboard for cypress to have no widgets on the loading page and switch to a dashboard with graphs/charts for tests for dashboards.

cc @Fryguy (note, I think Jason had the idea for the start page)

[asirvadAbrahamVarghese]

1. Disable code reloading in dev mode, by running the server with CYPRESS=true
   This is an easy way to speed up cypress by disabling code reloading manageiq#23461
2. Disable UI miq_debug notifications in cypress mode.

Tried with CYPRESS=true (CYPRESS=true bin/rails server, CYPRESS=true yarn cypress:open) and that too has a minimal impact in the time taken in local environment: