Added automated tests with cypress for Namespace form

[asirvadAbrahamVarghese]

CP4AIOPS-16490
Pr that adds cypress tests for Namespace form(Automation -> Embedded Automate -> Explorer -> {Any-created-domain} -> Namespace form)

• Validates add Namespace form behaviour
• Validates adding a Namespace
• Validates editing a Namespace
• Validates deleting a Namespace
• Validates edit Namespace form behaviour
• Validates duplicate namespace creation restriction

@miq-bot assign @GilbertCherrie
@miq-bot add-label cypress
@miq-bot add-label test

[miq-bot]

Checked commit asirvadAbrahamVarghese@1338c30 with ruby 3.1.7, rubocop 1.56.3, haml-lint 0.64.0, and yamllint
0 files checked, 0 offenses detected
Everything looks fine. ⭐

[jrafanie]

Seeing the method above, interceptAccordionApi, we have to be concerned with overengineering things where we try to generally handle all intercept, get, click, wait calls. I think this is fine as a one-off as you did here. As we see lots of repetition, like you saw previously, we can do methods that generalize the user's UI actions like you did in interceptAccordionApi.

[asirvadAbrahamVarghese]

I’ll need to figure out a way to handle API intercepts and waits without duplicating stuff or using aliases in a way that might make the test flaky.

[jrafanie]

Note, I'm fine with doing multiple validations in the same test function. We have to be aware these are more integration/component tests so they need not be small unit tests. See also: https://docs.cypress.io/app/core-concepts/best-practices#Creating-Tiny-Tests-With-A-Single-Assertion

If the setup is much different, it might be better to keep them seperate like they are here.

[asirvadAbrahamVarghese]

Tried clean up using api calls - b682c59

[asirvadAbrahamVarghese]

@GilbertCherrie @jrafanie @Fryguy @elsamaryv
Used @replace-with-api to tag spots to replace UI-driven data creation with API requests - b42e5f20 - Tag name not final, suggest if something else fits better - Replaced with general TODO: tag

[jrafanie]

Are all the validations here needed? Did they help you reduce sporadic test failures where some field labels weren't yet in the correct state?

Finally, validation/asserts like this are fine but please make sure that if they fail, it's not too hard to locate the problem. There's a lot of them in this function so it might be hard to tell what's the problem at a glance.

[asirvadAbrahamVarghese]

This isn’t aimed at resolving or minimizing sporadic test failures. I added these tests (Validate Add Namespace form fields, Validate Edit Namespace form fields) specifically to validate the form components, based on our earlier cypress discussion about deeper Cypress component testing.

If you feel this level of testing, where the entire component structure is verified is necessary, I can refactor them into separate modules to keep things readable and make debugging easier. I was also planning to add similar tests for other forms as well - schedule, tenant, collect-logs
If this is not required we can drop this change.

[jrafanie]

Some we check visible and enabled and others just visible but not checking disabled. I'm mentioning it here now because it's hard to tell which of these are more important and it could make reading and debugging the test difficult.

Obviously, we can't test every element on a form in every way possible so we should start with the most important tests/assertions and add others as you find them being important to check. I'll leav that to you to check which of these assertions are important.

[asirvadAbrahamVarghese]

Updated the element validation for consistency across all elements

[jrafanie]

Ok, I understand this. This is a todo/fixme to try to replace this with an API way to create the data needed for the test.

Note, we'll need to manually delete things if the afterEach() is not currently functional.

[asirvadAbrahamVarghese]

This is a todo/fixme to try to replace this with an API way to create the data needed for the test.

Yes correct.

Note, we'll need to manually delete things if the afterEach() is not currently functional.

afterEach is behaving as intended only the data creation step (domain creation) was triggering the abort error we discussed.
Deleting works correctly with the UI session, I just have to append csrf token into the request headers.

[jrafanie]

Let's keep iterating on this type of cleanup. I really don't know of a good way to do typical setup/teardown unit testing conventions since we need both the UI process and sometimes a backend worker process to create things in the UI (report preview in reports page requires a worker process to generate the preview data) so it's really difficult to hook into the database's transaction to rollback changes in a test.

The difficulty with this type of delete is:

• If errors occur anywhere in the setup or test code, the UI page could be in a different state so navigating to the right place to remove data is tricky. Is the x_button you are POST'ing to guaranteed to be there if the test failed at a random place before it got to the correct screen?

So, please verify if you have a fatal error in random places in the before and in the test, that the after still runs and is able to handle errors such as not being on the page.

We may need to find a way to do this through the API at some point so we can get away from needing to be in the correct page "state" to run these.

[asirvadAbrahamVarghese]

Is the x_button you are POST'ing to guaranteed to be there if the test failed at a random place before it got to the correct screen?

From what I've observed so far (particularly with domain data), the delete operation doesn't appear to depend on the UI state. I tried hitting the delete-domain endpoint right after logging in, as well as after switching to another menu, for instance:
"Test-Data" is a domain created manually through UI, id is 449

I’m making a cy.request to the delete-domain endpoint with ID 449 hardcoded in the URL, and I’m using the CSRF token obtained from the meta tag., right after navigating to Settings -> Application Settings instead of Automation -> Embedded Automate -> Explorer

"Test-Data" is now deleted from DB:

please verify if you have a fatal error in random places in the before and in the test, that the after still runs and is able to handle errors such as not being on the page.

Here I'm intentionally causing the test to fail after the domain is created - using an incorrect assertion.

In the afterEach hook, request is fired to the delete-domain endpoint.

Since domain named "Test_Domain" is created in beforeEach, failure to delete the domain would have caused the other test cases to fail with a duplicate name error.

This raises a follow-up question - should API access be restricted based on the current UI state, as observed during domain creation (edit abort error), while delete requests are not. Could this be a bug?

[jrafanie]

This raises a follow-up question - should API access be restricted based on the current UI state, as observed during domain creation (edit abort error), while delete requests are not. Could this be a bug?

Sorry, I didn't see this question. I'm not sure what you mean. Domain creation and domain deletion should be using miq_ae_domain_new and miq_ae_domain_delete product feature. There should be something like assert_privilege with those product features. The other features are seeded from the file here: https://github.com/ManageIQ/manageiq/blob/5f169315ad1a790d45a9a7b001f037160ca77434/db/fixtures/miq_product_features.yml#L2344-L2355

[asirvadAbrahamVarghese]

I’ve removed cy.request usage due to the overhead of retrieving and setting the csrf-token in request headers, I feel that adds unnecessary complexity to the tests...

[jrafanie]

It looks like this is for extracting the csrf token for the deletes in the afterEach below I'm ok with this for now but I do wonder if we need a generic way to do this through the actual API in the cypress tests. Thoughts on this @GilbertCherrie @elsamaryv @Fryguy

[jrafanie]

Thoughts on this @GilbertCherrie @elsamaryv @Fryguy? @asirvadAbrahamVarghese is this PR ready to go? It was failing in schedule form. I'll close and open the PR so it's tested against the latest merge point.

[asirvadAbrahamVarghese]

I’d like to drop this change for now, which uses cy.request with a CSRF token to delete data, but I’m unsure if we have a viable alternative. Not sure if we can hit our current endpoints without the token for create/delete operations. For creation, we’d also need to handle the 'Edit aborted: multiple tabs or windows' error if we reuse the same session.

For now, I’m thinking we drop this and just stick with the UI-based cleanup, What do you all think?

[jrafanie]

I'm looking at a some ability to use factories from the UI tests in cypress. I think we can either wait on that or have a workaround to manually delete stuff in the test until we can do it. If you have a simple test that needs creation and deletion, I can add them as ones to try a POC of the factory stuff for that test.

[asirvadAbrahamVarghese]

Switched to UI-based deletion for now. We can add cy.app('clean') later when it's ready, maybe in a follow-up PR.
We're only deleting the created domain(Automation -> Embedded Automate -> Explorer) for cleanup. Could be worth trying as a POC for the factory stuff you mentioned here.

[jrafanie]

LGTM. I have some concern about the csrf extraction for doing the deletes via the UI routes for miq_ae_domain_delete. I haven't had a chance to see how we can do this through the API more generically from the cypress tests.

[jrafanie]

Looks like the namespace tests are failing @asirvadAbrahamVarghese, please take a look

[jrafanie]

This looks good so far @asirvadAbrahamVarghese, is this still WIP?

[asirvadAbrahamVarghese]

This is good for now, dropping WIP.

[jrafanie]

LGTM