Cypress code coverage (#623)

* start code coverage * excluded tests files * codecov * ran style and added comment section in codecov.yml * fix api key test * Added codecov * Update codecov * fix font change * Updated tests * updated cypress tests * uploaded cypress test #11 * ran style * lightmode working * added layout regions * changed back applitools keys * fixed layout regions * node modal test * updated test files * removed CTRL and DEBUG from cy test * Added node edit test * fixed merge conflict * updated settings test * modified canvas btn test-id * updated plotly nodes tests * fixed tests and removed jest test * restore jest tests * dropdown test * node change * style * added run script test * edit s3modal * removed console.error() if statements * excluded web-socket and uncommented jest test * working on paramfield * fixed run script test * paramfield test * edit paramfield * test edit modal * update editmodal * edit number * update edit * fix the path * adjust ci * modified ci * changed to nyc * working on custom node test * removed --cov flag * fix path * included codecov token * added custom nodes test * fixed test * change report file * excluded web-socket from nyc and fixed test #17 * cleaned up tests and added comments * excluded app.tsx, errorpage.tsx * included .ts files * removed unexposed files * attempting to add cypress flag * update codecov yml * update file name for codecov * style * update codecov * validated codecov.yaml * Update codecov * excluded nodes folder * fixed nodes folder exclusion * excluded nodes folder * Updated config * omitting nodes folder * added .coveragerc * fixed .coveragerc filepath * Update tests * fixed filepath * restored filepath * Updated tests * updated tests * added pytest.ini * fixed nodes folder exclusion * Updated comments * cypress documentation * Updated doc Co-authored-by: Youngsun Yoo <[email protected]> * Update test files and doc * Updated docs * added description lines for few tests * edit codecov file * format --------- Co-authored-by: Youngsun Yoo <[email protected]> Co-authored-by: Youngsun Yoo <[email protected]>
flojoy-ai · Jul 5, 2023 · dcb4406 · dcb4406
1 parent 378e511
commit dcb4406
Show file tree

Hide file tree

Showing 41 changed files with 1,514 additions and 2,541 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -12,40 +12,6 @@ env:
   APPLITOOLS_BATCH_ID: ${{ github.event.pull_request.head.sha || github.sha }}
 
 jobs:
-  jest-tests:
-    runs-on: ubuntu-latest
-
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          submodules: recursive
-
-      - uses: actions/setup-node@v3
-        with:
-          node-version: 16
-
-      - uses: actions/setup-python@v4
-        with:
-          python-version: "3.10"
-          cache: "pip"
-
-      - name: Start Redis
-        uses: supercharge/[email protected]
-        with:
-          redis-version: 4
-          redis-port: 6379
-
-      - name: Run Flojoy startup script
-        run: bash flojoy -c
-
-      - name: Run Jest tests
-        run: npm run test
-
-      - name: Codecov report
-        uses: codecov/codecov-action@v3
-        with:
-          flags: typescript
-
   code-style-check:
     runs-on: ubuntu-latest
     steps:
@@ -85,7 +51,7 @@ jobs:
       - name: pytest
         run: |
           cd PYTHON
-          python -m pytest --cov --cov-report=xml
+          python -m pytest --cov --cov-report=xml --ignore=nodes
 
       - name: Codecov report
         uses: codecov/codecov-action@v3
@@ -125,3 +91,10 @@ jobs:
           start: npm run start-project
           browser: chrome
           install: false
+
+      - name: Codecov report
+        uses: codecov/codecov-action@v3
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          file: coverage-cy/coverage-final.json
+          flags: cypress
diff --git a/.nycrc b/.nycrc
@@ -10,6 +10,7 @@
     "src/**/*.cy.ts",
     "src/**/*.cy.tsx",
     "src/**/*.d.ts",
-    "src/reportWebVitals.ts"
+    "src/reportWebVitals.ts",
+    "src/web-socket/socket.ts"
   ]
 }
diff --git a/CYPRESS.md b/CYPRESS.md
@@ -0,0 +1,161 @@
+# Creating a new test
+
+When we create a new component on our frontend, we must create a cypress test in order to maintain the functionality of the code. There are certain steps that we should follow when we create a test.
+
+We should first distinguish whether or not we are going to use Applitools. Applitools is an application that takes a snapshot of our website to verify if the components are displaying correctly or not.
+If you created a component that needs a visual testing, it is probably a good idea to use Applitools. If not, you can only use cypress to test the component’s functionality.
+
+## Connecting Cypress to Applitools
+
+To connect any visual tests results from cypress to applitools:
+
+1. visit applitools
+2. click user-info icon on the very top right ⇒ click on “**My API Key”.** The key should be present if you are in the system.
+3. Next, within studio repository, search for `applitools.config.js` and paste the key over `APPLITOOLS_API_KEY`.
+
+`Note: DO NOT PUSH applitools.config.js TO DEVELOP/MAIN BRANCH`
+
+Now, you are ready to use applitools in cypress.
+
+## Template - With Applitools
+
+```ts
+/// <reference types="cypress" />
+describe("[Description of the test]", () => {
+  const layoutRegions = [
+    { selector: '[data-cy="app-status"]' },
+    { selector: '[data-cy="btn-play"]' },
+  ];
+
+  // This method performs setup before each test.
+  beforeEach(() => {
+    // Open Eyes to start visual testing.
+    // Each test should open its own Eyes for its own snapshots.
+    cy.eyesOpen({
+      // The name of the application under test.
+      // All tests for the same app should share the same app name.
+      // Set this name wisely: Applitools features rely on a shared app name across tests.
+      appName: "[Name of the repository that is being tested]",
+
+      // The name of the test case for the given application.
+      // Additional unique characteristics of the test may also be specified as part of the test name,
+      // such as localization information ("Home Page - EN") or different user permissions ("Login by admin").
+      testName: Cypress.currentTest.title,
+    });
+  });
+
+  // The interactions use typical Cypress calls,
+  // but the verifications use one-line snapshot calls with Applitools Eyes.
+  // If the page ever changes, then Applitools will detect the changes and highlight them in the Eyes Test Manager.
+  // Traditional assertions that scrape the page for text values are not needed here.
+
+  it("[Name of the test file]", () => {
+    // Test your component
+  });
+
+  // This method performs cleanup after each test.
+  afterEach(() => {
+    // Close Eyes to tell the server it should display the results.
+    cy.eyesClose();
+  });
+});
+```
+
+## Template - Without Applitools
+
+```ts
+/// <reference types="cypress" />
+
+describe("[Description of the test]", () => {
+  const layoutRegions = [
+    { selector: '[data-cy="app-status"]' },
+    { selector: '[data-cy="btn-play"]' },
+  ];
+
+  it("[Name of the test file]", () => {
+    // Test your component
+  });
+});
+```
+
+If you created a UI component, you must include `data-testid` attribute or `data-cy` attribute to test your component. By including these attributes, you are able to grab your component in cypress and test it such as click, hover, and etc.
+
+## How to use test attributes
+
+- Grab the component that you created and test its behaviour.
+
+```ts
+cy.get('[data-testid="new_component"]').click();
+```
+
+For more information, visit [here](https://docs.cypress.io/guides/end-to-end-testing/writing-your-first-end-to-end-test) and check our example.
+
+### Example
+
+```ts
+/// <reference types="cypress" />
+
+describe("Verify clear canvas button", () => {
+  const layoutRegions = [
+    { selector: '[data-cy="app-status"]' },
+    { selector: '[data-cy="btn-play"]' },
+  ];
+
+  // This method performs setup before each test.
+  beforeEach(() => {
+    // Open Eyes to start visual testing.
+    // Each test should open its own Eyes for its own snapshots.
+    cy.eyesOpen({
+      // The name of the application under test.
+      // All tests for the same app should share the same app name.
+      // Set this name wisely: Applitools features rely on a shared app name across tests.
+      appName: "studio",
+
+      // The name of the test case for the given application.
+      // Additional unique characteristics of the test may also be specified as part of the test name,
+      // such as localization information ("Home Page - EN") or different user permissions ("Login by admin").
+      testName: Cypress.currentTest.title,
+    });
+  });
+
+  // The interactions use typical Cypress calls,
+  // but the verifications use one-line snapshot calls with Applitools Eyes.
+  // If the page ever changes, then Applitools will detect the changes and highlight them in the Eyes Test Manager.
+  // Traditional assertions that scrape the page for text values are not needed here.
+
+  it("clear canvas test", () => {
+    cy.visit("/").wait(1000);
+
+    // Clear canvas
+    cy.get('[data-testid="clear-canvas-button"]').click();
+
+    cy.eyesCheckWindow({
+      tag: "dark flow page without any nodes",
+      target: "window",
+      layout: layoutRegions,
+      fully: true,
+    });
+
+    // Verify there aren't any nodes
+    cy.get('[data-testid="node-wrapper"]').should("have.length", 0);
+  });
+
+  // This method performs cleanup after each test.
+  afterEach(() => {
+    // Close Eyes to tell the server it should display the results.
+    cy.eyesClose();
+  });
+});
+```
+
+# Writing comments
+
+Make sure to describe your e2e test in 1-2 sentences in `[Description of the test]` and 1 sentence for `[Name of the test file]` sections.
+
+```ts
+describe("[Description of the test]", () => {
+  it("[Name of the test file]", () => {
+    // Test
+  });
+});
+```
diff --git a/PYTHON/nodes b/PYTHON/nodes
diff --git a/codecov.yml b/codecov.yml
@@ -1,20 +1,21 @@
 codecov:
   notify:
     after_n_builds: 2
-    wait_for_ci: yes
+    wait_for_ci: true
 
 coverage:
   status:
-    project: off
-    patch: off
-
+    patch: false
+    project: false
 flag_management:
   individual_flags:
-    - name: typescript
+    - name: cypress
       statuses:
-        - type: project
-          target: 30%
+        - target: 30.0
+          type: project
     - name: python
       statuses:
-        - type: project
-          target: 30%
+        - target: 30.0
+          type: project
+comment:
+  layout: "reach, diff, flags, files"
diff --git a/cypress/e2e/01_main_page.cy.ts b/cypress/e2e/01_main_page.cy.ts
@@ -1,6 +1,10 @@
 /// <reference types="cypress" />
 
-describe("studio", () => {
+describe("main page", () => {
+  const layoutRegions = [
+    { selector: '[data-cy="app-status"]' },
+    { selector: '[data-cy="btn-play"]' },
+  ];
   // This method performs setup before each test.
   beforeEach(() => {
     // Open Eyes to start visual testing.
@@ -22,12 +26,13 @@ describe("studio", () => {
   // but the verifications use one-line snapshot calls with Applitools Eyes.
   // If the page ever changes, then Applitools will detect the changes and highlight them in the Eyes Test Manager.
   // Traditional assertions that scrape the page for text values are not needed here.
-  it("should open flojoy studio's main page", () => {
+  it("main page", () => {
     cy.visit("/").wait(1000);
 
     cy.eyesCheckWindow({
       tag: "dark flow page",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
 
@@ -39,20 +44,24 @@ describe("studio", () => {
     cy.eyesCheckWindow({
       tag: "dark flow page with SINE menu",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
 
+    // Click add new node button
     cy.get('[data-testid="add-node-button"]').click();
     cy.eyesCheckWindow({
       tag: "dark flow page with add node sidebar",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
 
     cy.get('[data-testid="darkmode-toggle"]').click();
     cy.eyesCheckWindow({
       tag: "light flow page",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
   });

diff --git a/cypress/e2e/02_add_delete_node_page.cy.ts b/cypress/e2e/02_add_delete_node_page.cy.ts
@@ -1,19 +1,27 @@
 /// <reference types="cypress" />
+
 require("cypress-xpath");
+
 describe("Verify Add and Delete node", () => {
+  const layoutRegions = [
+    { selector: '[data-cy="app-status"]' },
+    { selector: '[data-cy="btn-play"]' },
+  ];
+
   beforeEach(() => {
     cy.eyesOpen({
       appName: "studio",
       testName: Cypress.currentTest.title,
     });
   });
 
-  it("should open flojoy studio's main page", () => {
+  it("Verify Add and Delete node", () => {
     cy.visit("/").wait(1000);
 
     cy.eyesCheckWindow({
       tag: "dark flow page",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
     // Click add node
@@ -22,6 +30,7 @@ describe("Verify Add and Delete node", () => {
     cy.eyesCheckWindow({
       tag: "dark flow page with add node sidebar",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
     //Select container Loaders
@@ -33,6 +42,7 @@ describe("Verify Add and Delete node", () => {
     cy.eyesCheckWindow({
       tag: "dark flow page with node loader",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
     // Click on added container LOADER
@@ -42,6 +52,7 @@ describe("Verify Add and Delete node", () => {
     cy.eyesCheckWindow({
       tag: "dark flow page",
       target: "window",
+      layout: layoutRegions,
       fully: true,
     });
     cy.xpath("//div[contains(text(), 'LOADER')]").should(