Merge branch 'master' into 187809206-add-tolerance-statements

.vscode/launch.json

@@ -14,6 +14,15 @@
       "console": "integratedTerminal",
       "program": "${workspaceFolder}/node_modules/.bin/jest",
       "args": ["${fileBasenameNoExtension}"]
+    },
+    {
+      "name": "Debug functions-v2 test",
+      "request": "launch",
+      "type": "node",
+      "console": "integratedTerminal",
+      "program": "${workspaceFolder}/functions-v2/node_modules/.bin/jest",
+      "args": ["${fileBasenameNoExtension}"],
+      "cwd": "${workspaceFolder}/functions-v2"
     }
   ]
 }

README.md

@@ -59,109 +59,49 @@ To deploy a production release:
 
 ## Developing/deploying cloud functions
 
-CLUE uses several Google cloud functions to implement certain features that would be difficult (or impossible) to implement entirely client-side.
-|Function|Purpose|
-|--------|-------|
-|_getImageData_|Retrieves image data that may reside in other classes and hence is not accessible client-side, e.g. for supports published to multiple classes or documents retrieved via the teacher network.|
-|_getNetworkDocument_|Retrieves the contents of a document accessible to a teacher via the teacher network.|
-|_getNetworkResources_|Retrieves the list of resources (documents) available to a teacher via the teacher network.|
-|_postDocumentComment_|Posts a comment to a document in firestore, adding metadata for the document to firestore if necessary.|
-|_publishSupport_|Publishes a document as a support that is accessible to all of a teacher's classes (including any referenced images).|
-|_validateCommentableDocument_|Checks whether a specific commentable document exists in firestore and creates it if necessary.|
-
-The code for the functions is in the `functions` directory. You should be able to cd into the
-`functions` directory and perform basic development operations:
-```
-$ cd functions
-$ npm install     # install local dependencies
-$ npm run lint    # lint the functions code
-$ npm run test    # runs jest (unit) tests for the functions code
-$ npm run build   # build the functions code (transpile TypeScript)
-```
-### Note 1
-There seems to be an uneasy relationship between the `node_modules` folder in the
-`functions` directory and the one in the parent directory. I had to explicitly specify the
-path to typescript in the `build` function. There's probably a better configuration available,
-but in the meantime this seems to mostly work.
-
-### Note 2
-When running `npm run test` with node 16, the following error is shown
-```
-TypeError: Cannot read properties of undefined (reading 'INTERNAL')
-```
-This error is triggered by the following line in `test-utils.ts`
-```
-import { useEmulators } from "@firebase/rules-unit-testing";
-```
-The current work around is to use node 14 to run the tests.
+CLUE uses several Google cloud functions to implement certain features that would be difficult (or impossible) to implement entirely client-side. There are two folders of functions `functions-v1` and `functions-v2`. We are trying to incrementally migrate the v1 functions into the v2 folder.
 
-See functions/dependency-notes.md for more on this.
+Each folder has its own readme:
+- [functions-v2](functions-v2/README.md)
+- [functions-v1](functions-v1/README.md)
 
-### Testing cloud functions
-
-Google recommends (requires?) that [firebase-tools](https://www.npmjs.com/package/firebase-tools) be installed globally:
-```
-$ npm install -g firebase-tools
-```
-This should be run periodically to make sure you're running the latest version of the tools.
+## Testing/Deploying database rules
 
-#### Running tests locally (without running functions in the emulator)
-```
-$ npm run serve                 # build and then start the emulators
-$ npm run test                  # run all tests in `functions` directory
-$ npm run test -- some.test.ts  # run a particular test
-```
-The existing tests currently work this way. They test the basic functionality of the cloud functions by importing and calling them directly from node.js test code. This is a simple and efficient way of testing the basic functionality without all the overhead of the functions emulator. The downside is that the node.js test environment is not the same as the hosted function environment. For instance, it's possible to return objects in node.js that can't be JSON-stringified which will throw an error when the function is hosted. That said, you can't beat the convenience of simply calling the functions directly.
+### Requirements
 
-#### Running local tests against functions hosted in the emulator
-To run jest tests against functions running in the emulator requires [serving functions using a Cloud Functions Shell](https://firebase.google.com/docs/functions/local-shell#serve_functions_using_a_cloud_functions_shell). Currently, all of our functions are `HTTPS Callable` functions, which [can be called](https://firebase.google.com/docs/functions/local-shell#invoke_https_callable_functions) in this shell mode, but:
->Emulation of context.auth is currently unavailable.
+- The tests currently only run with Node.js version 16.x
+- You need the firebase CLI. Version 12 is compatible with Node 16: `npm install -g firebase-tools@12`
+- You should be logged in to firebase: `firebase login`
 
-#### Running CLUE against functions running locally in the emulator:
-```
-$ npm run serve   # build and then start the functions emulator
-```
-and launch CLUE with url parameter `functions=emulator`.
+Java is also required for running the emulators. There are various ways to install it; I did this:
 
-### To deploy firebase functions to production:
-```
-$ npm run deploy                        # deploy all functions
-$ npm run deploy:getImageData           # deploy individual function
-$ npm run deploy:postDocumentComment    # deploy individual function
+```shell
+brew install java
+echo 'export PATH="/opt/homebrew/opt/openjdk/bin:$PATH"' >> ~/.zshrc
 ```
 
-By convention, our firebase functions have an internal version number that is returned with any results. This should be incremented appropriately when new versions are deployed. This will allow us to determine whether the current code in GitHub has been deployed or not, for instance. Also by convention, our firebase functions accept parameters of `{ warmUp: true }` which can be issued in advance of any actual call to mitigate the google cloud function cold-start issue.
-
-### Serving CLUE from https://localhost
-To test the deployed function(s) from your local development environment, you may need to run your local dev server with https to avoid CORS errors. To do so, [create a certificate](https://www.matthewhoelter.com/2019/10/21/how-to-setup-https-on-your-local-development-environment-localhost-in-minutes.html) in your `~/.localhost-ssl` directory and name the files `localhost.pem` and `localhost.key`. To use the certificate:
-```
-$ npm run start:secure
-```
-
-## Testing/Deploying database rules
-
-### Requirements:
-
- * You should install the firebase CLI via: `npm install -g firebase-tools`
- * You should be logged in to firebase: `firebase login`
-
 Firestore security rules are unit tested and realtime database rules could be with some additional work.
 
 ### To test database rules
-```
-$ cd firebase-test
-$ npm run test
+
+The emulator must be running when the test is invoked.
+
+```shell
+cd firebase-test
+npm run start &
+npm run test
 ```
 
+### To deploy database rules
+
 You deploy firebase functions and rules directly from the working directory using
 the `firebase deploy` command. You can see `firebase deploy help` for more info.
 
 See which project you have access to and which you are currently using via: `firebase projects:list`
 
-### To deploy database rules:
-```
-$ npm run deploy:firestore:rules    # deploys firestore rules
-$ npm run deploy:firebase:rules     # deploys firebase (realtime database) rules
+```shell
+npm run deploy:firestore:rules    # deploys firestore rules
+npm run deploy:firebase:rules     # deploys firebase (realtime database) rules
 ```
 
 ## Debugging
@@ -174,6 +114,7 @@ To enable per component debugging set the "debug" localstorage key with one or m
 - `docList` - this will print a table of information about a list of documents
 - `document` this will add the active document as `window.currentDocument`, you can use MST's hidden toJSON() like `currentDocument.toJSON()` to views its content.
 - `drop` console log the dataTransfer object from drop events on the document.
+- `firestore` turn on Firestore's internal debugging, this logs all queries to Firestore.
 - `history` this will: print some info to the console as the history system records changes, print the full history as JSON each time it is loaded from Firestore, and provide a `window.historyDocument` so you can inspect the document while navigating the history.
 - `images` this will set `window.imageMap` so you can look at the status and URLs of images that have been loaded.
 - `listeners` console log the adding, removing, and firing of firebase listeners
@@ -214,7 +155,6 @@ There are a number of URL parameters that can aid in testing:
 |`fakeClass`     |string                   |Class id for demo, qa, or test modes.|
 |`fakeUser`      |`(student\|teacher):<id>`|Configure user type and (optionally) id.|
 |`qaGroup`       |string                   |Group id for qa, e.g. automated tests.|
-|`qaClear`       |`all\|class\|offering`   |Extent of database clearing for automated tests.|
 |`firebase`      |`emulator\|<URL>`        |Target emulator for firebase realtime database calls.|
 |`firestore`     |`emulator\|<URL>`        |Target emulator for firestore database calls.|
 |`functions`     |`emulator\|<URL>`        |Target emulator-hosted firebase functions.|
@@ -242,11 +182,11 @@ The Standalone Document Editor also supports a `readOnly` url param. If you spec
 
 ### QA
 
-Along with `dev`, `test`, `authed` and `demo` modes the app has a `qa` mode.  QA mode uses the same parameters as demo mode with two additional parameters:
+Along with `dev`, `test`, `authed` and `demo` modes the app has a `qa` mode.  QA mode uses the same parameters as demo mode with one additional parameter:
+
+qaGroup - the group to automatically assign the fake user to after connecting to the database.
 
-1. qaGroup - the group to automatically assign the fake user to after connecting to the database.
-2. qaClear - either "all", "class" or "offering".  When this parameter is present the QA database is cleared at the level requested based on the user parameters.
-   This is useful to clear data between automated QA runs.  When complete the app will display `<span className="qa-clear">QA Cleared: OK</span>`.
+Additionally in `qa` mode the "root" in Firestore and the Realtime database is based on the Firebase user uid. This user is stored in session storage so each new tab will start a new root. In Cypress session storage is cleared between tests so each new test will have its own root.
 
 ### To run Cypress integration tests:
 - `npm run test:local`
@@ -268,7 +208,6 @@ implementation simpler. Existing commands at the moment:
 
     - setupGroup
     - upLoadFile
-    - clearQAData
 
 ## License
 

cypress.config.ts

@@ -35,6 +35,7 @@ export default defineConfig({
   qaMothPlotUnitStudent5: "/?appMode=qa&fakeClass=5&fakeUser=student:5&qaGroup=5&problem=1.1&unit=./demo/units/qa-moth-plot/content.json",
   qaNoSectionProblemTabUnitStudent5: "/?appMode=qa&fakeClass=5&fakeUser=student:5&qaGroup=5&problem=1.1&unit=./demo/units/qa-no-section-problem-tab/content.json",
   clueTestqaUnitStudent5: "/?appMode=demo&demoName=CLUE-Test&fakeClass=5&fakeUser=student:5&problem=1.1&unit=./demo/units/qa/content.json&noPersistentUI",
+  clueTestNoUnitStudent5: "/?appMode=demo&demoName=CLUE-Test&fakeClass=5&fakeUser=student:5&problem=1.1&noPersistentUI",
   clueTestqaUnitTeacher6: "/?appMode=demo&demoName=CLUE-Test&fakeClass=5&fakeUser=teacher:6&problem=1.1&unit=./demo/units/qa/content.json&noPersistentUI",
   clueTestqaConfigSubtabsUnitTeacher6: "/?appMode=demo&demoName=CLUE-Test&fakeClass=5&fakeUser=teacher:6&problem=1.1&unit=qa-config-subtabs&noPersistentUI",
   e2e: {

cypress/e2e/cleanup/remove_teacher_comment_spec.js

@@ -56,7 +56,6 @@ function beforePortalTest(url, clueTeacher, reportUrl) {
 
 function beforeTest() {
   const queryParams = `${Cypress.config("qaUnitTeacher6Network")}`;
-  cy.clearQAData('all');
   cy.visit(queryParams);
   cy.waitForLoad();
   cy.openTopTab("problems");
@@ -78,7 +77,7 @@ describe('Delete Teacher Comments In chat panel', () => {
       // Teacher 1 tile comment
       chatPanel.deleteTeacherComments();
     });
-    
+
     cy.log("login teacher2 and setup clue chat");
     cy.logout(portalUrl);
     beforePortalTest(portalUrl, clueTeacher2, reportUrl2);
@@ -93,7 +92,7 @@ describe('Delete Teacher Comments In chat panel', () => {
       cy.clickProblemResourceTile(tab.sectionCode);
       // Teacher 2 tile comment
       chatPanel.deleteTeacherComments();
-    });  
+    });
   });
   it('Delete chat panel comment tags', () => {
     beforeTest();
@@ -102,7 +101,7 @@ describe('Delete Teacher Comments In chat panel', () => {
     cy.openTopTab("problems");
     cy.openProblemSection("Introduction");
     chatPanel.deleteTeacherComments();
-    
+
     cy.log('Delete comment tags on tile comment');
     cy.openTopTab("problems");
     cy.clickProblemResourceTile('introduction');

cypress/e2e/functional/document_tests/bookmark_test_spec.js

@@ -12,7 +12,6 @@ const queryParams1 = `${Cypress.config("qaConfigSubtabsUnitStudent5")}`;
 const queryParams2 = `${Cypress.config("qaConfigSubtabsUnitTeacher1")}`;
 
 function beforeTest(params) {
-  cy.clearQAData('all');
   cy.visit(params);
   cy.waitForLoad();
 }
@@ -76,7 +75,7 @@ context('Bookmarks', function () {
     // resourcesPanel.getCanvasStarIcon('class-work', 'workspaces', copyDocumentTitle).should('have.class', 'starred');
     // cy.openSection('class-work', 'bookmarks');
     // resourcesPanel.getCanvasItemTitle('class-work', 'bookmarks').contains(copyDocumentTitle).should('exist');
-  })
+  });
   it('Test bookmarks for teacher', function () {
     beforeTest(queryParams1);
     let copyDocumentTitle = 'copy Investigation';
@@ -120,5 +119,5 @@ context('Bookmarks', function () {
     resourcesPanel.getCanvasStarIcon('class-work', 'workspaces', title).should('have.class', 'starred');
     cy.openSection('class-work', 'bookmarks');
     resourcesPanel.getCanvasItemTitle('class-work', 'bookmarks').contains(title).should('exist');
-  })
-})
+  });
+});

cypress/e2e/functional/document_tests/canvas_test_spec.js

@@ -28,7 +28,6 @@ const queryParams4 = "?appMode=demo&demoName=BrokenDocs&fakeClass=1&fakeUser=stu
 const title = "QA 1.1 Solving a Mystery with Proportional Reasoning";
 
 function beforeTest(params) {
-  cy.clearQAData('all');
   cy.visit(params);
   cy.waitForLoad();
 }

cypress/e2e/functional/document_tests/copy_doc_test_spec.js

@@ -31,7 +31,6 @@ let canvas = new Canvas;
 
 function beforeTest() {
   const queryParams = `${Cypress.config("qaUnitStudent5")}`;
-  cy.clearQAData('all');
   cy.visit(queryParams);
   cy.waitForLoad();
 }

cypress/e2e/functional/document_tests/exemplar_test_spec.js

@@ -18,7 +18,6 @@ const exemplarName = "First Exemplar";
 const exemplarInfo = "Ivan Idea: First Exemplar";
 
 function beforeTest(params) {
-  cy.clearQAData('all');
   cy.visit(params);
   cy.waitForLoad();
 }
@@ -53,6 +52,41 @@ context('Exemplar Documents', function () {
     clueCanvas.getStickyNotePopup().should("not.exist");
   });
 
+  it('Exemplars show up in the correct place in the sort work view', function () {
+    beforeTest(queryParams2);
+    cy.openTopTab('sort-work');
+
+    // With no secondary sort, the full exemplar tile should show up in the right sections.
+    sortWork.openSortWorkSection("No Group");
+    sortWork.checkDocumentInGroup("No Group", exemplarName);
+
+    sortWork.getPrimarySortByMenu().click();
+    sortWork.getPrimarySortByNameOption().click();
+    sortWork.openSortWorkSection("Idea, Ivan");
+    sortWork.checkDocumentInGroup("Idea, Ivan", exemplarName);
+
+    sortWork.getPrimarySortByMenu().click();
+    sortWork.getPrimarySortByTagOption().click();
+    sortWork.openSortWorkSection("Varies Material/Surface");
+    sortWork.checkDocumentInGroup("Varies Material/Surface", exemplarName);
+
+    sortWork.getPrimarySortByMenu().click();
+    sortWork.getPrimarySortByBookmarkedOption().click();
+    sortWork.openSortWorkSection("Not Bookmarked");
+    sortWork.checkDocumentInGroup("Not Bookmarked", exemplarName);
+
+    sortWork.getPrimarySortByMenu().click();
+    sortWork.getPrimarySortByToolsOption().click();
+    sortWork.openSortWorkSection("Text");
+    sortWork.checkDocumentInGroup("Text", exemplarName);
+
+    // With a secondary sort, "simple documents" (little boxes) should show up for exemplars.
+
+    sortWork.getSecondarySortByMenu().click();
+    sortWork.getSecondarySortByNameOption().click();
+    sortWork.checkSimpleDocumentInSubgroup("Text", "Idea, Ivan", exemplarInfo);
+  });
+
   it('Unit with exemplars hidden initially, revealed 3 drawings and 3 text tiles', function () {
     beforeTest(queryParams1);
     cy.openTopTab('sort-work');

cypress/e2e/functional/document_tests/group_chooser_spec.js

@@ -20,10 +20,6 @@ const defaultSetupOptions = {
   problem
 };
 
-function beforeTest() {
-  cy.clearQAData('all');
-}
-
 function setup(student, opts = {}) {
   const options = { ...defaultSetupOptions, ...opts };
   cy.visit('/?appMode=qa&fakeClass=' + fakeClass + '&fakeUser=student:' + student + '&problem=' + options.problem + '&unit=./demo/units/qa/content.json');
@@ -40,8 +36,6 @@ function setup(student, opts = {}) {
 
 context('Test student join a group', function () {
   it('Test student join a group', function () {
-    beforeTest();
-
     cy.log('Student 1 will join and will verify Join Group Dialog comes up with welcome message to correct student');
     setup(student1);
     cy.get('.app > .join > .join-title').should('contain', 'Join Group');

cypress/e2e/functional/document_tests/group_test_readonly_spec.js

@@ -25,7 +25,6 @@ context('Test group functionalities', function () {
   it('4-up view read-only', function () {
 
     cy.log('students to check each others tiles in 4-up view read-only');
-    cy.clearQAData('all');
 
     setupTest(0);
     setupTest(1);

cypress/e2e/functional/document_tests/group_test_spec.js

@@ -45,7 +45,6 @@ context('Test group functionalities', function () {
   it('4-up view tests', function () {
 
     cy.log('will set up groups');
-    cy.clearQAData('all');
     setupTest(0);
     setupTest(1);
     setupTest(2);

cypress/e2e/functional/document_tests/header_test_spec.js

@@ -4,7 +4,6 @@ const header = new Header;
 
 function beforeTest() {
   const queryParams = `${Cypress.config("qaUnitStudent5")}`;
-  cy.clearQAData('all');
   cy.visit(queryParams);
   cy.waitForLoad();
 }

cypress/e2e/functional/document_tests/nav_panel_test_spec.js

@@ -17,7 +17,6 @@ const queryParams4 = `${Cypress.config("qaNoSectionProblemTabUnitStudent5")}`;
 const queryParams5 = `${Cypress.config("qaConfigSubtabsUnitStudent5")}`;
 
 function beforeTest(params) {
-  cy.clearQAData('all');
   cy.visit(params);
   cy.waitForLoad();
 }

cypress/e2e/functional/document_tests/student_teacher_4up_readonly_spec.js

@@ -144,8 +144,6 @@ function setupTestBrain(studentIndex) {
 context('Test 4-up and 1-up views tiles read only functionalities', function () {
   it('4-up and 1-up views read-only text, table, geometry, drawing, expression, numberline, image, datacard tiles', function () {
 
-    cy.clearQAData('all');
-
     setupTest(0);
     setupTest(1);
 
@@ -174,8 +172,6 @@ context('Test 4-up and 1-up views tiles read only functionalities', function ()
   });
   it('4-up and 1-up views read-only dataflow, expression, xy plot tiles', function () {
 
-    cy.clearQAData('all');
-
     setupTestBrain(0);
     setupTestBrain(1);