Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

connect to chatgpt #2419

Merged
merged 22 commits into from
Oct 8, 2024
Merged

connect to chatgpt #2419

merged 22 commits into from
Oct 8, 2024

Conversation

bgoldowsky
Copy link
Contributor

@bgoldowsky bgoldowsky commented Oct 1, 2024

Includes:

This causes a special database key to be written out when a document is updated in a unit that has AI evaluation configured. Firebase functions notice this key and put the document into a queue that goes through 2 further stages:

  • a screenshot of the document is taken.
  • the screenshot, along with a prompt, is submitted to ChatGPT, and its categorization response is added as a comment to the original document.

For now, the functions only trigger if the user is within the AI demo space, so that is required along with being in a unit that configures AI Evaluation. The only such unit at the moment is ./demo/units/qa/content.json

Copy link

cypress bot commented Oct 2, 2024

collaborative-learning    Run #13971

Run Properties:  status check passed Passed #13971  •  git commit 1b715aa2ba: Fix test
Project collaborative-learning
Run status status check passed Passed #13971
Run duration 14m 44s
Commit git commit 1b715aa2ba: Fix test
Committer Boris Goldowsky
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 1
Tests that did not run due to a developer annotating a test with .skip  Pending 3
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 112

Copy link

codecov bot commented Oct 2, 2024

Codecov Report

Attention: Patch coverage is 97.50000% with 1 line in your changes missing coverage. Please review.

Project coverage is 86.27%. Comparing base (df5f4ab) to head (1b715aa).
Report is 23 commits behind head on master.

Files with missing lines Patch % Lines
src/lib/firebase.ts 95.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2419      +/-   ##
==========================================
- Coverage   86.29%   86.27%   -0.02%     
==========================================
  Files         742      742              
  Lines       38353    38379      +26     
  Branches     9796     9801       +5     
==========================================
+ Hits        33096    33112      +16     
- Misses       4957     4967      +10     
  Partials      300      300              
Flag Coverage Δ
cypress ?
cypress-regression 78.10% <100.00%> (+0.03%) ⬆️
cypress-smoke 27.96% <74.35%> (+0.03%) ⬆️
jest 48.86% <32.50%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@scytacki scytacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good.

I left a few comments. The only one that I'd recommend addressing in this PR is changing AIEvaluation to aiEvaluation.

docs/unit-configuration.md Outdated Show resolved Hide resolved
const message = reply.parsed.discussion +
` Key Indicators: ${reply.parsed.keyIndicators.join(", ")}`;

const commentsPath = `demo/AI/documents/${docId}/comments`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This simplified path to comment might not work in all cases.

For non-networked teacher comments they are stored under uid:[someId]_[docId]. I can't remember if someId is the id of the teacher that is commenting or the student that owns the document.

For networked teacher comments they are stored under [networkName]_[docId].

The important thing to figure out is if the CLUE runtime will show both the comments stored under the [docId] and the comments under [prefix]_[docId]. If it doesn't show both the comments, then making do that is probably the right thing do. That seems safer than having this system figure out which prefix to store the comments under.

} else {
logger.info("Creating comment for", event.document);
// NOTE we are leaving the "network" and "tileId" fields empty in the comment doc.
await firestore.collection(commentsPath).add({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something general that we are going to have to deal with is that the list of strategies in the Firestore metadata documents are currently updated by the CLUE runtime instead of via a Firebase function that is monitoring the comments. So just adding this comment with a strategy is not going to update the sort work view.

So in a separate story and PR we should add a Firebase function that monitors the comments and updates the Firestore metadata. And in that same PR we can delete the client side code which is updating the strategies. There is a story about handling deleted strategies: https://www.pivotaltracker.com/story/show/188144655 This Firebase function can take care of that too.

@bgoldowsky bgoldowsky marked this pull request as ready for review October 4, 2024 16:23
Copy link
Member

@scytacki scytacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me. I think we can try deploying these functions and see what if they work. The deploy script in the functions-v2 folder will just deploy the v2 functions.

const response = await fetch(shutterbugURL,
{
method: "POST",
body: JSON.stringify({content: html, height: 1500}),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This height might be a little short for some documents, but it seems like a good place to start until we can use dynamic heights.

A local fake secret is provided for  firebase-functions-test to use.
@bgoldowsky bgoldowsky merged commit 8bef86c into master Oct 8, 2024
15 of 17 checks passed
@bgoldowsky bgoldowsky deleted the 188092997-connect-to-chatgpt branch October 8, 2024 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants