Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#514 Implement voting on Community note #516

Merged
merged 13 commits into from
Dec 14, 2024

Conversation

keenlim
Copy link
Collaborator

@keenlim keenlim commented Dec 4, 2024

Pull Request to Deploy to UAT

##Issue Reference

Example:
Resolves #[514]

Description

Implementing voting on community note and showing the results in terms of a pie chart.

Implementation

  1. Frontend Changes: Voting frontend interface becomes an accordion form, where users are forced to select a community note category before they are able to select the messages category. To submit the form, users will need to select both the community note category and the messages category.
  2. Backend Changes: When users submit the selection of both community note category and the messages category, it will be patched onto the database and count the number of community note category users select.
  3. Results Page: A pie chart of the results that counts the number of community note category will be shown on the results page.

Testing

Additional Notes


Checklist

  • I have linked the issue above using closing keywords if this PR resolves the issue
  • I have provided a description and implementation details
  • I have tested the changes and ensured that they work

@keenlim keenlim requested a review from sarge1989 December 4, 2024 14:12
@keenlim keenlim self-assigned this Dec 4, 2024
@@ -21,6 +21,7 @@ const patchVoteRequestHandler = async (req: Request, res: Response) => {
//confirm category in body
const { category, communityNoteCategory, truthScore, reasoning, tags } =
req.body as updateVoteRequest
console.log(communityNoteCategory)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can look thru and replace console.log with logger.log from the firebase logger?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other files as well

Copy link
Collaborator

@sarge1989 sarge1989 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks generally good but comments inline in code!

The key feature addition needed is to show the links hahah.

But since you're so fast, can u help me with the following implementation as well:

  • If more than 50% of the votes cast are "unacceptable", AND more than 50% of the checkers have voted, set the CommunityNote.downvoted field in Message to true
  • When that happens, send a correction note to each user that received the community note, replying to the community note in WhatsApp. This means that we should add a communityNoteMessageId field to instances so we can track the messageId to reply to via WhatsApp.
  • The exact wording of the correction note can be set later, but it should be something like:

Sorry, our checkers have determined that the AI got that wrong!

{today's category-based reply reply}

U can add a row in the translations file for this new response hahah, just put whatever u think makes sense first, I'll clean up the words later.

Also, make sure that everything works end to end if the community note generation fails and sends u null too k. To check that, go to functions\src\definitions\common\machineLearningServer\operations.ts, in getCommunityNote function just throw an error instead of returning the this is a test. Basically, the behaviour should be exactly the same as today if this occurs.

@@ -21,6 +21,7 @@ const patchVoteRequestHandler = async (req: Request, res: Response) => {
//confirm category in body
const { category, communityNoteCategory, truthScore, reasoning, tags } =
req.body as updateVoteRequest
console.log(communityNoteCategory)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other files as well

return parts;
}

export default function CommunityCard(prop: PropType){
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can change the "CommunityCard" name to "CommunityNoteCard"? Same for categories etc. I'm always very concerned about getting the nomenclature meaningful so ppl can easily understand the code

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used liao right this one?

@@ -21,6 +21,7 @@ const patchVoteRequestHandler = async (req: Request, res: Response) => {
//confirm category in body
const { category, communityNoteCategory, truthScore, reasoning, tags } =
req.body as updateVoteRequest
console.log(communityNoteCategory)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can remove all the console.log in the code. If want, use the firebase logging functionality logger.log

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a section to show the links too? These will be the links identified by the GenAI system. Will help the checkers review the links to make sure the note is congruent with the link

@@ -308,6 +311,10 @@ async function updateCounts(
const currentCategory = after.category
let previousScore = before.truthScore
let currentScore = after.truthScore
const previousCommunityCategory = before.communityNoteCategory
console.log(previousCommunityCategory)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the console.logs here and other files hahah

@@ -162,6 +168,14 @@ const onVoteRequestUpdateV2 = onDocumentUpdated(
thresholds.endVoteAbsolute //10
))

const isAssessedUnacceptable =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this, to only switch downvoted to true if 50% have voted

@@ -225,6 +239,11 @@ const onVoteRequestUpdateV2 = onDocumentUpdated(
}
}

if (isAssessedUnacceptable) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this as well

}
}

if (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only reply community note instances under 2 circumstances

  1. downvoted tag switched to true and messageData is assessed
  2. isAssessed tag switched to true and pendingCorrection tag (new thing I added) is true
  3. Also means there's a case where we need to switch pendingCorrection to true if downvoted tag switched to true but messageData not yet assessed

@@ -1155,7 +1161,14 @@ async function respondToInstance(
await userSnap.ref.update({
numSubmissionsRemaining: FieldValue.increment(1),
})
await sendTextMessage("user", from, responseText, data.id)
if (communityNote.downvoted) {
Copy link
Collaborator

@sarge1989 sarge1989 Dec 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes shouldn't be made in respondToInstance, it should be a separate function imo. Because respondToInstance is also called whenever subsequent users send in new instances, even when they didn't get the community note in the first place. In that case, we don't want to have this correction statement. Either implement another specific function like correctCommunityNote (off my mind that's preferred), but feel free to let me know if you disagree, or otherwise we need to check isCommunityNoteSent for this instance.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also because the logic needs to set communityNote.pendingCorrection to false.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we using another function, can think about whether it makes sense to split out the line 1065 onwards of respondToInstance into another function because it'll be repeated in respondToInstance and correctCommunityNote.

@@ -52,6 +52,7 @@ const onVoteRequestUpdateV2 = onDocumentUpdated(
const messageSnap = await messageRef.get()
const beforeTags = preChangeData?.tags ?? {}
const afterTags = postChangeData?.tags ?? {}
const currentCommunityCategory = postChangeData.communityNoteCategory
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename this currentCommunityNoteCategory pls! Paiseh for being pendantic.

@sarge1989 sarge1989 merged commit 9c5d69d into feat/checkmate-v2 Dec 14, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants