feat: add grading method support #299

BryanttV · 2023-11-02T22:39:05Z

Description

This PR adds a new field in the Problem settings for choosing a Grading Method. Currently, the only Grading Method is the Last Score. From now on, the new grading methods allow are:

Last Score (Default): The last score made is taken for grading.
First Score: The first score made is taken for grading.
Highest Score: The highest score made is taken for grading.
Average Score: The average of all scores made is taken for grading.

Dependencies

This PR works with the legacy Problem Editor interface, but if you use the new interface with course-authoring MFE, you need the changes in this PR.

Legacy Interface

MFE Interface

Testing Instructions

Go to Studio and create a problem component from a Unit.
In component settings you should see a new field: Grading Method. This field is a dropdown list with the Grading Methods mentioned above.
Choose a Grading Method and save changes.
From the LMS answer the problems and check with the different Grading Methods. You can also check from the progress section.

Rescoring

Go to Studio and create a problem component from a Unit. e.g. Advanced > Blank Problem. In the settings you can add this example of a problem:

<problem>
  <h3 class="hd hd-2 problem-header">Multiple Choice</h3>
  <multiplechoiceresponse>
    <choicegroup type="MultipleChoice">
      <choice correct="true">correct</choice>
      <choice correct="false">incorrect 1</choice>
      <choice correct="false">incorrect 2</choice>
    </choicegroup>
  </multiplechoiceresponse>
  
  <h3 class="hd hd-2 problem-header">CheckBox</h3>
  <choiceresponse>
    <checkboxgroup>
      <choice correct="true">correct 1</choice>
      <choice correct="true">correct 2</choice>
      <choice correct="false">incorrect</choice>
    </checkboxgroup>
  </choiceresponse>
  
  <h3 class="hd hd-2 problem-header">Drop-Down List</h3>
  <optionresponse>
    <optioninput options="('resp 1', 'resp 2', 'resp 3')" correct="resp 3"></optioninput>
  </optionresponse>
</problem>

In component settings choose a grading method, e.g. Highest score.
From the LMS answer the problems and check your score.
From Studio edit the correct answers to the problem.
From the LMS as an instructor, make the rescore of the problem in STAFF DEBUG INFO > Rescore Learner's Submission
The final score should have changed depending on the answers.

xmodule/capa_block.py

xmodule/capa/capa_problem.py

xmodule/capa_block.py

mariajgrimaldi · 2023-11-24T15:06:04Z

xmodule/capa_block.py

+
+            new_score = self.score_from_lcp(self.lcp)
+            self.score_history.append(new_score)
+            grading_strategy_handler = GradingStrategyHandler(
+                self.grading_strategy,
+                self.score_history,
+                self.max_score(),
+            )
+            score = grading_strategy_handler.get_score()
+            self.set_score(score)


Can we move this to its method? called _set_score or something like it, so we can encapsulate the functionality

mariajgrimaldi · 2023-11-24T15:09:15Z

xmodule/capa_block.py

+            grading_strategy_handler = GradingStrategyHandler(
+                self.grading_strategy,
+                self.score_history,
+                self.max_score(),
+            )
+            calculated_score = grading_strategy_handler.get_score()


Should we move this piece of code to its own method? so we can re-use it elsewhere

mariajgrimaldi · 2023-11-29T19:57:35Z

xmodule/capa/capa_problem.py

+            if 'filesubmission' in responder.allowed_inputfields and student_answers is not None:
+                results = responder.evaluate_answers(student_answers, oldcmap)
+            else:
+                results = responder.evaluate_answers(student_answers, oldcmap)


what's the difference between these two conditions?

No difference, conditional would not be necessary

can we update it then? thanks!

mariajgrimaldi · 2023-11-29T19:58:57Z

xmodule/capa_block.py


    # enforce_type is set to False here because this field is saved as a dict in the database.
    score = ScoreField(help=_("Dictionary with the current student score"), scope=Scope.user_state, enforce_type=False)
+    score_history = ListScoreField(


why do we need a new field for this?

Because if it is stored as a List field, when querying the value it will have a dictionary, and it should have an instance of Score

mariajgrimaldi · 2023-11-29T20:06:21Z

xmodule/capa_block.py

@@ -2206,13 +2266,35 @@ def update_correctness(self):
        new_correct_map = self.lcp.get_grade_from_current_answers(None)
        self.lcp.correct_map.update(new_correct_map)

+    def update_correctness_list(self):
+        """Updates the correctness map list of the LCP."""


can you explain in detail what does this do?

mariajgrimaldi · 2023-11-29T20:09:29Z

xmodule/capa_block.py

+        if new_correct_map_list:
+            self.lcp.correct_map.update(new_correct_map_list[-1])


I don't get this part in particular

After updating each correct map in correct_map_history, take the last correct map in the history and assign it to the correct_map field, to update the field with the current correct map.

mariajgrimaldi · 2023-12-01T13:33:57Z

xmodule/capa_block.py

+            self.correct_map_history.append(correct_map.get_dict())
+
+            new_score = self.score_from_lcp(self.lcp)
+            self.score_history.append(new_score)
+            grading_strategy_handler = GradingStrategyHandler(
+                self.score,
+                self.grading_strategy,
+                self.score_history,
+                self.max_score(),
+            )
+            score = grading_strategy_handler.get_score()
+            self.set_score(score)


I'm still missing here moving this into its own method.

refactor: add grading strategy handler class feat: show grading strategy in student view fix: reset score history when reset student attempts fix: add missing argument in handle_average_attempt and fix typing hint chore: remove inline comments and print statements docs: add class and method docstring feat: add ListScoreField refactor: update grading strategy handler feat: consider grading strategy in rescore functionality feat: reset score history when reset student attempts feat: add correct_map_history and student_answers_history for rescore functionality fix: reset correct_map_history and student_answers_history when reset attempts test: fix unit tests refactor: address PR review fix: update correct_map_history from lcp instance for each attempt refactor: add _set_score and get_rescore methods chore: rename grading strategies (attempt -> score) refactor: address PR review fix: show grading strategy with unlimited attempts chore: add es_419 translations

BryanttV changed the title ~~feat: add grading strategy support~~ [POC] feat: add grading strategy support Nov 3, 2023

Ian2012 reviewed Nov 3, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

BryanttV force-pushed the bav/add-grading-strategy branch from c79d133 to ef8b811 Compare November 9, 2023 15:01

BryanttV marked this pull request as ready for review November 15, 2023 16:57

BryanttV requested a review from a team November 15, 2023 16:57

Ian2012 reviewed Nov 15, 2023

View reviewed changes

xmodule/capa/capa_problem.py Outdated Show resolved Hide resolved

Ian2012 reviewed Nov 15, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

Ian2012 reviewed Nov 15, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

Ian2012 reviewed Nov 15, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

Ian2012 reviewed Nov 15, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

Ian2012 reviewed Nov 15, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

mariajgrimaldi reviewed Nov 16, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

mariajgrimaldi reviewed Nov 16, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

mariajgrimaldi reviewed Nov 16, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

mariajgrimaldi reviewed Nov 16, 2023

View reviewed changes

xmodule/capa_block.py Outdated Show resolved Hide resolved

BryanttV changed the title ~~[POC] feat: add grading strategy support~~ feat: add grading strategy support Nov 17, 2023

BryanttV force-pushed the bav/add-grading-strategy branch from 2b0860e to 652f132 Compare November 23, 2023 17:53

mariajgrimaldi reviewed Nov 24, 2023

View reviewed changes

BryanttV mentioned this pull request Nov 27, 2023

feat: add grading strategy field in problem editor eduNEXT/frontend-lib-content-components#2

Closed

BryanttV force-pushed the bav/add-grading-strategy branch 4 times, most recently from 6ac85ba to fa59c65 Compare November 29, 2023 17:10

mariajgrimaldi reviewed Nov 29, 2023

View reviewed changes

mariajgrimaldi reviewed Dec 1, 2023

View reviewed changes

BryanttV force-pushed the bav/add-grading-strategy branch from 5d5adf7 to 7b580ff Compare December 4, 2023 18:26

BryanttV force-pushed the bav/add-grading-strategy branch 4 times, most recently from ec0aaeb to dfe9032 Compare December 11, 2023 20:04

BryanttV force-pushed the bav/add-grading-strategy branch 2 times, most recently from cf084d3 to d3416c9 Compare December 14, 2023 16:45

BryanttV force-pushed the bav/add-grading-strategy branch 2 times, most recently from 3afaeae to ae8a024 Compare January 9, 2024 19:26

BryanttV force-pushed the bav/add-grading-strategy branch from ae8a024 to 7f54dd2 Compare January 26, 2024 17:16

BryanttV force-pushed the bav/add-grading-strategy branch from 7f54dd2 to 33bb89d Compare February 19, 2024 15:17

BryanttV changed the title ~~feat: add grading strategy support~~ feat: add grading method support Feb 19, 2024

BryanttV added 2 commits February 19, 2024 13:12

chore: rename grading strategy -> grading method

2b2f1b5

BryanttV force-pushed the bav/add-grading-strategy branch from dad9812 to 2b2f1b5 Compare February 19, 2024 18:12

BryanttV closed this Feb 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add grading method support #299

feat: add grading method support #299

BryanttV commented Nov 2, 2023 •

edited

Loading

mariajgrimaldi Nov 24, 2023 •

edited

Loading

mariajgrimaldi Nov 24, 2023

mariajgrimaldi Nov 29, 2023

BryanttV Nov 29, 2023

mariajgrimaldi Nov 29, 2023

mariajgrimaldi Nov 29, 2023

BryanttV Nov 29, 2023

mariajgrimaldi Nov 29, 2023

mariajgrimaldi Nov 29, 2023

BryanttV Nov 29, 2023

mariajgrimaldi Dec 1, 2023

		if new_correct_map_list:
		self.lcp.correct_map.update(new_correct_map_list[-1])

feat: add grading method support #299

feat: add grading method support #299

Conversation

BryanttV commented Nov 2, 2023 • edited Loading

Description

Dependencies

Legacy Interface

MFE Interface

Testing Instructions

Rescoring

mariajgrimaldi Nov 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BryanttV commented Nov 2, 2023 •

edited

Loading

mariajgrimaldi Nov 24, 2023 •

edited

Loading