Emit multiple xAPI events for a multi-question submission

[pomegranited]

Description:

Updates the base event processor and tests to support transforming a single event into multiple events, and
modifies the server-side problem_check xAPI event transformer to transform multi-problem submissions into:

• 1 event for the problem submission, plus
• 1 event for each sub-problem

Resolves #219

JIRA: FAL-3430 (OpenCraft internal link)

Testing instructions:

This change should be covered by tests, but to test it manually:

1. Install this branch on your Tutor LMS
2. Login, and submit various responses to the Multiple Choice Questions block in the Demo course, or create your own.
3. Watch multiple xAPI events flow through into Aspects from a single submission.

Merge checklist:

• All reviewers approved
• CI build is green
• Version bumped
• Changelog record added
• Documentation updated (not only docstrings)
• Commits are squashed

Post merge:

• Create a tag
• Check new version is pushed to PyPI after tag-triggered build is
  finished.

Author concerns:

• Suggest reviewing this commit by commit to understand the reasoning.
• This change only affects xAPI, not Caliper events.
• Some minor changes to single question problem_check events occurred: added object.definition.name and interactionType.

[openedx-webhooks]

Thanks for the pull request, @pomegranited! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[pomegranited]

@bmtcril CC @Ian2012 @ziafazal This is ready for review :)

I think the coverage tests are failing because I hit the Actions too frequently when re-pushing my individual commits. I'll try that again tomorrow.

[bmtcril]

This is looking really good! Couple of things:

1. I think this should be a major version bump as it's a pretty big breaking change for downstream data handling
2. The coverage failures just happen in this repo pretty frequently, we just retry failed jobs when they come up
3. When testing with the demo course I ran into this error when submitting the "Multiple Choice Questions" problem (single answer problems worked great):

2023-08-07 10:37:23 2023-08-07 14:37:23,344 ERROR 19 [eventtracking.backends.routing] [user None] [ip None] routing.py:146 - Unable to send edx event "problem_check" to backend: xapi
2023-08-07 10:37:23 Traceback (most recent call last):
2023-08-07 10:37:23   File "/openedx/venv/lib/python3.8/site-packages/eventtracking/backends/routing.py", line 137, in send_to_backends
2023-08-07 10:37:23     backend.send(event)
2023-08-07 10:37:23   File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/backends/events_router.py", line 158, in send
2023-08-07 10:37:23     event_routes = self.prepare_to_send([event])
2023-08-07 10:37:23   File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/backends/events_router.py", line 82, in prepare_to_send
2023-08-07 10:37:23     processed_events = self.process_event(event)
2023-08-07 10:37:23   File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/backends/events_router.py", line 191, in process_event
2023-08-07 10:37:23     events = processor(events)
2023-08-07 10:37:23   File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/processors/mixins/base_transformer_processor.py", line 37, in __call__
2023-08-07 10:37:23     transformed_event = self.transform_event(event)
2023-08-07 10:37:23   File "/openedx/venv/lib/python3.8/site-packages/event_routing_backends/processors/xapi/transformer_processor.py", line 48, in transform_event
2023-08-07 10:37:23     event_json = transformed_event.to_json()
2023-08-07 10:37:23 AttributeError: 'list' object has no attribute 'to_json'

[pomegranited]

Thank you for testing this @bmtcril !

1. I think this should be a major version bump as it's a pretty big breaking change for downstream data handling

Done: ea3d27d

2. The coverage failures just happen in this repo pretty frequently, we just retry failed jobs when they come up

👍

3. When testing with the demo course I ran into this error when submitting the "Multiple Choice Questions" problem (single answer problems worked great):

Ach sorry.. I missed that during a refactor, since the tests were passing.. 40f7422 covers that error and adds a test.

[bmtcril]

Almost there, I think. What I see now is that the statements all get created with the same event id. I'd expect the child statements to each have their own id, but that the StatementRef would look up to the parent. As it stands, the child events get de-duplicated down to 1 event in the database because they have identical id/actor/object/verb/timestamps.

[pomegranited]

@bmtcril Gotcha. I decided to do this in two separate commits:

• 885f476 -- removes the mock uuid5 from the test wrapper so that our tests validate the actual generated event ID for all existing xAPI tests.
  This touches a lot of files, and I'm happy to move it into a separate PR if you'd prefer. But it was necessary for the next step, because the mocked uuid5 caused the parent event to have the same ID as its child events.
• 9482075 -- makes child events use a different event ID from the parent.

[pomegranited]

Hiya @bmtcril , when you get a chance -- did I fix the de-duplication issue you noted?

[bmtcril]

Hey there, I'll run again tomorrow. I've been flat out trying to get the asset import pr in shape

[pomegranited]

No worries at all, thank you for your thorough testing @bmtcril !

[bmtcril]

Ran into some new issues here today, but haven't had time to sort them out. Ralph is failing insert on all three child events with:

2023-08-15 16:48:30 Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type: while converting source column org to destination column org: while executing 'FUNCTION _CAST(org :: 5, String :: 8) -> _CAST(org, String) String : 10': while pushing to view xapi.xapi_events_all_parsed_mv (778579dd-f27a-4ef7-a7d1-0a137ee42cba): While executing WaitForAsyncInsert. (CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN) (version 23.3.4.17 (official build))

This is what the child event looks like in the db:

{"actor": {"account": {"homePage": "http://local.overhang.io", "name": "f7f6e171-9a01-480a-b330-7132f02f292e"}, "objectType": "Agent"}, "id": "324cbcfa-6e40-593b-8241-1efc604a03e0", "object": {"id": "http://local.overhang.io/xblock/block-v1:edX+DemoX+Demo_Course+type@problem+block@a0effb954cca4759994f1ac9e9434bf4_4_1", "objectType": "Activity", "definition": {"description": {"en-US": ""}, "type": "http://adlnet.gov/expapi/activities/cmi.interaction", "interactionType": "choice", "extensions": {"http://id.tincanapi.com/extension/attempt-id": 2}}}, "verb": {"id": "https://w3id.org/xapi/acrossx/verbs/evaluated", "display": {"en": "evaluated"}}, "version": "1.0.3", "timestamp": "2023-08-15T20:48:29.026858+00:00", "context": {"contextActivities": {"parent": [{"id": "http://local.overhang.io/xblock/block-v1:edX+DemoX+Demo_Course+type@problem+block@a0effb954cca4759994f1ac9e9434bf4", "objectType": "Activity", "definition": {"description": {"en-US": ""}, "type": "http://adlnet.gov/expapi/activities/cmi.interaction", "interactionType": "other", "extensions": {"http://id.tincanapi.com/extension/attempt-id": 2}}}], "grouping": [{"id": "http://local.overhang.io/course/course-v1:edX+DemoX+Demo_Course", "objectType": "Activity", "definition": {"name": {"en-US": "Demonstration Course"}, "type": "http://adlnet.gov/expapi/activities/course"}}]}, "statement": {"objectType": "StatementRef", "id": "1eef567e-8ded-52cc-b7de-1d1efd432708"}, "extensions": {"https://w3id.org/xapi/openedx/extension/transformer-version": "[email protected]", "https://w3id.org/xapi/openedx/extensions/session-id": "7fd5f7f471a9a2a95709accad1467f80"}}, "result": {"success": true, "response": "['a piano', 'a guitar']"}}

[bmtcril]

Looks like it's trying to get the course id and failing since it's now in "grouping":

"grouping": [{"id": "http://local.overhang.io/course/course-v1:edX+DemoX+Demo_Course", "objectType": "Activity", "definition": {"name": {"en-US": "Demonstration Course"}, "type": "http://adlnet.gov/expapi/activities/course"}}]}

I think we'll need a migration to update the sql, it's trying to do:

       if(JSON_VALUE(event_str,
                     '$.context.contextActivities.parent[0].definition.type') =
          'http://adlnet.gov/expapi/activities/course',
          JSON_VALUE(event_str, '$.context.contextActivities.parent[0].id'),
          JSON_VALUE(event_str, '$.object.id'))                   AS course_id,

[bmtcril]

This is working for me from the xAPI side, but we need to update Aspects before we can upgrade do a new version!

[Ian2012]

@bmtcril what is needed to include this change in Aspects?

[bmtcril]

@Ian2012 this will need a big rebase and testing again. We need to add a migration to xapi_events_all_parsed_mv to look for the course id in the new location. I think we'll need some changes to problem_events_mv to roll these up into whatever the logical structure is there, but I'm not sure what off hand.

[bmtcril]

I'm working to get this over the line in #352

[openedx-webhooks]

@pomegranited Even though your pull request wasn’t merged, please take a moment to answer a two question survey so we can improve your experience in the future.

[openedx-webhooks]

@pomegranited Even though your pull request wasn’t merged, please take a moment to answer a two question survey so we can improve your experience in the future.