Add: Dynamic problems for fill-in-the-blank questions

[bjones1]

@dbrianwalton, this is the initial PR for our work on dynamic problems. I'll be force-pushing this until tests pass.

[bjones1]

It looks like unit tests are broken at this point, so I'll not worry about getting those passing.

[bnmnetp]

I think there are two separate issues with the unit tests.

1. I made some changes to the homepage that are causing some failures
2. More than that, is that deprecated functions in selenium were actually removed causing our code to fail.

[bnmnetp]

Can we test against the runestone/fitb/test ??

[bnmnetp]

First -- Super Cool! Its amazing to see this coming together.

OK, so I pulled this down and after an npm install I was able to get the test page working in my browser. I'm not sure I'm smart enough to write a question but I was able to still do enough algebra to get the tests correct.

[bnmnetp]

Are the changes to the problem representation backward compatible?

Currently the problem statement is rendered outside the script tag that contains the json. But it appears that this implementation moves it to json in the "problemHtml" key.

The big issue with this is the problems currently in the runestone question bank would all need to be rebuilt somehow or they would cease to work.

[bnmnetp]

Can we write a code based problem using this dynamic stuff?

For example write a for loop to add the numbers starting with 3 and up to and including 11? Where we could randomize 3 and 11 ?

I guess one way of doing it would be to have blanks in a call to range.

[bnmnetp]

@rbeezer I don't think the html is very different from what we have now. With the exception of the problem statement moving to an attribute of the json.

        <div class="runestone">
            <div data-component="fillintheblank" data-question_label="7" id="test_fitb_dynamic_1"  style="visibility: hidden;">
                <script type="application/json">
                    {"problemHtml": "<p>Before-14-After: What is [%= a %] + [%= b %]? <input type=\"text\" name='c' /></p>\n", "dyn_vars": "v.a = Math.floor(rand()*10);\nv.b = Math.floor(rand()*10);\nv.types = {c: Number};", "blankNames": {"c": 0}, "feedbackArray": [[{"solution_code": "c === a + b", "feedback": "<p>Correct; [%= a %] + [%= b %] is [%= c %]. Note that [%= ans %] or [%= ans_array[0] %] also works.</p>\n"}, {"solution_code": "c === a - b", "feedback": "<p>That\u2019s subtraction.</p>\n"}, {"solution_code": "c === a * b", "feedback": "<p>That\u2019s multiplication.</p>\n"}, {"solution_code": "x", "feedback": "<p>I don\u2019t know what you\u2019re doing; [%= a %] + [%= b %] is [%= a + b %], not [%= c %].</p>\n"}]]}
                </script>
            </div>
        </div>

[rbeezer]

The big issue with this is the problems currently in the runestone question bank would all need to be rebuilt somehow or they would cease to work.

Fix them all by hand? ;-)

Not clear immediately in my head about the implications of the PreTeXt statement traveling in the JSON. Perhaps a good discussion at Tuesday Drop-In.

[bnmnetp]

Not for N > 500 😄

[bnmnetp]

Actually I just ran across a Jupyter notebook I wrote a couple years ago that solves this problem. I used it the last time we made a breaking change in the parsons problem html.

We just have to launch this at a time when I can build all the books and run this script to reparse all of the problems in the database.

[bjones1]

I think there are two separate issues with the unit tests.

1. I made some changes to the homepage that are causing some failures
2. More than that, is that deprecated functions in selenium were actually removed causing our code to fail.

True. It looks like all the unit tests (bookserver, Runestone components, Runestone server) need some maintenance. I'd like to think through these with you: instead of just fixing these, do they need to be re-though/re-directed? Are these tests helpful, or just a maintainance burden?

[dbrianwalton]

Are the changes to the problem representation backward compatible?

Currently the problem statement is rendered outside the script tag that contains the json. But it appears that this implementation moves it to json in the "problemHtml" key.

The big issue with this is the problems currently in the runestone question bank would all need to be rebuilt somehow or they would cease to work.

Can the dynamic problem code test if problemHtml is absent, and if so the statement is implicitly presumed to have already been written. This would be the case if there were no dynamic substitutions required, because there is no dynamic code. All that really needs to be sure happens is that the feedback testing still works correctly. As far as I understand, that doesn't really depend on the dynamic code—just dynamic code is responsible for creating appropriate scripted functions to do the tests.

[dbrianwalton]

Can we write a code based problem using this dynamic stuff?

For example write a for loop to add the numbers starting with 3 and up to and including 11? Where we could randomize 3 and 11 ?

I guess one way of doing it would be to have blanks in a call to range.

I would like to see how we can extend the dynamic structure to a variety of different problem types. It would be ideal if whatever structure is present for fixed coding problems was present.

The trick to the dynamic coding problem is that you need to define variables in dyn_vars for those two values (first and last numbers in a sequence), and then be able to write the test functions for validity/feedback of the solution to depend on those two variables.

[bjones1]

Can we test against the runestone/fitb/test ??

You mean, just run these tests for now, rather than the entire test suite? If so, I was able to do this locally. The dynamic problem didn't pass, so I'll work on that.

[bjones1]

First -- Super Cool! Its amazing to see this coming together.

Thanks! I've really enjoyed working with @dbrianwalton on this.

OK, so I pulled this down and after an npm install I was able to get the test page working in my browser. I'm not sure I'm smart enough to write a question but I was able to still do enough algebra to get the tests correct.

Glad it's working!

[bjones1]

Are the changes to the problem representation backward compatible?

It's the same data for non-dynamic problem, but stored in a different way, as you noted below.

Currently the problem statement is rendered outside the script tag that contains the json. But it appears that this implementation moves it to json in the "problemHtml" key.

The big issue with this is the problems currently in the runestone question bank would all need to be rebuilt somehow or they would cease to work.

Good point...

[bjones1]

Can we write a code based problem using this dynamic stuff?

Such as an ActiveCode problem? Probably, but I'd need to think about how to put these pieces together...

For example write a for loop to add the numbers starting with 3 and up to and including 11? Where we could randomize 3 and 11 ?

I guess one way of doing it would be to have blanks in a call to range.

[bjones1]

Actually I just ran across a Jupyter notebook I wrote a couple years ago that solves this problem. I used it the last time we made a breaking change in the parsons problem html.

We just have to launch this at a time when I can build all the books and run this script to reparse all of the problems in the database.

Fantastic! (I assume it's not easy to simply rebuild all the books using runestone build --all?)

[bjones1]

Can we test against the runestone/fitb/test ??

You mean, just run these tests for now, rather than the entire test suite? If so, I was able to do this locally. The dynamic problem didn't pass, so I'll work on that.

The tests now pass locally for me.

[bjones1]

I've added code per today's drop in that:

1. Changes the dynamic problem type from "fillb" to "dyn-fillb".
2. Logs all evaluated dynamic variables in the act field.

[bjones1]

Note that this test failure is from codelens, not from fitb.