error on PUT to /api/v2/forms/{uuid_or_slug}/variables issues (investigating)

[LaurensBurger]

Product versie / Product version

2.7.x and higher (reproduced on our test env)

Customer reference

No response

Omschrijf het probleem / Describe the bug

Seeing cases where errors while saving a form ("variables") lead to a form where no input will be stored in de submission for the new variables.

This can bee seen in the form itself as navigating back to these steps will simply show empty fields that previously held data.
These specific variables are also missing in the variables tab of the form.

These variables are "present" in the form builder, they also show in the front-end of the form but they are missing in the submission, and consequently trigger a validation error if they are required.

This is either showing up more frequent now because of BE validation, or 2.7 introduced a bug where saving of variables has changed?

suspected ValidationErrors look something like this
[{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {'key': [ErrorDetail(string='Ongeldige componentvariabele: er bestaat geen component met de opgegeven sleutel in de formulierdefinitie.', code='invalid')]}, {'key': [ErrorDetail(string='Ongeldige componentvariabele: er bestaat geen component met de opgegeven sleutel in de formulierdefinitie.', code='invalid')]}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}]
This would suggest this form now contains 2 "ghost" variables that unless refreshed by the from builder and saved again, will be missing from every submission

maykin-test form 608 is currently in this state - it contains 2 fields "dummyveld" and "dummyveld1" these were added with a "bad" component that triggered a error.

Both fields show up in the form designer and front-end, but are not present when creating a submission.

Stappen om te reproduceren / Steps to reproduce

No response

Verwacht gedrag / Expected behavior

No response

Screen resolution

None

Device

None

OS

None

Browser

No response

[stevenbal]

@LaurensBurger is the form with ID 608 the right one? I see many more fields in that one and no dummyveld or dummyveld1:

Also if I save the form I get a different error than the one you got:
rest_framework.exceptions.ValidationError: [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {'name': [ErrorDetail(string='This field may not be blank.', code='blank')]}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}]

[LaurensBurger]

@stevenbal sorry I've messed a bit with it in the meantime - i'll bring it back to the original state. (should be back to the state of showing 2 dummy fields)

The error is just a way to trigger the state, looking at Sentry there are many ways to trigger it - i used the "softRequiredErrors" to trigger the state because i have hard time reproducing the most common issue in my example above.

(also the way i trigger it seems to also break navigating the form. But you can still observe the effect if you pauze the form after filling in all fields and looking at the submission, dummy and dummy 1 should be missing)

[joeribekker]

This might affect submissions as well as building the form? Hard to estimate the impact. We'll make a database dump to reproduce the exact issue since locally this didnt seem to happen.