Migrate Calculations to DX

[toropok]

Description of the issue/feature this PR addresses

This PR migrates the AT Calculation to Dexterity for Python 3 compatibility.

Current behavior before PR

Calculation is a content type based on the Archetypes framework

Desired behavior after PR is merged

Calculation is a content type based on the Dexterity framework

--
I confirm I have tested this PR thoroughly and coded it according to PEP8
and Plone's Python styleguide standards.

[toropok]

Hi @ramonski @xispa,

Here's my preliminary delivery for the calc migration! 😊 This migration turned out to be quite a tough nut compared to the previous ones—no chance it would’ve worked without the invaluable help of @Bugerman58! 🚀

Currently, we’re stuck at a point where the tests are failing. Specifically:

We’re not using HistoryAwareReferenceField as expected.
Was there any thought about using an alternative?
Or is there no longer a need to maintain version references?
Looking forward to your input!

[ramonski]

Hi Leonid, hi Anton,
many thanks for your continuous efforts and support that we really appreciate!
Yes, indeed, you picked a tough nut with the calculations 🤯
Regarding the HistoryAwareReferenceWidget, Jordi and I discussed once that we want to drop it and maybe lookup the values from the Snapshot storage? And even combine this with our UIDReferenceField?
Anyhow, we'll find a solution for this for sure.
Thanks again for your good work and we'll review as soon as possible!

[xispa]

Excellent work @toropok and @Bugerman58 🥇 . I can imagine how challenging the migration of Calculation has been!. Your amazing work is really appreciated! Willing to review the Pull Request already and maybe we open another one later for the removal of the HistoryAware relic!

[xispa]

Hi @toropok , tests are failing https://github.com/senaite/senaite.core/actions/runs/11948903328/job/33307294669

Can you please take a look before I start to review?

[toropok]

Hi @toropok , tests are failing https://github.com/senaite/senaite.core/actions/runs/11948903328/job/33307294669

Can you please take a look before I start to review?

Hi @xispa

Sure, I'll fix the failed Python tests (they only test object validation) and get back to you. The remaining failing tests are doctests, which are related to HistoryAwareReferenceField.

[toropok]

@xispa

Sorry for the long pause.

• I’ve added FormulaValidator, and it works! 🙂
• Fixed the Python tests — all tests are now passing successfully.

That leaves us with two remaining issues that need your guidance and vision:

1. HistoryAware behavior for UIDReferenceField: All the failing doctests are related to this feature.
2. Formula wildcards: How are they supposed to work? It seems like they’ve never worked as intended.

[xispa]

Pheew, a lot of work here... @toropok you must be exhausted. I've done a first review and requested some changes though.

Haven't gone through the validators details yet, but I think you can simplify a lot there. I feel are overwhelming, will review them in the next round.

I've seen tests that are failing are because of the HistoryAwareReferenceField. Don't worry too much about this for now.

[xispa]

Really nice, so system "detects" if an interim field or service for a given keyword does exist each time a form value is edited.... definitely this helps a lot!!

However, would you mind to simplify the code a bit? something like below would be easier to read imho:

form = data.get("form")

validator = FormulaValidator(self.context, self.request, self, ICalculationSchema, None)
value = {
    "formula": form.get(FIELD_FORMULA) or "",
    "interims": self.get_interim_fields(data)
}
errors = validator.validate(value)

Note I would rely on a self.get_interim_fields(data) that returns well-formed interim field records, containing not only with the keyword, but also the rest of subfields (you never know the subfields the validator will rely on).

[xispa]

Please use "Variables" for title and description, like we have for AnalysisService: https://github.com/senaite/senaite.core/blob/2.x/src/bika/lims/content/analysisservice.py#L112-L119

I think is cleaner. From a user perspective, "interim" sounds confusing imo.

[xispa]

1. HistoryAware behavior for UIDReferenceField: All the failing doctests are related to this feature.

Yes, I've seen. I've requested some changes, we will handle this HistoryAware behavior afterwards.

2. Formula wildcards: How are they supposed to work? It seems like they’ve never worked as intended.

They are supposed to work, yes. For instance, if the user types this formula:

[CHOL] + [FAT] if [CHOL] >= [CHOL.LDL] else "Invalid"

where:

• CHOL is the keyword of a service (e.g. Cholesterol)
• FAT is the keyword of a saervice (e.g. Fat)
• <service_keyword>.LDL is the wildcard representing the value of the field LowerDetectionLimit for the prefixed service

Therefore, if the result of CHOL is below the LowerDetectionLimit, the result of the calculation will be Invalid.

See e.g. the doctests here: https://github.com/senaite/senaite.core/blob/2.x/src/senaite/core/tests/test_calculations.py#L83-L137