Unit-aware conversion of unit

[henrikt-ma]

In the ongoing intense work on unit handling driven by #3255, the need for a unit-aware conversion between units has come up on several occasions. Applications include:

• Unit conversions without bypassing unit checks as currently hidden away in functions like Modelica.Units.Conversions.to_degF (explained below).
• Unit conversions with the safety of the unit handling system.
• A concrete syntax for describing things happening implicitly in unit checking.

Functions like Modelica.Units.Conversions.to_degF would become unnecessary except as documented and annotated wrappers around the built-in operator. It is more interesting to see how a unit conversion block could be defined:

partial block PartialConversionBlock "Partial block defining the interface for conversion blocks"
  parameter String to "Unit of the output 'y', that is unit to which the input 'u' shall be converted";
  RealInput u "Connector of Real input signal to be converted (unit determined by inference)";
  RealOutput y(unit = to) "Connector of Real output signal containing input signal u in another unit";
equation
  y = inUnit(u, to);
end PartialConversionBlock;

block To_degF = PartialConversionBlock(to = "degF") "Convert temperature to degrees Fahrenheit";

Compare this to the need to introduce a specific block that calls the to_degF function, where the function's implementation is a complete mess with regards to units:

function to_degF "Convert from kelvin to degree Fahrenheit"
  extends Modelica.Units.Icons.Conversion;
  input SI.Temperature Kelvin "Value in kelvin";
  output Modelica.Units.NonSI.Temperature_degF Fahrenheit "Value in degree Fahrenheit";
algorithm
  Fahrenheit := (Kelvin + Modelica.Constants.T_zero) * (9 / 5) + 32;
end to_degF;

(Changing the function so that the result is computed via unit "1" would be possible, but it would end up being a very complicated way of expressing something that the unit system should already know how to do.)

[henrikt-ma]

@bilderbuchi, I couldn't add you a a reviewer, but this might be of interest to you as well.

[HansOlsson]

I don't see this as a good idea as it seems a way towards mixing units in models. That is not our previous intention, and it will be really problematic for units with offsets like Fahrenheit.

[bilderbuchi]

What are the problems you anticipate w.r.t. offset units?
Are those specific to Modelica limitations? I assume so, because offset units are integrated well (imo) in other unit frameworks like Python's pint.
I'll grant you that handling offset units well needs detailed considerations, and probably involves more changes than one simple conversion to integrate well. For example, we might need a separate quantity for offset-unit differences like TemperatureDifference.

[HansOlsson]

What are the problems you anticipate w.r.t. offset units?

In particular knowing when they apply - as the difference between delta and non-delta units isn't part of the unit in Modelica, and isn't really part of the SI system either.

Are those specific to Modelica limitations? I assume so, because offset units are integrated well (imo) in other unit frameworks like Python's pint.

I would say this is different as changes in the unit-system will have to be analyzed in greater detail than the introduction of a new one so that we don't break existing things. E.g., it seems pint has limitations on adding temperatures (due to the possibility of delta), but adding two values and dividing by two is a common operation for computing an average and it works for temperatures as well. Taking the conversion operator without considering that seems dangerous.

[bilderbuchi]

changes in the unit-system will have to be analyzed in greater detail than the introduction of a new one so that we don't break existing things.

I agree, hence my comment

handling offset units well needs detailed considerations, and probably involves more changes than one simple conversion to integrate well.

I suspect a design discussion would be good to have, to settle the design before a PR is made.

difference between delta and non-delta units isn't part of the unit in Modelica

Maybe the way FMI does it with a relativeQuantity type attribute could be instructive?

and isn't really part of the SI system either.

I'm not sure what the "really" qualifier implies for you, but the International Temperature Scale of 1990 (ITS-90), available from BIPM, at least makes explicit reference to temperature differences: "A difference in temperature may be expressed in kelvin or degrees Celsius."
The units are the same for delta and non-delta quantities, but the meaning is different! That's why I think unit (strings) are only half the picture.

If all you have available are unit strings, I'm not sure a flexible treatment is possible. I think this will boil down to a question of types, not units, in the end (or leveraging quantity/absoluteValue?).

E.g., it seems pint has limitations on adding temperatures (due to the possibility of delta), but adding two values and dividing by two is a common operation for computing an average and it works for temperatures as well.

AIUI, the limitations are there to avoid ambiguities, whose resolution might be clear to a practitioner from context (or not), but are not clear on a programming language level. For your example, how to resolve the addition of two Celsius temperatures, or dividing a Celsius temperature by 2 is clear in the context of a "mean", but how should the computer know?

[bilderbuchi]

In any case, this is a gnarly problem to tackle (this was also not an easy process in the Pint project, from what I saw), and I think that personally I'll concentrate the little time I have on getting #3266 finished.

[HansOlsson]

difference between delta and non-delta units isn't part of the unit in Modelica

Maybe the way FMI does it with a relativeQuantity type attribute could be instructive?

I believe that is historically based on how it is done in Modelica; so it's not part of the unit-string - but a separate annotation.

That actually makes sense based on how it is currently used:
You bring up the parameter dialog and then switch from input in Kelvin to Celsius or Fahrenheit and input the values.
Or similarly for a plot.

The implied conversion will depend on whether the value is relative or absolute and even though you can easily change displayUnit you cannot easily change if it is relative in those dialogs. That separation means that it just works!

and isn't really part of the SI system either.

I'm not sure what the "really" qualifier implies for you, but the International Temperature Scale of 1990 (ITS-90), available from BIPM, at least makes explicit reference to temperature differences: "A difference in temperature may be expressed in kelvin or degrees Celsius." The units are the same for delta and non-delta quantities, but the meaning is different! That's why I think unit (strings) are only half the picture.

True, I should have been clearer that even if the concept exists there isn't a standard way to specify it.

[casella]

I agree with @HansOlsson

I don't see this as a good idea as it seems a way towards mixing units in models. That is not our previous intention

I totally agree with @HansOlsson.

@henrikt-ma you write

Unit conversions without bypassing unit checks as currently hidden away in functions like Modelica.Units.Conversions.to_degF

I think that to the contrary the role of these functions is precisely the opposite, i.e., to make the unit conversion explicit, typically in the context of signal processing. E.g. you get Celsius readings from a data file, which is not Modelica so it doesn't have use SI, and you use to_K to convert it to SI units in a block diagram. Not when writing physical equations, you should not use those functions in there.

Your proposal instead hides some unit conversion magic behind the scene. To be honest, I find it a tad too dangerous. I'm fine at having units for documentation and dimensional consistency checking. Adding more semantics to them, that actually changes the results of the models by introducing automatic unit conversions behind the scene, doesn't seem wise to me.

[henrikt-ma]

difference between delta and non-delta units isn't part of the unit in Modelica

Maybe the way FMI does it with a relativeQuantity type attribute could be instructive?

I believe that is historically based on how it is done in Modelica; so it's not part of the unit-string - but a separate annotation.

Modelica has the absoluteValue-annotation, which I believe is all one needs, provided that it is properly applied. In the same way as absoluteValue is essential for not performing incorrect display unit conversions, it is essential for any kind of unit conversion. Fortunately, a unit system can detect errors with regards to relative vs absolute quantities.

A key safety measure that tools must obviously take when implementing a unit conversion operator such as inUnit is that they don't guess units when in doubt. If the unit of the argument expression to inUnit cannot be determined on solid grounds, the inUnit conversion must be rejected.

[henrikt-ma]

Are those specific to Modelica limitations? I assume so, because offset units are integrated well (imo) in other unit frameworks like Python's pint.

I would say this is different as changes in the unit-system will have to be analyzed in greater detail than the introduction of a new one so that we don't break existing things. E.g., it seems pint has limitations on adding temperatures (due to the possibility of delta), but adding two values and dividing by two is a common operation for computing an average and it works for temperatures as well. Taking the conversion operator without considering that seems dangerous.

I wouldn't be surprised if there is Modelica code out there that is performing additions that are invalid if applied to quantities in units with offset. Similar to other unit errors that we detect as our unit checkers improve, I think that such errors should be fixed instead of preventing development of the specification.

For the particular problem of computing a mean, it could be interesting to consider a built-in mean array reduction, similar to sum, but which can be applied also units with offset. Alternatively, it could be provided as a function:

model MeanTest
  function mean
    input Real[:] u;
    output Real y;
  protected
    Real[:] deltas;
  algorithm
    assert(size(u, 1) > 0, "There must be at least one value.");
    if size(u, 1) == 1 then
      y := u[1];
    else
      deltas := u - u[1]; /* Unit checker will understand that 'deltas' is relative. */
      y := u[1] + sum(deltas) / size(u, 1); /* OK to sum relative quantities. */
    end if;
  end mean;

  Real[3] x(each unit = "degC") = {10, 11, 13};
  Real y = mean(x);
end MeanTest;

[henrikt-ma]

I think that to the contrary the role of these functions is precisely the opposite, i.e., to make the unit conversion explicit, typically in the context of signal processing. E.g. you get Celsius readings from a data file, which is not Modelica so it doesn't have use SI, and you use to_K to convert it to SI units in a block diagram. Not when writing physical equations, you should not use those functions in there.

This sounds like a matter of modeling practice that the specification shouldn't interfere with, but I didn't say you have to hide the conversions. The top comment shows how to construct a block that applies the conversion:

block To_degF = PartialConversionBlock(to = "degF") "Convert temperature to degrees Fahrenheit";

Your proposal instead hides some unit conversion magic behind the scene. To be honest, I find it a tad too dangerous. I'm fine at having units for documentation and dimensional consistency checking. Adding more semantics to them, that actually changes the results of the models by introducing automatic unit conversions behind the scene, doesn't seem wise to me.

This PR is not about automatically applying unit conversions. This is about explicit conversion that relies on the sanity of the unit handling systems which we already use both for unit checking and for handling display units. Before considering automatic conversions we'd need to build even more trust in these systems, and this I believe we can only do put putting them to work.

[casella]

I wouldn't be surprised if there is Modelica code out there that is performing additions that are invalid if applied to quantities in units with offset.

The only units with offsets I am aware of are degC (degree Celsius), barg (bar-gauge, i.e. relative to atmospheric pressure) and psig (ditto in US customary units). These are not SI units, so they shouldn't be used in writing Modelica code in the first place. Using these units for computations can lead to very fishy results and subtle bugs, so we should discourage their use, rather than encourage it by trying to support it with sophisticated automatic conversion features.

In the 21st century, people should just use SI units. At least for engineering use, of course I'm fine with km/h speed limits on the roads or using liters for milk cartons in the supermarket 😄

I understand the consensus of the Modelica community is that the only tolerated use of of such non-SI unites is for pre- and post-processing of input and output data, e.g. in displayUnits. I fully agree with this and I don't think there is any need to make changes.

[henrikt-ma]

I understand the consensus of the Modelica community is that the only tolerated use of of such non-SI unites is for pre- and post-processing of input and output data, e.g. in displayUnits. I fully agree with this and I don't think there is any need to make changes.

I can't say I feel that consensus in my surroundings. It's a fact that some external data exists in other units, and to not have a unit system for handling that safely is a language design failure in my opinion, not a problem with the external data.

[bilderbuchi]

The only units with offsets I am aware of are degC (degree Celsius), [...]. These are not SI units

Surely you mean "SI base units", here? AFAIK, degrees Celsius were not removed from the list of SI derived units?

[HansOlsson]

The only units with offsets I am aware of are degC (degree Celsius), [...]. These are not SI units

Surely you mean "SI base units", here? AFAIK, degrees Celsius were not removed from the list of SI derived units?

It's complicated. The SI standard includes degree Celsius, but primarily as an alternative for temperature differences (when it is exactly the same as kelvin); so just because you see degree Celsius doesn't mean you can use it for the intended purpose. For a temperature value it is stated that it isn't coherent, but I'm not sure how legal that is according to BIPM (as the section is about coherent units).

[casella]

I understand the consensus of the Modelica community is that the only tolerated use of of such non-SI unites is for pre- and post-processing of input and output data, e.g. in displayUnits. I fully agree with this and I don't think there is any need to make changes.

I was in fact referring to the consensus about 10 years ago, when these decisions were taken. Hard to say now, since we've not been meeting for a long time in person 😭

I can't say I feel that consensus in my surroundings.

This is one of those things that should be discussed in a larger assembly, not just in a small MAP-LANG working group.

It's a fact that some external data exists in other units, and to not have a unit system for handling that safely is a language design failure in my opinion, not a problem with the external data.

This is a different requirement than being able to handle unit conversion automatically anywhere.

I would say that what you need for this are blocks/functions that perform unit conversion from/to custom units to/from SI units. Maybe this is best handled using libraries that provide such block functions, and include them in the MSL, rather than embedding fancy unit handling in the language, which has a much broader (and possibly controversial) impact.

I don't see any problem with a conversion block that is totally generic, and makes the conversion based on the unit attributes of inputs and outputs. Or maybe, even more simply, on unit strings declared for inputs and outputs as String parameters.

[casella]

@henrikt-ma, BTW, some years ago we had a discussion in the MA about trying to split the MLS into a core part and some "libraries" or "packages" specified more formally by some kind of code. For example, the stream connector specification is quite tricky, it would have been nice if we could have handled that with some kind of code or meta-code. As we did with the Clocked semantics, which is formally specified in terms of Modelia code.

This was to avoid bloating the specification (which is already over 300 pages) in favour of code that can be executed automatically, rather than interpreted by tool developers, and managed in a more straightforward and flexible way than textual plain English specification.

I guess we should try to go in this direction for the purpose you stated above.

[casella]

Surely you mean "SI base units", here? AFAIK, degrees Celsius were not removed from the list of SI derived units?

Yes. This is essential to be able to write physical equations without the need of any conversion factors, which are always tricky. The idea is to push conversion factors to pre- and post-processing as much as we can, and do the core modelling in SI base units only, so that one need not bother with them in the first place in that context.

Which AFAIK is exactly what is done in every sensible industrial engineering context, including US-based ones.

[henrikt-ma]

I would say that what you need for this are blocks/functions that perform unit conversion from/to custom units to/from SI units. Maybe this is best handled using libraries that provide such block functions, and include them in the MSL, rather than embedding fancy unit handling in the language, which has a much broader (and possibly controversial) impact.

In a setting where custom user symbols are present this functionality needs tool-specific implementation, and I don't want to end up with another loadResource – a function delivering key functionality in the language, but which avoids proper standardization in the specification by instead popping up in ModelicaServices.

[casella]

In a setting where custom user symbols are present this functionality needs tool-specific implementation, and I don't want to end up with another loadResource – a function delivering key functionality in the language, but which avoids proper standardization in the specification by instead popping up in ModelicaServices.

What I mean is, the unit conversion rules will be programmed in some Modelica blocks/functions that should be included in the MSL. Not just the interface, like ModelicaServices. Customizations could be handled by inheritance. We could do this entirely using Modelica, there is no need to make it a language feature.

[henrikt-ma]

Which AFAIK is exactly what is done in every sensible industrial engineering context, including US-based ones.

I'm afraid you are mistaking context lacking proper tool support for unit handling with sensible contexts. That context might be the Matlab community, or the Simulink community ten years ago. One could also say that in a context without proper tool support, it is a sensible decision by the community to try to mitigate the shortcomings of the tools.

To be clear, I don't expect Matlab's solution based on the Symbolic Math Toolbox to be integrated well enough in the language to count as proper tool support: https://se.mathworks.com/help/symbolic/units-of-measurement.html

Simulink, on the other hand (based on what I find by a quick search), may not have had proper unit support ten years ago, but appears to have it now: https://se.mathworks.com/help/simulink/ug/units-in-simulink.html
Since version R2016a they have a unit conversion block that appears to be performing unit conversion based on unit information in the connected signals (Simulink experts, please correct me if I'm wrong): https://se.mathworks.com/help/simulink/slref/unitconversion.html

In the Wolfram world, we are proud to deliver strong support for unit handling. To us, even adding quantities expressed in different units is a perfectly fine thing to do, thanks to the safety of the underlying unit system:

In[21]:= Quantity[1, "m"] + Quantity[2, "mm"]
Out[21]= 1002mm

Similarly, the famous pint in python has no issue mixing units:

>>> ureg = pint.UnitRegistry()
>>> 3 * ureg.meter + 4 * ureg.cm
<Quantity(3.04, 'meter')>

Neither Maple has any problem mixing units, see https://www.maplesoft.com/support/help/maple/view.aspx?path=units

Given the current limitations of Modelica, I can see that it is a sensible design decision for the MSL to agree on which base units to use, but this mustn't be mistaken for a wise decision about the development of the Modelica language. It seems like a very odd idea to defend that Modelica should be the only remaining technology where it isn't even possible to get unit conversion by explicitly asking for it.

[henrikt-ma]

What I mean is, the unit conversion rules will be programmed in some Modelica blocks/functions that should be included in the MSL. Not just the interface, like ModelicaServices. Customizations could be handled by inheritance. We could do this entirely using Modelica, there is no need to make it a language feature.

Such a solution will obviously not be able to take advantage of the powerful unit handling systems we already have in tools, and that the tools will always need to have regardless of which functions and blocks for conversion that the MSL provides. A unit system can automatically handle endless combinations of conversions that would be madness to try to cover with error-prone hand-coded conversion functions. Both madness to maintain, and madness for the users that need to search for the name of the block that converts from "g/ml" to "kg/m3".

[casella]

Which AFAIK is exactly what is done in every sensible industrial engineering context, including US-based ones.

I'm afraid you are mistaking context lacking proper tool support for unit handling with sensible contexts. That context might be the Matlab community, or the Simulink community ten years ago. One could also say that in a context without proper tool support, it is a sensible decision by the community to try to mitigate the shortcomings of the tools.

@henrikt-ma thanks for providing this context information, I've not been using Simulink for ages, since I'm fine using Modelica for my job 😄.

I based my statement on what a former PhD colleague of mine working in the R&D of a large US industrial conglomerate told me, about 10 years ago. I should probably ask him the same question, it could be that the availability of symbolic tools with strong unit handling made using a hodgepodge of units in large engineering projects a desirable thing. I personally doubt it, but I could be mistaken. I guess I should ask him again.

Regarding what Simulink does, I agree that having such a block available could be useful for the purpose of input/output data handling. I'll comment on that separately.

Regarding what symbolic manipulation tools like Mathematica or Maple do, I take note that you can do stuff like Quantity[1, "m"] + Quantity[2, "mm"] and get correct results. This is fine for me as long as the units are clearly visible. I would be very much scared by something like this

model Scary
  Real L1(unit = "m") = 1;
  Real L2(unit = "mm") = 2;
  Real L3(unit = "mm");
  Real L4;
equation
  L3 = L1 + L2;
  L4 = L1 + L2;
end Scary;

where L3 gets a value of 1002 and L4 gets a value of ?

What I mean is, for 25 years the semantics of L3 = L1 + L2 in Modelica has been very clear: the numerical value of L3 is bound to be the sum of the numerical values of L1 and L2. Units were there for the purpose of documentation, display, and to allow dimensional consistency checks, in which case L1 + L2 would give an error because you can't sum two numbers with different units.

What you are apparently proposing (also in other places, e.g. 2127 and links therein) is to change the semantics of the language, giving unit information first-class citizenship. I would be very much in favour of that if we were designing a new language. The problem I see is that if we do that, 25+ years of legacy Modelica code, that were written without this first-class citizenship, may become invalid, or broke, in the worst possible way, i.e., because of unexpected (and originally unintended, when the code was written) automatic conversions. I find this perspective very scary indeed.

The bottom line is that we should have a debate about this issue in the larger MA context, involving people that are responsible for actual industrial usage, and hear what they say. This is not something that should be resolved in a conversation between us two 😅.

Given the current limitations of Modelica, I can see that it is a sensible design decision for the MSL to agree on which base units to use, but this mustn't be mistaken for a wise decision about the development of the Modelica language. It seems like a very odd idea to defend that Modelica should be the only remaining technology where it isn't even possible to get unit conversion by explicitly asking for it.

I agree, as long as the unit conversion is explicit, not as in the Scary model above. I would also add that if we want to do that, we should standardize the actual content of the unit string much more explicitly than we do now, otherwise the risk of different tools doing different things is almost a certainty.

[casella]

What I mean is, the unit conversion rules will be programmed in some Modelica blocks/functions that should be included in the MSL. Not just the interface, like ModelicaServices. Customizations could be handled by inheritance. We could do this entirely using Modelica, there is no need to make it a language feature.

Such a solution will obviously not be able to take advantage of the powerful unit handling systems we already have in tools, and that the tools will always need to have regardless of which functions and blocks for conversion that the MSL provides. A unit system can automatically handle endless combinations of conversions that would be madness to try to cover with error-prone hand-coded conversion functions. Both madness to maintain, and madness for the users that need to search for the name of the block that converts from "g/ml" to "kg/m3".

I could agree to this, provided that the MLS defines the content of those unit strings much more explicitly than it does now, particularly regarding non-SI units. With the contents of the current Section 19, the likelyhood that two different tools end up doing different things seems to me pretty high.

[henrikt-ma]

where L3 gets a value of 1002 and L4 gets a value of ?

Actually not a big deal, if you ask me. All I require is that L4 gets a unit attached, so that I can't make the mistake of using it incorrectly in expressions/equations, or interpret it incorrectly in the simulation result.

A more elementary example of this mechanism would be something like this:

Real x(unit = "m") = 42.0;
Real y = x;

or the slightly more advanced:

Real a(unit = "m") = 42.0;
Real b(unit = "s") = 5.0;
Real c = a / b;

In these examples, it seems rather fundamental that the unit handling system keeps track of the units of y and c. If it doesn't, how would it detect the problem in a model like this?

Real x(unit = "m") = 42.0;
Real y = x;
Real z(unit = "s") = y;

To summarize the above, what scares me is not that the unit system figures out the units of variables that haven't been annotated with a unit-attribute. What scares me is unit systems not keeping track of units, especially in basic examples like above.

Maybe it didn't go without saying, so I clarified the proposed text to make it clear that inUnit is safe in this regard: inUnit(L4, "mm") would be an error unless the unit of L4 is defined in some tool-specific sense. Hence, while you might be able to bypass your unit checking system by throwing away units (that is, turning a defined unit into undefined, for example via a function call) in many other situations, this isn't a trick you can use on inUnit.

However, what is essential to the safety is that the tool understands the crucial difference between expressions with empty unit and expressions with undefined unit. Compare binding equations: Expressions with empty unit can safely be used to initialize a parameter with a concrete unit, whereas a (sloppy) tool might allow expressions with undefined unit to disable unit checking:

/* Note: This example assumes tool-specific meaning of well-defined/undefined. */
Real p(unit = "m") = 1 + 2; // Safe: Binding expression has well-defined empty unit.
Real q(unit = "m") = foo(1, 2); // Unsafe if unit of foo(1, 2) is undefined.

As it would probably be too restrictive and backfire if a tool would reject any model where not all expression units are well-defined, the tool is more or less forced to at most warn about the problem with the binding equation of q as long as expressions with undefined unit aren't extremely rare. (In my opinion, reducing the number of expressions with undefined unit should be a responsibility of the specification, but as long as the specification doesn't do its job, tools need to fill this gap.)

If expressed with inUnit, on the other hand, the situation is safe:

Real p(unit = "m") = inUnit(1 + 2, "m"); // OK: Binding expression has well-defined empty unit.
Real q(unit = "m") = inUnit(foo(1, 2), "m"); // Error if unit of foo(1, 2) is undefined.

[henrikt-ma]

I could agree to this, provided that the MLS defines the content of those unit strings much more explicitly than it does now, particularly regarding non-SI units. With the contents of the current Section 19, the likelyhood that two different tools end up doing different things seems to me pretty high.

Yes, one day we'll need to talk about the radian, and that "the compatible FMI standard" has chosen to treat it as a base unit: https://fmi-standard.org/docs/3.0/#_physical_units

Apart from that, I would expect tools to generally implement this part in consistent ways. I believe that the much bigger problem with unit checking differences between tools has to do with the amount of unit inference and propagation being performed, the sophistication of function call unit checking, unit handling in arrays, etc, all of which are problems outside the scope of this PR.