-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Serialization inaccuracy #72
Comments
I've traced it to Using Here is a simple test script: using F = fpm::fixed<int16_t, int32_t, 8>; // fixed_8_8
F v0{0.9};
F v1{0.89};
while(static_cast<double>(v1) < 0.89)
{
v1 = F::from_raw_value(v1.raw_value() + 1);
}
std::cout << "v0: " << v0 << std::endl;
std::cout << "v1: " << v1 << std::endl;
std::cout << "diff: " << (v0 - v1) << std::endl;
{
std::stringstream ss("0.898438");
F result{};
ss >> result;
std::cout << ss.str()<< " -> " << result << std::endl;
}
{
std::stringstream ss("0.890625");
F result{};
ss >> result;
std::cout << ss.str() << " -> " << result << std::endl;
} and its result:
To have consistent output, you can use But with current parsing code ( I would like to write a function that does not require the large type but I am not sure how (with good enough performance).
|
There is a problem with serialization to/from stream.
I've started with
0.9
infpm::fixed_16_16
which is serialized as0.899994
which is OK but if you de-serialize it, you get0.899887
making it different by0.000106812
which are 7 "raw values".EnableRounding
has no effect.Simple test: https://godbolt.org/z/68qW64v93
Converting to
float
ordouble
before/after stream serialization "fixes" this problem in this case but it is not a true fix.Using
std::setprecision
withstd::fixed
to make sure the serialization provides all the digits did not help.Some other values:
I understand that there is inaccuracy by the conversion to/from string but I think difference of 7 is little too big.
If you use
std::setprecision
andstd::fixed
, you may get value like1.1000061035
instead of1.1
but for the table I went with unmodifiedstd::cout
.If you want to use approach like this, you need string long enough to represent
0.0000152587890625
(1/(2^16)
) for 16 fractional bits as it is the epsilon.0.00390625
for 8 bits and0.00000005960464477539063
for 24 bits.Probably linked to #37 as stream serialization is not valid replacement for raw values.
The text was updated successfully, but these errors were encountered: