-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MONGOCRYPT-702 FLE2 range floating point correctness issue when using precision mode #868
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RFAL. I updated the errors for decimal128. PTAL at subtraction closely in Decimal128. I think it's correct but maybe worth making it more resilient.
src/mc-range-encoding.c
Outdated
if (precision > 255) { | ||
CLIENT_ERR("Invalid value for precision. Must be less than 255. precision: %" PRIu32, precision); | ||
return false; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where does 255 come from?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mlib_int128_pow10
takes a uint8_t, so we have to ensure that precision can fit into there. I have changed it to uint8_t max.
No description provided.