Skip to content

Commit

Permalink
Enhancing GDS2 writer float conversion such that less rounding occurs
Browse files Browse the repository at this point in the history
Matthias Koefferlein committed Oct 4, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent 7953cad commit 1289db7
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/plugins/streamers/gds2/db_plugin/dbGDS2Writer.cc
Original file line number Diff line number Diff line change
@@ -107,7 +107,7 @@ GDS2Writer::write_double (double d)
tl_assert (e >= -64 && e < 64);
b[0] |= ((e + 64) & 0x7f);

uint64_t m = uint64_t (d + 0.5);
uint64_t m = uint64_t (round (d));
for (int i = 7; i > 0; --i) {
b[i] = (m & 0xff);
m >>= 8;

0 comments on commit 1289db7

Please sign in to comment.