-
Notifications
You must be signed in to change notification settings - Fork 18
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Bug
- The
nlohmann/json
library, whichclp-ffi
uses to serialize kv-pair log events into JSON, acknowledges there are implementations whose number types can store up only to 2^53-1 : https://json.nlohmann.me/features/types/number_handling/#number-interoperability; however, it does serialize numbers out of that range: https://github.com/nlohmann/json/blob/4a602df34e71afca2242b34c743d4cc2486cd071/tests/src/unit-serialization.cpp#L168-L183 - When numbers in such ranges are parsed by
JSON.parse()
, loss of precision is expected. Below sample code illustrates the problem:const myNumStr = "9007199254740993"; // 2^53 + 1 console.log(JSON.parse(myNumStr)); // 9007199254740992
- Therefore, it is expected that numbers in that range, when returned as part of the messages by
clp-ffi-js
, cannot be accurately represented in the formatted messages displayed in the log-viewer.
Potential solutions
- Override the serialization / deserialization behaviours for big numbers:
const myNum = 1234567890123456789n BigInt.prototype.toJSON = function () { return JSON.rawJSON(this.toString()); }; const jsonStr = JSON.stringify(myNum) console.log(jsonStr) console.log(JSON.parse(jsonStr)) // loss of precision: 1234567890123456800 const isNumberRepresentable = value => { const num = Number(value); return ( false === isNaN(num) && Number.isInteger(num) && num <= Number.MAX_SAFE_INTEGER && num >= Number.MIN_SAFE_INTEGER ); }; console.log(JSON.parse(jsonStr, (_, __, context) => { if (false === isNumberRepresentable(context.source)) { return BigInt(context.source) } return value; })) // 1234567890123456789n
- Instead of JSON, return the decoded structured log event in a byte-string format with another serialization method. e.g., MessagePack.
yscope-log-viewer version
Environment
The sample code was run with Node.js 22 which uses the V8 engine which is also used by any Chromium-based browsers such as Google Chrome and Microsoft Edge.
Reproduction steps
- Run above sample code with Node.js 22 and observe the loss of precision in the console output.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working