You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently depending on what kind of invalid JSON record it encounters clp-s will either report a failure directly before the invalid message, or before a (large) batch containing the invalid message. The latter case makes it difficult to find the invalid JSON and solve the issue.
Possible implementation
Instead of feeding batches of raw log data to simdjson feed simdjson one log message at a time. We can do this by using something like std::getline (assuming the underlying content is ndjson), or by using some of the same simd tricks used by simdjson.
The text was updated successfully, but these errors were encountered:
Request
Currently depending on what kind of invalid JSON record it encounters clp-s will either report a failure directly before the invalid message, or before a (large) batch containing the invalid message. The latter case makes it difficult to find the invalid JSON and solve the issue.
Possible implementation
Instead of feeding batches of raw log data to simdjson feed simdjson one log message at a time. We can do this by using something like
std::getline
(assuming the underlying content is ndjson), or by using some of the same simd tricks used by simdjson.The text was updated successfully, but these errors were encountered: