GODRIVER-3090 Optimize logging truncation for large documents

[timothy-kim-mongo]

GODRIVER-3090

Summary

The goal of this pull request is to optimize the logging truncation process for large documents in the Go Driver. This optimization aims to improve driver performance when handling and truncating large extensive BSON documents and payloads, specifically addressing issues related to logging large documents, which can adversely affect driver performance and can cause resource containment and are critical for maintaining application performance and operational efficiency.

Key changes introduced include the implementation of a StringN method for bsoncore.Document. This method allows BSON documents to be stringified up to a specified byte limit (N), incorporating precise truncation logic that accounts for multi-byte characters. By leveraging the existing logging truncation algorithm, the method ensures accurate and efficient handling of BSON documents in extended JSON format, thereby maintaining data integrity and aligning with BSON specifications.

Furthermore, the pull request updates various BSON element string methods (Array.StringN, Value.StringN, etc.) to utilize the new StringN functionality. This ensures consistent and optimized truncation across different BSON element types, such as arrays and nested documents, when exceeding specified byte limits. Comprehensive unit tests and benchmark tests accompany these changes, validating the performance improvements in terms of reduced execution time (ns/op), decreased memory allocations (B/op), and enhanced efficiency in handling large BSON payloads.

Therefore, this pull request significantly improves driver performance by optimizing BSON document truncation for logging purposes. It addresses scalability challenges and enhances the reliability of the MongoDB Go Driver in handling large BSON payloads, thereby benefiting applications with critical logging and operational needs.

[timothy-kim-mongo]

Still looking for additional ways of optimizing

[mongodb-drivers-pr-bot]

API Change Report

./x/bsonx/bsoncore

compatible changes

Array.StringN: added
Document.StringN: added
Element.StringN: added
Value.StringN: added

[timothy-kim-mongo]

Addition of this code, fixed a bug that was making truncation 11 instead of the length of 10 in one of the test cases.
It also improved the benchmarking speed of truncating massive arrays.
Before Change-
BenchmarkRawString/massive_arrays_StringN-10 642 1860169 ns/op 632576 B/op 411 allocs/op

After Change-
BenchmarkRawString/massive_arrays_StringN-10 105404 10738 ns/op 15744 B/op 396 allocs/op

[prestonvasquez]

Great work @timothy-kim-mongo 🔧 . If you replace the _StringN benchmarks with the old solution:

str := bsoncore.Document(bs).String()
bsoncoreutil.Truncate(str, 1024)

and report allocations, you get a 3000x allocation improvement!

BenchmarkRawString/massive_arrays_StringN-10_old                          39          31489319 ns/op        60814318 B/op    1200137 allocs/op

BenchmarkRawString/massive_arrays_StringN-10                      117501             10122 ns/op           15744 B/op        396 allocs/op

[matthewdale]

Looks good! 👍