Store objects in "header+payload" format #2925
Labels
enhancement
Improving existing functionality
I4
No visible changes
neofs-storage
Storage node application issues
performance
More of something per second
S1
Highly significant
U3
Regular
Is your feature request related to a problem? Please describe.
I'm always frustrated when I look at how we allocate a lot of memory to fetch data from storage. #2316, #2178, #1398. #2814 also reminds about it.
Describe the solution you'd like
#1932 has some valid input for these problems, although it's localized to CLI, whereas I'm more concerned about SN. We can store/fetch objects in a bit different manner, serializing header and payload separately right at the FSTree level. Then object get operation would return a header and an
io.ReaderCloser
pointing to the file (if it's not too small). This can then be streamed to the outside as needed (but likely API changes are required as well).Describe alternatives you've considered
Keep pulling MBs of data from the storage is no fun.
The text was updated successfully, but these errors were encountered: